WO2008057504A2

WO2008057504A2 - Load tracking system based on self- tracking forklift

Info

Publication number: WO2008057504A2
Application number: PCT/US2007/023310
Authority: WO
Inventors: James A. Aman; Delbert Jerard Aman; Paul M. Bennett
Original assignee: Aman James A
Priority date: 2006-11-06
Filing date: 2007-11-06
Publication date: 2008-05-15
Also published as: WO2008057504A3

Abstract

Self-tracking forklift (10) employs various onboard technologies including either machine vision (102), RF tag reader (104), or accelerometers with RF reset (106) to perform less costly 'inside-out' self-tracking verses 'outside-in' tracking such as GPS, RF triangulation or UWB cell network with onboard transponders. Forklift (10) further employs load dimension sensor (30), implemented as stereoscopic cameras (108), 2d cameras with laser line generators (110), or 3d area sensors (112) in order to constantly scan for loads (4) being either engaged, lifted, lowered or disengaged by fork lift arms (10L). In addition to times of load (4) engagement and disengagement, sensor (30) also captures outer dimensions of load (4) along with its image. Onboard computer (50) records times of load (4) engagement and disengagement along with load (4) outer dimensions, image and identity (input by asset tag reader (114)) to be combined with the current location and orientation of lift (10) for creating a virtual warehouse database (102).

Description

Load Tracking System based on Self-Tracking Forklift Related Applications

The present invention is related to US 60/857,351 , a provisional application filed on November 6, 2006 entitled LOAD TRACKING SYSTEM BASED ON SELF-TRACKING FORKLIFT, of which the present application claims priority. Field of the Invention

The present invention relates to apparatus and methods for determining the current location and orientation of a forklift as well as the presence, dimension and unique identity of any engaged load. This automatically sensed information is then used in combination to track the load in a three dimensional virtual warehouse where the load may then be re-identified simply be first determining the three dimensional coordinates of some central spot on any of the loads outer surfaces. Background and Summary of the Invention

In referenced U.S. Patent No. 5,604,715 entitled Automated Lumber Unit Tracking System, the present inventors taught the technique of tracking the location of loads, such as a packaged unit of lumber, by implication based upon a combination of the determined location of the carrying forklift and the load engaged / disengaged times. In order to best determine the current X, Y location of the forklift, the present inventors preferred the use of fork-lift external, triangulation based local positioning systems; taught examples included the use of GPS and / or RF triangulation. Indeed such systems are currently used in the market as anticipated in order to follow the ongoing locations of vehicles. In order to determine the load engaged / disengaged times, the present inventors preferred sensing the pressure level changes in the hydraulic lift system. And finally, to determine the "Z" height dimension indicative of how high or low the load was either being placed or picked up from a potential "stack" of loads, the present inventors preferred using some form of a distance measuring technology such as ultrasound to continually sense the height of the forks with respect to the lift. While the prior teachings are still instructive and beneficial for showing how load tracking may be accomplished via implication from forklift tracking, further opportunities are herein taught for alternative means of:

1. determining the ongoing forklift position without an external tracking system such as GPS or RF triangulation, and

2. determining load engage / disengage times simultaneous with the load height along with the additional information of load dimension.

With respect to tracking the forklift's ongoing position, in addition to GPS and RF triangulation systems, there are now also available local positioning systems based upon ultra-wide band, microwave and surface acoustical waveforms. With respect to ultra-wide band, at least the following companies currently offer local positioning systems:

• Time Domain Corporation of Huntsville, AL, and

• Ubisense of Denver, CO;

With respect to microwave based system, at least Trakus, Inc. of Boston, MA offers a LPS. And, finally, with respect to surface acoustical waveforms, there are several companies at least including RF SAW, Inc. of Richardson, TX and RF Monolithics, Inc. of Dallas, TX.

Furthermore, with respect to RF ID based local positioning systems, the present inventors are aware of WhereNet of Santa Clara, CA. Each of these methods for local positioning systems, i.e. UWB, microwave and RF all share the common features of:

1. At least one "tag" (i.e. passive) or "beacon" (i.e. active) whose location is tracked by;

2. One or more transceivers that receive either emitted, reflected, resonated or similar signals from the "tags" or "beacons."

The range of accuracy varies from within several feet (for WhereNet's RF ID tags) down to inches (for RF SAWs saw tags.) Typical use for these devices with respect to asset tracking is to place the tag on the asset that may then be tracked as it is moved about within a broader sensing area covered by one of more transceivers. This approach calls for minimal cost tags / beacons that are ideally available for placement onto a large number of items. All of these systems may be used, and are considered functionally equivalent to, the GPS / RF triangulation techniques taught by present inventors in U.S. Patent 5,604,715. Specifically, the use of each of the prior mentioned technologies or any functionally similar approach would in practice require the calibrated placement of multiple transceivers throughout the tracking area, e.g. a warehouse or lumber yard in the examples preferred by the present inventors. This transceiver network may easily become cost prohibitive and difficult to install. Regardless, once the network is in place, the forklift would then be fitted with one or more tags / beacons that are then tracked thus providing the lift's current location. As taught in U.S. Patent 5,604,715, the lift's current location along with times of load engagement and disengagement as well as fork height and load ID may be used to imply a three dimensional load tracking dataset. However, when used in this way to track the forklift / load from the "outside-in," all of the above technologies quickly become cost prohibitive as the desired tracking area increases.

It is therefore taught in the present invention that these same technologies are made more cost efficient by reorienting to track from the "inside-out;" hence a self-tracking forklift. More precisely, the present inventors prefer placing the transceiver on the forklift while then creating a calibrated matrix of tags / beacons throughout the tracking volume thus allowing each forklift to continually self determine its own location. In this approach, the more expensive transceiver portion of the tracking system is limited in quantity based upon the number of forklifts while the significantly less expensive tags / beacons are used to fill the larger expanse of tracking area.

Also taught herein, machine vision techniques may be use to accomplish similar goals. Specifically, one or more cameras may be affixed to the forklift in such a way to always be in view of multiple calibrated markings placed throughout the warehouse / lumber yard. As the forklift travels, the images captured by the one or more cameras are analyzed using standard image processing in order to extract the unique markings from the background scenery. Once located within the image, these multiple markings are sufficient to perform image based triangulation, thus providing the equivalent ongoing forklift location.

And finally, also with respect to the purposes of less expensive techniques for forklift self-tracking, the present inventors also disuses the use of accelerometers with local frequent resets to account for accuracy drift. Again, what is similar to all of these approaches, i.e. energy triangulation (UWB, microwave or RF₁) marker triangulation (machine vision,) or gravitational sensing (accelerometers) is that the forklift is made responsible for actively determining its own current X, Y position within the warehouse or yard. Also taught within the present invention, but now with respect to apparatus and methods for determining the load engage / disengage times required by U.S. Patent 5,604,715, the present inventors teach the use of a new class of three dimensional, depth-to-pixel cameras such as those sold by: • Canesta, Inc. of Palo Alto, CA, and

• 3DV Systems, LTD. of Yokneam, Israel

Both of these cameras provide a somewhat typical 2D area image with the added benefit of knowing the depth from the camera to each pixel. Similar to these single imager systems, the present inventors are aware of at least two companies that provide real-time stereoscopic devices for capturing equivalent depth-to-pixel image information:

1. Valde Systems, Inc., of Brookline, NH, and

2. TZYX, Inc. of Menlo Park, CA.

Regardless of the technology chosen, the present inventors teach the use of these or similar devices to automatically detect that a load is presently positioned on the forks. With this information and the knowledge that the lift is now moving up (i.e. lifting the load prior to movement,) the time of load engagement may be deduced without sensing the forklift's hydraulics. Of course, the same depth-to-pixel cameras may also be used to detect the raising motion of the fork and its load. Furthermore, once the forklift begins traveling this additional motion information becomes confirmation of the engage time.

With respect to disengagement, once the forklift becomes stationary this will be detected by the self-tracking apparatus. Furthermore, the depth-to-pixel imager will then subsequently detect the lowering of the fork and its load as the load is set down prior to the forklift backing up. Then, as the forklift continues its movement (in this case in reverse) this self-tracking information is combined with the determination that the load is getting further away via changing depth-to-pixel data. Thus the time of disengagement is also determined. The present inventors prefer such imagers over sensing changes in the forklift's hydraulics since at the very least the depth-to-pixel information may be additionally used to determine load size and shape data. As will be taught, the load's size and shape are of great use when creating a virtual warehouse dataset for later load locating and movement planning. Furthermore, the associated 2D image, at least as provided by the 3DV and Valde systems, may be used to:

1. provide the basis for standard image analysis detection of load identification markings such as, but not limited to, bar-code ID's and / or tag numbers;

2. immediately communicate with front office personal regarding load questions, and

3. creating a visual record of each load item as it is placed.

As will be understood by those skilled in the art of local positioning systems, it is possible to place the more expensive transceiver portion of the system on the forklift and use the less expensive tags or beacons to fill the warehouse or yard. Thus, regardless of the technology the teaching of the like provides a method for continuously self-determining the location of any given forklift. Furthermore, as will be understood by those skilled in the art of 3D imaging, it is possible to use such imagers to detect the presence or absence of a load on the forks as well as its movement up or down, or forward (on) and backward (off) the forklift. As will also be understood, these same imagers will be useful for determining the size and shape of each load. As will be understood by those skilled in the art of software databases and virtual warehouse animation, with sufficient knowledge of load dimensions and last placement X, Y, Z coordinates; it is possible to recreate a representation of all loads within the warehouse thus providing many benefits at least including load locating and movement planning. Furthermore, as taught in prior U.S. Patent 5,960,413 entitled Portable System for Inventory Identification and Classification, and as will be understood by those skilled in the art of 3D object databases, it is possible to use the remotely determined location of a laser spot projected onto any side of any tracked load to parse the virtual warehouse and determine the load's identification; as will be discussed herein.

It is therefore the object of the present invention to:

1. provide apparatus and methods for continually tracking loads carried by forklifts by first providing a less expensive "inside-out" means of forklift self-tracking, the forklift locations of which may then be used to imply the initial, current and final load locations, and

2. provide apparatus and methods for automatically determining load engagement and disengagement by the forklift while simultaneously determining load size and shape information as well as optionally a captured image for record keeping.

Still further objects and advantages of the present invention will become apparent from a consideration of the drawings and ensuing description. Description of the Drawings

Fig. 1 is a perspective side view of a forklift preparing to place its load, a pack of lumber, onto a stack of two other such packs. Also depicted are various data gathering devices attached to the forklift for triangulating the lift's ongoing position based upon the remotely detected and determined positions of multiple tags or beacons. These tags or beacons are depicted in a matrix type arrangement either attached to the ceiling support members or the floor. Also depicted are sensing devices for measuring the dimensions and presence of the load on the lift's forks as well as a portable bar code scanner for scanning any tag attached to the load in order to determine its identity.

Fig. 2a depicts the preferred and alternate base technologies that may be used to continually determine the forklift's current location from the lift's "inside-out" looking point of view. The technologies include machine vision, RF tag readers and gravitational motion detectors (accelerometers.) Machine vision uses one or more forklift mounted cameras viewing reflective or retroreflective markers strategically placed as shown in Fig. 1. RF tag readers use antennas to emit to, and receive responses from, one or more saw tags, RF ID'S or similar devices that have likewise been strategically placed as shown in Fig. 1. Gravitational motion detectors require multiple strategically placed reset devices as depicted most specifically in Fig. 5. Fig. 3 is a two dimensional view of a warehouse ceiling depicting a preferred strategic arrangement of reflective or retroreflective markers whose information is detectable using camera(s) and image analysis. The detected information includes a marker identifier, perhaps encoded as an alpha-numeric sequence that may be translated using standard OCR or even more general pattern matching, or encoded using something like a traditional one or two dimensional barcode. Along with the marker's identity, the markers specific orientation can also be determined. This combination of marker location and rotation will provide sufficient data to triangulate the forklift's current location and orientation. Further depicted is a series of captured image (rectangular) outlines taken along the path of movement curve of an example forklift. What is shown is that most images contain a view of one or more markers thereby facilitating either triangulation (via two markers) or location estimation based solely upon marker pixel size as well location and orientation within the image. Fig.4a is exactly similar to Fig. 3 except that the machine vision technology has been replaced by an RF reader and antenna gathering response energy from the matrix of tags, the energy of which can be used to determine the tags identity and location within the sensing antenna's field of detection. In Fig. 4a, the series of captured image rectangular outlines have been replaced by a series of omni-directional antenna signal circles. Fig.4b is time series depicting an alternative mechanically swept RF antenna. The swept antenna allows the

RF reader to look forward, overhead and backwards with respect to the forklift cab. This sweeping of the detection field collects additional data that is helpful for determining both location and orientation.

Fig.4c is a perspective view looking down, from the back, on the cab-top of a forklift. Shown on the forklift cab-tpp is an RF reader controller four antenna. Using at least two antennas with sufficiently overlapping detection fields so that they pick up the same marker is helpful for determining both location and orientation.

Fig.4d is exactly similar to Fig.4a except that the RF reader configuration now includes multiple shaped detection fields as would be created using either mechanical (or electrical) field sweeping or two or more antennas alternately placed on the lift. In Fig.4d, the series of omni-directional antenna signal circles have been replaced by a series of directionally shaped antenna signals.

Fig. 5 is similar to Fig. 3 and Fig.4 in that it depicts a regular arrangement of preferably passive markers.

However, this arrangement in Fig. 5 is placed on the warehouse floor as an alternate possibility to the ceiling and is used to perform intermittent resets for the "inside-out" forklift tracking technology that for example might include accelerometers whose accuracy tends to drift over absolute distance traveled.

Fig. 6a and Fig. 6b depict the use of a 3d imaging to both sense load presence and dimensions.

Fig.7a is a flow chart depicting the anticipated steps involved in the process of receiving new loads into warehouse storage.

Fig. 7b is a flow chart depicting the anticipated steps involved in the process of retrieving a known load from storage to take to an intended drop off point.

Fig. 7c is a flow chart depicting the anticipated steps involved in the process of rearranging loads in a bay in order to expose a desired load for retrieval.

Fig. 8 is a side view depiction of the concepts involved with translating from fixed locations of markers into a current location and orientation of a self-tracking forklift into the outer edge points of a load being placed into storage.

Fig. 9 is a perspective side view of a forklift preparing to place its load, a pack of lumber, onto a stack of two other such packs, similar to Fig. 1. In this case the forklift is no longer equipped with a sensing device for measuring the dimensions and presence of the load. Additionally, each pack of lumber also has placed on it a poach containing a unique bar code ID matching a unique RF ID, and more particularly a unique RF Saw

Tag.

Fig. 10a is a perspective diagram of a virtual warehouse, shed or storage area as might be rendered based upon the general and specific types of load location data determined and / or calculated by the self-tracking forklifts. Specifically represented are solid figures of stored loads made possible by the determination of the load's outer surface edges and corners. Also shown are three dimensional representations of load tags.

Fig. 10b is similar to Fig. 10a except that the solid figures of stored loads have been replaced by some indication of the loads' centroids.

Fig. 11 a shows three possible variations of databases that may be used to store some or all of the specific location data of each load, either determined or calculated by the present invention, and / or the general location data of each load, either directly input by the forklift operator, automatically determined by apparatus or indirectly derived from the specific location data from the self-tracking forklift.

Fig. 11 b is similar to Fig. 11a and shows further possible combinations of specific and general load location data. Fig. 12a is similar to Fig. 2 from related U.S. Patent No. 5,960,413, entitled Portable System for Inventory Identification and Classification and depicts a block diagram of a portable device that could be used to remotely identify any given load for which specific or even general location data is determined, calculated and stored in the virtual warehouse database as herein taught. The device of Fig. 12a differs from related Fig. 2 in that it depicts a self-tracking ("inside-out") portable device vs. a warehouse tracked ("outside-in") portable device.

Fig. 12b is a block diagram similar of an alternate embodiment for a portable device. However, unlike the device of Fig. 12a, the alternate embodiment does not require an understanding of its own X, Y, Z current location, as would be determined by either "outside-in" or "inside-out" tracking, in order to remotely identify any given load for which specific or even general location data is determined, calculated and stored in the virtual warehouse database as herein taught.

Fig. 12c is a perspective drawing of the portable device depicted in Fig. 12b.

Fig. 13a depicts the conceptual use of a portable device such as discussed in relation to Fig. 12a being used to first remotely select a load by projecting a visible spot somewhere onto its outer surface, after which the location of the projected spot is translated into at least that load's identity using the information stored in the virtual warehouse.

Fig. 13b depicts the same portable device of Fig. 12a being used to first remotely select a load by projecting a visible spot onto its tag after which the location of the projected spot is translated into at least that load's identity using the information stored in the virtual warehouse.

Fig. 13c depicts the portable device of Fig. 12b and Fig. 12c, being alternatively used in place of the device in Fig. 12a, to first remotely select a load by projecting a visible spot onto its RF ID / tag after which the lidar measured distance to the projected spot is translated into at least that load's identity by comparison with the RF determined distance to all RF ID / tags within the given detection field, using the information stored in the virtual warehouse. Specification

Referring to Fig. 1 there is shown self-tracking forklift 10 moving about within warehouse 2. Forklift 10 is currently carrying load 4 (with ID 5448,) on its forks 1OL that it is about to place onto load stack 4s. Forklift 10 has been fitted with ceiling orientated self-tracking device 20a that periodically detects one or more ceiling markers, for example 7-1 through 7-15. Ceiling markers 7-1 through 7-15 are affixed in known and calibrated locations to warehouse 2 support members 2-1, 2-2 and 2-3. In addition to or instead of ceiling oriented self- tracking device 20a, forklift 10 is optionally fitted with floor oriented self-tracking device 20b that periodically detects one or more floor markers, for example 6-1 through 6-11. Furthermore, in addition to or instead of ceiling oriented device 20a and / or floor oriented device 20b, forklift 10 is optionally fitted with floor oriented location reset device 20c that periodically detects one or more floor markers, for example 6-1 through 6-11. The preferred and alternate technologies employed by self-tracking devices 20a, 20b and 20c will be discussed in greater detail with respect to upcoming Fig. 2 and in general throughout the remainder of this application. What is important to note is that each of these devices, 20a, 20b and 20c are actively (i.e. powered) working to self-determine the forklift 10's current position within the warehouse by detecting the positions of the calibrated affixed markers, such as 7-1 through 7-15 and 6-1 through 6-11. As will be shown, the present inventors will teach three technologies that all work with passive (i.e. not powered) markers, which is preferred. The present inventors will also discuss alternative technologies that fall within the teachings and scope of the present invention but employ the use of active (i.e. powered) markers. As will be understood by those familiar with warehouse operations, it is preferable that markers 7-1 through 7- 15 and 6-1 though 6-11 be both minimal in quantity therefore requiring less time to install and non-powered (i.e. passive) thereby lasting indefinitely without requiring maintenance. As will also be appreciated by those skilled in the art of local positioning systems, all of the technologies suggested for devices 20a, 20b and 20c will work in an outdoor setting as well as within a warehouse. For instance, at a lumber yard there are often outside hangers where packs of lumber are stacked, such as 4s. Markers such as 7-1 through 7-15 may be placed on the ceilings or outer faces of these hangers. Markers such as 6-1 through 6-11 may be strategically placed in the ground throughout the yard area, especially in locations where lumber is typically piled in the open air. Again, it will be understood by those skilled in the art of the taught technologies for implementing devices 20a, 20b and 20c, that in general the markers should be strategically placed to fall within the requirements of the chosen technology. For instance, if camera based machine vision is to be used, the markers must maintain a line-of-sight to the cameras and therefore their placement is dependent upon where the camera, now acting as 20a, 20b or 20c, is placed and oriented. If surface acoustical waveform readers are used, then it not necessary to maintain a direct line-of-sight; however, it is preferable to maintain open air pathways for the tracking energies to transverse with minimal barrier absorption. Therefore, for the remainder of this application when the present inventors refer to warehouse 2 it will be understood that the teachings are not to be limited to warehouses or other indoor spaces, but that the teachings herein simply require that markers be affixed in some permanent and calibrated fashion to the local structures and / or environment surrounding the locations where the loads are carried and or stored.

Still referring to Fig. 1, self-tracking devices 20a and 20b are additionally capable of reading marker 8-1 placed on fork 10L, thereby periodically providing information on the current height of fork 10L. As will be discussed in the ensuing specification, determining the height of fork 10L by detecting the current location of marker 8-1 (or by any other apparatus or method) will be most desirable and necessary at the times of load engagement and disengagement. As will be understood by those skilled in the art of 3D positioning systems, the marker such as 8-1 will be in a fixed relative position in two orthogonal axis, e.g. X (width across, or side- to-side, with respect to forklift 10) and Z (depth away from forklift 10), with respect to self-tracking devices 20a and / or 20b and / or 20c, held in a fixed position of forklift 10. Therefore its movement will be limited to a single Y (height, i.e. up and down) plane perpendicular to the fixed X-Z plane. Marker 8-1¹S current position in this plane is easily calculated based upon a changing distance from devices 20a and / or 20b and / or 20c. It should be noted that at any given instant the distances from a single self-tracking devices such as 20a to a single marker such as 8-1 will yield potential ambiguity if, and preferably, only one self-tracking device, such as 20a, 20b or 20c is used and it is placed such that its fixed X-Z horizontal plane intersects the potential Y movement of marker 8-1 within its full range (therefore device 20a, 20b or 20c is not fully above or below marker 8-1 in terms of the possible movement of marker 8-1 in the Y (up-down) direction.) There are three sufficient means for remedying this situation as follows:

1. Preferably, use at least two markers such as 8-1 and 8-2 (not shown) separated by sufficient distance in the Y dimension, where both are attached to fork 10L, so that triangulation may be performed; a. Furthermore, each of these markers is ideally placed at different Y heights along fork 1OL such that they cross the X-Z horizontal plane intersection of the tracking device 20a, 20b or 20c at different times;

2. Alternatively, use at least two self-tracking devices such as 20a, 20b or 20c, or as a variant use two cameras or antennas with a single device 20a, 20b or 20c, so that triangulation may be performed, or

3. Alternatively, store multiple sequences of Y (up-down) readings for single marker 8-1 taken by single self-tracking device 20a, 20b or 20c, where the time sequence reveals additional information that may be used to correctly infer the forks position, specifically: a. As marker 8-1 travels upward towards the intersecting horizontal X-Z plane the distance from single tracking device 20a, 20b or 2c will be decreasing as it reaches its minimum distance exactly at the intersection of horizontal X-Z plane with respect to perpendicular marker 8-1 travel plane X-Y. Thereafter, as it continues to be raised above intersecting horizontal X-Z plane the distance from device 20a, 20b or 20c to marker 8-1 will reverse and begin to increase. b. Furthermore, as long as device 20a, 20b or 20c is not placed exactly or roughly in the middle of the possible full Y movement of marker 8-1 , then over time as the fork 10L is moved up and down it will at least reach its lowest point in the Y direction (i.e. fork 10L is resting on the ground) equaling its largest possible distance from tracking device 20a, 20b and 20c. Once this distance, or a threshold representing "further down" than possibly "up" is reached, from this point forward it is straightforward to deduce from the changing distances whether the marker 8-1 is moving up or down with the fork 10L, as will be understood by those skilled in the art of object tracking systems.

Still referring to Fig. 1 , as self-tracking devices 20a, 20b and 20c periodically gather the current X, Y location, direction of travel and orientation of the forklift 10, expressed in any appropriate coordinate system, this information is directly translatable to the current centroid location of loads such as 4. As will be understood by those skilled in the art, since the devices 20a, 20b and 20c are affixed to permanent locations on the forklift 10, then it is possible to extrapolate from the X, Y position of either device 20a, 20b or 20c to the X+n, Y+m location of the middle of forks 10L. Since any load such as 4 will be carried resting upon forks 10L, then the middle of forks 10L can safely be assumed to be the width-wise center of any load 4. After having extrapolated to this first width-wise center of load 4, the depth and height centers of load 4 may be presumed or detected, both methods of which will be taught herein. Furthermore, while determining the loads centroid with respect to the warehouse is sufficient for many beneficial functions including on-going load tracking and identification, it is preferable to also determine the size and shape of each load thereby allowing for an understanding of the locations of the loads 4 outer surfaces within the warehouse 2. These particular coordinates have been portrayed in Fig. 1 as points "p1 (x, y, z), p2 (x, y, z) ... p8 (x, y, z)" etc. on loads similar to 4 but in stacks 4s, and specifically for example identified as packs labeled 5124 resting on 6372, and 5447 resting on 5388.

Still referring to Fig. 1 there is also shown portable hand held bar code scanner 40 that is capable of remotely reading bar coded information that is typically used for tagging loads such as 4. Scanner 40 is preferable in wireless communication with either or both forklift computer 50 or inventory control computer 60 and is used to identify the load 4, as will be understood by those skilled in the art. As will be shown, this identification is most beneficial when receiving new loads 4 into the warehouse, for instance after delivery from a vendor or creation during a work-in-process step. As such initial junctures, it is typical for a warehouse / inventory control procedure to include a step of printing up a paper tag uniquely identifying the load 4; for example and in this case tag 5448. Such tags often additionally include a bar-code representation of the tag number for scanning purposes. Furthermore, the tag often at least includes a "product number." Whether or not such a product number is printed, it will be associated with the unique tag number of load 4, e.g. 5448, within inventory control computer 60. Using this association, it will be possible for forklift 10 to determine the general dimensions of a typical load via the given product code. For instance, the product code may indicate that the lumber of load 4 is at most up to 16 feet in length. In this case, there is an added understanding that lumber is typically packed in 4' wide by 4' high loads. Hence, it is only the length of 16' that is missing for a general understanding of the load 4's storage dimensions. In many cases, this approach of reading the tag number, e.g. 5448, with bar code scanner 40 and then using this information to look up the standard dimensions as stored in inventory computer 60 will be sufficient to adequately translate from forklift 10's current location to the edge / corner locations of the outer surfaces of a typical load 4, such as points "p1 (x, y, z), p2 (x, y, z) ... p8 (x, y, z)" etc. Note that the 16' load length becomes a "width-wise" dimension when the load 4 is picked up by the forklift 10, as will be understood by those familiar with forklift and warehouse operations. However, in many other cases, this information may not be sufficient or may be less than desirable. In this case, and still referring to Fig. 1, there is shown load dimensioning sensor 30 with example field-of-view 3Ov that projects onto the forks 10L and therefore their loads, such as 4. The specific technology preferred for implementing dimension sensor 30 will be discussed in more detail especially in reference to Fig. 2 and the subsequent specification. In general what is needed is an indication of both the size and shape of at least the surface of load 4 that is facing the cab of the forklift 10. Preferably, dimension sensor 30 is affixed to forklift 10 and therefore always available to gather load shape and size. As will be understood by those skilled in the art, it is also possible to build such sensors using the identical technologies preferred by the present inventors, as a drive-through configuration not attached to the forklift 10. This approach would allow multiple forklifts 10 to share one load dimensioning sensor 30 by establishing some central location where each forklift 10 would at sometime pass through while carrying a new load 4. Obviously, multiple drive-through stations could also be established. What is important to the present invention is that the forklift 10 be able to automatically determine both size and shape information concerning load 4 from which it may then extrapolate the load's outer surface dimensions. While not a limitation on the present teachings, it will be understood in the art that most loads take on only two basic shapes: the most prevalent being a rectangular volume, such as a pack of lumber, a storage box, a pallet of bags or a long beam, while also common is a cylindrical volume, such as a drum or roll of metal or paper or a bundle of extruded bars (turned on end roughly equals a cylindrical shape.) This knowledge is useful as will be discussed for determining load points to track "p1 (x, y, z), p2 (x, y, z) ... pδ (x, y, z)", etc. and it is typically pre-known and determinable by knowing the product code. And finally in reference to Fig. 1, as will be understood by those skilled in the art forklift computer 50 will be in communication, via either wire or wireless, will all self-tracking devices 20a, 20b and 20c, as well as load dimension sensing device 30 and bar code scanner 40. Computer 50 is also in communication via a wireless network with inventory control computer 60. Referring next to Fig. 2a, there is shown a table of several base technologies to accomplish the desired forklift 10 self-tracking via devices such as 20a, 20b and 20c. Specifically, each device 20a, 20b or 20c may employ a machine vision system 102 comprising one or more fixed cameras that together form either overlapping or separate views of some portions of the warehouse; preferably but not limited to either the ceiling or floor, or both. As will be understood by those skilled in the art of machine vision, these cameras may be either monochromatic or color and may operate in the visible or non-visible spectrums. What is important is that the total field-of-view of devices 20a, 20b or 20c, includes at least one marker and preferably two at the necessary forklift 10 position determining time. In reiteration, only one device 20a (oriented for instance towards the ceiling,) 20b (oriented for instance towards the floor) or 20c (oriented towards the floor and in close proximity to the floor) needs to be present on the forklift 10. As previously stated, it may be beneficial to have more than one device, such as 20a and a 20b, or 20a and a 20c, or 20b and 20c, or 20a, 20b and 20c, so that markers such as 7-1 through 7-15 and 6-1 through 6-11 can be detected in different settings as optimal for the warehouse layout.

With respect to an implementation using machine vision 102, as will be understood by those skilled in the art, since the camera, or cameras making up device 20a, 20b or 20c will be affixed to the forklift 10 in a permanent and pre-known position, and since the forklift will tend to maintain a constant height to the floor, then any cameras used will therefore also have a fixed pre-known z height off of the warehouse floor. Furthermore, each one or more cameras orientation with respect to its optical axis and the north-south directional axis of the forklift 10 will be pre-known. Therefore, by first determining the cameras current x, y location within the warehouse, the x, y location of the forklift's centroid may also be known. Additionally, by determining the angle of orientation of each one or more camera's optical axis to one or more markers, then this may be translated into the forklift 10's angle of orientation and therefore current instantaneous direction of travel, all as will be understood by those skilled in the art of mathematical systems especially including trigonometry.

As will be further understood by those familiar with the art of machine vision, it will be possible to read even a single encoded marker and from this determine any fixed camera's current x, y position and optical axis rotation with respect to the markers. Again, since the camera's height z and orientation with respect to the forklift 10 are pre-known, the camera(s) acting as 20a, 20b or 20c may then have their determined position translated to the forklift's centroid, warehouse orientation and instantaneous direction of travel. For instance, this single encoded mark is preferably a triangle of pre-known dimensions where one side is clearly distinguishable as the base. Using such a triangular mark with distinguishable base will allow the vision system to determine orientation, based upon the detected triangles rotation within the image, and distance based upon the triangles distortion and size. Such techniques are well known in the art. The present inventors additionally prefer some rough equivalent to a bar code that is perhaps within the triangle or just outside, perhaps adjacent to its base side.

Once located within the image, this identification code will itself be translatable into the pre-known x, y, z location and orientation of the individual marker within the warehouse. Ideally, after the markers are permanently affixed to either the warehouse 2 ceiling members 2-1 , 2-2 or 2-3 or the warehouse floor, or any other appropriate part of the warehouse 2, each maker's centroid is calculated along with the centroid of its detectable features (e.g. the triangle comers) and therefore via simple calculation the rotational angle of this group of features with respect to the warehouse 2. Using the markers determined identification code will then provide a means for recalling this associated pre-determined information so that the detected triangles centroid and corner centroids may provide the necessary numerical information to be translated into the cameras current x, y location and orientation. As will be understood, using basic trigonometry, the cameras position may be calculated based upon the determined image location of each of the triangles three corner points.

However, it is the preference of the present inventors that at least two such triangle markers be in the view of any camera based sensor used for 20a, 20b or 20c at the time when any forklift 10 s location needs to be determined. Having two or more triangles in view of either the same camera or of multiple cameras with a combined larger field-of-view allows the triangle's centroids to themselves form a shape. Note that three concurrently visible triangles together form a larger triangle and that since each individual triangle will carry a unique identifier, the combination of the three markers will form a uniquely oriented larger triangle with exactly known corner points (i.e. the centroids of the individual markers themselves, such as 7-1 through 7-15 or 6-1 through 6-11.) Using this information, in addition to the pre-known z distance of the camera off the warehouse floor (based upon its fixed position on the forklift 10) and its orientation with respect to the forklift, the forklift 10's current location and orientation with respect to the warehouse may be calculated - as will be understood by those skilled in the art of machine vision.

Also referring to machine vision 102 as a base technology, the markers, such as 7-1 through 7-15 or 6-1 through 6-11 are ideally either reflective or retroreflective. If retroreflective markers are used, then the one or more cameras employed for devices 20a and / or 20b and / or 20c will require ring lights, as will be understood by those knowledgeable with retroreflectors. Due to the significantly increased cost of retroreflectors vs. normally reflective inks, the present inventors prefer using a reflective marking, which may be large enough in the given setting and therefore more easily detectable without the increased energy reflection benefits offered by retroreflection. Finally, with respect to any markers used, whether on the ceiling or floor or some other part of the warehouse, whether inside or out, the total number of markers is expected to be well in excess of that depicted.

Still referring to Fig. 2a, rather than using machine vision 102 as a base technology, the present inventors prefer using an RF saw tag reader 104, such as manufactured by RF Saw, Inc. of Richardson, TX. In this case, one reader may be attached to the forklift 10 as device 20a, 20b or 20c. This reader may have itself attached one to four antennas. Using the attached antenna, the RF reader 104 is capable of detecting multiple individual saw tags at distances of up to 100' with current accuracies of +/- 2 inches. Exactly similar to the strategy and methods employed by machine vision, each saw tag is conceptually identical to each triangle's centroid. Therefore as previously taught, if the distance from the RF reader 104 to two or more saw tags is detectable, and the distance between these two or more saw tags is also pre-known, than using simple trigonometry the current x, y location of the RF reader may be calculated. All that is required is that the RF reader 104 and its antennas are first affixed in pre-known locations and orientations with respect to forklift 10 and that the each saw tag used as markers such as 7-1 through 7-15 or 6-1 through 6-11 are uniquely identifiable and first located in x, y, z space with respect to the warehouse such that by using their unique identification codes these x, y, z coordinates are recallable for calculation as the marker's centroid. As will be understood by those skilled in the art of local positioning systems, by using at least two antennas per reader, where each antenna is preferably separated by the largest convenient distance as will be discussed further in relation to Fig.4c, than it is possible to easily determine the forklift 10's orientation. Essentially, each antenna plus a single detected marker then form a triangle with the distances of all sides either detectable or pre-known. Even if a forklift 10 could then spin in a single point just below the exact center between the two detecting antenna, while the determined forklift 10 location would not change, the detected distance from each antenna to the same marker would change thereby providing forklift 10's orientation. Although technology pricing is quickly dated and tends to decrease significantly over time, current market conditions place the cost of a single camera / capture card and computer at no less than half the cost of an RF reader 104. Furthermore, lower cost cameras still typically capture images only at 30 fps or at best 60 fps whereas the current RF reader can operate at 10,000 reads per second. The output of the RF reader 104 is easy to process in that it is a stream of found marker id's along with a distance to each marker. The output of a camera is its captured image which is not only significantly more information but must also be processed via image analysis algorithms in order to detect the presence, size, shape, etc. of any markers. Furthermore, because of the extreme uniqueness of the energy waveforms, the RF reader 104 is not as susceptible to background noise within the surface acoustical signal space, since a typical warehouse will neither generate ambient signals resembling a saw tag resonance nor will it have surfaces capable of resonating similar to a saw tag, thus creating false positives. Most difficulties, if any, might be expected from signal reflections but these should be easily handled as the shortest distance to any detected saw tag will always be assumed as the proper distance. Otherwise, the only potential interference will come from other wireless electrical devices sharing the same frequency space of 2.4 GHz. In this case, the readers fast cycle time is expected to overcome this potential interference.

However, with a machine vision system 102 operating broadly across the visible spectrum, a warehouse is expected to contain many potential reflecting surfaces that can disrupt or confuse an image processing algorithm. As will be understood by those skilled in the art of machine vision, this potential can be greatly minimized by using unique shapes or colors. For instance, given proper warehouse lighting, the markers could all be in red and any detecting cameras could use frequency matched red band-pass filters thus somewhat removing ambient noise, especially since there are potentially very few read surfaces in a typical warehouse. Then, by also using triangular shapes and by having these on the ceilings, the machine vision technology can begin to approach the signal to noise ratios expected for RF saw tag technology. However, adding camera filters adds cost and making markers larger adds complication. Hence, the simplicity of placing fewer small saw tags on the order of 2.5" x 1" by 1/16^th" vs. more larger red triangular markers, is desirable (but not necessary).

And finally in comparison between the base technologies of machine vision 102 vs. RF reader 104, the machine vision 102 approach is more susceptible to variations in ambient lighting and requires a constant direct line-of-sight, at least complicating the design of marker placement. Conversely, the RF readers 104 generate their own detection energy output in the form of an interrogation pulse that is output through the antenna, resonated off the saw tags, and then recaptured by the antenna and therefore could actually perform in low to no warehouse lighting. While the RF readers 1042.4 GHz frequency does prefer some air path to each marker, it is not necessary that this path be direct. For all of these above stated reasons, and others that will be understood by those skilled in the art of both base technologies, the present inventors prefer, but does not require, the use of RF readers 104 vs. machine vision 102.

Other technologies could be employed that are similar to RF readers 104. Specifically, Trakus, Inc. of Boston, MA sells a similar reader - tag system. In this case the chosen frequency of energy is microwave which is generally perceived more negatively than 2.4 GHz. Additionally, the tags are powered and must emit signals rather than resonate emitted antenna energy. At least for these two reasons RF saw tags are preferred over Trakus's microwave beacons. There are several companies, such as Ubisense of Denver, CO that sell local positioning systems based upon UWB (ultra-wideband). In this case the tags are larger, more expensive and also powered. Furthermore, the accuracy of detecting these tags is still only within +/- 6" at best and more often +/- 12". And finally, at least Ubisense's system would prefer a cell of four transceivers to be built onto the forklift along with an attached computer for signal processing, the total cost of which currently exceeds the cost of an RF saw tag reader plus antennas. For these reasons, the present inventors again prefer using the RF saw tags. However, all of the alternatives would work, as will be understood by those skilled in the art. Therefore, for the purposes of the present invention, they are all examples of the general concept that the forklift 10 actively determines its own position, the position of which is then translatable into the load 4's stored position. None of the mentioned limitations and tradeoffs of the potential apparatus base technologies should be construed as a limitation on the methods or scope of the present invention. Still referring to Fig. 2a, there is another low cost technology that is currently suitable and in use for self- tracking of a vehicles current location. Specifically, accelerometers 106 may be used for this function within self-tracking devices 20a, 20b or 20c. While any placement on the forklift 10 is possible, the present inventors prefer locating the accelerometers 106 in device location 20c, i.e. near the warehouse floor. While not necessary, this facilitates also including in the device 20c a reset means such as RF chips embedded into the floor. As those familiar with the art of accelerometers 106 will understand, over the course of time and distance, their minor directional errors will tend to add up such that eventually there overall current error will be in excess of that necessary for accurate system performance. The well understood remedy for this increasing error drift is to introduce reset points along the expected path of the vehicle, such as forklift 10. Many such reset mechanisms are possible. All that is required is that as the vehicle passes over a given point, at that instant the point can be detected and translated into a more accurate current x, y location and orientation of the forklift 10 than may be currently known by only the accelerometers 106. The present inventors prefer a very low cost and durable reset mechanism where the embedded reset marks are accurately located. Machine vision 102 would be a good choice for this in that low cost ruggedized cameras could have their fιeld-of-view continuously swept across the warehouse floor looking for recognizable reset marks. However, for dirty environments these markers and the cameras themselves are more susceptible to "false" or "no reads." Furthermore, many marks might need to be placed on the warehouse floor because of the small distance between the lift's underside and the floor itself. The reset marks along with the accelerometers 106 could be placed into self-tracking device 20a and oriented towards the ceiling. However, in this case the cost of detecting the reset marks using either machine vision 102 or RF readers 104 would equal that of simply using these technologies as the only approach, i.e. without accelerometers 106. Therefore, due at least to the requirement for excessive placement of floor marks, such as 6-1 through 6-11, as well as the difficulty of placing these marks outside in the yard, whether they are visible marks for machine vision or RF chips for RF readers, the present inventors still prefers the use of RF Saw readers 104.

Referring next to Fig. 2b, there is shown a table of several base technologies including 3D machine vision systems A 108, systems B 110 and systems C 112 to accomplish the desired load 4 size and shape measurement via load dimensioning device 30. Specifically, when using system A 108, device 30 may comprise 3D stereoscopic cameras such as available from either Valde or TZYX. As will be understood by those skilled in the art of machine vision system, the camera based stereoscopic systems capture two 2D simultaneous overlapping images of the field-of-view from slightly separated positions and then perform parallax or similar calculations in order to provide estimates of the depth to each pixel or pixel group within the 2D images. For the purposes of the present invention all that is necessary is that the stereoscopic cameras functioning as load dimensioning device 30, be preferable fixed in a calibrated position on forklift 10, such that device 30 may preferably view a maximum load-view area 3Ov of the forks 10L through their possible movement range. Viewing this maximum area 3Ov allows device 30 to estimate with reasonable accuracy the following information:

1. Is there a load present on the forks 10L?, especially: a. during the time that the forks 10L are being lifted, or b. during the time the forklift 10 is moving.

2. For the load 4 that is present, what is its detectable: a. width along the X axis?; b. height along the Y axis?, and c. depth along the Z axis?

Using this information, the current forklift 10 position and orientation at the time of load 4 engagement or disengagement may be directly convertible to not only the X, Y, Z centroid of load 4, but further to the X, Y, Z points describing each of load 4's possible outward faces as will be discussed further especially in reference to Fig. 8. Referring back to Fig. 1 , example load 4 is a pack of lumber conforming to a rectangular volume that will have eight corner points, such as p1 (x, y, z) through p8 (x, y, z), as will be understood by those skilled in the art of geometric shape modeling. As was also originally taught in the referenced patents, by extrapolating to corner points then the resulting 3D warehouse model will contain an understanding of where the reflecting surfaces, or faces of each placed load 4 are with respect to any inquiring individual or forklift 10 at all times. Thus, as will be discussed, it is possible for an individual to project a 3D tracked laser spot on any of the exposed load 4 reflecting surfaces, after which the 3D warehouse model will be able to accurately determine which load is being inquired upon. (This technique was originally taught especially in U.S. Patent 5,960,413 by the same lead inventor and will be discussed again with respect to the use of self-tracking base technologies such as those discussed with respect to Fig. 2a.)

While the base technology of system A 108 is well understood and possible for the present purposes of load 4 measurements, using 3D stereoscopic cameras is not preferred for several reasons, including:

1. The current pricing on such devices is more expensive than the other solutions;

2. There is not a great stand-off distance between desirable possible locations on forklift 10 to affix device 30, meaning that a single stereoscopic device may not sufficiently view the entire possible load 4 (e.g. a 4ft x 4ft x 20ft pack of lumber), and

3. Being camera based this solution is susceptible to degradation as the field-of-view 3Ov becomes blocked over time, e.g. as dirt and dust accumulates;

Still referring to Fig. 2b and staying within the same purposes and concepts, an alternative base technology to stereoscopic cameras for use in load dimensioning device 30 is 3D machine vision system B 110, comprising 2D cameras plus projected laser line patterns. As will be understood by those familiar with 3D machine vision, projecting calibrated laser line patterns onto a target object from a fixed and known position at least with respect to a 2D imaging camera, may be used to determine load 4 size and shape. While this technique is well understood and possible for the present purposes of load 4 measurements, using a 2D camera plus laser line generators is not preferred for several reasons, including:

1. Unless the laser lines are swept, the information provided is limited in accuracy to J4 the distance between any two separated laser lines, thus tying increased accuracy to increased costs and complexity;

2. The creation of a special read-head onto which a 2D camera plus laser line generators would need to be mounted adds complexity and is not practical for installation on a forklift;

3. There is not a great stand-off distance between desirable possible locations on forklift 10 to affix device 30, meaning that a single stereoscopic device may not sufficiently view the entire possible load 4 (e.g. a 4ft x 4ft x 20ft pack of lumber), and

4. Being camera based this solution is susceptible to degradation as the field-of-view 3Ov becomes blocked over time, e.g. as dirt and dust accumulates;

Still referring to Fig. 2b and with respect to the same purposes and concept of automatically determining load 4 presence and dimensions on forks 10L, another alternative is to use 3D machine vision systems C 112, comprising 3D sensors such as provided by Canesta, Inc. These sensors operate within a traditional digital camera design and provide a 2D monochromatic image. Their added benefit is that these 3D sensors have been modified at the silicon level to perform sophisticated time-of-flight calculations per individual pixel. This information provides fairly accurately (to within inches) depth information per pixel out to a range of approximately thirty feet. Furthermore, they are relatively inexpensive, especially in comparison to 3D stereoscopic systems. For these reasons, the preferred configuration of load dimensioning sensor 30 with respect to system A 108, system B 110 or system C 112, is system C 112 and includes one or more cameras using Canesta's 3D imaging sensor to create field-of-view 3Ov that sufficiently covers the types of loads 4 anticipated by the warehousing application.

However, being a camera based system, like the prior base technologies of stereoscopic cameras and 2D cameras with line generators, the 3D sensor camera is also susceptible to degradation as the field-of-view 3Ov becomes blocked over time, e.g. as dirt and dust accumulates. Also, there is added cost and complexity for any such system and for at least these two reasons, the present inventors prefer using a more traditional asset tag reader 114, comprising bar code scanner 40 as the base technology for determining load 4 dimensions. As will be understood by those skilled in the art of inventory control systems, it is not unusual that all loads of a given product are packaged in exactly or at least roughly equivalent sized dimensions. Hence, product XYZ is always stored in containers of size Height = 999, Width = 999 and Length = 999. Occasionally, and especially for "natural" products such as lumber, the same load of a given product XYZ will have different dimensions. In practice and with respect to packs of lumber, all packs, of all products, tend to be the same width such as 4' because this is optimal for the forks 10L on a forklifts such as 10. Therefore, what remains to be known is the length and height of the pack, i.e. load 4. With lumber, the packages will either have a fixed or random length, but in any case they are typically pre-assigned a maximum length. Therefore, by using a bar code scanner to first identify the product code, computer 50 on forklift 10 may inquire into its local inventory database to determine the preset / standard dimensions. In the specific case of lumber, once the length is know and added to a standard pack width, all that remains is to ascertain the pack / load 4 height. For this purpose, the present inventors prefer assigning the height to a pre-known maximum for that product and adjusting this defaulted height if and when some second load is placed on top of the current load 4. As will be understood by those familiar with warehouse operations, it is customary to stack large sized loads similar to loads 4s depicted in Fig. 1. In that case, when a load such as 4 is placed onto any other given load or stack of loads, the fork 1OL height at the time of disengagement not only serves to mark the bottom of the current pack 4 but also the top of the load 4 it is being placed upon. Hence, if the first load 4 onto which the current second load 4 is placed, did not have an exactly measured height, this process of stacking itself now determines this height. The present inventors prefer to gather this height information without using load dimensioning device 30 because of its simplicity aηd inexpensive implementation (given the understanding that a warehouse is already likely to be using bar code scanners.) Obviously, there will be situations where it is desirable to have the actual dimensions including the height of load 4, and perhaps even a digital photograph of load 4 captured at the point of storage. In those cases the present inventors prefer the use of the Canesta or similar 3D image sensors, although as taught, several viable base technologies exist. Still with the purpose of determining load 4 dimensions, the present inventors also includes by reference the prior issued U.S. Patent No. 5,307,294 entitled Automated End Tally System. The company, to which this patent is assigned, Industrial Vision Systems, Inc. of Bryn Mawr, PA, has built and know sells the system as specified in this patent. As is often the case, after a pack of lumber undergoes a repackaging transformation, it is desirable to count the total number of boards along with their sizes within the pack prior to warehouse storage; the count of which is known in the lumber industry as a tally. As will be appreciated by those familiar with lumber yards, this tally that is automatically determined by the Automated End Tally System, inherently provides the outer dimensions of each pack / load 4. Therefore, given the use of the present invention at least within a lumber yard setting, the present inventors specifically teach and prefer that the load dimensions are first determined during this automatic tally step after which they are made available to the inventory control system 60 and therefore also to forklifts such as 10 for storage on forklift computer 50. Then, by using bar code scanner 40 that is in communication with forklift computer 50, the exact dimensions of load 4 become known without requiring any such load dimensioning device 30.

As will be understood by a careful reading of the teachings within U.S. Patent 5,307,294, the Automated End Tally System should not be considered restricted to scanning packs of lumber and in fact is applicable to the counting or "tallying" of other similar products such as bundles of metal rods. What is important is that the combination of the teachings of the Automated End Tally System and the present invention provide an ideal solution for both determining the board dimensions within a pack of lumber (or similar product) as well as the 3D storage location of that same pack within a lumber yard, where the yard may comprise potentially several warehouses and outdoor storage locations.

Still referring to Fig. 2b, as will be understood by those skilled in the art of asset tag reading systems 114, rather than using bar code scanner as the base technology for device 40, other well known technologies exist such as RF tag readers. Using this technology, inexpensive and RF tags / chips are placed onto the packs / load 4 paper tag. Using an appropriate reader, it is then possible to identify this tag by decoding the chips energy response to an interrogation pulse. In any case, all that is necessary is that some technology is used to translate from the pack / load 4 identity information that might typically be printed / encoded on a paper tag attached to load 4, is used to communicate to forklift computer 50 the unique identity of load 4. Referring now to Fig. 3, there is depicted a ceiling of a storage warehouse onto which a matrix of identification labels has been placed marking both a major transport aisle and accessible storage bays. As discussed in relation to Fig. 1 and Fig. 2, these visible markers are for use with a machine vision based technology 102 to implement ceiling oriented self tracking device 20a. It should be understood that while these markers are shown to be mounted onto the warehouse ceiling, this is a preference but not a limitation since the markers could easily be placed on any surface within the warehouse, whether ceiling, floor or wall, that is expected to remain in view of any of the self tracking devices such as 20a, 20b or 20c. These types of markers are preferable because they are very low cost and relatively easily mounted. In operation, as forklift 10 moves about the warehouse it may periodically capture images using device 20a such as ceiling view 20a-v1 if a single camera is being used, and additionally ceiling view 20a-v2 if a second camera is used. Show within view 20a-v1 are aisle maker 7-am-19 and bay marker 7-bm-20-4. As previously discussed and as will be familiar to those skilled in the art of machine vision, using standard machine vision algorithms these markers can be easily detected within image view 20a-v1. Once detected, their size, shape and identity can be used to ascertain the forklift 10's current location and orientation. If necessary, additional view 20a-v2 may also find markers such as aisle maker 7-am-19 and bay marker 7-bm-7-4. Depending upon the exact construction of the marker, multiple feature points may be sufficiently discernible within even a single marker such as 7-am-19, 7-bm-7-4 or 7-bm-20-4 so that self-tracking device 20a may determine forklift 10's current warehouse location and orientation. In this case only one camera in device 20a is necessary and fewer rather than more markers may be placed on the warehouse ceiling, thus easing installation and calibration. However, if machine vision is used, the present inventors prefer a matrix of ceiling markers that would typically provide two or more unique markers to any given single camera view 20a-v1 of device 20a. Still referring to Fig. 3, it is further noted that while continually tracking the forklift 10's movement as it transverses a warehouse has benefit, for the purposes of determining the final storage position of a given load 4, or for confirming an engaged load 4's identity by detecting its location at the time of engagement, it is not necessary to continuously determine the forklift 10's location and orientation. Furthermore, unless loads such as 4 may be temporarily stacked in the aisles, it may not be desirable to place aisle markers such as 7- am-19 on the ceiling due to the added effort. What is necessary is that after a forklift such as 10 moves from some current aisle location, such as 10-Ln, through locations such as 10-l_n-3 and 10-Ln+7 towards a storage bay such as 10, that upon arriving in bay 10 there be sufficient markers to determine its location wherever loads such as 4 might be placed or stacked, either permanently or temporarily. In Fig. 3 the final storage location is preferably only captured by a single camera with view 20a-v1 inside self- tracking device 20a. In this example, view 20a-v1 includes bay ceiling markers 7-bm-10-1 and 7-bm-10-2 as well as bay identification marker 7-bid-09. Again, a second optional camera might alternately be used within self-tracking device 20a providing second view 20a-v2 thereby detecting ceiling markers 7-bm-9-1, 7-bm-9-2 and bay identification marker 7-bid-09 In either case, where using one or two cameras in device 20a, there are preferably two markers always in view so that device 20a may easily and accurately translate from their pre-known positions as well as imaged size and shapes, the current location and orientation of forklift 10 as it engages or disengages loads such as 4. (Note that the pre-known positions of all ceiling markers within the warehouse and all associated inside or outside storage areas are ideally set, defined and calibrated once after which the information may then be carried about within each forklift 10's onboard computer 50 supporting this self-tracking function.) Referring to Fig.4a as well as in reference to both Fig. 2a and Fig. 3, as will be understood by those skilled in the art of machine vision and local positioning, a single image captured by a single camera looking at two or more discernable features is sufficient for determining both the location and orientation of the capturing camera device 20a and therefore also the forklift 10 to which 20a is affixed. This is an advantage of machine vision that employs area sensors vs. the preferred based technology of RF Saw Tags that uses what can be thought of as a point sensor. Without modification, the RF antenna first emits an interrogation pulse after which it then receives that signal's reflections off any in-range markers such as 7. Using this received information from only a single antenna, it is possible to determine distance to multiple markers but not also orientation. Hence, while exact location can be determined, the preferred self-tracking device 20a (in this case an RF antenna / reader) includes at least two antenna separated by a maximum convenient distance so that together their information may be used to also determine orientation. As will be shown and as is well understood in the art of RF signal tracking and antennas, when using only a single antenna, taking multiple readings with a preferably modified / limited scanning window, the RF antenna / reader combination can determine orientation as well as location.

Referring now specifically to Fig.4a yet similar to Fig. 3, there is depicted a ceiling of a storage warehouse onto which a matrix of RF Saw tags has been placed essentially marking both a major transport aisle and accessible storage bays. As discussed in relation to Fig. 1 and Fig. 2, these RF responsive tags are for use with a RF antenna and reader base technology 104 to implement ceiling oriented self tracking device 20a. As with the identification labels shown in Fig. 3, it should be understood that while these RF Saw tags are shown to be mounted onto the warehouse ceiling, this is a preference but not a limitation since the tags could easily be placed on any surface within the warehouse, whether ceiling, floor or wall, that is expected to remain within antenna range of any of the self tracking devices such as 20a, 20b or 20c. Also similar to identification markers of Fig. 3, RF Saw tags while not as inexpensive as paper, are typically in the neighborhood of $3 to $7 each and therefore still relatively inexpensive. However, since machine vision technology 102 has limited detection ability due primarily to line-of-sight requirements and limited field-of-view in comparison to RF technology 104, the present inventors prefer RF tracking technology 104 at least in part because it is expected to require significantly fewer overall tags. And, while the cost of tags / markers is a consideration, the cost of their installation and calibration is expected to dominate in comparison to the materials costs. Therefore, the fewer tags / markers required to accomplish forklift 10 self-tracking the less the overall implementation and calibration costs.

Referring to Fig.4a, in operation, as forklift 10 moves about the warehouse it may periodically emit interrogation pulses via at least one RF antenna incorporated into device 20a. Based upon the signal power of the interrogation pulses and the shape of the emission field, one or more RF Saw tags such as 7-bm-7-2 or 7-bm-8-2 may receive and then reflect this pulse with enough energy to be subsequently detected by the reader preferably housed in device 20a. Similar to camera field-of-view 2Oa-VI₁ RF reader based self-tracking device 20a will have detection field 20a-V1 based primarily upon the factors of the emitted signal strength of the interrogation pulse, the shape of the emission wave-front as controlled by the antenna configuration and the signal strength of the reflected signals from the Saw tags primarily based upon their size and shape; all as will be understood by those skilled in the art of RF positioning systems in general and RF Saw tag technology in particular. As will also be understood by those familiar with RF Saw tag technology, the antenna used to emit the interrogation pulse may be the same or separate from the antenna attached to the RF reader and used to receive the reflected pulses from the RF Saw tags. The present inventors prefer a single reader with at least two antennas that are used to both emit the interrogation pulse and receive its reflections from the Saw tags. Such readers are available in the market as sold by companies such as RF Saw, Inc. Based upon typical configurations, it is not unusual for the RF reader / antenna to be able to detect and identify unique tags up to 100 ft. away from the device 20a.

Hence, after emitting the interrogation pulse, RF self-tracking device 20a may receive reflected signals from preferably multiple Saw tags such as 7-bm-7-2 or 7-bm-8-2 strategically placed throughout the warehouse. As will be appreciated by those skilled in the art of local positioning systems, it is possible to triangulate the exact coordinates of each antenna in device 20a, and therefore attached forklift 10, at least based upon the detection of two or more Saw tags such as 7-bm-7-2 or 7-bm-8-2 per antenna. As previously stated, in order to determine the orientation of the forklift 10, the present inventors prefer using at least two antennas separated by the maximum convenient width, as will be discussed in more detail with respect to Fig.4c. Alternatively, as will also be appreciated by those familiar with local positioning systems, self-tracking device 20a may use a single antenna to capture the changing location of the forklift 10 over time with respect to multiple Saw tags which will also provide sufficient information to determine forklift 10 heading and orientation, albeit with more complicated assumptions and calculations. However, these heading and orientation calculations are further aided when the antenna is oriented within tracking device 20a such that it is facing only or portion of the full 360 degrees of overhead space. For instance, if a single antenna within device 20a is oriented to emit pulses and detect Saw tag reflections in the forward direction of the forklift 10, than this calibrated pre-knowledge can be used to more easily determine forklift 10 heading and orientation. Given these considerations, and referring to Fig.4a, forklift 10 is depicted as moving in from a location such as 10-Ln through locations 10-Ln+3 and 10-Ln+7 to a final location 10-Ln+10. During this movement, while at least passing through these locations, self-tracking device 20a may interrogate the surroundings and determine its current position. For instance, while at location 10-Ln RF based device 20a may read aisle Saw tags 7-am-19 and 7-am-20 as well as bay Saw tags 7-bm-7-2, 7-bm-8-2, 7-bm-20-2 and 7-bm-19-2 Given the distances from device 20a to preferably two or more Saw tags self-tracking device 20a, (or onboard computer 50 using data from device 20a) may calculate the forklift 10's unique current location. Again, by taking a second reading as the forklift 10 continues its motion, the multiple current locations over time will provide heading and orientation, especially when the antenna in device 20a is oriented for instance in the forward looking direction with respect to forklift 10.

Still referring to Fig.4a, eventually forklift 10 will end up in a final position such as 10-Ln+10 within a bay where loads 4 may either be picked up or left off. In either case, what is important is to either know because of continuous periodic tracking, or to then finally determine, the stationary position and orientation of forklift 10 as it either engages or disengages a load such as 4. If the load 4 is being disengaged, than the detected location and orientation of self-tracking device 20a can be conclusively translated to the location and orientation of lift forks 10L which in turn may be used to estimate the centroid location and orientation of load 4 as it is placed in storage. As previously discussed and as will be further taught with respect to Fig.6a and Fig.6b, if the outer dimensions of load 4 are determined with respect to forks 10L, than the storage centroid calculated for load 4 can be further translated into the corner or edge points of load 4 describing its outer surface. This method is discussed in detail with respect to Fig. 8. Referring now to Fig.4b, there is depicted the cab top 10-t to forklift 10 onto which an RF reader has been center-mounted as self-tracking device 20a. In this case, RF reader device 20a has four attached antenna 20a-a, 20a-b, 20a-c and 20a-d, each of which is capable of both emitting interrogation pulses and receiving their reflected signals from any of in-range Saw tags, such as 7-bm-x, 7-bm-y and 7-bm-z. For example, antenna 20a-a emits and detects within field 24df-a while antenna 20a-b emits and detects within partially overlapping field 24df-b. Focusing on a single tag 7-bm-y, Fig.4c shows how at least two antenna, 20a-a and 20a-b may both detected and calculate their distance to tag 7-bm-y as identical. As will be appreciated by those skilled in the art of local positioning systems, antenna's 20a-a and 20a-b as well as marker 7-bm-y now form a triangle where all sides are of a known length, thus allowing for the calculation of each interior angle. The calculations of these interior angles in turn provide the necessary information to determine forklift 10's orientation. Furthermore, it will be understood that if the forklift 10 where in the exact same x, y location but was somewhat rotated, for example by 5 or more degrees either clockwise or counter clockwise, than the detected length from antenna 20a-a to marker 7-bm-y would no longer equal the detected length from antenna 20a-b to marker 7-bm-y. Again, by knowing all lengths of the triangle sides, the interior angles will now have changed thus reflecting the change in forklift 10 orientation.

Referring still to Fig.4b, each individual antenna, such as 20a-a, may detect multiple tags, for example 7-bm- y and 7-bm-x. In this case, the data from the single antenna 20a-a is sufficient to calculate the exact location of antenna 20a and therefore by extrapolation forklift 10. As will be appreciated by those skilled in the art of location positioning systems, antenna 20a-a and markers 7-bm-y and 7-bm-x form a triangle where again all side lengths are known. It should be noted that if the forklift 10 where rotated about the center of antenna 20a- a, this would not necessarily change the detected distances to markers 7-bm-y and 7-bm-x, and hence antenna 20a-a using a single reading is not sufficient for determining forklift 10 orientation. As will be appreciated by those skilled in the art of location positioning systems and more particularly RF Saw technology, there are many alternatives for determining both forklift 10 location and orientation by some combination of:

• One or more antenna;

• Electronic or mechanical interrogation pulse sweeping;

• Using the forklift 10's motion to sweep the antennas, and / or

• Using multiple interrogation pulses separated by a sufficient distance to be detected by a single antenna under a controlled basis.

Other combinations or uses will be obvious to those familiar with RF or similar technologies. What is important is that a forklift 10 may be fitted with this or similar technology allowing it to self-track its own location and orientation in a more cost effective manner than building a warehouse and yard centered transponder matrix that actively tracks each forklift.

Referring now to Fig.4c, there is depicted RF based self-tracking device 20a being controllably swept through at least five different read positions, shown over the sequence of time Tn through Tn+4. In order to accomplish the changing of read positions, device 20a is preferable constructed so that at least sensing device such as antenna 24a is attached to movable mount 24m that is attached to pivot mount 24pv such that it is able to be controllably moved through and arc from a "forward looking" orientation, though an "overhead looking" to a "backward looking" orientation with respect to mount 24pv. Hence, if mount 24pv is itself affixed to forklift 10 such that its pivot axis runs perpendicular to the forward direction of travel, than in effect antenna 24a may be moved from a forward looking orientation with respect to forklift 10 to an overhead / ceiling looking direction and finally to a backward looking direction. As will be understood by those skilled in the art of RF scanning technologies, as antenna 24a is swept, multiple readings can be taken thus providing two or more separate readings without necessarily further moving forklift 10. Using this approach, RF based self-tracking device 20a does not need to continuously track the current position of forklift 10 but rather only its final stopped position at time of engagement or disengagement. Such ability is preferable when operating forklift 10 in an outdoor environment such as an air-drying lumber yard. For example, the forklift may travel significant distances in the open-air from a main storage warehouse to a remote stack such as 4s that is being housed either in the open-air or in a shed. In this case it may not be convenient to place sufficient RF Saw tags to continually determine the forklift 10's ongoing position. Using a swept antenna approach as shown in Fig.4c allows Saw tags to be limited to the areas of final storage only. (Note that using the preferred multiple antennas also supports this concept of only determining forklift 10 location and orientation at the pickup / drop off locations, at the time of engagement / disengagement, respectively.) As previously mentioned and especially true in this case, the placement of Saw tags may ideally be on nearby poles or posts or even on the ground surface. All that is necessary is that at least one tag be in range for each reading position during the sweeping of antenna 24a. In this case additional RF self-tracking devices such as side-oriented 20b or ground / floor oriented 20c may be used in conjunction with ceiling / upward orientated device 20a. As will be understood by those familiar with RF Saw tag readers, any such second and third self-tracking device only requires a second or third antenna and not an accompanying second or third reader. This is important since as of the time of application the readers tend to be priced ^• around $5,000 while the antennas are in the range of $50. In fact, RF Saw, Inc. currently sells a reader that works with up to four separate antennas, each of which may be separated via wire from the reader. As will also be appreciated by those skilled in the art of antenna design, it is possible to create a sweeping interrogation pulse and / or detection field by the use of electronics rather than mechanics, thus obviating the need for moving parts. In any case, what is important is that the use of swept antennas has at least the benefit of allowing an RF self-tracking device such as 20a to determine its orientation (as well as position) while forklift 10 is at rest, using only a single antenna. This in turn allows the placement of Saw tags to be limited to those warehouse and open-yard areas where actual loads 4 are in practice stored, rather than those areas where forklift 10 moves about when going between storage areas.

Referring now to Fig.4d, there is implied a forklift 10 stopped at a storage location such as 10-Ln+9. At this location, using either multiple antennas or a modified sweeping RF based self-tracking device 20a as depicted in Fig.4b, device 20a takes at least two distinct readings. For example, forward looking detection field reading 24df-Tn+1 and / or overhead looking reading 24df-Tn+2 and / or backward looking reading 24df-TN+3. During time Tn+1, detection field 24df-Tn+1 picks up two Saw tags 7-bm-10-1 and 7-bm-9-1. During time Tn+2, detection field 24df-Tn+2 picks up four Saw tags 7-bm-10-1 and 7-bm-9-1 as well as 7- bm-10-2 and 7-bm-9-2. And finally, during time Tn+3, detection field 24df-Tn+3 picks up four Saw tags 7- bm-10-2 and 7-bm-9-2 as well as 7-am-23 and 7-am-22. Using the combination of the pre-known and controlled deflection field 24df orientations with respect to forklift 10 as well as corresponding distances from device 20a to each of the prior listed detected Saw tags, either device 20a or preferably connected on board computer 50 is capable of calculating forklift 10's location and orientation; as will be understood by those skilled in the art of local positioning systems. Referring next to Fig. 5, an alternative approach to forklift 10 self-tracking is shown based upon accelerometers with RF resets, as discussed in relation to Fig. 2a. Unlike Fig. 3, Fig.4a and Fig.4d, Fig. 5 depicts the floor of a warehouse onto which multiple reset strips such as 6-rs-19, 6-rs-20 and 6-rs-35 have been placed. Again, as previously mentioned, accelerometers may be positioned in device 20a, 20b or 20c but are preferably housed in 20c that is oriented near the warehouse floor. This positioning is more ideal for the detection of RF markers that are secured to the floor. One or more RF markers are used to make a strip such as 6-rs-19, 6-rs-20 and 6-rs-35. It is anticipated that these RF markers might be similar to those sold by Texas Instruments as their TI-RFid Systems. These are inexpensive passive devices that can encode a unique warehouse location / address. In operation, accelerometers self-tracking device 20c continually provides changing location data to onboard computer 50 that is periodically reset when forklift 10 drives over or near enough to a reset strip such as 6-rs-35. In this case it is important that detection field 20c-v1 be very limited in size so that once detected the presence of reset strip such as 6-rs-35 is enough to provide valuable reset information, as opposed to also needing to calculate the strip's position and orientation with respect to tracking device 20c.

Referring next to Fig. 6a there is depicted forks 10L holding load 4 in an approximately centered position. Also depicted is asset dimensioning device 30 that has a field-of-view 3Ov. As mentioned in reference to Fig. 2b, there are several base technologies that may be used to accomplish asset dimensioning including but not limited to stereoscopic cameras 108, 2D camera and laser line generator 110 as well as 3D area sensors 112. Again, as previously mentioned, the present inventors prefer a 3D area sensor 112 such as that sold by Canesta to be incorporated into an appropriate camera design as will be understood by those skilled in the art. To accomplish larger field-of-view 3Ov a single camera with a wide angle lens or multiple cameras may be used. What is most important is that asset dimensioning device 30 first be able to detect the presence of load 4 as forks 10L move to engage or disengage. Of additional importance is the determination of load 4 shape, in this case rectangular and ideally as many corner points of load 4 as possible, such as p1 (x, y, z), p2 (x, y, z), etc. Given a detected rectangular form as well as a load height, width and depth, all eight corners may be calculated based upon a known load centroid 4C Load centroid 4C is easily assumed as the lowest mid-point on load 4 between forks 10L that is closest to forklift 10.

Note that the present inventors prefer not requiring asset dimensioning device 30 due to its extra cost. Jn the case of a well understood inventory, by first determining the product code that may be, for instance, printed on a paper tag associated with load 4, the standard dimensions for load 4 may be recalled from a local inventory database stored within on-board computer 50. Two common methods for communicating load 4's product code to computer 50 include keypad entry and bar code scanner. In either case, it is typical that at least the shape, e.g. rectangular, cube or cylindrical may be pre-known and storable within computer 50's inventory database. It is also usual that at least the largest maximum dimensions of height, depth and width are also pre-known and may or may not vary in actuality. In any case, using this pre-known information along with additional fork 10L height information gathered when a second load may be stacked upon first load 4 (as previously discussed,) it is possible to provide reasonable calculations for edge points such as p1 (x, y, z), p2 (x, y, z), etc., as will be discussed in greater detail with respect to Fig.9.

Referring next to Fig. 6b there is for instance depicted a smaller load 4 that is so detected and measured within field-of-view 3Ov. What is important is that the present inventors teach the value of determining load 4 dimensions in addition to its presence on forks 10L. As will be further discussed, by storing the three- dimensional coordinates of the faces defining each load 4 held in storage, it is possible to provide a virtual warehouse simulating the current position of all loads. Such a virtual warehouse serves many valuable functions as will be discussed and others that will be understood by those skilled in inventory control operations. For example, by knowing the relative positions of all loads with respect to the warehouse and each other:

1. An order processing system can check to determine which of any two or more equivalent or interchangeable load 4s may be the most accessible, the identification of which may then be used to direct load 4 pulling for order fulfillment;

2. An order pulling system can sort all loads 4 to be pulled within a given time frame for a given set of orders into a sequence list best optimizing forklift 10 movement and time spent pulling;

3. An inventory control system can suggest the rotation of older loads 4 that may become buried over time in less accessible warehouse spaces under multiple other loads 4;

4. The computer 50 on-board forklift 10 may direct and verify each load 4 lifted by forks 10L against a plan of load 4 movements automatically designed to minimize load 4 shuffling as some loads 4 are removed from and replaced to storage in order to pull other loads 4, and

5. The computer 50 on-board forklift 10 may verify each load 4 pulled against a target list for a given order or set of orders, thereby both confirming the pull to the forklift operator and also indicating to the computer 50 that the targeted load 4 was pulled in satisfaction of a given work or pick ticket, where such confirmations are typically provided as hand written notes on a picking document or perhaps a scanned bar code off of load 4's paper tag.

As will be appreciated by those familiar with the control of large warehouse inventories, especially those that cannot ideally be stored in a binning system, the ability to create a virtual warehouse defining the current positions of all loads 4 is of significant value and has many uses beyond those listed above. Furthermore, the present invention is not limited to rectangular shaped loads 4 but rather is capable of functioning with any physical object being transported by forklift 10. Such other load 4 shapes might include pallets with bags or boxes or roll of paper or steel (i.e. cylindrical shapes.)

Referring next to Fig. 7a there is shown a flow chart depicting the anticipated steps involved in the process of receiving new loads 4 into warehouse storage. In the case of a lumber yard, a new load 4 would typically be one coming off a delivery truck from a supplier, such as a saw mill or another wholesaler. Most often in this situation a delivery truck or rail car will bring multiple loads 4 at a time where each is typically bearing a unique identification according to the supplier. Individual yard operations will vary, but first each load 4 must be removed from the transportation vehicle via forklift 10 where it is often placed in a staging area for identification and counting. As would be understood by a careful reading of this specification, it is possible to use the present invention to identify the incoming loads 4 with respect to the shipping vendor's product tags and then to subsequently track their removal from the transportation vehicle and placement into temporary storage. However, the present inventors prefer to begin load 4 tracking after this first step of receiving. In that case, each received load 4 will typically have been retagged with the receiving lumberyard's product code and a uniquely identifying tag number. It will usually have been counted in some capacity for inventory control purposes. As previously mentioned, the present inventors prefer using a system such as taught in prior issued U.S. Patent No. 5,307,294 entitled Automated End Tally System. Such a system, now in existence and being sold by Industrial Vision Systems, Inc., will provide both the functions of counting the individual boards within load 4 / pack as well as measuring the outer dimensions of the load 4. Also as previously mentioned, having these outer dimensions removes the need for asset dimensions device 30.

In any case, assuming that a device similar to the Automated End Tally System is not being used, after a new load 4 has gone through the receiving process, it is usually taken from the initial staging area for warehouse or yard storage. One other typical source of new loads 4 for lumber yards other than vendor receiving's is intermediate yard processing operations such as grade sorting, kiln discharging and milling operation. In all of these situations it is possible that some existing loads 4 going into the intermediate process will be transformed coming out of the process such that it may be classified and therefore re-identified and tagged. In such cases, these loads 4 can be treated as new loads 4 similar to those just received from a vendor. In both cases of vendor receiving's or yard internal operations, once a new load 4 is ready to be picked up and placed into storage, the present inventors prefer a series of steps similar to those depicted in Fig. 7a. First, in step 100, forklift 10 passes by a wireless data link whereby job and route instructions for picking up and storing new loads may be optionally downloaded into forklift 10 onboard computer 50 as step 100-A. As will be understood by those skilled in the area of wireless networks, it is possible that the network be built so that the forklift 10 is in constant communications throughout all of its movement areas both within all warehouses and were applicable outdoors in a yard setting. And while this is both possible and anticipated by the present inventors, for the practical aspect of cost savings it is sufficient to have some few areas often returned to by the forklift(s) 10 where wireless communication can be accomplished. As will be understood by those skilled in the art of inventory control systems, the apparatus and methods of the present invention make it feasible for the forklift 10 picking and storing routes to be pre-planned and even automatically laid out for each forklift driver thereby offering significant efficiencies. Ideally, if using any such pre-planned forklift 10 picking and storing (job) routes, these should be downloaded to each forklift 10 prior to beginning a work session. (It should be noted that the present invention functions perfectly well for it intended purposes of load tracking if this step is replaced by verbal or paper-based instructions given directly to the forklift operator.) While in the presence of the wireless data link during step 100, forklift computer 50 may also communicate with inventory control computer 60 to determine if there are any pertinent inventory data changes that should be exchanged as step 100-B. As will be understood by those skilled in the art of inventory and warehousing control systems, pertinent data might less often include new product codes and more often include new loads 4 being removed or added to storage by other forklifts 10, along with there location data. Once receiving up- to-the-minute information on the current state of the stored warehouse loads 4, forklift 10 may then proceed to its intended pickup location as step 102. The driver may either be using paperwork or verbal instructions as a guide to the pickup locations, or as mentioned may optionally be using job and route instructions acquired in step 100-A. During this step the forklift 10 may optionally engage its self-tracking device(s), such as 20a, 20b and 20c as previously taught using any of the described or similar base technologies. As will be understood, tracking the forklift 10's ongoing position has several benefits including the ability to guide the forklift 10 in a manner similar to that used by automobiles with GPS tracking while riding on the highway and road systems. It is also useful to know the current forklift 10's position in real-time as a means of coordinating any unplanned interruptions to routed work. However, this real-time updating of forklift 10's position back to a controlling office would require a larger wireless network or its equivalent to feed each forklift 10's currently changing locations back to the main inventory control computer 60. As previously mentioned, it may also be necessary based upon the technology chosen to perform self-tracking to continuously monitor forklift 10's current position so that both location and orientation can also be determined. However, using the preferred RF Reader 104 base technology with either the preferred multiple antennas or an electronically or mechanically swept antenna as previously taught, it is possible to only require forklift 10 position detecting at the moments of load 4 engagement and disengagement rather than continuously during point-to-point movements.

In any case and still referring to Fig. 7a, the forklift 10 will eventually arrive at the intended pickup location to pick new load in step 104. Once at this location, the forklift 10 operator ideally determines its identity in step 104-B and engages the targeted new load 4 in step 106. As will be understood by those skilled in inventory control systems, the exact order in which these two steps take place is immaterial as are similar combinations in this and next Fig. 7b and Fig. 7c. It is only important that for new loads 4 that they be identified and associated with the forklift 10 as it conducts its next step of transporting. In order to identify load 4, all that is typically necessary is to read the identifying code on the tag that is placed on load 4. This tag will often be a unique alpha-numeric code that will match one of those codes currently held in the onboard computer 50's stored inventory. After finding the match this tag number may easily be translated to a product code, description, typical storage location, typical dimensions, etc.

As will also be appreciated, there are several very good means for the forklift 10 operator to efficiently determining load 4's tag number including but not limited to, entry by the forklift operator into an onboard keyboard / pad associated with computer 50 or use of a bar code scanner or RF ID reader. One of the major goals and benefits of the apparatus and methods taught herein, are to provide a new additional method for identifying load 4 other than these traditional methods; that is essentially to simply engage or even just approach load 4 with self-tracking forklift 10. After the locations of stored loads 4 become known to the virtual warehouse system, then the current location and orientation of forklift 10 can be used with and without actually engaging loads 4 to exactly determine the tag number and associated data of load 4. It is even anticipated that any onboard forklift 10 routing system with display screen associated with computer 50 can list all loads 4 bay-by-bay, row-by-row at any necessary time to assist the forklift operator. However, as of yet the new loads 4 have not been tracked to a storage location for updating into the virtual warehouse database so that this novel load 4 identifying option herein taught is not yet available.

Still referring to Fig. 7a, during engaging step 106 it is preferable to sense the time of engagement as step 106-B. As previously discussed herein, and as previously taught in the lead inventor's U.S. Patent No. 5,604,715, entitled Automated Lumber Unit Tracking System, there are several good options for determining the point and time of load 4 engagement by forks 10L. These at least include detecting the increase in hydraulic pressure on forks 10L as they lift load 4 or the use of asset dimensioning device 30 to sense presence. Of these options, if asset dimensioning device 30, preferably using 3D area sensors is employed, the present inventors anticipate that this device is either continuously monitoring its view 3Ov or at least monitoring this view when forklift 10 speed is below a certain rate and / or forklift 10's location is near the pickup location. In this way asset dimension device 30 is essentially always looking to see if a load 4 is moving into position onto forks 10L, as will be appreciated by those skilled in the art of object tracking. Of course, if forklift 10 is continuously self-tracking its position, then it may use its own current location and orientation to determine if its forks 10L are underneath a given load 4. (Again, this assumes that the load 4 has already been identified and is now tracked by the virtual warehouse, which is not the case for the new loads 4 discussed in step 104 in particular and Fig. 7a in general.) What is important is that the time of load 4 engagement is detected in step 106-B. It should be noted that for this new load 4 pickup, it is less important to determine the actual fork 1OL height as compared to load 4 disengagement during upcoming step 110. At the time of engagement or shortly thereafter, onboard computer 50 may use the sensed engagement time to automatically trigger asset dimensioning device 30 to also determine load 4's outer dimensions as a part of step 106-B. Once load 4 is engaged it may then be transported to the intended drop-off location in step 108 where the forklift 10's current location is optionally tracked as step 108-B. Eventually, forklift 10 will arrive at the intended drop-off / storage location at which time it may or may not be necessary to raise or lower forks 10L in order to place load 4 in step 110. In step 110, similar to 106, the time of disengagement can be determined via many aforementioned methods. What is important is that at the time of sensed disengagement, forklift 10 accomplishes step 110-B to do the following:

• Self-determine the current X₁Y location and orientation of the forklift 10 using any of the herein taught or similar methods;

• Self-determine the current Z height of the forks 10L using any of the herein taught or similar methods;

• Record this X₁Y₁Z information and the time of disengagement along with determined or assumed load 4 dimensions onto onboard computer 50, and

• Optionally translate this X, Y, Z information into the load 4's centroid and outer dimension corner point locations as previously discussed especially in relation to Fig. 1 and Fig. 6a and as will be discussed in more detail with respect to upcoming Fig. 8.

As first taught in U.S. Patent No. 5,604,715, entitled Automated Lumber Unit Tracking System, this critical ability of the prior taught and present invention to automatically determine the storage location of a load 4 by tracking forklift 10 and its forks 10L enables the establishment of a virtual warehouse built upon the pre- known dimensions and wall locations of the warehouse and yards themselves as well as the current storage locations and outer dimensions of each load 4. As will be understood by those skilled in the art and as previously discussed, this information has many valuable uses and is not easy to determine by traditional means and methods.

Still referring to Fig. 7a, forklift 10 may now either seek its next new load 4 to pick up and store by looping back to step 102 or it may proceed to step 112, where it transfers stored load 4 data held in onboard computer 50 to inventory control computer 60 via wireless data link. The result of this transfer is depicted as step 112-B, where all newly stored loads 4 are recorded within the virtual warehouse / inventory database of computer 60 along with their automatically determined storage locations and time of storage. As will be appreciated by those skilled in the art of warehouse operations, it is also valuable to store the forklift 10's movements and times of engagement and disengagement of all loads 4 to be used for efficiency statistics and comparisons. Of course, having this data requires that forklift 10 be continuously tracked during its movements, which is entirely consistent with the present teachings.

Referring next to Fig.7b, there is shown a flow chart depicting the anticipated steps involved in the process of retrieving a known load 4 from storage to take to an intended drop off point. Here, it is both assumed and important to note that the loads 4 to be retrieved were originally placed in storage by forklift 10 and therefore their locations are known to the virtual warehouse and each will typically also have attached a uniquely identifying bar coded tag. While the present invention provides for a load 4 locating system that does not require a forklift operator to be looking at individual load 4 tags in order to find the desired load(s) 4, the present inventor prefers keeping the tag system in place as a matter of familiarity and practical backup for the automated system.

Still referring to Fig.7b and similar to step 100, in step 200, forklift 10 passes by a wireless data link whereby instructions for retrieving and then dropping off loads 4 may be optionally downloaded into forklift onboard computer 50 as step 200-A Also similar to step 100-A and as will be understood by those skilled in the art of inventory control systems, the apparatus and methods of the present invention make it feasible for the forklift 10 picking and storing routes to be pre-planned and even automatically laid out for each forklift operator thereby offering significant efficiencies. Ideally, if using any such pre-planned forklift picking and storing routes, these should be downloaded to each forklift 10 prior to beginning a work session. While in the presence of the wireless data link during step 200, forklift computer 50 may also communicate with inventory control computer 60 to determine if there are any pertinent inventory data changes that should be exchanged as step 200-B, as previously discussed in relation to step 100-B. Once receiving up-to-the- minute information on the current state of the stored warehouse loads 4, forklift 10 may then proceed to its first / next retrieve location as step 202. The driver may either be using paperwork or verbal instructions as a guide to the pickup locations, or as mentioned may optionally be using job and route instructions. Similar to step 102, during this step the forklift 10 may optionally engage its self-tracking device(s), such as 20a, 20b and 20c as previously taught using any of the described or similar base technologies as step 202-B The present inventors also note that ideally inventory computer 60 assigns to a given forklift 10 a set of instructions that restrict a given bay or set of bays to the exclusive management of that single forklift 10. As will be understood, unless the virtual warehouse is being updated in real-time, thus requiring a wireless network covering the entire warehouse and yard, then having multiple forklifts 10 operate in the same bay may cause difficulties if the forklift 10 operators do not follow directions. Therefore, to save on installation cost associated with a larger wireless network, the present inventors teach the method of assigning a bay or set of bays to a given forklift 10 for a specific work-shift; where the work shift starts at step 200 and goes until step 216. As will be understood, this rationale equally applies to Fig. 7a (but not to Fig. 7c which is essentially taking place within the steps of Fig.7b.)

Still referring to Fig. 7b, whether or not self-tracking is used during the forklift's 10 travel during step 202, the forklift 10 will eventually arrive at the intended location to retrieve stored load 4 in step 204. During this step 204 a significant new opportunity exists because of the virtual warehouse data accumulated by the present invention. Specifically, the present inventors prefer that the onboard computer 50 include a display screen that can be used to show the forklift 10 operator the exact loads 4 that are in the given bay directed by the retrieval route. Hence, the operator might see that this bay has the typical three stacks 4s. These stacks 4s might be portrayed from a side view showing the deepest stack 4s to the right and back against the warehouse wall and showing after this in scaled dimensions each of the next two stacks 4s and then also a portrayal of the forklift 10 in its correct current location and orientation. It is further anticipated that the present invention will be able to show each load 4 in a stack 4s in its relative proportion, which from the side will look like rectangles. Each rectangle will by tradition be the same width of approximately four feet while the height will vary from that of a typical "full pack" / load 4, e.g. four feet high, to a "half pack" / load 4, e.g. two feet high. Using this visualization of all three (more or less) stacks 4s, with each load 4 shown in proper proportion, the lift operator will have significant information for assisting in this step of retrieving, and specifically determining if a load 4 is exposed for pickup, i.e. step 206. It is further anticipated that each rectangular representation of the side view of a load 4 in stack 4s will at least include that load 4's unique identifying number / tag number. Again, this is not necessary for the function of the present invention, only a helpful and important feature that can be provided for the operator's comfort. It is also anticipated that the load 4 which is to be retrieved will be colored or in some way easily differentiated on the onboard display screen associated with computer 50, thereby assisting the operator to quickly visualize their task. As will be discussed in more detail with respect to Fig. 7c, it is most often the case that the load 4 to be retrieved is not immediately available; where in this case available means on the very top of the front- most stack 4s (i.e. the stack that the forklift 10 encounters first as it enters any given bay.) In this most typical situation, the forklift operator must "dig out" the load 4 to be retrieved. Again, this procedure is reviewed in Fig. 7c

Still referring to Fig. 7b, it is first assumed that when the operator arrives on forklift 10 at a given storage bay, the load 4 is immediately available to be picked up without having to move / remove any other loads 4 that might be in its way. (This check of load 4 availability for immediate pickup is shown as step 206.) If the desired load 4 is not available and must be dug-out of the bay's stacks, then the operator proceeds to step 208 to rearrange stored loads in order to expose desired load 4, as covered in Fig. 7c. If the desired load 4 is available, then the operator will use forklift 10 to pickup / engage desired load 4 in step 210. At this same time, and as already taught in detail in the current specification, forklift 10 as preferably equipped will then automatically detect that the forks 10L are moving under a load 4s such that eventually the load 4s is fully over the forks 10L. Once the presence of a new load 4 is detected, the on-board computer 50 will activate (if not already activated) the self-tracking device(s) 20a, 20b and / or 20c to determine the forklift 10's current X, Y location and orientation with respect to the warehouse / yard's relative coordinate system. At this same time, either of devices 20a, 20b and / or 20c will also determine the height of forks 10L, also as previously taught both herein and within related U.S. Patent No. 5,604,715, entitled Automated Lumber Unit Tracking System. After having determined both the X, Y location and orientation of forklift 10 and the height of forks 10L, the onboard computer 50 will then check its local virtual warehouse database to determine which load 4 is located at the current fork 10L X, Y location, orientation and Z height. Note that fork 10L's current X, Y location and orientation can easily be extrapolated from the forklift 10's X, Y location and orientation because they have a fixed relationship, as will be understood by those skilled in the art and also further discussed in Fig. 8

Still referring to Fig. 7b, and in this case where the available load 4 is the desired load 4, the onboard computer 50 will determine and confirm for the forklift operator that the load 4 now being picked up / engaged is in fact the desired load 4. As will be understood by those skilled in the art of inventory control and visual 3D modeling software, many possible advantages occur when a virtual warehouse data can be established as taught first in prior related Patent No. 5,604,715 and again in variation herein. Thus it is conceivable that as the forklift operator passes each bay on forklift 10, the onboard computer 50 is continuously tracking forklift 10's movements by activating any of the self-tracking devices 20a, 20b and / or 20c and using this information to dynamically update a visual 3D "drive-though" display of the warehouse. The present inventors prefer that this load 4 storage location data taught herein is uniquely combined with the board-within-pack data taught by the lead inventor in related U.S. Patent No. 5,307,294, entitled Automated End Tally System. With this combination, it is possible to recreate a very accurate 3D image of the warehouse for display to the forklift 10 operator. As the operator turns into a given bay and faces the front-most stack 4s, the display preferably shows each unit with course by course detail (gathered from the Automated End Tally System.) The loads 4 within stack 4s can easily be further differentiated by alternating color shades so that every other load 4 is clearly different from the load above and below, including a display of its unique pack / load 4 number. As the forklift 10 faces the stack 4s and begins to raise its forks 1OL, onboard computer 50 preferably further tracks this changing fork 1OL height and gives a depiction of the forklift 10 in front of stack 4s with the forks rising and highlights the current load 4 directly in front of forks 1OL at their current height. All of these features are anticipated by the present inventors and it will be understood by those skilled in the art that other variations of these functions are conceivable. Therefore, what is most important is that the virtual warehouse data be easily and inexpensively collected and then made available, as presently taught, for a host of uses including but not limited to those herein mentioned such as:

• multi-order load pull planning;

• forklift travel route planning;

• forklift travel route tracking and analysis;

• load retrieval / stack "dig-out" planning;

• warehouse 2D and / or 3D visualization for at least both the front office and forklift operators, and

• load pull confirmation.

Still referring to Fig. 7b and step 210-B, as desired load 4 is engaged forklift 10 then records this within its local virtual warehouse database that load 4 is no longer in its original storage bay but now on the assigned forklift 10 and being transported to its intended drop-off location as step 212. Note that this change to the configuration in loads 4 held at a certain bay is critical to the function of the virtual warehouse. Two basic scenarios are anticipated by the present inventors. First, a wireless network is provided that covers the entire storage area so that forklift 10 is in constant communications with inventory control computer 60 via onboard computer 50. In this case, as each load 4 is engaged, transported and disengaged by any forklifts 10, the information is immediately updated in the virtual warehouse. Conversely, and preferably, in order to save the costs associated with such an expansive wireless network, each forklift 10 is for instance dedicated via pull / movement planning to a given bay or set of bays at any given time.

As previously discussed, while assigned to those one or more bays, no other forklift 10 will be given directives or confirmation for similar access. Using this approach, when each forklift 10 is assigned its jobs, such as in steps 100-A or 200-A (and as will be seen also 300-A of Fig. 7c,) it is preferable that only one forklift be assigned to any given bay in one time-frame. This is equivalent to saying that one or more bays are signed- out to a given forklift 10 from the time starting at steps 100-A, 200-A or 300-A and at least until the assigned forklift returns with updated reconfigured bay(s) information at the time of steps 112-B, 2116-B and 310-B. This control can be easily managed by the inventory control system 60 as will be understood by those skilled in the art and will ensure that the virtual warehouse data is accurate for any given forklift 10 at all times without requiring the expense of a larger wireless network.

Furthermore, it is even possible for this functionality to be accomplished without any wireless network as would be understood by those skilled in the art. For example, job instructions could optionally be provided to each forklift operator on a USB memory stick, which by today's standards could hold in excess of 1 GB of data, which is anticipated to be more than enough to hold the entire virtual warehouse dataset as well as routing instructions. This approach has advantages since the memory stick which is then placed into onboard computer 50 can also be used for redundant data storage with the onboard computer 50's disk drive or equivalent, thus acting as a fail-safe backup. With this approach, should the onboard computer 50 fail for any reason, the data is easily transferred to another forklift 10 for immediate transfer of responsibilities by simply removing the USB stick and placing it in another forklift 10. Therefore it should then be clear that the present invention may perform its unique functions with these or similar variations without departing from the scope of the invention.

Still referring to Fig. 7b, once the desired load 4 is engaged, confirmed and recorded as being carried on forks 10L in step 2010-B, and then carried to its intended destination in step 212, forklift 10 optionally self- tracks its own location as step 212-B After arriving at the drop-off destination, the operator may direct forks 10L to raise and / or lower as load 4 is set down / disengaged in step 214. As with step 110-B, onboard computer 50 now also works with tracking devices 20a, 20b and / or 20c to determine the forklift 10 location and orientation as well as fork height. This combination of data is sufficient to translate into at least the location of the approximately front-center-bottom of load 4 that rests between forks 10L closest to forklift 10, as will be further discussed in relation to Fig. 8.

While several automatic means are available and have been taught both in related U.S. Patent No. 5,604,715, entitled Automated Lumber Unit Tracking System as well as the present invention for determining load times of engagement and disengagement as well as fork 10L height, the present inventors anticipate using a manual method step as well. Specifically, the forklift 10 operator may assume responsibility for all of the following tasks:

1. At the approximate time of first engaging a load 4, the operator may indicate via and onboard console / keyboard that a particular load has now been picked up, thus obviating the need that the forklift 10 self-detect engagement. In the case of step 106 in Fig. 7a, if the load 4 is new to the virtual warehouse, the operator via the keyboard, a bar code scanner, or any similar method may then enter the load 4's unique pack / tag number. In the case of step 210 in Fig.7b, if the load 4 is already known to the virtual warehouse, its number is ideally listed on the forklift 10's display screen for easy confirmation of engagement as the onboard computer 50 displays all loads 4 in the current stack 4s and highlights the intended load 4.

^■ If this manual approach is taken, it is not necessary to know fork 10L height at the time of engagement since this measurement was only important as a selector for automatically determining which of several loads 4 in stack 4s was engaged, each of which has its own unique height off the warehouse floor.

^■ If this manual approach is taken, at the time of disengagement it is also possible to use a manual approach for determining fork 10L height. For instance, at the very least height ruler marks could be added to the stationary mast supports of fork 10L such that as the moveable forks 10L themselves go up and down, there current height at the time of disengagement can easily be read off the ruler marks and entered via the onboard console.

2. Also, at the time of engagement, the forklift operator may then also enter the approximate dimensions of load 4 if these are not to be determined automatically by device 30. For instance, entering the load 4 tag number may cause onboard computer 50 to automatically lookup the standard load width, length and height which the operator may then easily confirm / override. With lumber, the pack width is often fixed for all inventory loads 4 as previously mentioned and for example might be four feet. Furthermore, load 4 lengths are less important than load 4 widths. Specifically, at any given bay, knowing the stack 4s that a given load 4 resides within is easily determined by knowing the depth of the load 4's front facing side with respect to the warehouse's back wall. This information comes from the current location of the forklift 10. In fact, if forklift 10 is situated in front of three successive stacks 4s and has not yet engaged any load 4 in the front most stack 4s, simply knowing the forklift 10's current location indicates that it can only be in front of the outer-most stack 4s. As this stack is removed, as will be discussed in reference to Fig. 7c, then the forklift 10 may drive deeper into the bay at which time its current location will again be enough to indicate that it must be in front of the next outer-most stack 4s, even though this knowledge is also determinable following the steps to be discussed in Fig. 7c. Since the location of the stacks 4s within a bay is a function of load 4 width, this makes the accuracy of load 4 width more important than load 4 length. The dimension of load 4 length has more to do with the separation between bays, which is often fixed even if a given load 4 is shorter than the longest allowed load 4 length within a bay.

^■ As just mentioned, fork 1OL height may be determined by painting ruler marks on the fork 10L stationary mast. This technique can then double as a means for determining pack height whilst resting on forks 10L.

^■ The present inventors further teach that it is not necessary to even include load 4 height as an initial dimension in the accumulated virtual warehouse database since the height may be assumed as a standard based upon the current product code assigned to load 4. In the case of lumber loads 4, this assumed height might for instance be four feet. Therefore, when load 4 is dropped off either onto the warehouse floor or on top of another load 4, the height of the forks 10L serves as the bottom of the current load 4, while the top of that same load 4 will be assumed to be for example four feet higher. Note that at this same time, the current height of the forks 10L at disengagement serve to indicate that actual top of the load just underneath the current load 4. Hence, the height of forks 10L at disengagement are thus being used to retroactively update the actual height of the load 4 underneath the current load 4, which up to then was assumed to have been the default height (for example four feet.) Likewise, if some additional new load 4 is later rested on top of the current load 4 just now being dropped off, the actual height of the then current load 4 could be retroactively reset within the virtual warehouse.

As will be understood by those skilled in the art of warehouse and business operations, there is always a trade-off to be considered between the cost of equipment necessary to support personnel and the related complexity / ease of their job functions. Hence, while it is possible as herein taught to have several means for automatically determining load engagement and disengagement as well as load dimensions, the cost of this extra technology must be weighed against the simplicity of any reasonable, alternative manual steps. Therefore, while the present inventors prefer automatic means as taught herein, they also anticipate the aforementioned manual steps which fall within the scope of the present invention. Still referring to Fig. 7b, and in any case, whether using automatic or manual apparatus and / or methods, the changes to various loads 4 locations within the virtual warehouse are continually updated on forklift 10's onboard computer 50. At some point forklift 10 is anticipated to return to a central wireless node in step 216 so that the current locations of all loads 4 moved out of a given bay, as well as all loads 4 rearranged in a bay in order to retrieve another load 4 for removal, are then transmitted to inventory control computer 60. Again, this step 216-B may be performed as already discussed in other ways rather than a wireless node exchange. Referring next to Fig. 7c, there is shown a flow chart depicting the anticipated steps involved in the process of rearranging loads 4 in a bay in order to expose a desired load 4 for retrieval. As was previously discussed with respect to Fig. 7b, in practice it is expected that most desired loads 4 to be retrieved will not be the topmost load 4, on the outer-most stack 4s, in any given bay, or in fact even within the outer-most stack 4s. Hence, the forklift operator will almost always need to be rearranging loads 4 in a bay in order to retrieve a desired load 4. The series of steps discussed in relation to Fig.7c are the preferred procedure for this rearrangement. One of the additional benefits of the present invention is that for any given set of sales orders for which inventory loads 4 must be pulled during a given work-shift, it will now be a practical possibility to consider all of the sales orders at one. Note that the traditional approach has been to assign one sales order to a single forklift operator who would then travel about potentially the entire warehouse from bay to bay rearranging bays as needed in order to find the easiest loads 4 to pull to fill the customer's request. This approach was both wasteful with respect to forklift 10 movements, costly in terms of overall throughput and required handling loads 4 in any given bay once per sales order line. Using the concepts taught in this and prior related U.S. Patent No. 5,604,715, entitled Automated Lumber Unit Tracking System it is now possible to:

• Assign an set of bays to a single forklift 10 to be worked for an entire shift and then to further minimize movements between these assigned bays;

• Create a routing plan that takes the forklift 10 operator to any given bay only once per shift (for instance,) which is accomplished by making sure all loads 4 that will be needed from a given bay are pulled during the same and therefore a single rearranging effort;

• Limit the energy bills and wear and tear on the forklifts 10 as their overall travel distances are reduced, thereby potentially requiring fewer total forklifts 10 in operation;

• Limit unintentional damage to individual loads 4 by reducing the number of times each load is touched / moved / rearranged by any given forklift 10;

• Increase inventory rotation by intelligently pulling loads from the back-most stacks 4s in a given bay that may be exceeding a maximum desired age threshold, rather than simply taking the "first pack the operator finds," presumably from the outer-most stacks 4s, and

• Improving customer satisfaction by allowing the sales staff in combination with the inventory control software preferably resident on computer 60 to weigh out the different piece tallies (i.e. actual boards within the pack) between two loads 4 that may have significantly different "costs to pull." Hence, the sale staff will be able to see which load's 4 are good alternatives for their customers orders based upon their piece tally and then also know which are located in easier positions for retrieval within a given bay. In any case, as those skilled in the art of warehouse management will understand, the ability to strategically plan the minimum rearrangements necessary to retrieve all desired loads 4 from a given bay, over a given work shift, has significant value.

Still referring to Fig.7c, onboard computer 50 preferably contains bay rearrangement instructions that were included with job and route instructions downloaded during step 200-A Preferably, in step 300 the forklift operator will refer to these bay rearrangement instructions as well as the local virtual warehouse database included on computer 50, that together serve as a guide to pulling out and replacing blocking loads 4 in order to retrieve one or more desired loads 4. In essence, each time an individual blocking load 4 is first engaged / picked up, moved and then disengaged / dropped off (albeit temporarily) the overall process is exactly similar to steps 210, 212 and 214 as discussed in reference to Fig. 7b, and is shown as steps 302, 304 and 306 respectively. Furthermore, associated with engaging step 302 is sensing step 302-B that is exactly similar to respective step 210-B Similarly, steps 304-B and 306-B are exactly similar to steps 212-B and 214-B and therefore no further teaching is provided.

What is important to understand is that these temporary moves of blocking loads 4 out of a given bay into the nearby aisle ultimately result in the desired load 4 being exposed for pickup in step 308. As will be understood by those skilled in the art of warehouse operations, once exposed then the desired load 4 may be removed from the bay and itself set aside temporarily after which those originally blocking loads 4 now being held temporarily in the aisle may be strategically returned to bay storage in step 310. Of course, during this entire rearranging process, the picking up from bay, moving to aisle, setting down in aisle followed by the return trip is all tracked and recorded by onboard computer 50 using tracking devices 20a, 20b and / or 20c as previously discussed. It should also be noted that often there will be two or more desired loads 4 to be pulled from a bay during a given shift and for this reason there is no particular set order to the pulling out of blocking loads 4 and their return to storage, only that this be accomplished with the minimum of overall movements. Ultimately, between steps 200 through 214 and 300 through 312 the forklift operator will accomplish all job instructions by successfully pulling all desired loads 4 and rearranging any blocking loads 4 in the targeted bays as needed. Eventually, during step 216 forklift 10 will pass by the preferred wireless node connected to inventory control computer 60 and upload from computer 50 all locally stored changes to the virtual warehouse as caused by the steps shown in Fig. 7b and Fig.7c so that that the shared virtual warehouse held on computer 60 will be accurate.

Referring next to Fig. 8, there is depicted a review of the measurement concepts involved with translating from the detected fixed locations of multiple markers 6 and 7 into: first, a current location and orientation of a self-tracking forklift 10 and second, into the outer edge points of a load 4 being placed into physical storage for inclusion in a virtual warehouse database. As will be understood by a careful reading of the present application, the method steps and apparatus in support of Fig. 8 are independent of the type of base technology chosen for self-tracking or even whether a top mounted device 20a is used versus a side mounted 20b or even a bottom mounted 20c. Furthermore, the method steps and apparatus reviewed in the Fig. 8 are also independent of the type of base technology and the level of operator interaction chosen for detecting the times of load 4 engagement / disengagement, the height of forks 10L at these same times, or the shape and size of load 4. As taught herein, the following elements are of most importance for enabling the novel teachings of the present invention: 1. that first a multiplicity of passive markers 6 and / or 7 are placed either on the ceiling and / or walls and / or floors of the warehouse and / or any similar outside structure, and that these markers 6 and / or 7 be at pre-known fixed locations with respect to every other marker 6 and / or 7 placed on the warehouse or similar outside storage areas;

2. that second at least one self-tracking device 20a and / or 20b and / or 20c be placed on a self- tracking forklift 10 in such a way that the X, Y, Z offset distance from any of these devices 20a, 20b, 20c to the mast 10L-M (i.e. the stationary arms upon which the forks 10L bearing load 4 move up and down) is preferably fixed but always calculable, as would be the case if the self-tracking device 20a, 20b, 20c is permanently mounted at a fixed X, Y, Z offset from the bottom most point of mast 10L-M, or any other fixed spot of the mast 10L-M that is relatable to the current position of movable forks 10L;

3. that third any and all self-tracking device(s) 20a, 20b, 20c be able to self-detect the distance from themselves to some number of markers 6 and / or 7 via one or more distinct fields-of-view 20a-v1 and / or 20a-v2 or detection fields 24df-a and / or 24df-b, as would be the case if two distinct fields- of-view or detection fields are sensed simultaneously or one field-of-view or detection field is swept either electronically or mechanically (where mechanical includes the forklift's 10 own movement) over time to create two or more distinct datasets;

4. that also there is either a manual / operator based method with supporting apparatus for indicating the times of load 4 engagement / disengagement including but not limited to a keypad associated with onboard computer 50 for indicating such times, or that there is used some automatic method with supporting apparatus for detecting the same including but not limited to the sensing of pressure changes to the hydraulic (or similar) system associated with forks 10L, asset dimensioning device 30 viewing forks 10L via field-of-view 3Ov or self-tracking devices 20a, 20b, 20c generating the current extrapolated location and orientation of forks 10L that may be interpreted in light of accumulated virtual warehouse data to imply engagement and disengagement;

5. that also there is either a manual / operator based method with supporting apparatus for indicating the Z height of forks 10L at the time of engagement / disengagement including but not limited to a set of markings placed upon the fixed mast 10L-M allowing the up / down movement / height of forks 10L to be visibly determined by the operator and entered via a keypad associated with onboard computer 50, or that there is used some automatic method with supporting apparatus for detecting the same including but not limited to first placing markers such as 10L-m1 and 10L-m2 onto forks 10L for second detecting forks 10L Z height by self-tracking devices 20a, 20b, 20c, using information gathered by asset dimensioning device 30 to further detect fork 10L height or sensing fork 10L height via ultrasonic, lidar or similar apparatus;

6. that also there is either a manual / operator based method with supporting apparatus for indicating a sufficient approximation of load 4 dimensions including but not limited to using bar code scanner 40 or keypad, both associated with onboard computer 50, for entering a unique identifier associated with load 4 for recalling dimension information from a stored inventory database, or that there is used some automatic method with supporting apparatus for detecting the same including but not limited to receiving information from a forklift 10 external device such as an automated end tally system or similar load scanning apparatus that also gathers load 4 outer measurements or using asset / load dimensioning device 30 capable of self-determining the shape and size of a load 4 being engaged or disengaged from the forks 1OL, and

7. using an onboard computer 50 attached to forklift 10 in communication with both the operator, when and if needed, and all other sensing apparatus for calculating forklift 10's location and orientation and then translating this into load 4's location and orientation at least at the times of engagement and disengagement, where this information relating to the pick up and drop off of load 4 is either immediately or ultimately communicated to a central inventory control computer 60 for maintaining a virtual warehouse of all load 4 current locations and orientations.

As the present inventors have already taught, and as will be discussed again here in reference to Fig.8, these elements are sufficient to establish a database of load 4 locations and orientations within a virtual warehouse representative of the actual warehouse. Furthermore, these elements are sufficient to create a detailed location of each load 4 surface meaning the X, Y, Z relative coordinates of each edge point of each side of each load 4. This detailed information is of useful importance especially in reference to the teachings of the prior related U.S. Patent No. 5,960,413, entitled Portable System for Inventory Identification and Classification. This prior patent teaches remotely identifying individual loads 4 by first projecting a laser spot onto one of their sides where the X, Y, Z location of the laser spot is first calculated and then used to sort through the virtual warehouse to determine which load 4 surface / side is being shined upon. Still referring to Fig. 8, using device 20a as an example self-tracking device, there is shown preferably two self-tracking device center-points 20a-c1 and 20a-c2 representing either multiple fields-of-view such as 20a- v1 and 20a-v2, or detection fields such as 24df-a and 24df-b, or a single field-of-view 20a-v1 or detection field 24df-a that is either swept or transported. These center-points 20a-c1 and 20a-c2 are either fixed or dynamically calculable in distance between each other and with respect to some point such as 10L-Mc on forklift mast 10L-M. Via their fixed or calculable relationship with point 10L-Mc, center points 20a-c1 and 20a- c2 are further relatable to some point 10L-f-c on movable arms 10L-f, supported by mast 10L-M that bears load 4 up and down in a vertical plane; (where the combination of mast 10L-M and moveable arms 10L-f comprise forks 10L of prior paragraphs.)

These fixed relationships therefore imply that each center-point 20a-c1 and 20a-c2 can be extrapolated to the coordinates of any point along mast 10L-M, especially that point where movable arms 10L-f are detected to be vertically positioned. One method for determining the alignment height of arms 10L-f on mast 10L-M, as previously discussed, is to affix preferably two markers such as 10L-m1 and 10L-m2 onto some portion of movable arms 10L-f. Also as previously taught, using one or more fields-of-view or detection fields on self- tracking device 20a, the distance from either center-point 20a-c1 or 20a-c2 to each marker 10L-m1 and 10L- m2 could be determined. Because arms 10L-f are pre-known to be limited in movement to a single vertical plane, their detected movement can also be translated into the change in this single plane, i.e. single variable Z, which is of course the height of the arms 10L-f. Using this detected height Z of arms 10L-f the determined location and orientation of mast point 10L-Mc can therefore be translated to some fixed point such as 10L-f-c, which is preferably point 4-c the bottom-center of load 4, as will be obvious by those familiar with forklifts and geometric coordinate systems.

Hence, at any given time, self-tracking device centers 20a-c1 and / or 20a-c2 are immediately translatable into some chosen fixed spot on or between movable arms 10L-f, such as the front-to-back and side-to-side center 10L-f-c. The actual location point to use on arms 10L-f for calculations is immaterial, and therefore could just as easily be the near or far end of the arms 10L-f with respect to the forklift 10 cab, or some other location. What is important to see is that the forks 10L-f move in only a single Z dimension with respect to self-tracking device 20a and that any means for automatically or manually determining the current position of the forks 10L-f in that dimension at the time of engagement and disengagement is sufficient to accomplish the teachings of the present invention.

Still referring to Fig.8, now that the relationship between some fixed point 10L-f-c on movable arms 10L-f and self-tracking device centers 20a-c1 and / or 20a-c2 has been shown to be calculable, and based upon the teachings herein for both automatic and manual methods and supporting apparatus of doing the same, what remains is to be reviewed is how the final resting location and orientation of load 4 can be determined via this relationship, relative to the warehouse or yard coordinate system. A further object of the present invention is express this resting location and orientation of load 4 as the coordinates of the corner points of load 4's exterior surfaces, for example corners 4-p2, 4-p6, 4-p7 and 4-p3 as well as comers 4-p1 , 4-p4, 4-pS and 4-p8 (not depicted.) As will be understood by those skilled in the art of geometric systems, a rectangular shape such as formed by a pack of lumber and example load 4, can be fully described by its eight corners, i.e. 4-p1 through 4-p8. All that is needed to make this translation from load center 4-c to load corners 4-p1 through 4-p8 is the exact or approximate dimensions of the load 4 as well as its exact or approximate centering on forks 10L-f.

As previously discussed herein, these dimensions can either be manually determined and entered into onboard computer 50, or automatically determined by asset dimensioning device 30. What is important is that once determined, these dimensions are then sufficient to allow the determined relative warehouse location and orientation of load center 4-c to be translated into the relative locations and orientations of each corner point such as 4-p1 through 4-p8 using simple geometry, as will be understood by those skilled in the art. Obviously, loads 4 with shapes other than rectangles can be treated with a similar understanding, even when their shapes may not have corners, such as drums or rolls of paper. As will be understood by those skilled in the art, the outer surfaces of these non-rectangular loads must be described with other geometry, for instance centered circles.

Thus the present inventors have shown that by adding a self-tracking device such as 20a, 20b or 20c to a forklift 10, where this device may continuously, periodically or on demand calculate its own position by triangulating to low cost and passive markers, then sufficient data can be gathered to at least approximate the location and orientation of each load 4 in storage relative to a prepared warehouse, yard or similar space. Furthermore, the present inventors have shown that several base technologies exist as viable alternatives for self-tracking and that these same or other base technologies can be used to determine load presence and dimensions as well as pick-up and drop-off heights. And finally, the present inventors have also shown that manual methods may be used to alternatively determine load presence, load dimensions and fork height, thereby demonstrating a larger method beyond any select set of apparatus. Referring next to Fig.9, there is shown an alternative system comprising all of the same elements as depicted in Fig. 1 except for the following:

• The forklift 10 no longer includes load dimensioning sensor 30 capable of determining the presence of load 4 on forks 10L as well as the dimensions of load 4 for use in the calculations of corner points such as p1 (x, y, z), p2 (x, y, z) ... p8 (x, y, z); • Corner points p1 (x, y, z), p2 (x, y, z) ... p8 (x, y, z) are also now longer depicted on packs / loads labeled 5124 resting on 6372, and 5447 resting on 5388, thus indicating that they will not being directly sensed (although they might still be estimated via calculation,) and therefore potentially not stored in the virtual warehouse database;

• Now included on each pack / load 4, i.e. 5124 resting on 6372, and 5447 resting on 5388, there is shown RF ID poach 8-m (on pack labeled 5124,) 8-o (on pack labeled 6372,) 8-n (on pack labeled 5447,) 8-p (on pack labeled 5388,) and 8-q (on pack labeled 5448 still on forks 10L); o Where each poach 8-m, 8-n, 8-o, 8-p and 8-q is preferably a removable packet that contains a single RF ID, preferably passive, preferably an RF Saw Tag, as well as a printed bar code label uniquely identifying the RF ID. (However, it should be noted that the teachings herein work equally well if the RF ID is not a SAW Tag and / or if it is permanently affixed to the pack label, such as paper tags 5124, 6372, 5447, 5338 and 5448, excepting for the extra costs involved with the ongoing consumption of the more expensive Saw Tags.) Still referring to Fig. 9, in this alternative embodiment of the present invention, additional RF ID's are employed such as those contained in poaches 8-m, 8-n, 8-o, 8-p and 8-q, ideally one per load 4. Preferably, the RF ID's used can be sensed by their appropriate RF detector at longer distances such as forty or more feet away. For these reasons, the present inventors currently prefer RF SAWs Saw Tags, which as previously discussed can be detected at 100' to within 2" with current technology. While it is expected that this will add roughly $1 to $3 per load 4, there are several added benefits as will be herein taught. First, when forklift 10 has arrived at the intended drop-off / storage location such as in step 108 in Fig. 7a and then proceeds to disengage load 4 in step 110, it will still have at least three alternatives for describing the location of load 4, without fully describing the X, Y, Z location of load 4 corners such as p1 (x, y, z), p2 (x, y, z) ... p8 (x, y, z), such as:

1. At the time of disengagement, onboard computer 50 may still calculate and store load 4 center 4-c shown in Fig.8, where again load 4 center 4-c can easily be assumed as some equidistant point between forks 10L-f, either at or somewhat above determined fork 10L height (or any other similar strategy as will be obvious from an understanding of forklifts and the particular loads 4 they will be carrying);

2. Using any self-tracking device employed, such as 20a, 20b or 20c, (of course using the appropriate base technology such as RF tag readers 104 from RF SAW,) the onboard computer 50 may now also sense the relative distance to the appropriate pouch, e.g. 8-q attached to pack / load 4 labeled 5448, which contains a unique Saw Tag; a. Assuming device 20a, 20b and / or 20c detects the unique Saw Tag in poach 8-q with at least two antenna or two separate swept readings from one antenna, onboard computer 50 may then extrapolate from the forklift 10's current location (and not orientation) to the location (and not orientation) of poach 8-q, where this unique poach location 8-q-c may then be stored in the virtual warehouse database as the location of load 4, i.e. in place of first calculating the outer corners p1 (x, y, z), p2 (x, y, z) ... p8 (x, y, z) and then storing this information for describing load 4, and in place or in addition to the calculated load 4 center 4- c; 3. Or, onboard computer 50 may also associate the current load 4 and poach 8-q with the current warehouse number, bay number and stack number (all general location data with specific X, Y, Z coordinates) as well as the fork height at the time of disengagement, all of which can be determined without sensing the Saw Tag in poach 8-q, and where it is further realized that: a. Especially for packs / loads 4 of lumber, each bay within any indoor or outdoor storage space will tend to be a large area, e.g. 20' wide by 20' deep, (due to the longer length of the lumber packs / loads 4,) and therefore the current warehouse and bay are easily determined by understanding the current location of forklift 10 within the warehouse; b. At least for bays storing packs / loads 4 of lumber, there are typically three stacks formed starting from the back wall and working towards the aisle, as will be discussed further with respect to Fig. 11a and Fig. 11b, where each stack is therefore separated by the typically at least 4' (i.e. the standard pack / load 4 width) and therefore the stack number 1 , 2 or 3 is easily determined by understanding the current depth location of the forklift 10 within the current bay, or c. The fork 10L height may be determined via a number of methods both manual and automatic as previously described herein and therefore easily associated with load 4, however rather than an absolute height, it is also possible to simply translate this position into a pack / load 4 sequence number within the determined 1 , 2, or 3 stack 4s, where sequence 1 is the first / lowest load 4 in the stack resting on the warehouse 2 floor upon which is then placed sequence 2 load 4 and then sequence 3 load 4, etc., all of which will be further discussed in relation to Fig. 11a and Fig. 11b.

Fig. 9 depicts the use of poaches 8 to hold RF SAW Tags or similar detectable RF ID based upon the assumption that even at a lower cost of $1 it might prove cost effective to rotate these ID's as loads 4 come and leave the company's possession. It should be understood that under the right economic circumstances poaches 8 become instead disposable load labels / tags, onto which the disposable SAW Tag, RF ID chip or equivalent has been attached, and therefore any such arrangement or its equivalent are considered to be within the scope of the present invention.

Referring next to Fig. 10a, there is depicted virtual warehouse 100, for example laid out similar to a physical warehouse 2 or shed at a typical lumber wholesale company. As will be understood by those skilled in the art, other arrangements are possible. What is important to notice is that some products / loads 4 for storage by manufacturers, wholesalers, retailers, etc., may share at least two qualities that make them similar to packs of lumber and possible candidates for use with the teachings herein:

• First, the products are large enough that they would become loads 4 to be carried about only or most often using a forklift 10 (e.g. as opposed to warehouse personnel using a hand truck, although this should not be construed as a limitation since the hand trucks could also self-track.) Therefore, the present invention includes smaller products that are typically stacked together onto pallets, where the pallets themselves become loads 4, and

• Second, the products are not easily stored in a binning system and consequently it is often the case that they can become misplaced through repeated movements.

Regardless of these two characteristics, the teaching herein presented have significant use for tacking any loads 4 carried about by forklifts 10, even if the loads 4 are not coming or going into storage, as for instance would be the case as a part of a step in a manufacturing process. In the broader sense, the present inventors are teaching how self-tracking forklifts 10 can be used to track the pick-up, transportation and drop-off locations of their loads 4, where the type of load 4 and the need for knowing this are immaterial to the scope of the present invention.

Still referring to Fig. 10a, in depicted virtual warehouse 100, representative of a typical lumber yard, the physical warehouse 2 and therefore virtual warehouse 100 is typically broken into several consecutive bays, such as 109, 110 and 111. In such a warehouse 2, each bay such as 109, 110 and 111 typically runs along the sidewalls of warehouse 2 / 100, as previously discussed and depicted earlier in this specification. Running down the center of the warehouse 2 / 100, between bays such as 109, 100 and 110, are access aisles, which are not shown in Fig. 10a but are implied. For an example wholesale lumber storage warehouse, each bay such as 110 is typically 20' wide; that is from side-to-side running along the sidewall of warehouse 2 and 100. The side-to-side direction is assigned the "X" axis in the virtual warehouse 100. Each bay such as 110 runs up to the ceiling of warehouse 2 / 100, where the ceiling is typically at least 25' off the warehouse 2 / 100 floor. This bay height is assigned the "Y" axis in the virtual warehouse 100. And finally, each bay of a typical lumber warehouse 2 / 100 is deep enough to hold three stacks 4s as shown in bay 110; where each stack 4s comprises one or more packs / loads 4. For lumber, these stacks 4s are typically aligned lengthwise along the X axis. Given this lengthwise alignment, it is the pack / load 4 width that determines the depth or "Z" axis dimension of a given bay such as 109, 110 and 111 in the warehouse 2 / 100. While also immaterial to the specification, the present inventors are referring to these stacks 4s numerically as stack 1, 110-1, stack 2, 110-2 and stack 3, 110-3 starting from the back of the bay 110, or sidewall of warehouse 2 / 100 coming towards the aisle.

Still referring to Fig. 10a, each load 4 in stack 4s within a given bay such as 110, is depicted as a solid similar to its actual physical form. In order to accomplish this, the present invention may be used to automatically sense, determine and / or estimate the outer corners p1 (x, y, z), p2 (x, y, z) ... p8 (x, y, z) of each load 4, such as depicted for load 4-3-4, which is an example load 4 that is 4^th from the bottom on stack 3, 110-3, in bay 110. Within the present invention, especially the teachings in reference to Fig. 1 through Fig. 8, multiple alternatives for determining and / or estimating outer comers p1 (x, y, z), p2 (x, y, z) ... p8 (x, y, z) are taught. Also shown in Fig. 10a, for each load 4 such as 4-3-4, the present invention may also calculate the X, Y, Z coordinates of an attached poach, such as 8-3-4 centered at p9 (x, y, z), where the poach center is shown, such as 8-3-4-c. As previously taught, it is assumed that poach 8-3-4 attached to load 4-3-4 contains some form of remotely detectable RF ID, especially as discussed in reference to Fig.9. All of this determined and / or estimated information can be held in a computer database such as might be found on inventory control computer 60 and used to both recreate a virtual warehouse 100 likeness of the current state of physical warehouse 2 and to perform meaningful sorts and calculations on individual loads 4, stacks 4s and bays such as 110. The data held in support of the virtual warehouse 100 will be further discussed with respect to Fig. 11a and Fig. 11b.

And finally, while still referring to Fig. 10a, the present inventors prefer combining detailed "interior" pack / load 4 information, gathered by end tally devices such as that described in U.S. Patent No. 5,307,294, entitled Automated End Tally System, with the aforementioned outer corners, or "exterior" pack / load 4 information. Furthermore, as previously mentioned, using devices such as or similar to the Automated End Tally System, provides a separate means for determining asset / load 4 outer dimensions which is useful for the automatic extrapolation from the current location and orientation of forklift 10 and forks 10L at the time of disengagement to the location and orientation of outer corners p1 (x, y, z), p2 (x, y, z) ... p8 (x, y, z) as well as the location of any attached poach / label 8 containing a remotely detectable RF ID. The combination of this "interior" (i.e. board-by-board) and "exterior" (i.e. outer surface) information can be depicted in 3D graphics to form a more realistic virtual warehouse 100, as will be understood by those skilled in the art of real-time animation and the datasets generated from such scanning devices as the Automated End Tally System.

The present inventors are aware of at least one other system in the marketplace that is similar to the Automated End Tally System. This system is being marketed as the Picture Tally, by River City Software, of Exeter, NH. While the Picture Tally does not fully measure the outer dimensions of a unit, it is still useful for estimating the outer dimensions in that it first measures the dimensions of one end of a pack / load 4 and then allows the operator to indicate the length of the pack / load 4. Together, the measured dimensions provide the pack / load 4 width and height while the entered dimensions provide its length. This combination of width, height and length is similar to the measurements made by the Automated End Tally System and are incorporated for use by the present specification as both an alternative means of supporting the calculation of outer corners p1 (x, y, z), p2 (x, y, z) ... p8 (x, y, z) as well as providing useful graphical "interior" details about loads 4 to be depicted within virtual warehouse 100. Such graphical details include but are not limited to both the wireframe estimation of the "interior" of lumber pack / load 4 as well as a digital image of the same, all of which is likewise available from the Automated End Tally System.

Referring to Fig. 9 and Fig. 10a, the present inventors teach a method for restoring the virtual warehouse 100 database in the event that any information is lost. Specifically, self-tracking forklift 10 may drive up to a given bay, such as 110, where the virtual warehouse 100 database information needs to be restored. During this process, if for example RF reader technology 104 is being used, then self-tracking forklift 10 may now also detect poaches 8 that are attached to loads 4. In the case where multiple antennas are used such as 20a-a and 20a-b as shown in Fig.4b, then in a single reading tracking device 20a may determine the exact location of each detectable poach with respect to forklift 10.

Once determined, this information will allow onboard computer 50 to extrapolate from the currently detected location and orientation of forklift 10 to the current location of each detected poach 8 on loads 4 stored in a bay such as 110. Furthermore, as will be appreciated by a careful reading of the present invention, the location of each poach 8 may be used as an approximate replacement for the center point of each load 4. Hence, by knowing the location of each poach 8, and operating under the assumption that they have been placed upon each load 4 following a uniform procedure, than their individual locations may be further used as a means for extrapolating approximations for outer corners p1 - p8 using pre-known load 4 size information. It should be noted that at least for lumber packs, it is typical that poaches such as 8 might always be aligned in the middle of the left side of pack / loads 4 placed in stacks 4s within a given bay such as 110. As will be appreciated by those familiar with coordinate systems, these assumptions are sufficient along with the approximate pack dimensions to support the recreation of outer corner locations for p1 through p8. Of course, these detected poach 8 locations themselves could suffice as the load 4 centers and the virtual warehouse database can easily accommodate an understanding that some loads 4 are being identified by the location of their poach 8 while others have a more complete set of information. Using this same basic method, it is additionally possible to use poaches 8 to assist in the initialization of virtual warehouse 100. Hence, for existing warehouses 2 where inventory loads 4 are already in storage at the time self-tracking forklifts 10 are implemented, it is possible that as a first step a unique poach 8 may be strategically placed onto each load 4 in a manner similar to that described in the prior paragraph. This can be done with a minimum of labor and without moving any load 4. As each poach 8 is attached to a load 4, its unique poach number, preferably representing the RF ID tag number, may be scanned by a hand held such as 40 shown in Fig. 1. As a second step, after scanning the poach 8 RF ID, the unique label also attached to load 4 may be scanned thereby associating the companies product and load ID with the detectable RF ID code / poach 8 number. As a third step, forklift 10 may then traverse warehouse 2 going from bay to bay reading all of the attached poaches 8 to quickly form a virtual warehouse 100 database. As mentioned in the last paragraph, the database might only include the poach 8 locations that are themselves sufficient to differentiate loads 4 as they are picked up and removed from storage, or the database might additionally or alternatively include a further estimated load 4 center point such as 4-c shown in Fig. 8, or even estimated outer corner points p1 - p8. Again, as previously explained, outer corner location p1 - p8 can be extrapolated from poach 8 locations along with placement assumptions and standard pack / load 4 size estimates. (Note, in relation to upcoming Fig. 12a and Fig. 12b, it will be further explained how poach 8 locations alone will suffice to differentiate loads 4 for the uses benefits and uses specified herein.)

And finally, as will be understood by those familiar with base technologies 102 and 104, especially in light of prior related U.S. Patent No. 5,960,413, entitled Portable System for Inventory Identification and Classification, it is possible to make a portable self-tracking device that will perform similar functions as forklift 10 for self-determining poach locations. For the purposes of either initializing the virtual warehouse database or restoring some portion of lost data, it is therefore possible to use a portable hand held device rather than self-tracking forklift 10, as will be obvious by a careful reading of the present specification. The portable device taught in U.S. Patent No. 5,960,413 along with a new preferred alternative herein taught, will be discussed further in relation to Fig. 12a, 12b, 12c and 12d

Referring next to Fig. 10b, there is depicted the equivalent of Fig. 10a without the solid shapes of loads 4 made possible by an understanding of the X, Y, Z locations of their outer corners p1 (x, y, z), p2 (x, y, z) ... p8 (x, y, z). Instead, there is shown load 4 center point p10 (x, y, z), such as 4-3-4-c, for example load 4-3-4. In addition to load 4 center points such as 4-3-4-c there is also depicted the X, Y, Z coordinates of an attached poach 8-3-4, such as 8-3-4-c, shown as point p9 (x, y, z); exactly similar to what is shown in Fig. 10a. When taken together, Fig. 10a and Fig. 10b depict the following important load 4 coordinate data that is determined and / or calculated by the present invention, specifically including:

1. Multiple points p1 (x, y, z), p2 (x, y, z) ... p8 (x, y, z) representing the location of each of load 4's outer surfaces, first shown in Fig. 1 and then also in Fig. 6a, Fig. 6b, Fig.8 and Fig. 10a. a. As will be understood by a careful reading of the present invention, the exact geometric model chosen to represent each distinct load 4 shape stored in a warehouse is immaterial. What is preferable is that some consistent model, that is applicable to at least any load 4 with the same basic geometric shape, be used as a representation of where the current load 4 has been placed in storage, and specifically where its outer surfaces are located and how they are oriented. It is even possible that the rectangular shape of a pack of lumber or a full pallet of bagged materials be represented as a single rectangular face (i.e. two dimensions) as well as a second indication of depth (i.e. the third dimension), where the indication of the third dimension is simply a scalar that is used to extrude the rectangular face in a consistently understood direction.

2. A single point p10 (x, y, z) representing the location of each load 4 in storage, such as load center 4- c, first show in Fig. 8 and then also depicted in Fig. 10b as 4-C-110-3 a. As will be understood by a careful reading of the present invention, the exact location chosen to represent load center 4-c is immaterial. What is preferable is that some consistent location, that is applicable to any load 4, be used as a representation of where the current load 4 has been placed in storage. It should be understood that while Fig. 8 shows the load center 4-c to be the X-Z center on the bottom facing surface of a six sided lumber pack, this is not the only possible location. It is even possible, and in fact beneficial, that the chosen load center 4-c be different for different product types, especially where the different product types have different forms, such as rectangular vs. cylindrical. b. Note that while it is possible to generate a representative visualization for a given load 4 in the virtual warehouse 100 based solely upon that load's center point 4-c as well as an approximate understanding of the load's size and orientation, this virtual depiction is more readily accomplished by storing the outer corners of loads 4.

3. A single point p9 (x, y, z) representing the location of each load 4's associated poach 8, first shown in Fig. 9 and then also in Fig.10a and Fig. 10b.

As will be discussed in greater detail with respect to Fig. 11a and Fig. 11b, these three types of specific location data may be used separately or in any combination to provide significant novel value as already taught herein. Therefore, the present invention should not be limited to some specific combination of this data. As will be understood by those skilled in the art, the present invention has significant value even if it only determined and / or calculated load 4 centers, represented as p10, or instead load 4 outer corners represented as p1 - p8, or instead attached poach center represented as p9. Each of these three pieces of specific location data, or their equivalents, may be used to at least direct the picking up, transporting and dropping off of loads 4 within warehouse 2. Obviously, the differences in the granularity of their information (i.e. the precision and accuracy of their X, Y and Z dimension measurements) affect the display of loads 4 within virtual warehouse 100.

Referring next to Fig. 11a, there is shown a representative virtual warehouse database 1, 102-1, database 2, 102-2 and database 3, 102-3. Each database 1, 2 and 3 tracks progressively more detailed information regarding all loads 4 present in a given warehouse 2. Specifically referring to database 1, 102-1, there is shown that for each bay, such as 10, for each stack number 1, 2 or 3, a list of 10 possible loads is maintained. (Note that it is assumed that a warehouse code is also being kept but since bays may be number sequentially across all warehouses, sheds and otherwise any storage area either indoors or outdoors, storing load 4 location data by bay would be sufficient.) This list simply contains the load ID such as a tag number assigned by the company to each load 4. In this case, sequence 1 is assumed to be the bottom-most load 4 resting on the floor of warehouse 2. Obviously, the number of stacks and sequence codes is immaterial. Also as obvious, other warehouse arrangement are possible that do not have "bays" or "stacks." Such example might be for a warehouse with "rows," "aisles" and "bins." Therefore, as will be understood to those skilled in the art of warehouse systems and database design, other arrangements of information and naming conventions are possible. What is important to see is that all loads can be tracked by the herein taught invention with at least some portion of this minimal set of data:

A. Warehouse / Shed / Yard: a. This information indicates where a load 4 may be found in the larger view familiar to typical warehouse operations;

B. Bay / Storage Area: a. This information narrows down the location of load 4 to a smaller area within the Warehouse / Shed or Yard, which is also a typical practice; b. It is typical that a single Bay / Storage Area is limited to a single product or type of product for ease of human interaction;

C. Stack Number: a. Within the smaller Bay / Storage area, there are typically designated locations roughly the size of loads 4 intended for the placement of actual loads 4, where the loads 4 may or may not be stacked one upon another. If they are not stacked, than this is the same as a stack of one load and stack number ends up representing a specific area with the larger Bay / Storage Area, within the Warehouse / Shed / Yard; b. It is also possible that loads 4 are stored vertically but that they are not stacked. In other words, they may be resting in some form of a bin or on a structure designed to hold the next load 4 without resting it directly upon the load 4 beneath it.

D. Sequence Number: a. If loads 4 are stacked in order to take advantage of ceiling height to increase storage density, than this number indicates the order in which the loads in a given stack are arranged, again preferably from bottom to top.

It is important to note that this first set of information, e.g. Warehouse 5, Bay 18, Stack 2 and Sequence 3 can be gathered by the present invention using various apparatus and methods as previously discussed.

Therefore, while the present invention has taught how a self-tracking forklift 10 may determine and / or calculate specific location data including:

1. multiple points p1 (x, y, z), p2 (x, y, z) ... p8 (x, y, z) representing the location of each of load 4's outer surfaces;

2. a single point p10 (x, y, z) representing the location of each load 4 in storage, such as load center 4- c, and

3. a single point p9 (x, y, z) representing the location of each load 4's associated poach 8; this information may be further simplified into general location data (or its equivalents):

A. Warehouse / Shed / Yard:

B. Bay / Storage Area:

C. Stack Number:

D. Sequence Number:

As will be obvious to those skilled in the art of local positioning systems, the specific X, Y, Z coordinate based information (such as specific location data 1 through 3 above) can be abstracted and generalized into coded values (such as general location data A though D above.) For instance, at the time of disengagement, self- tracking forklift 10 may use its current coordinates and orientation to easily determine all of the information A, B, C and D above for updating the virtual warehouse database 1, 102-1 associated with the load 4's ID; where this ID is either already known to the forklift 10 or may be entered by the forklift operator by use of bar code scanner 40 or even into a keypad on onboard computer 50. It should be evident that after determining the general location data A through D, very possible using some or all of the specific location data 1 through 3, the specific location data 1 through 3 does not necessarily need to be stored into the warehouse database, such as 102-1. Therefore, the presence or absence of any one or more pieces of specific location data 1 through 3 within the virtual warehouse databases such as shown in Fig. 11a and upcoming Fig. 11b, should not be taken as a limitation of the present teachings. Conversely, and as previously stated, any one or more pieces of the specific data 1 through 3 is sufficient for the virtual warehouse database 102 to be of use, as long as this data is associated with the load 4 ID, and therefore none of the general data A through D necessarily needs to be stored in database 102. Therefore, it should be further evident that forklift 10 still provides value even if it is not self-determining all of specific location data 1 through 3 or general location data A through D. Other simplified variations of the present invention are possible such as, but not limited to:

• A forklift 10 that uses a keypad attached to onboard computer 50 to accept the current codes for A through D without any automatic detection, hence the forklift operator 10 is providing the equivalent of initial self-tracking determinations as each load is placed into storage, where for instance: o Information D, the sequence number is determined simply by human observation, where the forklift operator counts the number of loads 4 underneath the current load 4 being placed on a given stack 4s; o Information C, the stack number is determined simply by human observation, where the forklift operator perhaps views a marker placed on the warehouse structure or a painted symbol on the warehouse floor; o Information B, the bay / storage area number is determined simply by human observation, where the forklift operator perhaps views a marker placed on the warehouse structure or a painted symbol on the warehouse floor, and o Information A, the warehouse / shed / yard code that is determined simply by human observation, where the forklift operator perhaps views a marker placed on the warehouse or shed structure, or in the yard area;

• A forklift 10 that additionally determines any one or more pieces of information A through D in any combination with the above mentioned manual observations, using any apparatus or methods herein taught or their equivalents, as will be understood by those skilled in the various requisite arts.

As will be understood by those skilled in the art, given all of the various apparatus and methods taught herein, what is most important is that the general information A through D is determined as it is relevant to a particular warehouse arrangement. The present invention should not be limited to the requirement of determining all of this general information A through D automatically, (therefore without operator intervention.) As it has been shown herein, there are many different and sufficient fully automatic apparatus and methods for determining A through D. In practice however, for any number of reasons, it might be best to automatically determine some portion of general location data A through D while manually determining the remainder. For example, it may be best to automatically determine Sequence Number D, for instance based upon the detected current height of fork 10L-f, simply because this is economically cost effective to implement while at the same time frequently required and more difficult for an operator to judge, even if looking at a mast 10L-M that has ruler marking. At the same time, while general information D is automatically determined, having the operator fully determine the remaining information C, B and A via observation plus manual data entry into onboard computer 50 is still within the scope of the present invention. A similar example should be obvious to anyone skilled in the arts herein taught for best determining any one piece of general location data A through D while relying upon human observation and data entry for the remaining information.

Again, and for example, it may also prove beneficial to use simple RF ID resets (without the accelerometers,) such as discussed in relation to accelerometer base technology 106, only at the entrance areas to a given warehouse or shed that can be sensed by forklift 10 upon entering to provide data A, so that this information is automatically determined and provided to onboard computer 50, while the operator manually observes the bay B and stack locations C, and possibly even the fork 10L-f height and therefore sequence number D. Furthermore, RF ID resets could be used to generally mark entrance into a given bay area, data B, while the operator or machine vision technology 102 is used to observe the specific stack, data C. All of these combinations show what is possible by a careful reading of the present specification. Therefore, all of the various combinations are being discussed herein in order to establish that the scope of the present invention should not be limited to a self-tracking forklift 10 that must automatically determine all or indeed any of general location data A through D and / or specific location data 1 through 3. Many combinations are possible, all falling within the intended scope and purposes of the present invention.

Therefore, what is to be claimed by the present inventors is first that this general location data A through D, or its applicable equivalents, is determined for each load 4 being placed into storage. Once this is available, this data has many uses for preparing picking and load movement operations as well as confirming load 4 identity at the time of engagement. Furthermore, the determination and use of any one or more specific location data sets 1 through 3 is also to be claimed for use instead of, in combination with, or in support of, general data A through D

It is also noted that for some warehouse arrangements, information D sequence number, may not be important since loads 4 are not stacked. Also, note that for some or other warehouses, information C, stack number may not be important since loads 4 are simply segregated into a larger bay-like areas; or it may be good enough to simply know what warehouse, shed or yard a given load 4 is stored within. Therefore, even with respect to general location data A through D, the present invention should not be limited to being only applicable to warehouses laid out in such a fashion as to require any specific piece, or all of, this general location data A through D - for example like a typical lumber warehouse. Conversely, a more complex warehouse could be imagined where an additional piece of general location data E might be beneficial. All of this general location data A through D serves the same purpose; that is to uniquely differentiate the location of one load 4 from every other load 4 in a company's storage areas. The exact coding system of general location data is immaterial. The self-tracking forklift 10 is capable of determining any set of general data such as A through D that is desirable for uniquely denoting loads 4 within a database. It should also be obvious that ultimately self-determining specific location data 1 and / or 2 and / or 3 is sufficient for automatically deducing any system of general location data, such as A through D discussed herein.

Referring still to Fig. 11a, it can be seen that general location data A through D held in database 102-1 can be incrementally augmented by the inclusion of more and more specific location data such as 1 through 3. For example, the actual height of forks 10L-f might be added to database 102-1 forming database 102-2. While the sequence number for a given load 4 has not changed, by also storing the manually or automatically determined height of forks 10L-f at the time of disengagement into storage, the present invention may use this information to more accurately create depictions of virtual warehouse 100, as well as to more accurately confirm load 4 identity at the time of subsequent engagement / pick up, etc. Furthermore, if each load 4 included poach 8 or some equivalent preferably including a detectable RF ID, than the specific location of this poach 8 (data 3) might also be stored as the RF ID location in a database such as 102-3. Referring next to Fig. 11b, virtual warehouse database 4, 102-4 no longer includes poach 8 specific locations (data 3) but does now include the specific location of load 4 outer comers (data 1). And finally, in database S, 102-5, there is shown a database including both specific data 1 and data 3. Note that none of these databases show specific location data 2 that represents some location within load 4, preferably a center spot. This location data 2 could just as easily replaced specific load 4 outer corner locations data 1, or poach 8 location data 3. What is important to see from a review of databases 102-1 through 102-5 in Fig. 11a and Fig. 11 b is that any single piece of specific data 1 , 2 or 3, or its generalized translation into data A, B, C or D may be stored in any one or more combinations to provide different benefits as both herein discussed and as will be obvious to those skilled in the art of inventory and warehouse management. Therefore, the present invention should not be limited to any exact combination of specific location data 1 through 2 or generalized location data A through D as represented for example in Fig. 11a and Fig. 11b.

Referring next to Fig. 12a, there is depicted a block diagram of a portable observation point device 1000 as first taught within related U.S. Patent No. 5,960,413, entitled Portable System for Inventory Identification and Classification. This Fig. 12a is exactly equivalent to Fig. 2 of this prior U.S. Patent No. 5,960,413 except that the element numbers have been somewhat changed and specifically portable device 100 in the original application is herein referenced as portable device 1000. What is important to note is that this portable device 1000 allows an operator to press a button after which a laser spot will be projected onto any load 4 in warehouse 2 or any other tracked storage area. (A detailed description of the functions and operations of device 1000 are included in the referenced patent (as device 100) and will therefore will not be repeated herein, but are included by reference.) After projecting a laser spot on the sighted load 4, portable device 1000 then uses "outside-in" tracking to cooperatively determine its own X, Y, Z centroid position. Once it has determined its own centroid, device 1000 can extrapolate to the position of the projected spot, and therefore to some point on the outer surface of load 4. Once the position of the projected spot is calculated, its location can be used to sort the virtual warehouse and determine which load 4 is being pointed to by the operator. This is especially true using database 102-4 and 102-5 shown in Fig. 11b, since they include the measured corners of loads 4 that are sufficient for describing the location of all of a given load 4's outer surfaces upon which the spot might be projected. However, the other databases 102-1, 102-2 and 102-3 shown in Fig. 11a may still be used by converting the projected spot information into general location data A through D in a manner similar to the translation of forklift 10's current specific X, Y, Z location into the general location data A through D for load 4, as will be understood by a careful reading of the present specification. As will be further appreciated, portable device 1000 may be modified to use "inside-out" self-tracking in a manner similar to that of self-tracking forklift 10. For this, the present inventors prefer the use of RF reader base technology 104. In reference to Fig. 12a, base technology 104 would replace both transmitter 1000t and receiver 1000r as well as the shared attached antenna 1007 and unit tracking computer system 800 along with its multiple antennas 740a, 740b; as again will be obvious to those skilled in the art of RF SAW technology in light of the teachings herein. Similar to forklift 10, a so modified portable device 1000 may then determine its own location by sensing multiple markers 6 and / or 7 placed throughout warehouse 2 or equivalent indoor and outdoor storage areas.

Referring next to Fig. 12b, there is shown a preferred alternative to original portable device 1000 as taught in U.S. Patent No. 5,960,413, or even as modified by adding self-tracking similar to forklift 10. Specifically, alternate portable device 80 comprises portable identifying device 80c that has attached portable pointing device 8Op. Portable identifying device 80c comprises computer 1000c that controls and receives input from RF Reader 104r that itself controls the emission of interrogation pulses and the interpretation of the reflected signals so that it can both detect the presence of SAW Tags as well as measure the distance to each tag; all of which is explained in the prior paragraphs and well understood in the art. Note that the use of preferred alternative portable device 80 presumes the use of SAW Tags or their equivalent as warehouse markers such as 6 and 7.

Portable identifying device 80c further comprises some form of electronic storage for holding any variation of the virtual warehouse database such as 102-1 through 102-5, where the preferred storage is removable storage device 102-db which could be for instance a removable USB drive holding 1GB or more of data. Identifying device 80c also preferably comprises an LCD screen 1006 or some equivalent, keyboard 1003 or some equivalent and optionally speakers 1005. The input / output functions of LCD screen 1006, keyboard 1003 and speakers 1005 will be obvious to those skilled in the art of human interface design with respect to portable electronic devices and will not be further taught herein.

Still referring to Fig. 12b, portable identifying device 80c has attached to it by typical communications cable portable pointing device 8Op that itself comprises an antenna 104a which is in direct communication with RF Reader 104r. Antenna 104a is used by reader 104r for both emitting interrogation pulses and receiving their return signals for detection. It is assumed that antenna 104a has an ideally shaped detection field designed to roughly project in the forward direction over an extended distance as will be depicted in upcoming Fig. 12c. The means for shaping antenna fields are well understood in the art and the particular means used are immaterial to the present teachings. It should be noted that antenna 104a could just as easily be multiple antennas packaged together in such a way that one "points" generally forward, like the beam of a flashlight, while the other "points" generally upwards or downwards, in order to read other markers 7 and 6, respectively. And finally, portable pointing device 8Op also comprises a Lidar distance measurement unit 1008 for use as the projecting light source to shine upon load 4 poaches as will be discussed in reference to Fig. 12c and Fig. 13c

Referring next to Fig. 12c, there is depicted physical warehouse 2 into which is stored two representative stacks 4s-a and 4s-b, each of which has two stacked loads 4-a1, 4-a2, 4-b1 and 4-b2, respectively. Attached to each load 4-a1, 4-a2, 4-b1 and 4-b2 are poaches 8-o, 8-m, 8-p and 8-n, respectively; where each poach contains a uniquely identifiable RF responsive tag, preferably a SAW Tag. Also depicted is inventory control computer 60 that has been in communication with one or more self-tracking forklifts 10 as they pick-up, transport and drop-off loads 4, including for example 4-a1 , 4-a2, 4-b1 and 4-b2, thereby forming virtual warehouse database 102. Control computer 60 may copy any or all of virtual warehouse 102, in any of its depicted or anticipated variations such as 102-1 through 102-5, or any their equivalents, onto removable storage device 102-db that is preferably a USB drive. It should be noted that computer 60 may also copy onto storage device 102-db other relevant data that may be associated with each load 4 tracked in virtual warehouse database 102 that may be of interest to the operator of portable device 80. Portable device 80 is itself shown as both the portable identifying device 80c and portable pointing device 8Op. Removable storage device 102-db is inserted into portable identifying device 80c and carried about the warehouse 2 or equivalent indoors or outdoors storage areas. As will be appreciated by those skilled in the art of computer networking, other sufficient ways exist that allow computer 60 to exchange information with portable device 80 beyond a removable drive that include at least a wireless or wired communications link. Still referring to Fig. 12c, in operation some company personnel is expected to walk about warehouse 2 with identifying device 80c attached to their person and portable pointing device 8Op held in their hand. As will be obvious to those skilled in the art of electronics, at some time it may be feasible to make a single integrated unit comprising both 80c and 8Op without departing from the scope of the present invention. It should also be noted at this time that portable device 80 has some acceptable form of power source such as a lithium battery as is also well understood in the art. Using pointing device 8Op, the operator presses a button (on device 8Op but not depicted) that causes computer 1000c to direct lidar 1008 to project a visible beam 80-L onto the target. At the same time, computer 1000c also directs RF reader 104r to take one or more readings to determine what SAW Tags if any might be within its shaped detection field 80-df. It should be noted and is depicted that lidar beam 80-L roughly projects through the center of detection field 80-df. In the example shown in Fig. 12c, the detection field 80-df will sufficiently include multiple poaches 8-o, 8-m, 8-p and 8-n attached to loads 4-a1, 4-a2, 4-b1 and 4-b2, respectively. As will be understood by those familiar with warehouse storage and RF ID technology, there will often be multiple RF ID'S responding to a single interrogation pulse and as such providing confusion as to which RF ID is of interest to the operator. By using SAW Tags for the RF ID'S the present invention anticipates receiving a fairly accurate distance to tag measurement along with the list of detected SAW tags, all of which is returned by RF reader 104r to computer 1000c. Simultaneous with receiving a list of SAW tags (in poaches 8-o, 8-m, 8-p and 8-n,) computer 1000c will also input a distance measurement from lidar 1008. If the operator has shined lidar beam 80-L either onto or sufficiently close to a given poach such as 8-m in this example, than the distance from portable pointing device 8Op to poach 8-m will be determined. As will be obvious to those skilled in the art of computer software, the distance measured by lidar 1008 to a given poach such as 8-m is sufficient for selecting from the list of possible poaches 8-o, 8-m, 8-p and 8-n, as detected by RF reader 104r and sorted by distance to tag by computer 1000c.

For instance, RF reader 104r might indicate to computer 1000c that four SAW tags with codes equivalent to "6372," "5124," "5388," and "5447" are detected within the current reading of detection field 80-df. Furthermore, RF reader 104r will also provide a distance to each tag, in this case poaches 8-o, 8-m, 8-p and 8-n respectively, as might look like the following sorted list:

As will be obvious to the reader, if lidar 1008 returns a distance of for instance 58' 7" + / - 2.4" than computer 1000c will be able to accurately select load 4-a2 with attached poach 8-m representing label 5124. Once identified, computer 100Oc may provide many useful types of information as will be well understood by those familiar with inventory control databases and warehouse operations. Note that as the accuracy of RF reader 104r increases, than the required separation distance from pointing device 8Op to detected SAW tags within poaches such as 8-0, 8-m, 8-p and 8-n will also decrease. Based upon the current state-of-the-art, the lidar measurement devices are typically more accurate than the RF readers.

As will be understood by those skilled in the art of software design, there are many viable options for resolving potential ambiguities as might arise when two or more poaches 8 are detected to within 2" of the lidar 1008 measurement. (As previously mentioned, 2" is the current margin of error for RF SAW distance measurements, at least based upon technology being sold by RF SAW, Inc.) For instance, computer 1000c could simply display a graphic of the two or even three confused loads 4, either one on top of the other or side by side, depending upon the information within the virtual warehouse. Upon seeing the arrangement of possible loads 4 being pointed to, the operator is expected to easily understand which load 4 is of interest and then depress some appropriate button, or even tap a touch sensitive screen in order to select that load and then receive the desired information. Of course, many other methods are possible for communicating with the operator to resolve ambiguity.

An additional approach for resolving ambiguity would be to prompt the operator to change their location and then take a second reading where the distances to the poaches 8 may now change sufficiently to resolve on their own. Furthermore, it would be possible to combine the information from these two or more successive (ambiguous) readings to resolve to the correct load 4, as will be obvious to those skilled in the art of local positioning systems.

Another method for resolving ambiguity assumes that virtual warehouse 102 contains specific location data 3, therefore a single point p9 (x, y, z) representing the location of each load 4's associated poach 8. In this case, the simultaneous reading of at least three poaches will suffice to accurately triangulate the X, Y, Z coordinates of pointing device 8Op within warehouse 2, as will also be obvious to those skilled in the art of local positioning systems. This is especially obvious in light of the present teachings where in this example each stationary poach 8 affixed to stored load 4, with its predetermined X, Y, Z warehouse coordinate, provides multiple fixed reference points for pointing device 8Op in a manner exactly similar to how fixed and pre-known markers 6 and 7 provided reference points for self-tracking forklift 10. Once the X, Y, Z coordinate of the pointing device 8Op has been extrapolated from three or more single points p9, this will aid in creating a most accurate depiction of the possible loads 4 based upon the operator's view thereby helping resolve ambiguity. (Note that pointing device 8Op, either using a single antenna or multiple antennas as 104a, may also detect fixed markers 6 and / or 7. These fixed markers will provide additional information that might be used to help the operator correctly choose from amongst two or more poaches 8 at the same lidar measured distance away.) As will be obvious to those skilled in the art of RF ID technology, that alternate portable device 80 taught herein is anticipated to have significant uses and benefits outside of the virtual warehouse teachings of the present invention. As such, the device 80 in this simplest form depicted in Fig. 12c, may sufficiently resolve between multiple RF IDs / SAW Tags by comparing distances detected by RF with that detected by lidar; the resolution of which does not require either a virtual warehouse or predetermined X, Y, Z locations of each SAW Tag. Therefore, the scope of alternate portable device 80 should be considered broadly applicable to use with any system using RF ID's to mark multiple objects where the RF ID readers return only detected RF ID codes plus distance to IDs. The careful reader might also see that some elements from portable device 1000 could be added with benefit to device 80 for resolving ambiguities. For instance, pointing device 80c could be further equipped with a azimuth measurement unit such as 1000g. Thus, as each distance reading is taken by the lidar 1008 and RF reader 104r, an additional reading could be received by computer 1000c providing the elevation of projected lidar beam 80-L. As will be understood by those skilled in the art, this additional azimuth data could be combined with a preset typical distance from the floor to the pointing device to assume that the ambiguous poach 8 must be above or below the horizontal plane running parallel to the warehouse floor at the typical height of the pointing device 8Op. It is further possible that each operator could use the lidar on device 8Op to first measure and set within computer 1000c this typical height at which the pointing device 8Op will be held with respect to the warehouse floor while in use.

Other variations are possible that build upon the teachings of combining an RF distance measurement with a focused light (lidar) distance measurement without departing from the teachings of the present invention for using a portable light pointing device for identifying load 4 IDs. On such variation is a modification to the SAW tag itself that supports identification by project light energy. Specifically, as will be appreciated by those skilled in the art of SAW tag technology 104, each SAW tag receives energy via interrogation pulses that is converted by a RF transducer into the appropriate surface acoustical wave. The wave that then travels down a substrate with strategically placed and oriented wave reflectors. The exact combination of these reflectors serves to encode the unique return wave that is then picked up by the transducer for conversion back to RF. This return RF signal may then be received by any antennas within a range dependent upon the signal's strength; all of which is well known in the art. What has not been taught to the best knowledge of the present inventors is the inclusion of an optical-to-electrical transducer on the SAW tag that is sufficient for creating the surface acoustical wave by itself.

Using this type of SAW tag, that might for instance be activated by either or both RF or optical energy, the portable device 80 may then use a simple laser pointer to project beam 80-L, rather than a more expensive lidar. In this case, computer 1000c will activate the laser pointer version of component 1008, that will then shine upon the optical transducer of the RF ID / SAW tag within a poach 8; while at the same time computer 1000c will no longer direct RF reader 104r to emit an RF interrogation pulse. By so doing, the modified SAW tag will convert the optical energy received from the projected laser into a surface acoustical wave whose reflection will be returned via RF for pickup by antenna 104a. Once picked up by antenna 104a, reader 104r will decode the return wave as normal to therefore provide its unique identity to computer 1000c. Since only one modified SAW tag can be shined upon for any given reading, there will be no possibility for poach ambiguity. As will be understood by those skilled in the art of SAW technology, this modification to SAW Tags to include an optical transducer for producing surface acoustical waves for return by RF will have many possible applications beyond the stricter uses of tracking multiple loads 4 in a warehouse 2. Therefore, the present inventors consider that the scope of this modified SAW tag to include an optical transducer, as well as its accompanying modified portable device for detecting the tag using emitted light and received RF energy, is not to be limited by the larger warehouse teachings within the present specification, but rather is inclusive of the entire range of uses for SAW technology. Furthermore, the present inventors anticipate that other RF ID's, that may be either active or passive, and that receive RF energy which acts as a trigger for response, may similarly provide an option to receive projected light energy to alternatively act as a trigger for response. In general, the present inventors then are teaching the benefits of having an RF ID label that may responds to interrogation either by RF and / or light energy.

Referring next to Fig. 13a, there is depicted a physical warehouse 2 as virtual warehouse 100, especially as might be represented using virtual warehouse database 102-4 and 102-5, both databases of which contain specific location data 1, that is multiple points p1 (x, y, z), p2 (x, y, z) ... p8 (x, y, z) representing the location of each of load 4's outer surfaces. Although the location of any given load 4's outer surfaces may be estimated even simply by knowing specific location data 2, that is single point p10 (x, y, z) representing the location of each load 4 in storage, such as load center 4-c as well as some estimate of the typical orientation and size of load 4, it is preferable that the virtual warehouse database 102 contain specific location data that may be used to most accurately render each side of each load 4. In either case, as was originally taught in related U.S. Patent No. 5,960,413, with the ability to determine the location of each side of load 4, it then becomes possible to inquire upon virtual warehouse database 102 by simply providing the unique X, Y, Z location of some point on one surface of load 4.

Still referring to Fig. 13a, and more specifically device 1000 at position A, it can be seen that portable pointing device 1000p may project focused light beam 1000-L (e.g. from a lidar) onto some spot, e.g. 1000-s on a surface of some load, e.g. sequence 3 in stack 1 of bay 10 in warehouse 2. As first taught in related U.S. Patent No. 5,960,413, and as further modified herein to self-track its own location, portable device 1000 will be capable of extrapolating its self-tracked X1 , Y1 , Z1 location into the X2, Y2, Z2 location of spot 1000-s. Using this extrapolated X2, Y2, Z2 location of spot 1000-s, portable identifying device 1000c attached to pointed device 1000p (together comprising device 1000,) may inquire upon virtual warehouse database 102 to uniquely identify which load 4 is currently being pointed to by the operator; as will be understood by those skilled in the art of location positioning systems and based upon the teachings of the present specification as well as that of related U.S. Patent No. 5,960,413.

Also shown in Fig. 13a is device 1000 at position B, where it is likewise projecting spot 1000-s whose extrapolated X2, Y2, Z2 location is used to inquire upon virtual warehouse 102 in order to determine, for example, that the operator is now pointing at load sequence 2 in stack 3 of bay 10 in warehouse 2. Referring next to Fig. 13b, there is shown a similar use of device 1000 at two positions A and B. In this drawing, the only difference is that projected spots 1000-s at positions A and B are specifically directed to land on poaches 8 that are shown as floating marks 4-10-1-1 through 4-10-3-4 representing each load 4 within the example stacks 1 through 3 and bay 10 of warehouse 2. By restricting device 1000 to only point at a poach 8, the resulting extrapolated X2, Y2, Z2 location of spot 1000-s will be easily matched with specific location data 3, that is the single point p9 (x, y, z) representing the location of each load 4's associated poach 8, as will be understood by those skilled in the art of local positioning systems. Referring next to Fig. 13c, and in comparison to Fig. 13b, alternative portable device 80 is shown at both position A and B projecting spot 80-s from pointing device 80-p attached to identifying device 80-c, both of which comprise device 80 as previously taught. However, in this case alternate portable device 80 does not need to self-track in order to calculate the X1 , Y1 , Z1 location of pointing device 8Op for extrapolation into the X2, Y2, Z2 location of spot 80-s. Rather, device 80 may use the list of RF IDs (preferably SAW Tags) detected within field 80-df, along with their RF determined distance measurements, to directly select from based upon the light determined distance measured by lidar 1008; as taught in relation for Fig. 12b and Fig. 12c Conclusions and Ramifications

Thus the reader will see that the present application for a Load Tracking System Based On Self-Tracking Forklift teaches:

1. apparatus and methods for continually tracking loads carried by forklifts by first providing a less expensive "inside-out" means of forklift self-tracking, the forklift locations of which may then be used to imply the initial, current and final load locations, and

2. apparatus and methods for automatically determining load engagement and disengagement by the forklift while simultaneously determining load size and shape information as well as optionally a captured image for record keeping.

While the above description contains many specifications, these should not be construed as limitations on the scope of the invention, but rather as an exemplification of preferred embodiments thereof. Many other variations are possible. It is evident from the description of the Load Tracking System Based On Self-Tracking Forklift that it has applicability beyond that of tracking the location of units of lumber within a lumber yard. For example, lumber yards also handle large timbers and engineered wood product beams which must also be moved via fork lift and can be tracked in a similar means as described herein. There are other industries, such as metal, which handle large products which must be transported via fork lifts about geographic areas. Metal I-beams, bundles of extruded bars, bundles of sheets, coils of steel, plates, etc. are all examples of such products. Furthermore, the paper industry also stores large rolls of paper in stacks and columns within warehouses and shipyards store containers in large open yards - all of which is applicable to the teachings herein.

All of the above alternative uses are examples of large products (in load form) that have individual identities and must be moved by forklift. In this situation, the present application teaches new apparatus and methods for tracking the stored location and identity of these products by first having the forklift self-track its own location using pre-placed passive markers and second using 3D imaging to determine times of engagement, disengagement, load dimensions and images.

From a careful reading of the present invention, it should be obvious to the reader that these same concepts are applicable to vehicles other than forklifts, such as cranes, hoists, dump trucks, etc. All of these vehicles carry loads. In fact, the teachings for a self-tracking vehicle may also have novel application simply to follow the vehicle's location and orientation even if no loads are either being, or capable of being transported. Such "inside-out" vehicle self-tracking using the methods taught herein is expected to provide greater positional accuracy than GPS tracking now allows, even with additional ground based signals. Also, in many situations GPS is simply not feasible and other ground based vehicle tracking methods remain cost prohibitive over larger areas.

From the foregoing detailed description of the present invention, it will be apparent that the invention has a number of advantages, some of which have been described above and others that are inherent in the invention. Also, it will be apparent that modifications can be made to the present invention without departing from the teachings of the invention. Accordingly, the scope of the invention is only to be limited as necessitated by the accompanying claims.

Claims

Claims:

Claim 1. A vehicle tracking system where each vehicle is capable of self-determining its current location and orientation based solely upon energies reflected by passive markers surrounding the vehicle's potential path of travel, comprising: passive markers for reflecting energy that have been pre-placed in fixed known positions surrounding the vehicle's potential path of travel; an energy receiving sub-system attached to the vehicle for receiving energy reflected by the passive markers and for determining the relative location of each marker with respect to the vehicle, and a computer for determining the current location and orientation of the vehicle based upon the detected relative locations of the markers in combination with the pre-known location of each detected marker.

Claim 2. The system of claim 1 where the markers are RF responsive tags capable of being uniquely encoded and where the receiving sub-system is an RF reader, further comprising: an RF energy emitter attached to the vehicle for directing RF energy in the anticipated direction of the tags.

Claim 3. The system of claim 2 where the markers are Saw tags.

Claim 4. The system of claim 2 where the markers are RF ID tags.

Claim 5. The system of claim 1 where the markers are substrates with uniquely encoded and oriented surface markings that reflect energies capable of being detected by camera sensors and where the receiving sub-system is a camera.

Claim 6. The system of claim 5 where the energy is visible, ultraviolet or infrared.

Claim 7. The system of claim 6 where the surface markings are either reflective or retroreflective.

Claim 8. An automated load identification system for differentiating loads by pick-up and drop-off locations, as transported by self-tracking vehicles capable of self-determining the load pick-up and drop-off locations by first tracking their own current location and orientation based solely upon energies reflected by passive markers surrounding the vehicle's potential path of travel, comprising: engaging and disengaging apparatus attached to the vehicle for controllably picking up, holding during transportation and dropping-off loads; a load detection sub-system affixed to the vehicle for determining the time of pick-up and drop-off of a load by the engaging and disengaging apparatus; passive markers for reflecting energy that have been pre-placed in fixed known positions ^• surrounding the vehicle's potential path of travel; an energy receiving sub-system attached to the vehicle for receiving energy reflected by the passive markers and for determining the relative location of each marker with respect to the vehicle; a load identifying sub-system for determining the unique identity of the asset, and a computer for determining the current location and orientation of the vehicle based upon the detected relative locations of the markers in combination with the pre-known location of each detected marker and for recording in a database the current location and orientation of the load, as extrapolated from the vehicle's location and orientation, at the time of drop-off along with its identity, and for determining the load's identity based upon its determined pick-up location and orientation, as extrapolated from the vehicle's location and orientation, as queried into the database at the time of engagement.

Claim 9. The system of claim 8 where the markers are RF responsive tags capable of being uniquely encoded and where the receiving sub-system is an RF reader, further comprising: an RF energy emitter attached to the vehicle for directing RF energy in the anticipated direction of the tags.

Claim 10. The system of claim 9 where the markers are Saw tags.

Claim 11. The system of claim 9 where the markers are RF ID tags.

Claim 12. The system of claim 8 where the markers are substrates with uniquely encoded and oriented surface markings that reflect energies capable of being detected by camera sensors and where the receiving sub-system is a camera.

Claim 13. The system of claim 12 where the energy is visible, ultraviolet or infrared.

Claim 14. The system of claim 13 where the surface markings are either reflective or retroreflective.

Claim 15. The system of claim 8 where the load detection system determines the outer dimensions of the load in addition to its times of engagement and disengagement and where the computer stores this information in the database associated with each load's identity, location and orientation.

Claim 16. The system of claim 15 where the load detection system further captures an image of the load along with its outer dimensions and where the computer stores this information in the database associated with each load's identity, location, orientation and outer dimensions.

Claim 17. The system of claim 16 where the load detection system uses stereoscopic cameras.

Claim 18. The system of claim 16 where the load detection system uses cameras and one or more laser line generators.

Claim 19. The system of claim 16 where the load detection system uses three-dimensional detecting area scan cameras.