MXPA01005100A - Multiple object tracking system - Google Patents

Abstract

A system (100)for tracking the movement of multiple objects within a predefined area using a combination of overhead X-Y filming cameras (25) and tracking cameras (24) with attached frequency selective filter (24f). Also employed are perspective Z filming cameras (25) and tracking (24) cameras with filter (24f). Objects to be tracked have been marked with a frequency selective reflective material, such as patches (7r and 71), sticker (9) and tape (4a). System (100) radiates selected energy (23a) throughout the area of tracking to reflect off said reflective materials. Reflected energy such as (7m, 9a and 4b) is then received by tracking cameras (24) while all other ambient light is blocked by filter (24f). Local Computer System (60) captures images from tracking cameras (24) and locates said markings. Using said location information along with preknowledge concerning said multiple objects maximum rate of speed and maximum size as well as calculated movement information, system (60) is able to extract the background of the unfiltered images (25) that represent said multiple objects.

Description

MULTIPLE OBJECT TRACKING SYSTEM FIELD OF THE INVENTION The present invention relates to systems for tracking the movement of multiple objects within a predefined area.

BACKGROUND OF THE INVENTION As camera, microelectronic, and computer systems technology continues to advance at high speed, there has been an increasing supply of machine vision systems with which it is intended to replace well-defined repetitive vision / recognition tasks that had previously been performed by humans. The first systems were designed to recognize parts that moved along assembly lines to aid the manufacturing process. More recently, many inventions have been applied to recognize humans and their movement. The variability of humans and their clothing, as well as the complexity of the background within which they act, has presented a significant challenge to advanced technology. Considerable attention has been given to several techniques to distinguish the human form from its background using edge detection techniques that seek to remove fixed, ie background, information. There are two main factors that make the success of these techniques diverse The first is image resolution, which drives the amount of information and, consequently, detail that is available to the accompanying computer system to differentiate the foreground from the background. Ideally, the higher the resolution, the better. However, as the resolution increases it also increases the cost of the camera as well as the accompanying computer system. Even even more importantly, as the resolution increases, the time to process increases significantly. And as the time to process increases, the ability for these systems to perform operations in real time becomes difficult. The following are seven examples of machine vision systems designed in some way or another to recognize the movement of humans or objects from within a predefined area. In November 1994 the US patent 5,363,297 entitled "Automated Camera -Based Tracking System for Sports Contests" issued to Larson et al. This system used multiple cameras to continuously monitor the area of a sporting event that was taking place. Each camera fed their information to an accompanying computer system for analysis that consisted of extracting the players from the fixed background and thus trace their silhouettes. The inventor had anticipated problems when individual players collided or otherwise interacted with each other and, consequently, merged their individual silhouettes. The need to initialize the system by identifying each player first was also recognized. as they appeared inside the vision capo of the system. Larson et al. He specified two solutions for these problems. First, it was proposed to attach monitors to the tracking system that would be operated by humans who would perform the initial recognition as well as subsequent reidentifications when the system lost track of a player due to a fusion of silhouettes. Second, it was proposed to couple electronic tracking devices and employ triangulation of received signals to identify and track individual players. There are at least four major problems with the Larson patent. First, the amount of digital processing required to perform player extraction in real time greatly exceeds today's cost-effective computer technology, let alone 1994. Second, in order to perform this extraction, a large amount of amount of detail that would therefore increase the cost of implementation by requiring more cameras and related computer systems. And of course, the additional detail would only tend to further delay the response capacity of the system. Third, the requirement of one or more operators to initially recognize and then reidentify players is extremely limiting and expensive. This requirement essentially made the patent impractical from the economic point of view to monitor sporting events of young non-professionals, where the cost of the system that includes the current cost of the human operator would considerably exceed smaller revenue streams. It should be mentioned that it is more than likely that this Operator would be a father of one of the young people who probably would not be familiar with all the players and who would probably cause tension to make so many decisions in real time. This approach would also require training and re-training of operators, which would also be prohibitive. Fourth, the type of electronic components needed to track players in real time would have to operate at higher frequencies, which would also mean that it would be more expensive, providing an additional economic disadvantage. The fifth major problem is that it may be difficult for the system to determine the orientation of the player. For example, although Larson's invention could detect the direction of an individual's movement, he could not determine whether the person was looking forward or backward, much less if the person's head was turning in another direction. In December 1995, U.S. Patent No. 5,473,369 entitled "Object Tracking Apparatus" issued to Keiko Abe, this system referred to the actual image processing techniques used to follow a picture-to-picture object.The inventor described prior art which he compared The images block by block from one frame to the next, where it is assumed that a block is one or more pixels of the image.It was pointed out that such systems depended on statistical calculations prone to error and that took a long time that were especially susceptible to interpretation erroneous when the object changed size within the field of view or disappeared altogether, Abe proposed taking the same video frames but first separate them into luminance and color histograms that must then be compared frame by frame. Comparing the histograms instead of the blocks, Abe argued that the system would be more accurate and efficient than block matching systems. However, there are at least five major problems with Abe's patent. First, the effectiveness and reliability of this technique depends to a large degree on lighting conditions initially and over time within the field of vision that is being tracked. For example, if the initial frame was taken under a well-lit condition, the object's luminance histograms may be ideal. However, when lighting conditions are poor to start with, or worse still change from frame to frame as can happen with a sudden burst of ambient light, the luminance histograms will be subject to considerable error. Second, relying on color histograms is equally uncertain due in part to the susceptibility of color detection to lighting conditions, which again may vary from frame to frame, and in part to the potential for the object and background to become blurred when the color schemes overlap. A third problem is that the Abe system does not lend itself to tracking multiple objects that may have identical luminance / color information and that can be superimposed from frame to frame. A fourth problem is discussed in Abe's specification that indicates a requirement for a human operator to initialize the system by selecting the portion of the video image that contains the object to be tracked, the one called Region designation box. This requirement would be even more restrictive when considering the tracking of multiple objects where objects can enter and leave the field of view of temporal overlap with each other. And finally, in the specification reference is made to a fifth problem where it is expressed as an opportunity of the system to automatically control the rotation for panning, tilting and fast approaching or distancing a camera. In doing so, Abe mentions that the system is capable of "copying with any change in the size of the object and that it can photograph the target object always in a desirable size, thus achieving a substantial improvement in terms of the ease with which it can be used. the device. " Therefore, it is recognized that this method / apparatus is still very dependent on the resolution similar to the block methods on which an improvement is being attempted. In April of 1997, patent of E.U.A. No. 5,617,335 entitled "System for and Method of Recognizing and Tracking Target Mark" issued to Hashima et al. This invention is attempting to address the problem of determining the three-dimensional coordinates of an object with respect to a tracking camera and processing mechanism, for example, a robotic arm of a single two-dimensional image. These coordinates are expressed as the position and placement of an objective mark that has been placed on the object to be traced. In the Hashima review of the prior art several existing methods are listed, many of which require too many calculations and / or have problems with multiple objects and background image noise. A technique is described for marking the object to be traced with a white triangle within a black circle. Once the special marks are captured, they are quickly converted into histograms projected in the X and Y directions of the triangle mark image after which the centers of gravity as well as the maximum histogram values in the X and Y directions they are determined too. All this information is then used collectively to "determine which of the classified and prefixed position patterns the position of the triangle of said target brand belongs to based on the placement of the centers of gravity, the maximum histogram values, the values of the X and Y axis, and the known geometrical data of said target mark ". Even taking Hashima's claim of increased efficiency and accuracy, his technique has at least three main limitations. First, the object to be tracked must be marked in a very accurate manner and this mark must be visible to the tracking camera at all times. No condition has been described as to how the object can be tracked if the marks are temporarily blocked from the vision of the tracking camera. Second, trying to determine three-dimensional information from a single two-dimensional image, Hashima is focusing its solution in situations where additional perspective cameras may not be available. Given such additional cameras, there are even more efficient and accurate methods to determine the third dimension. Third, this invention teaches a system that works well "even when an image contains either many objects or has many noises. "However, if each of these multiple objects needed to be tracked within the same image, the invention of Hashima would not be optimal performance since at any given time the preferred orientation of the camera to the object can not be maintained simultaneously for multiple objects scattered in three dimensions In March 1998, U.S. Patent No. 5,731,785 entitled "System and Method for Locating Objects Including an Inhibiting Feature" issued to Lemelson et al. This invention teaches the tracking of objects by "an electronic code generating system or device carried by the object in a portable housing." This "system or device" is specified to receive location signals such as from the GPS constellation or an installation. It uses these signals to determine its own location, Lemelson anticipates that at some point or over time the operators of a remote tracking system may be interested in the exact location of an individual object among the multiplicity of objects that are being housed by said tracking devices. In order to determine the location of the objects, the tracking system will first transmit a single "investigation signal" encoded for a particular device on a particular object. All individual tracking devices will then receive this signal but only the device whose identification code corresponds to the "investigation signal" will respond. This response is in the form of a transmission that includes the Currently determined location of the tracking devices. The tracking system then receives this signal and displays on a computer system monitor information related to the object identified / located. The invention of Lemelson et al is mainly applicable to the tracking of many objects over a very large area, so wide that these objects are out of range of any reasonably sized camera tracking system. As an apparatus and method for tracking objects that are within a suitable range for a camera network, this invention has at least three major problems. First, it requires that each object has the ability to constantly monitor and track its own location. Such a requirement involves the use of a computing device that must be installed to receive GPS or other tracking signals and also transmit location signals. Such a device will typically occupy more space than a mark or signals that can be placed on an object and then track with a camera. further, this device will require energy. Second, the invention of Lemelson et al. assumes that the remote tracking station is interested in only one or a fraction of all the potential objects at a given time. However, there are many situations when you want to track the exact and continuous movements of all tracked objects as they move around a predefined area. While it is possible that this invention could constantly transmit "research signals" for all objects and constantly receive location signal responses, it is anticipated that this amount of information would unacceptably limit the resolution of system movement. Third, such a system based on electronic components lacks an understanding of the orientation of an object with respect to its direction of movement. Therefore, although it is possible to determine the direction in which a car or person being tracked is moving, the way in which the system could determine whether the same car or person was facing its current direction of travel is not shown. traveled or turned away from it. In June of 1998, the patent of E.U.A. No. 5,768,151 entitled "System for Determining the Trajectory of an Object in a Sports Simulator" issued to Lowery et al. This invention teaches the use of stereoscopic cameras focused in a limited field of vision to follow the path of an anticipated object. As the object crosses the field of view, the cameras capture images at an appropriately slower speed so that the object creates a blur as it moves. This blurred path is then analyzed and converted into the path vectors of the object within the field of view. Another key means of the apparatus of Lower et al. is the ability to determine when you should start tracking. As such, a sound detection device is specified to detect the presence of the object within the field of view after which the image capture system is activated immediately. There are at least four main limitations with the invention of Lower et al. that could hinder their wider application capacity. First, the invention expects a very narrow range of motion of the object and as such has a significantly restricted field of view. If the concept is to be applied to a larger area, then multiple perspective cameras would be needed. The system would also need to determine which cameras should be activated once it is detected that the object is moving within the tracking region. However, without first actually determining where the object is located, for example, by trying to triangulate the sound emitted by the object, the system may have no idea which cameras to activate. Therefore, all cameras would need to capture images creating a very large data set that would need to be analyzed by the tracking computer to determine the location of the object. The second limitation is that this system does not attempt to identify only each object it detects. Therefore, although it is capable of determining a path vector for individual objects, it does not anticipate a need to, nor describe a method to determine the unique identity of each object as it passes. The third limitation has to do with the ability to track multiple objects simultaneously within their same field of vision. Because this invention anticipates only one object at a time, it is simply determined path vectors, and not the identity of the object. Therefore, if two or more objects are moving throughout the tracking region and collide in such a way as to affect the path of the other, then the system will be left to determine which object continued on which path after the fusion event. The fourth limitation has to do with the inability of the system to take the object from its background when there is insufficient color and / or luminescence difference between the two. All prior art listed above in one way or another was attempting to track the movement of at least one object within a predefined area. When taken in combination, their limitations that must be overcome in total are the following: 1- If the tracking system tries to differentiate between the object and its background purely on the basis of comparison of pixel by pixel as does Larson et al. , then the video image must have a higher resolution to be accurate and the resulting computer processing time prohibits the operation in rtime. 2- If the tracking system tries to reduce the processing time by performing averaging techniques based on color information and separate luminescence as Abe does, then accuracy is compromised especially as colors merge between multiple objects and their background or lighting conditions fluctuate substantially between image frames. Said reduction techniques are further hindered as the size of the object decreases which essentially reduces the amount of object-against-background information thus increasing the "noise". The only solution is to focus on the object that is being tracked in the foreground in order to maintain the appropriate relation of object information thoroughly. So this implies that each object that is being Tracking should have its own camera thus greatly reducing the effectiveness of these techniques to track either more objects and / or larger fields of vision. 3- If the tracking system such as that of Lowery et al. employs two perspective cameras and a technique to blur an image to capture three-dimensional trajectory information, reduces image processing requirements but loses important video image detail. 4- If the tracking system such as that of Hashima et al. uses detailed signals placed on the object to be tracked, this can be effective to reduce the amount of image processing. However, Hashima faces significant problems when trying to determine three-dimensional information from a single two-dimensional image, which is one of its mandatory requirements. Their resulting techniques prevent the tracing of multiple objects of rapid movement in wider fields of view where the signals of the object can sometimes be blocked from viewing or at least be in significantly changing perspectives from the tracking camera. 5- All techniques based only on video / camera such as Larson, Abe, Hashima and Lowery are prone to error if they were to track multiple objects whose trajectories intersperse and / or collide. Only Larson specifically anticipates this type of multi-object tracking and suggests the use of human operators to solve the superposition of objects. Such operators are prohibitive in terms of cost and are also limited in their ability to go through the passage of multiple objects that are moving rapidly in real time. Although, as Larson suggests, it is possible to use passive electronic compounds to help identify objects once the system determines that their identities have been lost, these devices will have their own resolution / response speed constraints that are cost-sensitive. 6- In addition, the video / camera solutions of Larson and Abe anticipate the requirement of a human operator to initialize the system. Larson required the operator to identify each object for the system. These objects were then automatically tracked until they merged in some way with another object at which time the operator needed to reinitialize the tracking system. Abe required the operator to cut the initial image down to a "region designation framework" which essentially reduces the processing requirements to at least find, if not also track, the object. The intervention of any operator is both prohibitive in terms of cost and limiting response in real time. 7- The Lowery video / camera solution anticipates automatic tracking activation based on the detected presence of sound from an object within the field of view. This technique is inherently limited to objects that make distinctive sounds. He is also unable to track objects multiple that could make similar noise simultaneously within the given field of view. 8- If the tracking system tries to eliminate image processing requirements using active electronic tracking devices such as Lemelson et al., Then it is required that the objects house devices that function with electric power capable of receiving and processing signals of both location and research. Such devices limit the range and type of objects that can be tracked based on the practical utility and cost of embedding computing devices. In systems such as Lemelson's which use only electronic components, the wealth of information available from the image data is lost. Also, such systems may be able to track location but can not track object orientation, for example, if the object moves forward or backward. 9- With the exception of the Hashima signaling technique, all these solutions are not yet capturing object orientation information. Such information can be extremely important to anticipate movement of the future object. 10- All video / camera-based solutions will have difficulty picking up fast-moving objects whose color and / or luminescence information is sufficiently close to that of other tracked objects or the background of the image, no matter what technique is used. All Solutions that are not based on video will provide valuable image information.

BRIEF DESCRIPTION OF THE INVENTION Although the present invention will be specified with reference to a particular example of multi-object tracking as will be described below, this specification should not be construed as limiting the scope of the invention, but rather as an exemplification of preferred embodiments of the invention. the same. The inventors envision many related uses of the apparatus and methods that are described herein, but only a few of which will be mentioned in the conclusion of this application specification. For purposes of teaching the novel aspects of this invention, the example of multi-object tracking is that of a sporting event such as hockey. The particular aspects of hockey that make a series of events difficult to trace and therefore a good example of the strengths of the present invention over the prior art are the following: 1- There are no other human-based activities known to the inventors of the present where humans as objects can move at a faster speed or change directions and orientation more quickly than hockey. In skates, the speed of a player can approach 46.33 Km / h, which is considerably faster than Any activity that involves walking or running that is also carried out on the ground without the help of a vehicle of any kind. Tracking these faster movements, especially given the variability of the human form, challenges aspects of the system's real-time performance. 2- The speed of the object for which the players contend, that is, the disc, can travel at speeds of up to 185.32 Km / h and is also subject to sudden and rapid changes of direction. This combination of speed and re-direction presents a difficult tracking problem and is unique in athletics. Tracking the disk is easier than tracking a player considering that the disk will not change shape even though it travels at approximately 4 times the speed, is in the magnitude of 100 times so small and can travel in three dimensions. 3.- Individual players are constantly entering and exiting the field of view of tracking and as such should be identified efficiently and automatically by the tracking system for real-time performance. 4- Although they are in the field of vision, both the disc and the players are routinely totally or partially hidden from view when they merge with the trajectories of other players. This creates a challenge to follow movements with often limited or non-existent image data. 5- It is difficult to work with lighting conditions because the ice surface will create a very reflective background that could tend to saturate the CCD elements of the camera while the area as such may be subject to sudden bursts of light coming either from flashes of on-camera cameras or internal lighting systems. This places limitations on the luminescence-based tracking techniques. 6- The colors of the players of the same team are identical and can often correspond with the marks on the surface of ice and boards of the surrounding skating rink. This places limitations on color-based tracking techniques. 7- It is not usual for a hockey game to take place while a certain level of fog exists inside the arena. This challenges any camera-based system because it could significantly reduce the visibility of players and the disc. 8- Hockey is a filmed event and as such presents the opportunity not only to track the movement of multiple objects but also to determine a center of interest which is changing constantly and abruptly. Once this center is determined, there is an additional advantage of automatically steering the tilt, turn for panning and fast approaching or distancing from a camera of the broadcasting company to follow the action from a perspective view in real time. Automatically directing a camera that is located for a perspective view presents a difficult problem for a machine vision system since it is considerably more difficult to track objects in three dimensions in real time as a perspective view would require. 9- Each individual player as well as the coaches in a game can at any time be instantly eager to obtain information about themselves, a group of players, their complete team and / or the other team. This information may also belong to all or a subset of the duration of the activities from the beginning to the present moment. Such requirements put a demand on the tracking system to quickly and efficiently store information in a way that can be easily extracted from many points of view. 10- The sand of cement blocks and enclosed metal prevents the use of GPS and presents difficulties for the use of passive electronic tracking devices due to the many potential reflections of internal triangulation signals. Players as such and the nature of the game and its potential for significant high-impact collisions limit the convenience of placing active electronic devices within your team. Because these devices must carry a source of energy, in practice they will cover enough space to present a potential danger to players. In addition, such devices would be extremely prohibitive in terms of cost at the local skating rink level where literally hundreds of children are playing games each week and would each need their own devices or share devices. 11- The player's orientation and location of members with respect to the body are very important information to track. A player may be moving forward while facing forward while Turning your head to the side to see the development of the game. Turning the head becomes important information for training analysis since it is an indication of an ice knowledge of a player. In addition, a player can turn the orientation almost instantaneously while continuing in the same direction so that it is now moving backward instead of forward. This is also very decisive information. A player's head may tilt down towards the ice, which is undesirable for long periods or if it is repeated frequently. All this information is important to track but presents problems in addition to simply tracking the player as a whole. 12- Limiting the size of the tracking area is desirable during practice sessions where individual trainings can be carried out on a limited portion of the ice with a small number of players at a time. Under these conditions it would be advisable to easily restrict the tracking area of the system within your field of vision. 13- The number and speed of player changes and collisions is so great that using human intervention to identify and reidentify players would cause significant tension and be prone to errors, if not practically impossible, especially at the level of skating local. Given the systems of advanced technology cameras, non-visible energy sources and filters, digital image processing and automated camera controls it is possible to create an object tracking system Fully automated multiples that operate within a predefined area and track the location, orientation and constant direction of movement of each and every object within the field of vision. This system considerably increases the ability of participants and observers to understand, analyze and enjoy the given activity. Accordingly, the objects and advantages of the present invention are to provide a system for tracking multiple objects within a predefined area with the following capabilities: 1. - provide a system for tracking multiple objects, of fast movement, of variable forms such as humans in real time without the help of an operator intervention either to initially identify or conditionally re-identify the objects during the tracking period; 2.- reduce the time required to process image data by first separately tracking special signals that have been added to the object after which the location of the signals can then be used together with the direction, acceleration and velocity vectors to extract efficiently the object of your fund; 3.- provide a system that does not depend on color distinctions within the objects; 4.- provide a system that does not depend on electronic tracking devices either passive or active; . - provide a system that determines orientation information around the objects as a whole and potentially their individual parts; 6.- provide a system that can work under various lighting conditions and air humidity that may not be optimal for recognition techniques based on visible light; 7.- provide a system where the tracking field is easily compressed within the field of vision of the system; 8.- provide a system that can create * and maintain in real time a database of accessible movement either by time and identity of the object (s); 9.- provide a system that can output all or some of the captured data either immediately within the ice arena or after the event to one or more remote computers in a communication network such as the Internet; and 10.- provide a system in such a way that the output of information to remote computer systems over remote communications such as the Internet can be used for controlled repetition of the event as well as critical analysis. Other objects and advantages are to provide a cost-efficient system to build, install and maintain with a minimum of moving parts that are capable of operating under a range of operating conditions. temperature. Other objects and advantages of the present invention will be apparent from a consideration of the drawings and following description: DESCRIPTION OF THE DRAWINGS Figure 1 is a top view drawing of the preferred embodiment of the present invention illustrating an arrangement of the upper XY tracking cameras that, when taken together to form a field of vision that includes the skating and banking area within an ice hockey arena. Scouting camera equipment Z of perspectives behind each camera of prospective shooting in the goal, turning for automatic panning, tilt and fast approach or remote as well as a single player and disc representative are illustrated. Figure 2a is a series of three perspective drawings illustrating a typical player's shirt, shoulder pads with tracking patches in place, and then a combination of the shirt over the shoulder pad with patches. Figure 2b is a series of two perspective drawings illustrating a hockey puck as well as a typical player hockey stick, where each has been enlarged to include tracking ink on at least a portion of its outer surfaces. Figure 2c is a series of two perspective drawings illustrating a typical hockey player helmet that has been augmented for include tracking decals in at least the upper portion of its outer surface. Figure 3a is a perspective drawing of shoulder pads, helmet, walking stick and typical hockey player disk being captured from a top X-Y shooting camera and displayed on a viewing screen. Figure 3b is a perspective drawing similar to Figure 3a except that tracking ink has now been added to the hockey stick and disc, tracking patches have been added to the shoulder pads and helmet tracking stickers. In addition, a tracking energy source as well as a frequency matching filter have been added to the upper X-Y camera making it a tracking camera. Figure 4a is a perspective drawing similar to 3b except that a filming chamber without additional filter has now been added to the upper X-Y tracking camera to efficiently combine both sets of information. Figure 4b is a top view illustration of a key element of the novel method of the present invention for efficiently extracting the video image of the object being tracked by first locating aggregated signals and then calculating away from the signals by comparing each pixel of the image with a previously known background to effectively trace the tracked object. Figure 4c is a top view of a portion of an ice arena showing a series of tracked and traced movements of a typical hockey player, baton and discus by the upper X-Y tracking and filming cameras illustrated in Figure 4a. Figure 5 is a perspective drawing of a mountable camera arrangement frame with upper X-Y filming and tracking chambers coupled to the roof of a typical ice arena. Figure 6 is a top view drawing of an ice arena where area restriction cones with tracking ink have been placed to indicate to the upper XY tracking cameras that only a sub-portion of the entire field of view is to be tracked . Typical players are also shown, one is inside the tracking zone while many are outside the zone. In addition, a portable device that can be used by ice training personnel to control the functions of and to inquire about the information generated by the present invention is illustrated. Figure 7 is a previous view drawing of a hockey players bench that has been equipped with a series of display devices with keys that can be used by typical hockey players during the course of a hockey game to find out the information generated by the tracking system. Figure 8 is a block diagram illustrating all the tracking and computing elements of the invention of the present proposal.

DETAILED DESCRIPTION OF THE INVENTION Referring to Figure 1, a top view drawing of the preferred embodiment of the Multi-object Tracking System 100 is shown. System 100 comprises an arrangement of x and higher camera assemblies 20c that individually track all movement of objects within of a fixed area such as 20v. In total, the arrangement of upper assemblies 20c track all movements on the ice playing surface 2, equipment boxes 2f and 2g, penalty boxes 2h as well as a portion of the entry way 2e. The assembly 20c further comprises a film camera 25, power source 23, tracking chamber 24 in which an energy filter 24f is coupled, all of which is housed in the assembly cover 21 and has a view to the surface of ice 2 below through the plexiglass assembly 21a. The energy source 23 emits selected energy 23a that radiates down into the surface 2 and away from the objects moving on this surface such as player 10 and disk 3. It should be mentioned that the selected energy 23a was specifically chosen to be of a frequency that is not only outside the normal ambient light range but also unaffected by humidity conditions such as fog. A specially assembled upper assembly 20c1 is shown as covering the entry path 2e of the ice surface 2. This assembly 20c1 is identical in construction to 20c and is important and will be specifically referred to below. because all the objects that are to be tracked must enter and exit through this field of vision of the camera. Scanning camera z perspective cameras 30 which are located as a pair at both ends of ice 2 are also tracking movements on a selected portion of the ice surface 2. And finally, there are automatic shooting cameras 40 which are constantly being directed to the game center as represented by player 10 which is currently controlling disk 3. Automatic cameras 40 are in continuous communication with and are receiving their instructions from the local computer system for video processing and analysis 60. The system 60 as such is also in continuous communication with the arrangement of tracking camera assemblies x and higher 20c and tracking camera equipment z of perspective 30. The local system 60 is also in optional communication with the remote computer system to review the events captured 70, which has a vision monitor 27 that displays scene 28. Optional communication stations 60 of equipment output 80 are also in optional communication with the local system that provide information on the movements of objects tracked from the beginning of the event to the present moment. Referring now to Figure 2a, a typical player's shirt 5 and player's shoulder pads 6 are shown. Adhering to the shoulder pads 6 are the right shoulder equipment patch 7r and the left shoulder player patch 71. The patch 7r comprises the orientation mark 7r1, which is an arrowhead pointing away from the head towards the arm and signals of equipment 7r2 which is a unique barcode. The patch 71 comprises the orientation mark 711, which is also an arrowhead pointing away from the head towards the arm and player signals 7I2, which is a unique number. It is worth mentioning that the signals in patches 7r and 71 are created from a selective frequency reflective material, preferably an ink. Also referring to Figure 2a is illustrated a t-shirt 5 placed on shoulder pads 6. Note that shirt 5 is also shown cut off for a full view of the underlying player patch 71. Also shown in Figure 2a is a reflected energy 7m which is shows irradiating through the transmissive shirt 5. These techniques to see through the clothes are not new and were demonstrated by Sony Corporation when they put a shirt on a police plaque and then took two photographs, one without a filter and another with a visible light filter. Because the clothing reflects light in the visible spectrum so intensely, the plate could not be seen by the unfiltered image. But with the infrared pass filter only the infrared light was captured making visible the numbers and letters on the police plaque. It should be mentioned that the present invention teaches the use of special frequency selective reflective material which will further improve the contrast, ie the signal to noise ratio, of the desired image above and beyond the filtering techniques discussed by Sony.

Referring now to Figure 2b a typical hockey disc 3 is shown where its top surface (and in practice all external surfaces) has been coated with a reflective ink 3a similar to the frequency selective reflective material used in patches 7r and 71. In response to a particular energy frequency as it would be emitted, for example, by the energy source 23, the ink 3a causes the reflected energy 3b. Also shown is a typical hockey stick 4 where its blade has been wrapped with a special reflective hockey ribbon 4a which has been produced to include the same special reflective ink. In response to a particular energy frequency as would be emitted, for example, by the energy source 23, the reflective tape 4a causes the reflected energy 4b. Referring now to Figure 2c there is shown a top and perspective view of a typical hockey player helmet 8 where a reflective sticker 9 has been applied to its top surface and has been produced to include the same special reflective ink. In response to a particular energy frequency as would be emitted, for example, by the energy source 23, the decal 9 causes the reflected energy 9a. Referring now to Figure 3a, a first embodiment of an upper x-y tracking camera assembly 20a is shown. In this embodiment, the assembly 20a, has been limited to tracking chamber 24 (without energy filter 24f) which is enclosed within an assembly cover 21 and has a view to the ice surface 2 below through the plexiglas of assemble 21a. The unlabeled player 10 is illustrated below the assembly 20a, unmarked cane 4 and unlabeled disk 3. Also shown is a cable 26 that connects the assembly 20a to the local computer system 60 (not illustrated), to the remote computer 70 (not illustrated), and therefore to the monitor of vision 27 displayed by the scene 28. Referring now to Figure 3b, a second embodiment of the tracking camera assembly x and upper 20b is shown. In this embodiment, the assembly 20b has been enlarged to include the energy source 23 that emits the selected energy 23a that radiates downward in the objects moving towards the ice surface such as player 10, disk 3 and cane 4. Observe that the pads 6 of player 10 have been increased to include the right shoulder equipment patch 7r and left shoulder player patch 71. Also note that the disc 3 now includes a reflecting ink 3a and that the stick 4 has been wrapped with a special reflective hockey tape 4a. Scene 28 now illustrates a much smaller series of information to be analyzed and tracked. Referring now to Figure 4, a third and the preferred embodiment of the x-top tracking camera assembly 20c is shown. In this modality, the assembly 20c has been enlarged to include the shooting camera 25 which captures unfiltered images of all movement on the ice surface below. Figure 4b illustrates a key element of the novel method of the present invention for efficiently extracting the video image of the object being tracked by first locating aggregated signals and then calculating away from the signals comparing each pixel of the image with a iously known background 2r. Once it is determined that the background pixels have been reached, the image tracing path 10p continues around the object until a closed path is completed. Around this closed path 10p a minimum bound rectangle 10r is calculated to quickly extract the portion of the video image containing the tracked object. Referring also to Figure 4c, the processed movement of the player 10 on the ice surface 2 is illustrated using the erred assembly 20c. The player 10 is shown to cross through the four movement points 10a1, 10a2, 10a3 and 10a4 leading the disc 3 along its way towards a shot in the goal 2b within the calculated shooting triangle 2c. The center of the ice surface 2 is shown as the point 2a from which the first angle of movement 10a1o and radius of movement 10a1 r have been calculated to reent the first movement of player 10 to point 10a1. The angle of movement 31 o and the radius of movement 31 r that describes the initial movement of the disc 3 to the point 10a1 are also calculated. Around the central point 2a is shown a circle of maximum limit 10b within which is calculated where the next location of an object will be based on the last known position of the objects (in this case point 2a) as well as its last ones calculated vectors of direction, acceleration and speed. And finally the maximum initialization search rectangle 10e that covers the outermost portion of the custom field of view is illustrated. which is tracked by the upper assembly 20c1 which is located on the only entry path 2e to the ice surface 2. Referring now to figure 5, the upper mounting frame 50 is shown which can be variably coupled to different constructions of master beams of the skating rink 2d. The arrangement of upper x-y tracking camera assemblies such as 23c that emit selected energy 23a downward into the ice surface 2 below is also coupled to the frame 50. The assemblies 23 are connected to the local computer system for video processing / analysis 60. Referring now to Figure 6, the ice surface 2 is illustrated whose tracking area has been restricted by the placement of 4 typical cones 11 that have been coated with a reflective ink 11a. In response to a particular energy frequency as it would be emitted, for example, by the energy source 23, the ink 11a causes the reflected energy 11 b. A single player 10 whose current movement is to be tracked while remaining within the 2t region is shown within the restricted region 2t. 2t multiple players such as 10 whose movements are not being tracked are shown outside the region. Also shown are the marker 91 and portable tracking control device 90 that are in optimal communication with the local computer system 60.

Referring now to Figure 7 there is shown an approaching view of equipment output stations 80 that are coupled to equipment boxes 2f (shown) and 2g, as well as penalty box 2h. Referring now to Figure 8, a block diagram of the entire system is illustrated. With respect to the tracking camera assembly x and upper 30c, an additional optional rf modem 25m is shown which can be used in place of the cable 26 (not illustrated) to link the camera and recording 25 to the local computer system 60. It is also shows the optional additional RF modem 24m that can be used in place of the cable 26 (not shown) to join the tracking camera 24 to the local computer system 60. Also, additions are illustrated for tracking camera equipment z perspective 30. The local system 60 has been decomposed into its parts of constituent blocks starting with the video capture for the tracking camera unit (s) 61 which accepts input from all the tracking cameras 24 mounted within the upper 30c perspective equipment assemblies 30. The capture unit 61 then feeds its flow of data to the signal / object tracking analysis unit 62. In parallel time operation, video capture is shown for the camera unit 63 which accepts input from all the cameras 25 mounted within upper assemblies 30c or equipment of perspective 30. The capture unit 63 then feeds its data flow to the object / scene extraction analysis unit 64. The unit 64 also receives simultaneous input from the signal / object tracking analysis unit 62.

Both the extract analysis unit 64 and the trace analysis unit 62 simultaneously provide their data streams to image composition that overlap multiple in the individual complete frame 65 unit. The unit 65 also receives conditional input of the portable tracking control device 90 and provides its data flow to the game center analysis / vision choice unit 66 and storage of tracking and video frames indexed by the object unit and frame number / time sequence 67. The vision choice analysis unit 66 in turn provides real-time instructions to the automatic shooting cameras 40 through the conventional cable or optional RF modem 45m. Such instructions are then input to the computer steerable event recording camera 45. The camera 45 then feeds its data stream back to the storage and tracking unit 67 via conventional cable or optional RF modem. The unit 67 subsequently provides its data flow to quantization and motion analysis of objects stored in an interlaced database unit 68. The unit 68 optionally conducts two-way communications with the portable tracking control device 90, stations 80 device output and remote computer system to review captured events 70. The optional link to device 90 will be made through the Rf connection (not illustrated) while the link to the remote system 70 will be made through remote communications devices 69 and 79. The information provided by the local system 60 to the remote system 70 will be provided to the end user through remote access to the stored database of tracking and filming frames as well as the object movement quantification and analysis unit 71.

Operation Referring first to Figures 1 and 5, the normal operation of the preferred embodiment is initiated after the system 100 has been properly installed in an ice arena such as 2. One of the most decisive aspects of installation is the coupling of the arrangement of the tracking chamber assemblies x and upper 30c to the roof of the arena 2. The mounting frame 50 has been designed to be variably coupled to the main beams of the skating track 2d in such a way that the coupled arrangement subsequently of the upper assemblies 30c form a field of view that is superimposed through the ice surface 2 below. It is preferable that each assembly 30c maintains a perpendicular position with respect to surface 2. It is anticipated that either fixed lenses with preselected depths of field or electronically controllable fast moving or approaching lenses will be used to properly establish the overlapping fields. . Overlap is important to ensure that any object that is to be tracked can be followed from camera to camera as it moves across the predefined area. As part of the initial installation and assembly of the system 100, a calibration procedure will be operated to define the limits of the field of view of each of the cameras 25 and tracking cameras 24. Once known, the system will restrict its search to regions that do not overlap in order to minimize duplicate processing. All assemblies 30c will communicate with either the local system 60 through cables such as 26 or optional RF modems such as 25m and 24m. Referring now to Figures 2a, 2b, and 2c, the preferred embodiment provides several methods for marking the objects to be tracked with a material! selective reflector of specially chosen frequency such as an ink. This ink is then used to be included in the disc 3 as a reflective ink 3a, to produce the reflective tape 4a, to be included in the patches 7r and 71, and to produce reflective stickers 9 for the helmets 8. It is also used for creating cones 11 with reflective ink 11a as shown in figure 6. It is decisive for the novelty of the present invention to observe that this ink has been chosen specifically to reflect energy 23a as output from the energy source 23. This reflection has been illustrated in 3b, 4b, 7m, 9a and 11b and will be received by the tracking cameras 24 through the energy filter 24f. By using the specially coupled energy filter 24f in each chamber 24, the amount of information required to be processed to track all object movement is minimized thereby considerably increasing the speed to determine the position of the object. To further illustrate the novelty of this point, Figures 3a, 3b and 4a have been created to dramatize the effects resulting from the addition of This tracking ink. Figure 3a shows a conventional camera system that captures ambient light and displays its corresponding field of view as scene 28 on monitor 27. Note that in this view there is considerably more information to be processed including the object such as player 10 and the bottom such as the ice surface 2. By adding reflective materials as previously discussed in the form of 3a, 4b, 7r, 71 and 9, figure 3b now shows a much smaller amount of information in scene 28. Again, this it is achieved in real time without the additional computer processing requirements by applying power filter 24f to track the camera 24 which was specially chosen to pass a narrow band of frequency as the power source 23 outputs as selected energy 23a. The energy 23a radiates through the predefined area so that the ice surface 2 and its back reflected as 3b, 4b, 7m and 9a. The orientation marks 7rl and 711, equipment signals 7r2, player signals 7I2 as well as the reflective sticker 9 have been produced to include special markings that will be easily distinguishable from the signal / object tracking analysis unit 62. The size of these marks will be matched to the objective resolution of the tracking cameras 24 in the top assemblies 30c. Therefore, the lower the resolution against the field of vision, the larger the mark will have to be to facilitate recognition. The preferred embodiment also includes filming cameras without a filter 25 that will capture all ambient light frequencies, as described in FIG. figure 4a. However, as will be discussed at a later point, due to information provided by the tracking analysis, only a small portion of this data should be carefully examined to extract all the information from the relevant object. Before operating the system 100 it will also be necessary to install perspective tracking camera equipment z. A device will be mounted at each end of the ice surface 2 to be in constant view of the area immediately surrounding each goal 2b. These equipment 30 are placed in anticipation of the path of travel of the disc 3 which is expected to leave the surface 2 and travel in the Z direction when approaching the goal 2b. Note that due to the superior nature of assemblies 30c, they will be unable to pick up any object movement in Z dimensions which is acceptable and even desirable since it reduces the amount of processing that must be carried out to track all objects. However, in the restricted area closest to the goal 2b, namely defined by the triangular area 2c, it can be very beneficial to trace the element Z of the path of travel of the disc 3. It is important to note that the equipment is not required 30 and the information that they collect allow the main aspect of what the inventors of the present feel that is novel regarding this invention. Specifically, practice the following movements of specially marked objects and apply this information to normal video images to quickly extract information from the object.

Also optional and yet novel for the present invention is the inclusion of automatic filming cameras 40 at discretionary locations within the ice surface arena 2. Until now, camera operators who intuitively determined the game center controlled filming. for the transmission of events such as hockey and self-directing the turn for panning, fast approaching or distancing and tilting of its cameras to capture the restricted scene. Now that the system 100 will have all the information regarding the movement of objects on the ice surface 2, the local computer system 60 is planned., through its game center analysis / vision choice unit 66, automatically directs panning for panning, zooming in or zooming out and tilt of automatic shooting cameras 40. Once all x and above tracking camera assemblies 30c, all of the perspective camera z tracking equipment 30 and all the automatic shooting cameras 40 have been installed and calibrated correctly and all the objects to be tracked have been appropriately increased with marks such as 3a, 4a, 7r , 71 and 9, the system 100 is ready for operation. The following discussion will describe how the movements of a single player 10, staff 4 and disc 3 will be traced from frame to frame during the event through the entire predefined field of view in dimensions XY and Z. Next, the inventors of the present they will teach how additional players 10 with their canes 4 they can also be tracked simultaneously and how problems due to overlapping of objects, changing object size, disappearance of the object of sight, sudden changes in ambient light and otherwise reductions in visibility, for example, will be handled to fog. Referring now to Figures 1, 2, 4a, 4b, and 8, the tracking of multiple objects by the system 100 begins after the system has been turned on and the first player 10 proceeds down the driveway 2e and crosses into the field of view of the arrangement of upper assemblies 30c. Within the entry way 2e player 10 will appear first in a single frame captured by video capture for the tracking camera unit (s) 61. Unit 61 will capture a total of 40 positive frames per second (40 + fps) of each tracking camera in the assembly 20c and will feed each frame to the signal / object tracking analysis unit 62. Note that given the current state of the technology a camera controller board such as would be required within the unit 61 can control up to eight cameras. The actual number of frames per second that needs to be captured depends on the desired motion resolution. For example, the fastest object in our example is disk 3 that will sometimes travel up to 185.32 km / hr. At 40 frames per second, 1609.34 m by 1.85324 km and 60 * 60 = 3,600 seconds per hour, the maximum travel distance for disk 3 per second is: (185.32 km / hour * 1609.34 m /1.85324 km) / (3.600 sec. / hour) = 44.71 m. At a speed of 40 frames per second, the maximum distance traveled between frames would be 1.12 m. Note that a player such as 10 on skates will get maximum speeds of approximately 46.33 km / hr, or a quarter of the speed of the disc. At 40 frames per second, the maximum distance traveled by a player 10 would be approximately 0.281 m. At 20 frames per second, disc 3 will travel no more than 2.43 m at maximum speed while player 10 will be limited to approximately 0.609 m in distance. Considering that both player 10 and disk 3 rarely travel at top speed, the resolution of movement will be considerably higher in practice. Also, note that there is an anticipated area, namely around goal 2b where disc 3 is most likely to reach maximum speeds. Both the upper camera assemblies 20c and the perspective camera equipment 30 can be operated at an increased frame capture rate to increase the resolution of movement in these specific zones. In any case, the present invention has the ability to exchange the cost of computing power against motion resolution by exchanging the frame rate and the number of tracking cameras 24 that will be coupled to a single computer and video capture card. It should be recognized that the use of multiple computers to join and analyze data is not considered novel, nor a limitation of the present invention. As each frame of the tracking camera 24 of the upper assembly 20cl is accepted by the analysis unit 62, the scale of The gray of each pixel is compared against a threshold value where those pixels that exceed the threshold indicate the presence of any form of the special mark such as 3a, 4a, 7r, 71 and 9. Because each player 10 must both enter and exit of the ice surface 2 of the entry path 2e that is always in view by the 20cl assembly, and because the maximum movement between frames can be calculated based on the anticipated maximum speed and frame capture speed, it is possible calculating a minimum number of pixels that must be completely searched in order to first detect the presence of a new player such as 10. This minimum number of pixels is shown as 10e in Figure 4c and consists of those pixels running parallel along the length of the pixel. outermost edge of the assembly field of view 20cl inward several rows towards the main ice surface 2. The depth of this rectangular area will again depend on the maximum distance that a player 10 is expected to be able to travel between frame captures. Observe that even in other sporting events such as basketball or soccer, there is usually a tunnel through which each team must travel to gain access to the main arena. If the field of view of the system is properly extended to be continuous in that tunnel, then this minimum search technique can be used to first detect any new player to be tracked. As will be shown, once detected, the processing requirements to continue tracking are significantly reduced since the extent of a player's movement between frames is limited.

Referring now further to Figure 4c, once a marked object is detected, either a patch 7r or 71 on the shoulder pads 6, or the reflecting sticker 9 on the helmet 8, or the reflecting ink 3A on the disk 3, or the reflective hockey tape 4A wrapped around a baton spade 4 is tracked individually. The first detected movement of each mark is expressed in polar coordinates relative to the central point 2a of the ice surface 2. Therefore, the first movement of each object is expressed as an angle of 0o and a distance from the central point 2a along said angle, these calculations are constantly performed by the object tracking analysis unit 62. Once unit 62 has detected a given object, it will continue to search for that object within its field of vision based on the latter. known coordinates, the last known vectors of direction, acceleration and speed and the distance of maximum travel calculated between frames. The last known coordinates combined with the maximum possible travel distance will work to define a maximum limit circle, illustrated as 10b in Figure 4c, of possible movement that must be searched to find the next location of the object. In order to cut the average search time within this boundary circle, unit 62 will first look at the last known travel direction based on prior movements out of a distance equal to the last known unit speed divided by the frame rate known From this most likely new point within the boundary circle, unit 62 will continue to search making this point grow in all directions until the circle of the entire boundary has been analyzed. In case the mark is not found and it is known that the object has completely passed through the minimum rectangle of pixels surrounding the edge of the field of vision covering the entry path 2e, then the object will be searched in the next frame . In this case, the radius of the new maximum limit circle will be double that of the previous search. If a marked object is detected and then lost to the system 100, the trace analysis unit 62 will first communicate with the object / scene extraction analysis unit 64 to determine whether the presence of information based on additional ambient light will discover the location of the object. Before reviewing this technique, it should be mentioned first that once a marked object is detected, its type and location are passed to the extraction unit 64 from the tracking unit 63, the extraction unit 64 then analyzes the video frame without corresponding filter taken by the camera 25 which is housed in the same upper assembly 20c whose tracking camera 24 is currently viewing the located object. Knowing the type of object, for example a patch 7r or reflective sticker 9, indicates to the extraction unit 64 the maximum expected size of the real object, for example, the shoulder pads 6 or the helmet 8. Given this maximum size together with the location In the current state of the reflecting signals, the unit 64 will start at the location of the added signals and then move away from the signals by comparing each pixel of the image with a known background. previously 2r, as illustrated in Figure 4b. Once it is determined that the background pixels have been reached, the image-plot path 10b continues around the object until a full path is completed. Around this closed path 10p a rectangle of minimum limit 10r is calculated to quickly extract the portion of the video image containing the tracked object. Knowing the maximum expected size of the object related to the type of signals found in the object (for example, shoulder 6 or helmet 8 or player 10), this process can be limited to a circle of maximum limit to find the edge of the object. Note that this object defined by the closed path 10p may contain and often contains two or more signals such as patches 7r and 71 as well as the decal 9. This surface is extracted as defined by 10r., as well as the tracked location of each reflector object within that surface is then passed to the composition of multiple overlapping images in the unit of individual full-view frames 65. Unit 65 then catalogs in a complete series all the reflective objects detected such as 3b, 4b, 7m and 9a and their corresponding extracted object pixels that have been detected throughout the field of view. As a matter of prac, the extracted pixels will be expressed as the minimum bound rectangle as illustrated by 10r more than a list of pixel coordinates. Within this minimum-bound rectangle, all the background pixels will be set to a null value by the extraction unit 64 to clearly differentiate them from the first object.

Flat during later revision. A center of gravity as well as polar coordinates for that center point are also calculated by the extraction unit 64 and the unit is passed to the composition unit 65. And finally, the unit 64 will determine a starting edge point to be associated with each object that can be used by subsequent routines to quickly trace the object's outline from within its minimum bounding rectangle to perform a final extraction. It is also the responsibility of the composition unit 65 to join the pieces of the reflecting surfaces such as 3b, 4b, 7m and 9a as well as the objects to which they are attached such as shoulder pads 6 or helmet 8 or player 10 that can be superimposed on squares. taken by 25 cameras or 24 separate tracking cameras. Once the composition unit 65 has created the known series of polar coordinates for the centers of gravity for all the known reflector marks and their corresponding objects and has also defined the minimum limit rectangles and a starting edge point, this series data is passed to the storage of tracking and video frames indexed by the object unit and frame number / time sequence 67. As the entire series of frames through the field of view created by the layout of upper assemblies 20c and perspective equipment 30 are continuously introduced by the capture units 61 will be passed to the tracking unit 62 which will follow each new object as it enters the 20c1 field of view and finally comes out by the same assembly 20c1. The inventors hereby anticipate that after identifying the first occurrence of a surface with ink or corresponding object it may be more efficient to allow the tracking unit 62 to express subsequent centers of gravity using the same polar coordinate method except that the center point and it is not the center of ice 2a but rather the previous center of gravity for the same surface with ink or corresponding object. This changing center of reference strategy is shown in Figure 4c by the sequence 10a1 to 10a4. The storage unit 67 will receive this continuous stream of surface information with ink and corresponding object and will create several databases for further analysis. First, each individual surface and object will have its own movements cataloged from the point where it enters the field of vision until at a later point it comes out. Note that the field of vision is extended to cover the equipment boxes 2f and 2g as well as the penalty box 2h. Therefore, for the system 100, there is no difference in tracking these surfaces and objects in these areas, where they are not active in the current event, as opposed to on the ice surface 2, where they are active, the storage unit 67 He is also responsible for creating and storing information in a group. For example, unit 67 expects to find the following associations: 1.- It will always be true that a player 10 will have a right patch 7r and left patch 71 associated. 2. - It will often be true that a player 10 will have a helmet 8 associated with decal 9 and a staff 4 with tape 4a. 3.- Sometimes it will be true that a player 10 will have a disc 3 inside his direct semicircle of control. This semicircle will be defined by the current location, direction, travel speed and orientation of player 10 and illustrated as 10s in Figure 4c. 4 .- Sometimes it will be true that it may seem that a player 10 has additional patches, helmets or clubs in which case multiple players may have crashed. Any detected associations are used to form a player group database. Note that the storage unit 67 will distinguish between a stray baton or helmet and a player. As such, the lost baton or helmet will not create another case of a group of players. Thus, there will be a table-by-frame count of the total number of unique player groups that should remain the same unless: 1.- A new player enters the field of vision. 2.- An existing player leaves the field of vision. 3.- There is a collision between one or more players so that their forms come together temporarily. When two or more groups of players come together to form a single group somewhere within the field of vision, it is anticipated that eventually the players will separate. At this point the system that had Assigning the movements of the groups to each known player who had entered the group, he will now begin to track individual players and the total group count will have returned to his pre-collision count. A storage unit 67 continuously updates its various databases, this same information is then made available in real time to quantization and analysis of object movement stored in the interlaced database unit 68. The unit of quantification and analysis 68 will have a variable series of tasks that you can perform in the tracking databases that can be enabled either in real time or after the event. In any case, some of these tasks are listed below: 1.- Correlating group data of individual players with a previously known list of potential players. This is achieved by comparing the equipment signals 7r2 as well as the player signals 712 to a database of previously known values. 2.- Determine the orientation of the player based on the location of surfaces with ink inside the objects extracted in combination with the last direction, speed and orientation known both of the surfaces with ink and their associated objects. Accordingly, unit 68 will be able to identify that a player is skating backwards against forward, or that his head and shoulders are beginning to turn and therefore it is expected that he will also turn his direction of travel on the next captured picture.

It should also be mentioned that the reflective decal 9 was specifically designed to also assist the unit 68 to determine whether the associated helmet 8 (and hence, the player's head) is facing downward or forward. This is facilitated by the alternating black and white boxes of the decal 9 that will vary in count depending on the orientation of the hull 8. 3.- Update the statistics of the players such as the time they entered and left the ice surface ( known as change), duration of change, average speed, percentage of change used in defense zones, middle or front, percentage of disc control, number of shots, rollovers, interceptions, passes, etc. 4.- Update the statistics of the team such as number of changes, average duration of changes, percentage of the game in the defense zones, middle and front, percentage of disc control, number of shots, rollovers, interceptions, passes, etc. . Any and all of this information that was previously allowed to be tracked in real time is now available for research and deployment at team 80 exit stations. After the event has been completed, or during scheduled breaks in the event activity, the unit of quantification and analysis 68 can continue processing, stored databases to derive additional information that was not previously allowed for real time. Other events, whether they are sporting or not and regardless of the inclusion of objects humans, will have their own set of unique quantification and analysis requirements. At any time after the conclusion of the event and all its analysis by unit 68, it may be advisable for selected individuals to remotely investigate the information tracked and analyzed by the local computer system 60. Through the use of remote communications devices conventionally illustrated as 69 and 79 in Figure 8, a remote computer system for reviewing captured events illustrated as 70 can be used to access the event databases. The inventors of the present consider that the described technique of first tracking specially inked surfaces that have been added to objects that are to be traced and then extracting the minimum limit areas of those movements of objects is novel because at least for the following reasons : 1.- The technique operates in a minimum of time and requires a minimum of computer processing using a narrow band of selected energy to illuminate and capture a small amount of information that produces identity, orientation, direction of travel and speed of the object to a minimum, regardless of the background reflection of the environment. 2.- The technique uses the information determined quickly from the narrow energy band to efficiently extract a series largest information of the entire energy band typically found in ambient light. 3.- The technique creates a resulting movement database that can be used to reconstruct one or more portions or the entire event after it has concluded overlapping all object movements captured and extracted again on an ice surface background. previously known. This last point is essentially beneficial, since in practice it allows to save what would otherwise be a prohibitive amount of event video information as a minimum series of movement information that can then be transferred through the normal system to system connections such as the Internet. Therefore, several individuals involved with an event can separately and easily download this data to their respective remote systems 70 for their own selective repetition and analysis. It is even envisaged that only the centers of gravity need to be transmitted as opposed to all the object pixels in the minimum-bound box because in a repetition of arcade-type event the graphs of the body, helmet and staff of a player as well as The disk can be easily generated by the local system. A further novel aspect of the system 100 is illustrated in Figure 6. Specifically, by using predesigned marked objects such as the cone 11 with reflective ink 11a, it is possible to restrict the tracking field of the system 100 to some suporción of its field of vision such as 2t. In practice, an operator such as an ice trainer will initiate this reduced tracking mode of the system 100 using a special input code feed to the tracking control device 90. The device 90 transmits this control sequence via conventional RF to the system. of local computer 60. This information is then provided to multiple overlap unit composition 65 which then searches for two or more predesigned objects, e.g., cones 11, on the ice surface 2. When taken together, these two or more cones 11 will prescribe one or more geometric areas such as 2t within which all tracking is enabled and out of which all tracking is disabled. In cases where the cones have been placed in an ambiguous manner, the unit 65 will transmit for deployment all possible arrangements given the current cones 11 detected from which the operator of the control device 90 will select the appropriate arrangement. For example, the trainer can simply place two cones 11 on ice 2, one on each side of the red line that divides the ice surface into two parts. At this point, unit 65 will display two possible arrangements that cover either one side of the ice or the other. After restricting the tracking area of system 100, all other functionality is identical as described above. In addition, the quantization and analysis unit 68 is capable of outputting all the information calculated in the portable tracking control device 90 in response to the real-time inquiries of the coaches about ice. The unit 68 can be enabled by the device 90 also to output the speed of consecutive shots in the goal to a marker 91. Therefore, the reader will see that the multi-object tracking system provides a novel apparatus and method for : 1- tracking objects of variable form, of fast multiple movement, such as humans in real time without the help of intervention of an operator either to initially identify or conditionally reidentify the objects during the tracking period; 2- reduce the time required to process image data by first tracking separately special signals that have been added to the object after which the location of the signals can be used together with the direction vectors, acceleration and velocity to extract efficiently the object of your fund; 3- perform their tasks of independent recognition of color distinctions within the objects; 4- perform their tasks of independent recognition of electronic tracking devices, whether passive or active; 5- determine orientation information about objects as a whole and potentially their individual parts; 6- perform under various lighting conditions and air humidity that may not be optimal for recognition techniques based on visible light; 7- Easily shrink the tracking field within the field of vision; 8- create and maintain in real time a database of accessible movement either by time and / or object identity; 9- output either all or some of the captured data either immediately within the ice arena or after the event to one or more remote computers over a communication network such as the Internet; and 10- provide information to remote computer systems over the Internet that can be used for controlled event repetition as well as critical analysis. Although the above description contains many specifications, these should not be construed as limitations on the scope of the invention, but rather as an exemplification of the preferred embodiments thereof. Many aspects of system functionality are beneficial as such without other aspects being present. For example, separate filming cameras could be omitted for cost savings and / or speed improvement. The system would still provide a significant and novel benefit by tracking ink surfaces added to the objects to be traced. There does not have to be a provision of tracking cameras that are mounted directly above the event to be tracked. If it proves that this arrangement is inconvenient, then the cameras simply need to be placed so that together they can create an overlapping field of vision that is more likely to hold the surfaces in ink in vision. For example, on the plate of a film production, this system could be used to automatically track one or more aspects of the scene as the action unfolds. Therefore, if the actors were to use large markings made of special ink in patches hidden under their clothing, then this tracking system could follow the movements of the actors and automatically direct the turn for panning, tilting and zooming or fast removal of selected production chambers. Another example of a system aspect that is beneficial but not mandatory is the link to a remote computer system to review the captured events. Although the inventors of the present consider that this system is unique in the way it stores information that is especially beneficial for remote downloading, the remote system is not necessary for the system to have novelty and utility. It is evident from the description of the multi-object tracking system that it has application capability beyond tracking the movements of hockey players and the puck during an ice hockey game. For example, this same system could be installed on a roller hockey skating rink if the frame holding the upper assemblies were to be mounted as such on poles to hold it above the playing area. The system could also be used to track basketball in a way very similar to ice hockey and these games are almost always played in an indoor arena. Similar approaches could be used with other sports such as soccer and baseball as long as the field of vision is sufficiently covered with perspective tracking cameras because there will be some top assemblies. The system could also be used in convention sales or large auditoriums to track the security location, flow of support staff assistants. This could be achieved using the same upper tracking cameras while it is very likely that the cameras were unnecessary. Each type of person to be tracked could be asked to use a special patch that could even be coded based on statistically relevant criteria as determined by the hosts of the event. As individuals with patches moved around and visited different cabins, your choices could be automatically tracked including the time spent in each selected booth. Such a system could also help with the flow of the crowd if it were detected that large lines are formed around selected areas. Observe that in this application, it is less critical that each and every one of the movements of each and every one of the people that is to be tracked is followed, but rather that in total most of the movements of all individuals similar ones are determined from? or which could be derived useful decisions and statistics.

From the above detailed description of the present invention, the multi-object tracking system, it will be apparent that the invention has a number of advantages, some of which have been described above and others of which are inherent in the invention. Also, it will be apparent that modifications can be made to the multi-object tracking system without departing from the teachings of the invention. Accordingly, the scope of the invention should be limited only as the appended claims require.

Claims (23)

NOVELTY OF THE INVENTION CLAIMS

1. An automated system for tracking the movement of multiple objects within a predefined area comprising: a multiplicity of passive marks continuously reflecting a selected narrow band of energy and applied to each of the objects to be tracked; at least one energy source capable of irradiating the selected narrow band of energy throughout the predefined area; at least one camera with filter to accept only the selected narrow band of energy to receive the reflections of the radiated energy of the passive marks; and a location-tracking computer in communication with the camera to determine the location of the special marks and consequently the objects based on the reflections.

The system according to claim 1, further characterized in that the passive marks comprise an ink embedded in materials covering the objects.

3. The system according to claim 1, further characterized in that the passive marks are at least one of stickers and patches pre-marked with reflective material.

4. The system according to claim 1, further characterized in that the energy source is chosen from the non-visible spectrum.

5. The system according to claim 1, further characterized in that the filter chamber is in a fixed upper location above the objects to be tracked.

The system according to claim 5, further comprising a multiplicity of upper filter chambers which together form a grid and each having a field of view, the field of view of each chamber with fixed upper filter overlapping the field slightly vision of any and all cameras with adjacent fixed top filters.

The system according to claim 6, further comprising one or more movable perspective filter cameras located within a perspective view of the objects.

The system according to claim 7, further characterized in that each camera with a perspective filter is rotated for panning, tilting, and rapidly approaching or moving away in a controllable manner by the tracking computer to specifically track a chosen object.

9. The system according to claim 8, further characterized in that the tracking computer receives a multiplicity of video frames of each camera with upper filter and each camera with perspective filter.

The system according to claim 9, further characterized in that the tracking computer analyzes each individual frame to locate each individual mark within the frame and further calculates the location of the mark within the predefined area based on the known fixed position of the mark. the camera with upper transmission filter or camera with perspective filter and the location of the mark inside the frame.

11. The system according to claim 10, further characterized in that multiple marks are applied to a single object to be tracked.

The system according to claim 11, further characterized in that the tracking computer further determines the orientation of each marked multi-layer object by comparing the related changing locations of each mark applied to the same object.

The system according to claim 12, further characterized in that the tags include unique identification patterns that distinguish between tracked objects in a similar manner.

The system according to claim 13, further characterized in that the tracking computer detects the unique identification patterns created with the marks and creates an information database that interleaves each unique object with its tracked location and orientation data.

15. The system according to claim 14, which further comprises at least one display analysis computer further characterized in that the tracking computer communicates the interlaced information relating to each tracked object with the display analysis computer (at least one).

The system according to claim 13, further comprising at least one camera without a filter further characterized in that each camera with an upper filter each camera with a perspective filter is accompanied by a non-filtered camera with a field of vision that is overlays.

The system according to claim 16, further characterized in that each camera without a filter is a video camera whose images are visible to the human eye.

18. The system according to claim 17, further characterized in that the tracking computer receives corresponding overlapping frames of the cameras with both filter non-filter.

The system according to claim 18, further characterized in that the tracking computer first analyzes the information contained within the box of the camera with a filter to identify the location of all detected objects to be within the field of view subsequently applies this information to the overlapping frame corresponding to the unfiltered camera to extract more efficiently the image of the background object from the predefined area.

20. The system according to claim 13, further characterized in that the tracking computer is programmed to understand which portions of the fields of vision of the several upper filter chambers within the grid represent the entry and exit points of the predefined area for the tracked objects. The system according to claim 20, further characterized in that the tracking computer specifically monitors and at least the entry point to first detect and identify an object as it enters the predefined area. The system according to claim 21, further characterized in that the tracking computer minimizes its computational effort to determine the posterior locations of a detected object first as its location changes from one frame to the next using the location information and orientation contained in previous tables to predict a maximum possible travel area and therefore, a minimum search area for the current frame. The system according to claim 22, further characterized in that the location and orientation information contained in the above tables is used to specifically determine a direction vector and travel speed for each tracked object which is then combined with previously known information. regarding the size and potential range of acceleration and velocity of the object to better predict the location of the object in the current frame.