EP3877722A1 - Method for determining a target - Google Patents
Method for determining a targetInfo
- Publication number
- EP3877722A1 EP3877722A1 EP19801266.8A EP19801266A EP3877722A1 EP 3877722 A1 EP3877722 A1 EP 3877722A1 EP 19801266 A EP19801266 A EP 19801266A EP 3877722 A1 EP3877722 A1 EP 3877722A1
- Authority
- EP
- European Patent Office
- Prior art keywords
- target
- image
- classifiers
- recognition unit
- missile
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 53
- 238000010801 machine learning Methods 0.000 claims description 12
- 238000012706 support-vector machine Methods 0.000 claims description 8
- 239000011159 matrix material Substances 0.000 claims description 3
- 230000010006 flight Effects 0.000 claims description 2
- 230000015654 memory Effects 0.000 description 13
- 238000012545 processing Methods 0.000 description 5
- 230000001419 dependent effect Effects 0.000 description 4
- 238000001514 detection method Methods 0.000 description 4
- 238000013507 mapping Methods 0.000 description 3
- 238000012549 training Methods 0.000 description 3
- 238000013459 approach Methods 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 230000018109 developmental process Effects 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 230000008447 perception Effects 0.000 description 2
- 238000001454 recorded image Methods 0.000 description 2
- 230000003595 spectral effect Effects 0.000 description 2
- 238000012512 characterization method Methods 0.000 description 1
- 238000005314 correlation function Methods 0.000 description 1
- 238000003066 decision tree Methods 0.000 description 1
- 230000007123 defense Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 239000002360 explosive Substances 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 238000010191 image analysis Methods 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 238000002955 isolation Methods 0.000 description 1
- 238000003672 processing method Methods 0.000 description 1
- 230000000717 retained effect Effects 0.000 description 1
- 239000011435 rock Substances 0.000 description 1
- 238000010187 selection method Methods 0.000 description 1
- 239000013589 supplement Substances 0.000 description 1
- 238000012911 target assessment Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 description 1
Classifications
-
- F—MECHANICAL ENGINEERING; LIGHTING; HEATING; WEAPONS; BLASTING
- F41—WEAPONS
- F41G—WEAPON SIGHTS; AIMING
- F41G7/00—Direction control systems for self-propelled missiles
- F41G7/007—Preparatory measures taken before the launching of the guided missiles
-
- F—MECHANICAL ENGINEERING; LIGHTING; HEATING; WEAPONS; BLASTING
- F41—WEAPONS
- F41G—WEAPON SIGHTS; AIMING
- F41G7/00—Direction control systems for self-propelled missiles
- F41G7/20—Direction control systems for self-propelled missiles based on continuous observation of target position
- F41G7/22—Homing guidance systems
- F41G7/2253—Passive homing systems, i.e. comprising a receiver and do not requiring an active illumination of the target
-
- F—MECHANICAL ENGINEERING; LIGHTING; HEATING; WEAPONS; BLASTING
- F41—WEAPONS
- F41G—WEAPON SIGHTS; AIMING
- F41G7/00—Direction control systems for self-propelled missiles
- F41G7/20—Direction control systems for self-propelled missiles based on continuous observation of target position
- F41G7/22—Homing guidance systems
- F41G7/2273—Homing guidance systems characterised by the type of waves
- F41G7/2293—Homing guidance systems characterised by the type of waves using electromagnetic waves other than radio waves
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
- G06F18/2411—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on the proximity to a decision surface, e.g. support vector machines
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/24765—Rule-based classification
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10048—Infrared image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20092—Interactive image processing based on input by user
- G06T2207/20104—Interactive definition of region of interest [ROI]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30212—Military
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V2201/00—Indexing scheme relating to image or video recognition or understanding
- G06V2201/07—Target detection
Definitions
- the invention relates to a method for detecting a target, in which an image depicting the target is predetermined and a target recognition unit recognizes the depicted target as such from image features of the image and predetermined classifiers.
- Missiles with a seeker head which recognizes the target as such and can track it, are used to approach ground or air targets.
- the missile can independently track the target and fly towards it.
- the missile comprises a camera that records the target in the visual and / or infrared spectral range.
- the target is recognized from one or more recorded images and the position of the target in the image and, if necessary, an orientation of a pivotable optics of the camera are used to determine the position of the target relative to the flight direction or to the longitudinal axis of the missile.
- image processing software examines an image representing the target for image features using so-called classifiers, which describe typical image features of one or more targets.
- classifiers describe typical image features of one or more targets.
- the reliability of the view of the target is also dependent on the quality of the classifiers used to understand the goal.
- the target recognition unit uses the knowledge of the target to create new classes. identifiers determined from the image with which the target can be recognized as such with a higher recognition quality than with the classifiers used for recognition.
- the invention is based on the consideration that the determination of classifiers with which missile targets can be reliably identified as such is associated with a high outlay.
- Classifiers can be determined by extracting image features of the target from a multiplicity of images, each depicting a target, and forming classifiers from these, each of which recognizes their image feature. With the classifiers found in this way, the same and / or other images, each showing a target, can be examined with the task that the image processing software uses the classifiers to find the target in the images alone. Using a recognition quality that is the result of the recognition process, it can be checked how reliable the recognition of the target in the respective image is.
- images are required in which at least one target to be recognized is depicted.
- images are often taken during training missions in which the target and / or a missile pursuing a target flies an mission. Since such images often contain sensitive data, they are often kept under lock and key and are therefore not accessible to improve the classifiers.
- the target recognition unit can be made available to a user taking target images, for example to a military flying exercises.
- the user can use the target recognition unit to determine new classifiers from images that represent a target without the target images themselves having to be released.
- the classifiers which are usually free of sensitive data, can now be made available to the manufacturer of the target recognition unit, so that the target recognition unit can be further improved using the new classifiers.
- the reliability of the target perception can thus be increased considerably with relatively little effort.
- the target recognition unit In order to keep target images, that is to say images each depicting one or more targets, in a narrowly limited area and not to distribute them widely, it makes sense for the target recognition unit to be integrated in a missile or a platform, for example an aircraft.
- the platform expediently carries a missile.
- the missile can be an unmanned missile, in particular a missile with a rocket engine.
- the new classifiers can be determined by the target recognition unit while the platform or the missile is in use. The new classifiers can even be determined while a flying target is being pursued or the platform or the missile itself is flying. The images no longer have to be sent to a ground-based location.
- the understanding of a target can include the recognition of the target as such on the basis of image features in one or more images.
- the recognition of a target can include a classification that classifies an image part as a target image.
- the target recognition unit expediently comprises image processing software for analyzing one or more images on the basis of image features.
- the classifiers expediently each describe one or more image features.
- Methods for forming classifiers, also called descriptors are, for example, the SIFT method (Scale Invariant Feature Transform), the SURF method (Speded Up Robust Features), and / or the ORB method (Oriented Fast and Ro - tated letter) or further developments of one of these methods.
- Another known method for feature determination and automatic feature comparison are, for example, Laplacian of Gaussian (LoG) and Normalized Cross-Correlation Function (NCCF).
- LoG Laplacian of Gaussian
- NCCF Normalized Cross-Correlation Function
- the target recognition unit is given an image depicting the target.
- a partial area of the image, which images the target, hereinafter also called the target area, is expediently given to the target recognition unit.
- the image features of the target in the image are known.
- the new classifiers can be determined from these image features, expediently using the predetermined, that is to say known classifiers. New classifiers can be determined or the known classifiers are changed, which in the following is equated with the determination of new classifiers.
- a similar goal can now be achieved with a higher recognition quality than such can be recognized than with the predetermined classifiers used for recognition. This is because the new classifiers were determined using the image features of the depicted target, so they are specially tailored to such image features. If such or similar features appear in another image, they are reliably recognized by the new classifiers, so that the target is particularly reliable.
- the new classifiers can be determined using machine learning. In addition to the pure recognition of the target as such, it is also expedient to be able to classify the target more precisely using the classifiers, so that the target can be assigned to one of several target categories. In this way, a type of the identified target can be determined, which is advantageous, for example, for a friend-foe recognition or a selection of one of several approach or control options. Accordingly, the classifiers are appropriately divided into the target categories.
- the target recognition unit examines the image as a whole and recognizes the imaged target as such on the basis of the predetermined classifiers.
- a sub-area in which the target is depicted is selected from the target image, that is to say a target area.
- the search of the target recognition unit for the target on the basis of the predefined classifiers can be restricted to the target area, thereby reducing the risk of misdirection.
- the target area can be selected by an operator, for example a pilot of a flying platform.
- the operator can recognize the target from an image and mark the target area in the image.
- the target recognition unit can recognize the target as such in the target area and track it, for example, in a sequence of images taken in succession.
- the image is recorded by a camera and displayed to an operator.
- the operator can now recognize the target as such.
- the operator can make entries in an input system and thereby direct the missile or its target recognition unit to the target.
- This instruction can be done by the operator selecting a target area, that is to say an image partial area in which the target is depicted, and transferring this selection to the target recognition unit.
- a pilot looks at the target - either in a real scene or on a picture - and the line of sight data are transmitted to the target recognition unit, which is instructed with this data on the target.
- the target recognition unit can now search for the target using the specified classifiers exclusively in the target area.
- the operator When identifying the target, the operator is advantageously in a flying platform.
- the camera is expediently part of a seeker head of the missile, which is attached to the platform.
- the target area can be marked, for example, using a target area selection by an operator during a flight of the missile.
- a crew member of an aircraft hereinafter simply called a pilot, even if this operator does not have to perform any control activity, can direct the missile to the target by, for example, manually marking the target area.
- the missile conveniently transmitted the image to the operator.
- a target area is selected in an external image that was recorded from the ground or another aircraft, e.g. by a guide on the floor.
- the external image is compared with at least one image recorded by the platform - in particular the missile - and image areas are assigned to one another, so that the selection of the target area is transferred to the image.
- the target area selection can be automated using data from a preliminary instruction, e.g. from radar image data or IR / VIS image data.
- the target recognition unit is part of a flying platform, in particular part of a missile attached to a flying platform. It is particularly advantageous here if the missile has a seeker head that records the image.
- the image can be transmitted to an operator of the flying platform, who recognizes the target from the image and marks a partial area of the image as the target area.
- the target area of the image can now be transferred to the target recognition unit, which recognizes the target as such based on the predefined classifiers.
- the target recognition unit can determine the new classifiers based on the recognized target.
- the target is recognized in an image, for example manually by an operator, that is, by eye, or automated by another recognition unit.
- This can include part of the platform and, for example, very extensive software, the flow of which is high Computing capacity required.
- the already recognized target is recognized by the target recognition unit.
- the target or a target area of the target image is transferred to the target recognition unit.
- the new classifiers can now be determined. This can take place during the flight or later on the ground.
- a plurality of images of the target are recorded in succession, so that the images are a series of images of the target.
- the target becomes more and more visible from image to image.
- a target recognition unit with good classifiers finds a target much faster than an operator who looks at the picture.
- a trained target recognition unit could recognize the target in an image taken earlier than an operator viewing the series of images, for example as a film. If the operator recognizes the target and marks a target area, the target will presumably be quite easy to recognize.
- the target unit will also recognize the target with a high recognition quality using the predefined classifiers. However, the classifiers are particularly good if they also recognize the goal in previous pictures.
- the image is part of an image series whose images represent the target and the new classifiers are determined from at least one other image in the image series. It is particularly advantageous here if the target is more difficult to recognize in the other image than in the image in which it was previously recognized with the specified classifiers.
- the classifiers can now be created or changed in such a way that the target is reliably recognized at a very early stage. As a rule, the other image will therefore be recorded before the image in which the target was found by the target recognition unit using the predefined classifiers.
- the other image was only taken after this image, for example if the target is later difficult to see or if the target scene has changed, for example if decoys are dropped.
- the target recognition unit After a vision of the goal, it will usually be desirable to pursue the goal. In the row of images in which the target is depicted, the target recognition unit will therefore try to find the target again or even always in the subsequent images. This can be done with the given classifiers. Depending on the speed of the creation of the new classifiers, it can also make sense for the goal to be understood in the following pictures of the image series with the new classifiers. This enables the target to be recognized more reliably, so that the target can be tracked more reliably.
- the classifiers can make sense for the classifiers to be continuously re-determined in the course of the target assessment using the images in a series of images.
- the classifiers are changed using machine learning.
- the new classifiers are advantageously determined by machine learning.
- Machine learning is expediently guided machine learning, since the target to be recognized can be specified by another entity, for example the operator, and can also expediently be checked. In this way, an error detection, for example due to an incorrect development of mechanical learning, can be avoided.
- the Support Vector Machine method is particularly advantageous for this type of machine learning. Training objects for the creation of the support vector machine can be the target in each image of a series of images, expediently several targets in several series of images. For each of the goals shown, it is known which class they belong to, ie whether it is a goal or not. Each object can now be represented by a vector in the vector space of the Support Vector Machine. The Support Vector Machine can now create a level of hype that separates the classes.
- the Support Vector Machine it therefore makes sense to use a feature space with a large dimension.
- a feature space with such a high dimensioning is used that the image values of the entire pixel matrix of the target area of nxm pixels are used.
- the dimension can be nxm.
- a method for combining the image values or other processing of the image values can be dispensed with, so that high-quality classifiers can be formed even without a reliable preliminary method - which is generally not known beforehand. It may well be that the classifiers deviate greatly from one application to another, especially if different objectives are understood in the operations.
- the new classifiers are assigned to one of several usage profiles.
- This can be stored, for example, in a carrier platform that carries a missile with the target recognition unit, for example an aircraft.
- an assignment profile can be assigned to each assignment, so that the classifiers stored for this assignment profile are used. This can counteract misclassification or misrecognition.
- the deployment profile advantageously includes a description of the goal, for example a goal type, and / or a job characterization independent of the image.
- the predetermined classifiers were determined at least in part by means of machine learning during flights of a carrier platform which is intended to carry a unit with the target recognition unit, for example to carry a missile.
- the target can be reliably identified in this way, particularly in the case of similar deployment profiles.
- classifiers After long training sessions, a large number of classifiers will be available, each assigned to an assignment profile. In the case of a new mission, it can now be decided which mission profile is selected in order to use the cheapest classifiers.
- a particularly advantageous selection of classifiers can be achieved if the predefined classifiers are assigned to different usage profiles.
- the insert profiles are expediently weighted and the selection and / or weighting of the classifiers is carried out using the weighting of the insert profiles. If the selection is made using the weighting of the deployment profiles, classifiers can be selected from various deployment profiles and can now be used together as predefined classifiers. A predetermined selection of the classifiers is also possible, these being then weighted individually, so that these classifiers are used to a greater or lesser extent to identify the target. This enables a very differentiated target recognition to be achieved.
- the weighting is carried out using image data of the target area. If, for example, the image data of the target area suggest that the target area depicts a cloud area, a “clouds” application profile can be used without an operator having to specify this manually got to. The same applies to the scenario, for example, when a target is displayed against the background of water, forest or an urban background. The background can be automatically recognized and assigned to an assignment profile, so that the correct assignment profile is automatically and quickly selected.
- several insert profiles can also be used together, each with a weighting, so that the predefined classifiers can be assigned to different insert profiles and / or are weighted differently.
- the target area can contain information that can contribute to an advantageous selection of classifiers, but also, for example, target instruction.
- the application profiles are weighted and the weighting is carried out using image-independent target information.
- Image-independent target information can be a description of the target, a target briefing and / or mission data, which contain information about the target.
- a single mission profile can be selected, expediently that with the best classifiers, for example for the current mission. This is also possible without mission information, for example by trying out the classifiers mission profile after mission profile and selecting the classifiers with which the goal is most reliably recognizable.
- the usage profiles can be arranged in a tree structure.
- the branching of the tree structure can be formed from usage characteristics and / or target characteristics. In this way, the tree structure can be traced in accordance with the current application characteristics and / or target characteristics to compile the specified classifiers.
- the invention is also directed to an apparatus for detecting a target.
- the device expediently contains a camera, a selector for selecting a target area in at least one image of the camera and a missile with a seeker head for tracking a selected target.
- the missile advantageously contains a target recognition unit which is prepared to recognize the imaged target from image features of the target area and predetermined classifiers and to determine new classifiers based on the knowledge of the target, with which the target has a higher recognition quality than with those used for recognition classifiers can be recognized as such.
- the selector can be a screen on which an operator selects the target area.
- a detection unit for automated detection of the target is also possible.
- FIG. 1 shows a missile with a seeker head, a camera and a target recognition unit
- 2 shows an image taken by the camera of the missile with an illustrated target
- 3 shows a flowchart for understanding a target using classifiers and for determining new classifiers
- the flying platform 6 is, for example, an aircraft, under the wing of which the missile 2 hangs.
- the missile 2 is an unmanned missile 2 with a rocket motor 8 and a seeker head 10, which contains a camera 12 with optics 14 and a detector 16.
- the optics 14 are infrared optics and the detector 16 is an infrared detector as an imaging detector 16 in the form of a matrix detector.
- a target recognition unit 18 with image processing software for processing the image signals or pixel signals of the detector 16 is connected in terms of signal technology to the detector 16.
- the search head 10 also has a control unit 20 for controlling the flight of the missile 2 and a data memory 22 for recording digital ones Data from and for the target recognition unit 18 and the control unit 20.
- the missile 2 also has an active part 24 with, for example, an explosive charge and steering wing with control surfaces 26 with which the steered flight of the flying head 2 can be controlled.
- the control is carried out by the control unit 20 on the basis of data from the target recognition unit 18.
- FIG. 2 shows an image 28, which was recorded by the camera 12 of the missile 2, while the latter is firmly connected to the flying platform 6. It can be seen that the platform 6 flies over a sea 30, clouds 32 and land 34 are visible in the image 28. Also shown in Figure 28 is a target 36 to be flown to by missile 2. FIG. 2 indicates that the target 36 is barely recognizable at the time the image 28 was taken, since it is either very small or very far from the missile 2 or the platform 6.
- the image 28 contains the image of an aircraft 38 and further image regions 40 which, due to the objects depicted in them, are very similar to the image of the target 36 in terms of image processing and image analysis based thereon. For example, a sailboat is depicted in the sea 30 and a ship further back, both of these objects standing out clearly and objectively from the surrounding image background.
- Rocks on an offshore island or a point in the clouds 32 also form such an object-conspicuous image region 40.
- the aircraft 38 which is clearly easier to recognize in the moment of image acquisition than the target 36, but in principle is also very easily confused with a target 36 in terms of image processing.
- heat radiator objects which can easily be confused with the infrared signature of the target 36, also easily form false targets. If, for example, sunlight is reflected from the sea 30, image regions 40 with a target-like image signature can occur.
- a window pane reflecting the sunlight on the coast of the island, the aircraft 38 and also illuminated cloud regions can also form image regions 40 which an image processing unit could confuse with a target 36.
- FIG 3 shows a flow diagram of a method for detecting a target, in which the picture elements in the upper row of the figure are assigned to processes in the flying platform and picture elements in the rows below are assigned to processes taking place in the missile 2.
- the platform 6 or an operator operating the platform 6 receives a mission order 42 which, for example, describes what type of target 36 is to be grasped, followed and flown to by the missile 2.
- a mission order 42 which, for example, describes what type of target 36 is to be grasped, followed and flown to by the missile 2.
- Data from the mission order 42 determine essential characteristics of a flight 44 which is carried out by the platform 6.
- the missile 2, which is attached to the platform 6, is taken along and in this respect performs the same flight 44.
- camera 12 of missile 2 captures a series of images 28, which depict target 36. These images 28 are transmitted to the platform 6 and are presented to the operator in turn on a display of the platform 6, for example in the form of a film that views the images 28.
- the representation of the images 28 on the display is a real-time representation, so that the operator can see the imaged front surroundings of the flying platform 6 through the seeker head 10 or the camera 12 of the missile 2. For example, the operator cannot yet recognize the target 36, even if it is already shown in the images 28.
- the target recognition unit 18 could possibly recognize the target 36, but since there are so many other conspicuous image regions 38, 40 in the images 28, it can do so here easily misinterpret a supposed goal.
- the target recognition unit 18 could make suggestions, for example by marking all conspicuous image regions 36, 40.
- the operator can look at the images 46 and recognize the target 36 independently or - if present - select a marked image region 36, 40, whereby the target 36 is recognized as such, which is indicated in FIG. 3 as step 48 by a stylized crosshair.
- the display on which the operator selects the target area or a recognition unit which autonomously recognizes and marks the target 36 can be referred to as a selector, the illustration of which has been omitted in FIG. 1 for the sake of clarity.
- the operator marks the target in method step 50, for example by marking the target 36 with a finger on a touch screen or by using a mouse or another aid.
- the marking 50 of the target 36 takes place in that a target area 52 is selected by the operator.
- This target area 52 is part of the image 28, for example of such an image 28, which is currently displayed to the operator on the display.
- the target 36 is shown in the target area 52 of the image 28, as shown in FIG. 2 and FIG. 3.
- the target area 52 After the target area 52 has been marked 50, the target area 52 or the data characterizing it is transferred to the missile 2. This is shown in FIG. 3, in that the target area 52 now lies in one of the lower-lying rows of image objects that reproduce the processes in the missile 2.
- the target 36 is depicted in the image area 52, but is not yet known to the missile 2 or its target recognition unit 18, as is indicated by the dotted representation in FIG. 3.
- the target 36 in the target area 52 is recognized 54 by the target recognition unit 18 of the missile 2.
- classifiers 56 stored in the data memory 22 are selected in step 58 and then used.
- the classifiers 56 each describe image characteristics. Some or all of these image characteristics are found in the target area 52 in such a way that a recognition quality of the target 36 exceeds a recognition threshold, so that the target 36 in the target area 52 is regarded as recognized or as perceived.
- FIG. 3 in that the target 36 is shown as a solid line in the target recognition 54, that is to say the target 36 is recognized as such.
- Data of the target recognition 54 can now be transferred to the flying platform 6, so that the target 36 is shown marked in the image 28 in an image display 60 can. In FIG 3, this marking is indicated by the dotted circle around the image of the target 26. The operator can now see that the target 36 marked by him has been grasped by the missile 2 and can now be optically tracked by the missile 2 or its search head 10.
- a target recognition 62 is now carried out directly using the classifiers 56 from the data memory 22, the images 28 being made available to the target recognition unit 18, which results from the position of the target 36 in one of the previously recorded images 28 independently assess the target area 52 and can recognize the target 36 directly and independently using the classifiers 56.
- Images 28 are transferred to flying platform 6, so that target 64 of target 36 is thereby tracked over time, that is to say via the sequence of images 28. This is indicated in FIG. 3 by way of example, in that the target 36 wanders through the images 28 when the target 64 is being tracked and the image 60 shows the target tracking 64 at a different location than in the previous image display 60 .
- the target recognition unit 18 knows the image features of the target 36 in the target area 52 or in the image 28 through the target recognition 54 of the target recognition unit 18.
- the target 36 was recognized with a recognition quality that depends on the classifiers 56 and the mapping of the target 36.
- a recognition quality can be increased if more suitable classifiers 56 are available.
- a process of machine learning or machine learning is used in the target recognition unit 18.
- the method of the support vector machine is used for step 68 of determining the new classifiers 66. Since the target 36 to be recognized by the new classifiers 66 is already known, this process is a so-called supervised learning or guided learning.
- a plurality of images of the target 36 can also be used for the classifier determination 68 in order to increase the number of objects which are separated from one another by the support vector machine.
- the new classifiers 66 are now composed or ascertained in such a way that the known target mapping with them is carried out with the higher recognition quality than can be recognized by the classifiers 56.
- the new classifiers 66 are stored in the data memory 22.
- the new classifiers 66 can already be used in the step of target recognition 62 of the following images 28.
- the same target 36 which is represented by the image series, can thus first be determined as such with the predetermined classifiers 56 and in later images 28 with the newly determined classifiers 66.
- the classifiers 56 from the data memory 22 are predetermined classifiers 56 which are available as a priori data, for example from previous missions or as basic data of the target recognition unit 18.
- the classifiers 66 determined later can be dynamically optimized classifiers 66 which are characterized by the Target perception are improved dynamically and in particular are continuously improved or even optimized in the further course of target tracking 64.
- the new classifiers 66 can be transferred from the data memory 22 of the missile 2 into a data memory 70 of the flying platform 6, as shown by the arrow in FIG. 3. If the missile 2 is launched later, for example, from the platform 6, the data determined by the target recognition unit 18, in particular the new classifiers 66, are retained and stored in the flying platform 6, so that they are later stored in a data memory 22 of another missile 2 can be transferred.
- data from the mission order 42 are used to select the predefined classifiers 56 from a large number of predefined classifiers by selection or weighting. This is shown in FIG. 3 by the weighting step 72.
- weighting no further distinction is made between weighting and selection, since sorting out unused classifiers can be understood as zero weighting.
- the description of an exemplary embodiment is essentially limited to the differences from the previous exemplary embodiment, to which reference is made with regard to features and functions that remain the same. In order not to have to repeatedly carry out what has already been described, all features of a previous exemplary embodiment are generally adopted in the following exemplary embodiment without being described again, unless features are described as differences from the previous exemplary embodiments.
- classifiers 56 can be weighted, which in the following can also be understood to mean a selection that is assigned to this type of goal.
- Other target data such as airspeed and agility of target 36, can also be used to weight classifiers 56. Based on the weighting carried out, the classifiers 56 are selected weighted from the data memory 22 and used for target recognition 54.
- a further weighting 72 can result from image data of the target area 52.
- special classifiers 56 can be selected which are specially adapted to such a scenery .
- the target area 52 is evaluated for image characteristics in step 76 and these are used for weighting 72 of the predetermined classifiers 56 stored in data memory 22.
- the classifiers 56 are applied in accordance with the weighting 72 carried out. This weighting can be carried out alternatively and / or in addition to the weighting from the mission data.
- the target recognition unit 18 can clearly recognize the target 36 in an image 28 than is possible for a human operator. In image 28, in which the operator recognizes target 36, this can already be seen quite well. Good classifiers 66 for an early target acquisition 54 can therefore possibly be determined from images 28 which lie before the image 28 in which the human operator manually recognized the target 36 48.
- such preceding images 28 - or only their target areas 52, whose position is known from the position of the manually selected target area 52 - are examined.
- these images 28 there is again a target detection 78, either with the predefined classifiers 56 or with the already improved classifiers 66.
- the former is shown as an example in FIG.
- new classifiers 66 are now determined in step 80 and stored in the data memory 22. This can be done, for example, for a predetermined number of images 28 backwards from the manually marked image 28 or until the achievable recognition quality of the new classifiers 66 drops below a threshold, so that the target 28 can no longer be recognized sufficiently well even with improved classifiers 66.
- this assignment profile 42 can be assigned to the classifiers 66 determined in this assignment.
- the new classifiers 66 stored in the carrier platform 6 can be clearly assigned to one or more application profiles and, in the case of a similar application, can be preselected accordingly by weighting 72. It is considered equivalent whether the usage profiles are weighted or the classifiers 56, 66 are weighted. In the same way, the classifiers 56, 66 can also be assigned to sceneries or scenery profiles that were determined from scene characteristics of the target area 52 and can be used for weighting 72.
- FIG. 4 shows a simple example of a weighting 72.
- a first group 82 comprises predefined classifiers 56, which were determined, for example, in an offline process on the ground using image data. Such statically determined classifiers 56 can form a priori data for target recognition 54.
- further groups 84-90 can also be used, which differ, for example, in the mission profiles and or target classes. In the following it is assumed, for example, that groups 84 - 90 describe four different target classes.
- the target 36 to be flown to is then specified later in the mission order 42, so that one of the groups 84-90 or its classifiers 66 can be selected or several groups 82-90 are linked to one another by a more differentiated weighting.
- the individual groups 82-90 can be weighted in accordance with the mission order 42, in which the new target 36 is defined, as indicated by weighting factors G in FIG. These weighting factors G result from the ratio of the data from the mission order 42 to the target types corresponding to the groups 84-90 or the a priori group 82.
- the classifiers 56 stored in the groups 82-90 can be weighted by the weights G1 , 66 can now be used for target recognition 54, 78. Reference symbol list
Landscapes
- Engineering & Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Combustion & Propulsion (AREA)
- Data Mining & Analysis (AREA)
- Chemical & Material Sciences (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Physics & Mathematics (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Bioinformatics & Computational Biology (AREA)
- Artificial Intelligence (AREA)
- Life Sciences & Earth Sciences (AREA)
- Electromagnetism (AREA)
- Aiming, Guidance, Guns With A Light Source, Armor, Camouflage, And Targets (AREA)
Abstract
Description
Claims
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
DE102018008864.3A DE102018008864A1 (en) | 2018-11-10 | 2018-11-10 | Method of understanding a target |
PCT/EP2019/080336 WO2020094691A1 (en) | 2018-11-10 | 2019-11-06 | Method for determining a target |
Publications (1)
Publication Number | Publication Date |
---|---|
EP3877722A1 true EP3877722A1 (en) | 2021-09-15 |
Family
ID=68503103
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
EP19801266.8A Pending EP3877722A1 (en) | 2018-11-10 | 2019-11-06 | Method for determining a target |
Country Status (3)
Country | Link |
---|---|
EP (1) | EP3877722A1 (en) |
DE (1) | DE102018008864A1 (en) |
WO (1) | WO2020094691A1 (en) |
Family Cites Families (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
DE19716025B4 (en) * | 1997-04-17 | 2009-12-03 | Diehl Bgt Defence Gmbh & Co. Kg | Platform with launchable, target-tracking missiles, in particular combat aircraft |
US8582871B2 (en) * | 2009-10-06 | 2013-11-12 | Wright State University | Methods and logic for autonomous generation of ensemble classifiers, and systems incorporating ensemble classifiers |
DE102015004936A1 (en) * | 2015-04-17 | 2016-10-20 | Diehl Bgt Defence Gmbh & Co. Kg | Method for aligning an agent unit to a target object |
IL239191A0 (en) * | 2015-06-03 | 2015-11-30 | Amir B Geva | Image classification system |
CA2998956C (en) * | 2015-11-26 | 2023-03-21 | Sportlogiq Inc. | Systems and methods for object tracking and localization in videos with adaptive image representation |
-
2018
- 2018-11-10 DE DE102018008864.3A patent/DE102018008864A1/en active Pending
-
2019
- 2019-11-06 EP EP19801266.8A patent/EP3877722A1/en active Pending
- 2019-11-06 WO PCT/EP2019/080336 patent/WO2020094691A1/en active Application Filing
Also Published As
Publication number | Publication date |
---|---|
WO2020094691A1 (en) | 2020-05-14 |
DE102018008864A1 (en) | 2020-05-14 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
EP2399096B1 (en) | Method and system for detecting target objects | |
EP2381207B1 (en) | 3D targeting and target designation from IR data | |
DE69730733T2 (en) | FLIGHT BODY STEERING SYSTEM WITH LOCK AFTER SHOT AND RECONSTRUCTION OF THREE-DIMENSIONAL SCENES | |
EP0815464A2 (en) | Device for seeking, discovering and tracking air targets | |
WO2020094691A1 (en) | Method for determining a target | |
EP3081892A1 (en) | Method for guiding a missile | |
DE102019129029A1 (en) | OBJECT DETECTION SYSTEM AND METHOD | |
DE4205056A1 (en) | Scene scanning for classification of moving objects - involves assessment of average speeds deduced from processed images of subdivided scene obtd. during successive scans | |
DE60305345T2 (en) | IMAGE PROCESSING DEVICE WITH DETECTION AND SELECTION OF LIGHT SOURCES | |
DE102017223264A1 (en) | Method and device for driving an actuator | |
EP1116962B1 (en) | Method for the autonomous detection of helicopters | |
EP3284010B1 (en) | Method for classifying a vehicle | |
DE69310695T2 (en) | Procedure for the control aid of a low-flying missile | |
DE102016110691A1 (en) | Apparatus and method for determining a respective position of a plurality of aircraft located on an airport surface | |
DE102019007515B4 (en) | Image processing method for determining guidance commands for control wings of a guided missile | |
DE102010005198B4 (en) | Missile and method for detecting a target | |
EP3200154B1 (en) | Method for determining a position of an object | |
EP3637312A1 (en) | Method for detecting a target object by means of a camera | |
DE102020106703B4 (en) | Object recognition unit with sensor inputs in different spectral ranges | |
EP3089076A1 (en) | Method for aligning an agent unit on a target object | |
DE102021115140B4 (en) | Method for controlling a flight movement of an aircraft for landing or for dropping a charge, and aircraft | |
DE102016104021A1 (en) | Method for tracking a target vehicle on the basis of a reference geometry characterizing a wheel geometry of the target vehicle, camera system and motor vehicle | |
Zhao et al. | Onboard smart surveillance for micro-uav swarm: an experimental study | |
EP2392890B1 (en) | Method for directing a missile to a target | |
EP3200149B1 (en) | Method for detection of an object in a seeker image |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: UNKNOWN |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: THE INTERNATIONAL PUBLICATION HAS BEEN MADE |
|
PUAI | Public reference made under article 153(3) epc to a published international application that has entered the european phase |
Free format text: ORIGINAL CODE: 0009012 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: REQUEST FOR EXAMINATION WAS MADE |
|
17P | Request for examination filed |
Effective date: 20210419 |
|
AK | Designated contracting states |
Kind code of ref document: A1 Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR |
|
DAV | Request for validation of the european patent (deleted) | ||
DAX | Request for extension of the european patent (deleted) | ||
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: EXAMINATION IS IN PROGRESS |
|
17Q | First examination report despatched |
Effective date: 20240208 |