EP2251849B1 - Procédé de détection améliorée d'objets de type conduite - Google Patents

Procédé de détection améliorée d'objets de type conduite Download PDF

Info

Publication number
EP2251849B1
EP2251849B1 EP10002260.7A EP10002260A EP2251849B1 EP 2251849 B1 EP2251849 B1 EP 2251849B1 EP 10002260 A EP10002260 A EP 10002260A EP 2251849 B1 EP2251849 B1 EP 2251849B1
Authority
EP
European Patent Office
Prior art keywords
measurement points
straight line
profiles
matrix
line
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Not-in-force
Application number
EP10002260.7A
Other languages
German (de)
English (en)
Other versions
EP2251849A3 (fr
EP2251849A2 (fr
Inventor
Michael Hoyer
Oliver Ruebsamen
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Hensoldt Sensors GmbH
Original Assignee
Airbus DS Electronics and Border Security GmbH
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Airbus DS Electronics and Border Security GmbH filed Critical Airbus DS Electronics and Border Security GmbH
Publication of EP2251849A2 publication Critical patent/EP2251849A2/fr
Publication of EP2251849A3 publication Critical patent/EP2251849A3/fr
Application granted granted Critical
Publication of EP2251849B1 publication Critical patent/EP2251849B1/fr
Not-in-force legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G08—SIGNALLING
    • G08G—TRAFFIC CONTROL SYSTEMS
    • G08G5/00—Traffic control systems for aircraft, e.g. air-traffic control [ATC]
    • G08G5/0073—Surveillance aids
    • G08G5/0086—Surveillance aids for monitoring terrain

Definitions

  • the invention relates to a method for improved detection of line-type objects according to the features of patent claim 1.
  • the present method is suitable for low-flying aircraft to avoid collisions with power lines and other line-type objects such as e.g. Guy wires of masts, cable cars, etc.
  • a method for obstacle warning for low-flying aircraft In this case, distance images of the environment in front of the aircraft are generated by means of a sensor. Obstacle contours are extracted from the distance images by searching for discontinuities ("hops") between adjacent image pixels by means of high-pass filters. With a navigation system, the location of the Hindemisk correction can be superimposed graphically on the natural external view. A recognition of the obstacle in the true sense does not take place.
  • DE 10 2005 047 273 B4 Another method for supporting low-level flights is described, which is based on the examination of jumps in the recorded distance images. In this method, wire-like obstacles are detected in strong environmental conditions, such as clutter or when looking at the sky.
  • US 2008 / 0007708A1 also describes a method for line detection in which a Hough transformation is used.
  • a Hough transformation By means of a rangefinder distance images are taken during the flight and potential images in which lines could occur selected. Subsequently, a projection of the potential line measuring points in a horizontal Level. In a subsequent Hough transformation and subsequent shape transformation, the identification of straight line pieces takes place.
  • the object of the invention is to provide a method for improved line detection, with which lines are detected reliably and in real time.
  • a set of three-dimensional measurement points in a geodesic coordinate system are generated from the distance values of a distance image generated by a distance sensor of this environment, taking into account the position and location of the aircraft. From this set of measurement points, potential line measurement points are extracted using known filter methods.
  • the potential line measurement points are projected into a horizontal plane and identified by means of a polynomial phase transformation and a subsequent spectral analysis of line progressions.
  • the potential line measurement pixels are projected on and at a predeterminable distance from a straight line found in the first step into a vertical plane to the horizontal plane and the respective straight line from the first step, and catenoid or parabolic courses are identified by means of a quadratic shape transformation.
  • a mathematical analogy is established between a line fit and the determination of the spectral parameters of a signal having a plurality of components.
  • an assignment of suitable measuring points i. which are within a defined corridor, i. having a predetermined distance to the estimated in the first step straight line, instead.
  • the evaluation in the second method step according to the invention is generalized to the identification of catenoids.
  • the identification can also be done on parabolas.
  • the result of an evaluation of a distance image (data set) can be compared with the results of the temporally advanced images in addition to the confirmation.
  • the position of the lines relative to one another can be investigated on the basis of the parameterization found by the polynomial phase transformation, and implausible line courses can be eliminated.
  • the attitude relative to the extrapolated flight trajectory of the missile can be examined to generate a warning in critical cases.
  • the pilot can be informed, for example, about the distance, height and direction relative to the helicopter axis or flight vector.
  • the line can be drawn as a symbol, cloud of measurement point or katenoid based on the found parameterization as a video or FLIR overlay or in a digital map.
  • the sensor selected is an imaging front-end which actively scans the scene and whose image information consists of distance values. Distance images are advantageous for the subsequent automatic image evaluation for line detection.
  • the sensors used are a laser scanner, which preferably operates in the infrared range, or an imaging radar in the mm-wave range. Although the radar has certain advantages in terms of bad weather conditions (fog), according to the current state of the art, only one laser scanner has the desired vertical and lateral image resolution. Both sensors are equally suitable as active systems for day and night operation.
  • a suitable laser scanner is designed for long ranges (> 1km) and high image resolution (eg 0.5 horizontal * 0.1 degrees vertical). For each distance image, the spatial direction into which the laser measurement pulse was emitted must be clearly determinable in the sensor-oriented coordinate system, by the design of the scanner and possibly by additional measurements during the scanning process.
  • the navigation system is used to determine the position and position of the aircraft.
  • Each distance value is converted into a scene point using the coordinates of the associated spatial scanning direction and the current position and attitude parameters of the aircraft.
  • a scene point (measuring point) is defined by its three spatial coordinates in a terrestrial coordinate system. From each distance image, a so-called earth-fixed measuring point cloud is formed in this way, from which possibly existing lines can be extracted.
  • Data processing is handled by a mid-power processor. On this, the above-mentioned transformation of distance measurement values into measurement points is first carried out.
  • the resulting data serves as input for an evaluation unit.
  • the evaluation unit determines whether the individual lines are in dangerous proximity to the current position of the aircraft or the intended trajectory.
  • the result of this evaluation is communicated visually and / or acoustically to the pilot.
  • an imaging front-end can be used, which actively scans the surroundings of the aircraft and whose image information consists of distance values. The resulting distance images are used in the subsequent process steps for line detection.
  • Suitable sensors are a laser scanner or an imaging radar in the mm-wave range. In contrast to the radar, which has certain advantages with regard to bad weather operation (fog), a laser scanner has a comparatively better vertical and lateral image resolution. Both sensors are equally suitable as active systems for day and night operation.
  • a suitable laser scanner is designed for long ranges (> 1km) and high image resolution (eg 0.5 horizontal * 0.1 degrees vertical). For each distance image is determined by the design of the scanner and by any additional measurements during the scan, the spatial direction in which the laser measuring pulse was emitted clearly in the sensor-oriented coordinate system.
  • the navigation system is used to determine the position and position of the aircraft.
  • Each distance value is converted to a measuring point using the coordinates of the associated spatial scanning direction and the current position and attitude parameters of the aircraft.
  • a measuring point is defined by its three spatial coordinates in a geodetic coordinate system. From each distance image arises in this way a so-called earth-proof measuring point cloud from which existing lines can be extracted.
  • Data processing is handled by a mid-power processor. On this a transformation of the distance measurement values into measuring points is carried out. The resulting data serves as input for an evaluation unit. The evaluation unit determines whether the individual lines are in dangerous proximity to the current position of the aircraft or the intended trajectory The result of this evaluation is communicated visually and / or acoustically to the pilot.
  • the starting point for the PPT used in the invention is a reduced amount of three-dimensional measuring points.
  • the reduced quantity is due to the fact that only such measured values are subjected to a PPT which has not been eliminated by clutter or interference filters.
  • a first method step of the invention only the horizontal (x, y) component of the measured values is evaluated in order to identify the straight line courses in this point set. Horizontal lines are potential directions of passage of lines.
  • all vertical planes belonging to the straight lines found here are examined for catenoid pieces or parabolic pieces [Katenoide: cosh (x)]. That is, all measurement points lying in or near such a plane are projected into that plane, and the resulting set of two-dimensional measurement points is examined for catenoids using quadratic shape transformation (QFT).
  • QFT quadratic shape transformation
  • Polynomial phase transformation (polynomial phase transform - PPT)
  • the invention is based on the so-called polynomial phase transform.
  • the problem of the line detection according to the invention is converted into a problem of the frequency determination of a sinusoidal signal or the spectral analysis of a frequency mixture in the case of multiple lines.
  • known signal processing methods are available (Fourier Transformation, Matrix Pencil, FFT, Wavelets, Esprit, Pisarenco, MUSIC etc.) where the matrix Pencil has proven to be particularly efficient and will be explained in detail below.
  • the frequencies ⁇ j are determined according to the invention.
  • these include the Fourier transform (FT), the Maximum Likelihood Method (MLM), the least squares method (LSM) and the Matrix Pencil Method (MPM).
  • FT Fourier transform
  • MLM Maximum Likelihood Method
  • LSM least squares method
  • MPM Matrix Pencil Method
  • the matrix pencil method is used.
  • the other methods for frequency determination can be used.
  • T means transposed and L is a chosen parameter, the so-called pencil parameter, with d ⁇ L ⁇ N - d.
  • the continuity of each element of the truncated pseudoinverse is Y 0 + also at the point where the noise equals 0, and thus also the continuity of the z r 's.
  • Y 0 + is identical to X 0 + if and only if the noise is zero.
  • Y 0 + Y 1 If LM has singular values equal to zero that do not contain information about the z r 's, one can reduce the size of the matrix before calculating the singular values.
  • the singular values z r are the M eigenvalues of Z E * which are the same as the M non-zero singular values of Y 0 + Y 1 ,
  • Quadratic Form Transform (QFT)
  • the pixels of the line are transformed into the front view. In other words, you no longer look at the pixels of the straight line from a bird's eye view. Instead, one uses the first found pixel of the straight as the zero point of a new coordinate system, which uses the found straight line as one of the new axes, and the z axis as the second new axis. Thus, one obtains a graph in which the found straight should have catenoid shape if it corresponds to a high voltage line.
  • the transformation PPT is based on a polynomial of degree 1.
  • ⁇ # 0 expediently ⁇ is set to 1.

Landscapes

  • Engineering & Computer Science (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Remote Sensing (AREA)
  • Aviation & Aerospace Engineering (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Length Measuring Devices By Optical Means (AREA)
  • Image Processing (AREA)

Claims (4)

  1. Procédé destiné à la reconnaissance améliorée d'objets de type canalisations dans un environnement situé à l'avant d'un engin volant, à partir des valeurs de distance d'une image de distance dudit environnement, créée à l'aide d'un capteur de distance, en tenant compte de la position et de la situation de l'engin volant, une quantité de points de mesure tridimensionnels étant créée dans un système de coordonnées géodésique, à partir de laquelle en utilisant des méthodes de filtrage connues pour l'identification de points de mesure qui sont éliminés comme des encombrements ou des pannes, on extrait des points de mesure potentiels des canalisations,
    dans une première étape, les points de mesure potentiels des canalisations étant projetés dans un plan horizontal, des tracés de droites étant identifiés et dans une deuxième étape, les points de mesure potentiels des canalisations étant projetés à et dans un écart prédéfinissable par rapport à une droite trouvée lors de la première étape, dans un plan vertical par rapport au plan horizontal et à la droite concernée issue de la première étape et à l'aide d'une transformation de forme quadratique, des tracés caténoïdes ou paraboliques étant identifiés, dans la première étape, la projection horizontale des points de mesure potentiels des canalisations s'effectuant sur une matrice NxM selon R = q 1 1 … q M 1 ⋮ q 1 N … q M N avec q i k ∈ 0 1 , dans laquelle i = 1 , .. M et k = 1 , .. , N
    Figure imgb0058
     et les tracés de la droite étant identifiés à l'aide d'une transformation de phase polynomiale et d'une analyse spectrale successive, lors de la transformation de phase polynomiale, les points de mesure potentiels des canalisations étant transformés par la matrice R en un signal de forme sinusoïdale z(k) de la forme z k = ∑ j = 1 d e i ⋅ µ 1 k ⋅ tan ϕ j + x j
    Figure imgb0059
     avec
    j : variable entre 1 et d
    d : nombre de la droite
    µ1 : paramètre constant compris entre 0,01 et 1
    xj : tronçon d'axe de la jème droite
    ϕj : angle de la jème droite
    et lors de l'analyse spectrale, les fréquences fj = µ1*tanϕj étant déterminées.
  2. Procédé selon la revendication 1, caractérisé en ce que pour aboutir à un faible taux de fausses alertes, on compare des tracés caténoïdes ou paraboliques trouvés avec des identifications antérieures dans le temps.
  3. Procédé selon la revendication 1 ou la revendication 2, caractérisé en ce qu'on analyse une image de distance accumulée à partir de plusieurs images de distance successives dans le temps.
  4. Procédé selon l'une quelconque des revendications précédentes, caractérisé en ce que si plusieurs canalisations ont été trouvées dans une image de distance, on élimine les tracés de canalisations non plausibles.
EP10002260.7A 2009-05-09 2010-03-05 Procédé de détection améliorée d'objets de type conduite Not-in-force EP2251849B1 (fr)

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
DE102009020636.1A DE102009020636B4 (de) 2009-05-09 2009-05-09 Verfahren zur verbesserten Erkennung von leitungsartigen Objekten

Publications (3)

Publication Number Publication Date
EP2251849A2 EP2251849A2 (fr) 2010-11-17
EP2251849A3 EP2251849A3 (fr) 2012-08-08
EP2251849B1 true EP2251849B1 (fr) 2016-10-12

Family

ID=42289768

Family Applications (1)

Application Number Title Priority Date Filing Date
EP10002260.7A Not-in-force EP2251849B1 (fr) 2009-05-09 2010-03-05 Procédé de détection améliorée d'objets de type conduite

Country Status (2)

Country Link
EP (1) EP2251849B1 (fr)
DE (1) DE102009020636B4 (fr)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104849708B (zh) * 2015-05-18 2017-03-08 中国民航大学 基于频域多项式相位变换的高速机动目标参数估计方法

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DE19605218C1 (de) 1996-02-13 1997-04-17 Dornier Gmbh Hinderniswarnsystem für tieffliegende Fluggeräte
DE19828318C2 (de) 1998-06-25 2001-02-22 Eurocopter Deutschland Drahthervorhebung
DE10055572C1 (de) 2000-11-09 2002-01-24 Astrium Gmbh Verfahren zur Leitungserkennung für tieffliegende Fluggeräte
FR2888944B1 (fr) * 2005-07-20 2007-10-12 Eurocopter France Procede de detection par telemetrie d'objets filaires suspendus
DE102005047273B4 (de) 2005-10-01 2008-01-03 Eads Deutschland Gmbh Verfahren zur Unterstützung von Tiefflügen

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
ABED-MERAIM K ET AL: "Multi-line fitting using polynomial phase transforms and downsampling", 2001 IEEE INTERNATIONAL CONFERENCE ON ACOUSTICS, SPEECH, AND SIGNAL PROCESSING. PROCEEDINGS. (ICASSP). SALT LAKE CITY, UT, MAY 7 - 11, 2001; [IEEE INTERNATIONAL CONFERENCE ON ACOUSTICS, SPEECH, AND SIGNAL PROCESSING (ICASSP)], NEW YORK, NY : IEEE, US, vol. 3, 7 May 2001 (2001-05-07), pages 1701 - 1704, XP010802866, ISBN: 978-0-7803-7041-8, DOI: 10.1109/ICASSP.2001.941266 *

Also Published As

Publication number Publication date
DE102009020636B4 (de) 2017-09-28
DE102009020636A1 (de) 2010-11-18
EP2251849A3 (fr) 2012-08-08
EP2251849A2 (fr) 2010-11-17

Similar Documents

Publication Publication Date Title
DE102015100134B4 (de) Verfahren zum Detektieren und Verfolgen von Objekten unter Verwendung mehrerer Radarsensoren
EP2418510B1 (fr) Procédé d'évaluation d'une surface au sol concernant son caractère approprié en tant que zone d'atterrissage ou surface de roulement pour aéronefs
DE102018102884A1 (de) Lidar-radar-relative pose-kalibrierung
EP1531343B1 (fr) Procédé de suivi d'objets
DE10307580B3 (de) Verfahren zum Bestimmen und Verfolgen von Position und Orientierung eines Magnetfeldsensors
EP2417476B1 (fr) Détecteur radar à onde entretenue modulée en fréquence, et procédé d'accord de fréquences
EP2927844A1 (fr) Estimation de position et de pose des objects 3d
EP1770595B1 (fr) Procédé d'assistance d'avions volant à basse altitude pour la détection d'obstacles
DE102015107388A1 (de) Kollisionsvermeidung mit statischen Zielobjekten in engen Räumen
DE102015115786B4 (de) An einem computer realisiertes verfahren und system zur verarbeitung einer sequenz von bildern
DE102017128194A1 (de) Genaue Selbstortung unter Verwendung eines Automobilradars mit synthetischer Apertur
DE102018132805A1 (de) Verfahren für eine verbesserte Objekterfassung
EP1205765B1 (fr) Méthode de reconnaissance des fils conducteurs pour aéronefs volants à basse altitude
EP1634241B1 (fr) Procede et dispositif de determination d'objet dans des systemes d'assistance au conducteur pour vehicules automobiles
DE102019114723A1 (de) Abschwächen von schwingungen in einem radarsystem auf einer beweglichen plattform
DE102021100351A1 (de) Adaptive untersuchung für lidar-basiertes gruppieren
DE102006039104A1 (de) Verfahren zur Entfernungsmessung von Objekten auf von Bilddaten eines Monokamerasystems
EP3663881B1 (fr) Procédé de commande d'un véhicule autonome en fonction des vecteurs de mouvement estimés
EP2251849B1 (fr) Procédé de détection améliorée d'objets de type conduite
DE102018001533B4 (de) Echtzeitverfahren zur dynamischen Koordinaten- und Parameterschätzung mit gitterbasierten nicht-linearen probabilistischen Filtern
DE102019126631A1 (de) Verbesserte Trajektorienschätzung basierend auf Ground Truth
DE102018121158A1 (de) Verfahren zum Erfassen von Bodenabtastpunkten und Fahrerunterstützungssystem, das dafür konfiguriert ist, ein derartiges Verfahren auszuführen
WO2022129266A1 (fr) Procédé de détection d'au moins un objet d'un environnement au moyen de signaux de réflexion d'un système capteur radar
DE102019215440B4 (de) Erkennung von Verkehrsschildern
EP2699933B1 (fr) Procédé et dispositif de détermination de paramètres d'une cible

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

AK Designated contracting states

Kind code of ref document: A2

Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO SE SI SK SM TR

AX Request for extension of the european patent

Extension state: AL BA ME RS

PUAL Search report despatched

Free format text: ORIGINAL CODE: 0009013

AK Designated contracting states

Kind code of ref document: A3

Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO SE SI SK SM TR

AX Request for extension of the european patent

Extension state: AL BA ME RS

RIC1 Information provided on ipc code assigned before grant

Ipc: G08G 5/00 20060101AFI20120629BHEP

17P Request for examination filed

Effective date: 20130201

RAP1 Party data changed (applicant data changed or rights of an application transferred)

Owner name: AIRBUS DEFENCE AND SPACE GMBH

17Q First examination report despatched

Effective date: 20151130

GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

RAP1 Party data changed (applicant data changed or rights of an application transferred)

Owner name: AIRBUS DS ELECTRONICS AND BORDER SECURITY GMBH

GRAS Grant fee paid

Free format text: ORIGINAL CODE: EPIDOSNIGR3

INTG Intention to grant announced

Effective date: 20160718

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

AK Designated contracting states

Kind code of ref document: B1

Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO SE SI SK SM TR

REG Reference to a national code

Ref country code: GB

Ref legal event code: FG4D

Free format text: NOT ENGLISH

REG Reference to a national code

Ref country code: CH

Ref legal event code: EP

REG Reference to a national code

Ref country code: AT

Ref legal event code: REF

Ref document number: 837129

Country of ref document: AT

Kind code of ref document: T

Effective date: 20161015

REG Reference to a national code

Ref country code: IE

Ref legal event code: FG4D

Free format text: LANGUAGE OF EP DOCUMENT: GERMAN

REG Reference to a national code

Ref country code: DE

Ref legal event code: R096

Ref document number: 502010012526

Country of ref document: DE

REG Reference to a national code

Ref country code: LT

Ref legal event code: MG4D

REG Reference to a national code

Ref country code: NL

Ref legal event code: MP

Effective date: 20161012

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LV

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20161012

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 8

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20161012

Ref country code: GR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170113

Ref country code: NO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170112

Ref country code: SE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20161012

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: HR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20161012

Ref country code: PL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20161012

Ref country code: IS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170212

Ref country code: NL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20161012

Ref country code: ES

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20161012

Ref country code: FI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20161012

Ref country code: PT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170213

REG Reference to a national code

Ref country code: DE

Ref legal event code: R097

Ref document number: 502010012526

Country of ref document: DE

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20161012

Ref country code: EE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20161012

Ref country code: RO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20161012

Ref country code: DK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20161012

Ref country code: CZ

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20161012

REG Reference to a national code

Ref country code: DE

Ref legal event code: R081

Ref document number: 502010012526

Country of ref document: DE

Owner name: HENSOLDT SENSORS GMBH, DE

Free format text: FORMER OWNER: AIRBUS DS ELECTRONICS AND BORDER SECURITY GMBH, 82024 TAUFKIRCHEN, DE

PLBE No opposition filed within time limit

Free format text: ORIGINAL CODE: 0009261

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: BG

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170112

Ref country code: SM

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20161012

26N No opposition filed

Effective date: 20170713

REG Reference to a national code

Ref country code: CH

Ref legal event code: PL

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20161012

Ref country code: MC

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20161012

REG Reference to a national code

Ref country code: IE

Ref legal event code: MM4A

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LU

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20170305

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: CH

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20170331

Ref country code: IE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20170305

Ref country code: LI

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20170331

REG Reference to a national code

Ref country code: BE

Ref legal event code: MM

Effective date: 20170331

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 9

REG Reference to a national code

Ref country code: AT

Ref legal event code: MM01

Ref document number: 837129

Country of ref document: AT

Kind code of ref document: T

Effective date: 20170305

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: BE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20170331

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: AT

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20170305

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20161012

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: HU

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT; INVALID AB INITIO

Effective date: 20100305

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: CY

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20161012

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20161012

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: TR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20161012

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: GB

Payment date: 20220324

Year of fee payment: 13

Ref country code: DE

Payment date: 20220322

Year of fee payment: 13

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: FR

Payment date: 20220323

Year of fee payment: 13

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: IT

Payment date: 20220331

Year of fee payment: 13

REG Reference to a national code

Ref country code: DE

Ref legal event code: R119

Ref document number: 502010012526

Country of ref document: DE

GBPC Gb: european patent ceased through non-payment of renewal fee

Effective date: 20230305

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: GB

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20230305

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: GB

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20230305

Ref country code: FR

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20230331

Ref country code: DE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20231003

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IT

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20230305