EP2805313B1 - Procédé et dispositif permettant de déterminer et de régler une zone à surveiller par une caméra vidéo - Google Patents

Procédé et dispositif permettant de déterminer et de régler une zone à surveiller par une caméra vidéo Download PDF

Info

Publication number
EP2805313B1
EP2805313B1 EP12809829.0A EP12809829A EP2805313B1 EP 2805313 B1 EP2805313 B1 EP 2805313B1 EP 12809829 A EP12809829 A EP 12809829A EP 2805313 B1 EP2805313 B1 EP 2805313B1
Authority
EP
European Patent Office
Prior art keywords
camera
area
monitored
georeferenced
visual
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
EP12809829.0A
Other languages
German (de)
English (en)
Other versions
EP2805313A1 (fr
Inventor
Michael Hoeynck
Alexander Flaig
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Robert Bosch GmbH
Original Assignee
Robert Bosch GmbH
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Robert Bosch GmbH filed Critical Robert Bosch GmbH
Publication of EP2805313A1 publication Critical patent/EP2805313A1/fr
Application granted granted Critical
Publication of EP2805313B1 publication Critical patent/EP2805313B1/fr
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G08—SIGNALLING
    • G08B—SIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
    • G08B13/00—Burglar, theft or intruder alarms
    • G08B13/18—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength
    • G08B13/189—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems
    • G08B13/194—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems using image scanning and comparing systems
    • G08B13/196—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems using image scanning and comparing systems using television cameras
    • G08B13/19639—Details of the system layout
    • G08B13/19652—Systems using zones in a single scene defined for different treatment, e.g. outer zone gives pre-alarm, inner zone gives alarm
    • G—PHYSICS
    • G08—SIGNALLING
    • G08B—SIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
    • G08B13/00—Burglar, theft or intruder alarms
    • G08B13/18—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength
    • G08B13/189—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems
    • G08B13/194—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems using image scanning and comparing systems
    • G08B13/196—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems using image scanning and comparing systems using television cameras
    • G08B13/19678—User interface
    • G08B13/19686—Interfaces masking personal details for privacy, e.g. blurring faces, vehicle license plates

Definitions

  • the invention relates to a method for determining and setting an area to be monitored by a video camera. Furthermore, the invention relates to a corresponding device and a corresponding camera arrangement.
  • Position refers to the translational coordinates of the mounting location or center of the image recording
  • orientation are the solid angle, pitch, roll and roll angle, which describes the inclination of the camera in the three spatial directions.
  • Position and orientation together describe the pose of the camera.
  • So-called video surveillance systems usually comprise several surveillance cameras, which are to be directed to surveillance areas.
  • Such video surveillance systems can be found, for example, in companies or companies, but also in public places, intersections, railway stations or other buildings.
  • the monitoring areas are to be secured and, if necessary, a tracking of suspicious objects is made possible.
  • a collection of personal data in public areas is subject to strict regulations.
  • surveillance may be carried out under certain conditions if it does not extend to public space, such as a street or a square.
  • public space such as a street or a square.
  • the privacy of the person who is in the image area which is not primarily to be monitored but captured by the camera, for example, a street is protected.
  • Such a function is referred to in today's camera systems as a "Privacy Mask”.
  • a Primary Mask When installing the camera especially in static and dynamic (pan-tilt-zoom) cameras are under control of the corresponding Video image certain areas often selected in the form of polygons, which are then grayed out or rendered illegally in the transmitted and possibly recorded or displayed video image.
  • the masking module for a monitoring system, wherein the monitoring system comprises at least one monitoring camera and is suitable for the observation of monitoring areas with moving objects and arranged. Furthermore, the masking module comprises a selection device for selecting objects, wherein the masking module is designed to output the selection objects masked, which is limited to at least one selected spatial subregion of the monitoring region.
  • a camera surveillance system that includes a camera and a computing unit coupled thereto that is configured to obscure some areas of an image captured by the camera with a privacy mask.
  • a method for determining and setting an area to be monitored by a video camera.
  • a visible area resulting during installation and calibration of the camera and captured by the camera is thereby tailored to the area to be monitored so that the position and orientation of the camera are described with georeferenced coordinates.
  • the georeferenced coordinates of a georeferenced map of the area to be monitored are taken.
  • a mask is automatically determined, which masks on the view area contents of the field of view outside the area to be monitored.
  • Position and orientation of the camera are defined, for example, by a corresponding 3D camera pose.
  • a pose corresponds to a combination of position and orientation of the camera in three-dimensional space.
  • a programming interface API, Application Programming Interface
  • Google Map API® it is conceivable to use Google Map API®.
  • the mask is calculated on the basis of geometrical assumptions calculated visual beams of the camera. With the help of the mask, the contents of the field of view outside the area to be monitored are grayed out and / or removed and / or hidden.
  • the setting of the area to be monitored by the video camera is carried out fully automatically after an initial installation and calibration.
  • the present invention relates to a device for a camera arrangement with at least one camera, wherein the device has at least one module which has access to a georeferenced map and is configured to describe the position and orientation of the camera by means of georeferenced coordinates.
  • the device further comprises a computing unit which is configured to automatically determine a mask based on the georeferenced coordinates, which masks, when imaging on a field of view detected by the camera, contents of the field of view outside a defined area to be monitored.
  • the present invention relates to the use of a method according to the invention and / or a camera arrangement according to the invention for or in a video surveillance system.
  • a schematic representation of a camera arrangement 1 comprises a camera 2 having a first module 4 for recording and digitizing images of an area to be monitored by the camera 2. Another module 6 is used to evaluate images of the camera and to detect moving objects. Furthermore, the camera 2 has a further module 8, which via a wireless or wired interface 10 has a connection to an external memory or to the Internet 11, via which the module 8 has access to a georeferenced map.
  • the georeferenced card can therefore either be stored in an external memory or, for example, be retrievable from an external server via the Internet 11.
  • the interface can be realized via a so-called Transmission Control Protocol Internet Protocol (TCP-IP). However, other network protocols, such as bpsw. UDP, are used.
  • the access to the geo-referenced map makes it possible to describe the position and orientation of the camera 2 by means of georeferenced coordinates.
  • Google-Map API® can be used as a georeferenced map, for example.
  • the description of position and orientation The camera 2 by means of such georeferenced coordinates is unique worldwide and thus allows a clearly defined statement about the position and orientation of the camera. 2
  • the area to be monitored as well as the field of view actually detected by the camera 2 can be determined relatively accurately by means of the georeferenced map so that a mask can be automatically determined by means of a further module 12 on the basis of the geo-referenced coordinates the camera 2 detected area masked contents of the field of view outside a defined area to be monitored.
  • the present embodiment of the device according to the invention is thus arranged and integrated in the camera 2 and has at least the modules 8 and 12.
  • the field of view captured by the camera 2 is privacy-preserving using the position and orientation of the camera, i. H. a 3D camera pose, and using georeferenced maps to actively restrict to the defined area to be monitored, so that content outside the defined area to be monitored can be masked, for example, hidden or grayed out.
  • the position and orientation of the camera is not, as before, on a building or land on which it is installed, even mapped to geographic coordinates.
  • the area to be monitored such as a defined plot or an acceptable surveillance area beyond this, can be defined, for example, as far as the street or one meter into the street.
  • the area detected by the camera can then be grayed out or removed automatically from the area to be monitored. Such an operation can be performed automatically after an initial installation and calibration, and image data can thereby be actively suppressed from the outset.
  • a calibration of the camera is expanded with the aid of the device according to the invention taking into account the georeferenced data mentioned.
  • the defined area to be monitored is already to be marked on a digital, geo-referenced map, so that all contents outside of this area, but adjacent to it, are automatically removed from the field of vision covered by the camera, whether in the form of a graying out or a perfect one Distance.
  • the device according to the invention does not have to, as in FIG. 1 shown, inevitably be integrated into a housing of the camera 2.
  • FIG. 2 An example of a further embodiment of a device according to the invention is shown in FIG FIG. 2 shown.
  • a camera arrangement 15 comprises a first camera 20 and a second camera 30, which are designed to observe a region to be monitored.
  • the camera arrangement 15 comprises a device 25 with at least one module 26, which is designed to determine the respective position and orientation of the first camera 20 and the second camera 30.
  • the cameras 20 and 30 are respectively described with respect to their position and orientation by means of georeferenced coordinates, so that by means of a further computing unit 27 based on the georeferenced coordinates, if necessary, a common for the cameras 20 and 30 mask can be determined automatically when mapping to a
  • the viewing area captured by the two cameras conceals contents of the viewing area outside of a defined area to be monitored.
  • Communication for exchanging information takes place between the first camera 20 and the device 25 via a line 40.
  • the device 25 is connected to the second camera 30 via a further line 41.
  • the device 25 of this embodiment establishes a connection 42 to the Internet 28 in order to be able to provide information about it on a correspondingly stored georeferenced map.

Landscapes

  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Engineering & Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • Closed-Circuit Television Systems (AREA)
  • Studio Devices (AREA)
  • Alarm Systems (AREA)

Claims (5)

  1. Procédé de détermination et de réglage d'une zone à surveiller par une caméra vidéo, un champ de vision résultant lors de l'installation et de l'étalonnage de la caméra et balayé par la caméra étant adapté en fonction de la zone à surveiller, l'adaptation du champ de vision balayé en fonction de la zone à surveiller étant effectuée en décrivant la position et l'orientation de la caméra avec des coordonnées géoréférencées qui sont relevées sur une carte géoréférencée de la zone à surveiller, et un masque étant défini automatiquement à l'aide des coordonnées géoréférencées qui, lorsqu'il est représenté sur le champ de vision, masque les contenus du champ de vision en-dehors de la zone à surveiller, caractérisé en ce que
    le masque est calculé sur la base de rayons de vision de la caméra calculés en s'appuyant sur des hypothèses géométriques, la caméra possédant un module par le biais duquel est réalisé, par le biais d'une interface de programmation par le biais de l'Internet, un accès à une mémoire externe qui contient des données de la carte géoréférencée, la zone à surveiller étant déjà marquée sur la carte géoréférencée.
  2. Procédé selon la revendication 1, selon lequel la position et l'orientation de la caméra sont définies par la position et l'orientation de la caméra dans l'espace tridimensionnel.
  3. Procédé selon l'une des revendications 1 ou 2, selon lequel les contenus du champ de vision en-dehors de la zone à surveiller sont grisés et/ou supprimés et/ou masqués à l'aide du masque.
  4. Dispositif pour un arrangement de caméra, notamment pour mettre en oeuvre le procédé selon l'une des revendications 1 à 3, le dispositif possédant au moins un module qui dispose d'un accès à une carte géoréférencée et qui est configuré pour décrire la position et l'orientation de la caméra au moyen de coordonnées géoréférencées, le dispositif possédant en outre une unité de calcul qui est configurée pour adapter un champ de vision résultant lors de l'installation et de l'étalonnage de la caméra et balayé par la caméra en fonction de la zone à surveiller, et pour définir automatiquement un masque à l'aide des coordonnées géoréférencées qui, lorsqu'il est représenté sur un champ de vision balayé par la caméra, masque les contenus du champ de vision en-dehors d'une zone à surveiller définie,
    caractérisé en ce que
    l'unité de calcul est configurée pour calculer le masque sur la base de rayons de vision de la caméra calculés en s'appuyant sur des hypothèses géométriques, le module servant à accéder, par le biais d'une interface de programmation par le biais de l'Internet, à une mémoire externe qui contient des données de la carte géoréférencée, la zone à surveiller étant déjà marquée sur la carte géoréférencée.
  5. Arrangement de caméra comprenant une caméra et un dispositif selon la revendication 4.
EP12809829.0A 2012-01-17 2012-12-28 Procédé et dispositif permettant de déterminer et de régler une zone à surveiller par une caméra vidéo Active EP2805313B1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
DE102012200573A DE102012200573A1 (de) 2012-01-17 2012-01-17 Verfahren und Einrichtung zum Bestimmen und Einstellen eines durch eine Videokamera zu überwachenden Bereichs
PCT/EP2012/077011 WO2013107606A1 (fr) 2012-01-17 2012-12-28 Procédé et dispositif permettant de déterminer et de régler une zone à surveiller par une caméra vidéo

Publications (2)

Publication Number Publication Date
EP2805313A1 EP2805313A1 (fr) 2014-11-26
EP2805313B1 true EP2805313B1 (fr) 2018-12-26

Family

ID=47501277

Family Applications (1)

Application Number Title Priority Date Filing Date
EP12809829.0A Active EP2805313B1 (fr) 2012-01-17 2012-12-28 Procédé et dispositif permettant de déterminer et de régler une zone à surveiller par une caméra vidéo

Country Status (3)

Country Link
EP (1) EP2805313B1 (fr)
DE (1) DE102012200573A1 (fr)
WO (1) WO2013107606A1 (fr)

Family Cites Families (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6509926B1 (en) * 2000-02-17 2003-01-21 Sensormatic Electronics Corporation Surveillance apparatus for camera surveillance system
GB2404247B (en) * 2003-07-22 2005-07-20 Hitachi Int Electric Inc Object tracing method and object tracking apparatus
US8212872B2 (en) * 2004-06-02 2012-07-03 Robert Bosch Gmbh Transformable privacy mask for video camera images
DE102006042318B4 (de) 2006-09-08 2018-10-11 Robert Bosch Gmbh Verfahren zum Betreiben mindestens einer Kamera
US20080074494A1 (en) * 2006-09-26 2008-03-27 Harris Corporation Video Surveillance System Providing Tracking of a Moving Object in a Geospatial Model and Related Methods
DE102007029476A1 (de) 2007-06-26 2009-01-08 Robert Bosch Gmbh Bildverarbeitungsvorrichtung zur Schattendetektion bzw. -unterdrückung, Verfahren und Computerprogramm
US8098282B2 (en) * 2007-07-13 2012-01-17 Honeywell International Inc. Privacy zone algorithm for ptz dome cameras
DE102008007199A1 (de) 2008-02-01 2009-08-06 Robert Bosch Gmbh Maskierungsmodul für ein Videoüberwachungssystem, Verfahren zur Maskierung von selektierten Objekten sowie Computerprogramm

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
None *

Also Published As

Publication number Publication date
WO2013107606A1 (fr) 2013-07-25
DE102012200573A1 (de) 2013-07-18
EP2805313A1 (fr) 2014-11-26

Similar Documents

Publication Publication Date Title
DE102005063217B4 (de) Verfahren zum Konfigurieren einer Überwachungseinrichtung zum Überwachen eines Raumbereichsund entsprechende Überwachungseinrichtung
EP1454285B1 (fr) Systeme de surveillance video avec masquage d'objets
DE102005021735B4 (de) Videoüberwachungssystem
DE102010038341B4 (de) Videoüberwachungssystem sowie Verfahren zur Konfiguration eines Videoüberwachungssystems
WO2009095106A1 (fr) Module de masquage pour un système de vidéosurveillance, procédé de masquage d'objets sélectionnés et programme informatique
EP2104904B1 (fr) Procédé et dispositif pour surveiller un volume spatial, et procédé d'étalonnage
DE102008039130A1 (de) Durch ein neurales Netzwerk gesteuertes automatisches Verfolgungs- und Erkennungssystem und Verfahren
DE102007053812A1 (de) Konfigurationsmodul für ein Videoüberwachungssystem, Überwachungssystem mit dem Konfigurationsmodul, Verfahren zur Konfiguration eines Videoüberwachungssystems sowie Computerprogramm
DE102006042318B4 (de) Verfahren zum Betreiben mindestens einer Kamera
WO2015028294A1 (fr) Installation de surveillance et procédé de visualisation d'une zone surveillée
DE102012211298A1 (de) Anzeigevorrichtung für ein Videoüberwachungssystem sowie Videoüberwachungssystem mit der Anzeigevorrichtung
DE10151983A1 (de) Verfahren zur Dokumentation einer Unfallsituation
EP3236440B1 (fr) Dispositif, systeme et procede de surveillance de pente et/ou de batiment sans marquage
EP3021256A1 (fr) Procede de traitement d'images, detecteur de presence et systeme d'eclairage
EP2805313B1 (fr) Procédé et dispositif permettant de déterminer et de régler une zone à surveiller par une caméra vidéo
DE10049366A1 (de) Verfahren zum Überwachen eines Sicherheitsbereichs und entsprechendes System
WO2017144033A1 (fr) Procédé de détermination et de visualisation de changements dans un environnement réel comportant un terrain réel et des objets réels se trouvant sur ce terrain
DE102009000810A1 (de) Vorrichtung zur Segmentierung eines Objektes in einem Bild, Videoüberwachungssystem, Verfahren und Computerprogramm
EP3542528B1 (fr) Dispositif d'affichage pour une installation de surveillance d'une zone de surveillance
LU102252B1 (de) Computerimplementiertes Verfahren beim Bestimmen eines Beschattungszustands eines Objekts
DE102007056835A1 (de) Bildverarbeitunsmodul zur Schätzung einer Objektposition eines Überwachungsobjekts, Verfahren zur Bestimmung einer Objektposition eines Überwachungsobjekts sowie Computerprogramm
DE102014223433A1 (de) Dynamische Maskierung von Videoaufnahmen
DE102017216372A1 (de) Verfahren zur Ermittlung von Arbeitsschwerpunkten und/oder Gefahrenstellen auf einer Baustelle
EP3489842A1 (fr) Base de données forensique
DE10214306A1 (de) Verfahren zur Überwachung von Datenbewegungen

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

17P Request for examination filed

Effective date: 20140818

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

DAX Request for extension of the european patent (deleted)
17Q First examination report despatched

Effective date: 20161026

GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

INTG Intention to grant announced

Effective date: 20180730

GRAS Grant fee paid

Free format text: ORIGINAL CODE: EPIDOSNIGR3

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

AK Designated contracting states

Kind code of ref document: B1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

REG Reference to a national code

Ref country code: GB

Ref legal event code: FG4D

Free format text: NOT ENGLISH

REG Reference to a national code

Ref country code: CH

Ref legal event code: EP

REG Reference to a national code

Ref country code: AT

Ref legal event code: REF

Ref document number: 1082494

Country of ref document: AT

Kind code of ref document: T

Effective date: 20190115

REG Reference to a national code

Ref country code: DE

Ref legal event code: R096

Ref document number: 502012014067

Country of ref document: DE

REG Reference to a national code

Ref country code: IE

Ref legal event code: FG4D

Free format text: LANGUAGE OF EP DOCUMENT: GERMAN

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: NO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190326

Ref country code: FI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20181226

Ref country code: BG

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190326

Ref country code: LT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20181226

Ref country code: HR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20181226

Ref country code: LV

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20181226

REG Reference to a national code

Ref country code: NL

Ref legal event code: MP

Effective date: 20181226

REG Reference to a national code

Ref country code: LT

Ref legal event code: MG4D

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20181226

Ref country code: RS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20181226

Ref country code: AL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20181226

Ref country code: GR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190327

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: NL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20181226

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: ES

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20181226

Ref country code: PL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20181226

Ref country code: IT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20181226

Ref country code: CZ

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20181226

Ref country code: PT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190426

REG Reference to a national code

Ref country code: CH

Ref legal event code: PL

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20181226

Ref country code: IS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190426

Ref country code: RO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20181226

Ref country code: EE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20181226

Ref country code: SM

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20181226

Ref country code: LU

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20181228

REG Reference to a national code

Ref country code: IE

Ref legal event code: MM4A

REG Reference to a national code

Ref country code: DE

Ref legal event code: R097

Ref document number: 502012014067

Country of ref document: DE

REG Reference to a national code

Ref country code: BE

Ref legal event code: MM

Effective date: 20181231

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: DK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20181226

Ref country code: IE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20181228

Ref country code: MC

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20181226

PLBE No opposition filed within time limit

Free format text: ORIGINAL CODE: 0009261

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: BE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20181231

26N No opposition filed

Effective date: 20190927

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LI

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20181231

Ref country code: CH

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20181231

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20181226

REG Reference to a national code

Ref country code: AT

Ref legal event code: MM01

Ref document number: 1082494

Country of ref document: AT

Kind code of ref document: T

Effective date: 20181228

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20181226

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: TR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20181226

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: AT

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20181228

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MK

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20181226

Ref country code: HU

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT; INVALID AB INITIO

Effective date: 20121228

Ref country code: CY

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20181226

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: GB

Payment date: 20231220

Year of fee payment: 12

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: FR

Payment date: 20231219

Year of fee payment: 12

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: DE

Payment date: 20240227

Year of fee payment: 12