EP2681722A1 - Method and system for detecting a fall and issuing an alarm - Google Patents
Method and system for detecting a fall and issuing an alarmInfo
- Publication number
- EP2681722A1 EP2681722A1 EP12712975.7A EP12712975A EP2681722A1 EP 2681722 A1 EP2681722 A1 EP 2681722A1 EP 12712975 A EP12712975 A EP 12712975A EP 2681722 A1 EP2681722 A1 EP 2681722A1
- Authority
- EP
- European Patent Office
- Prior art keywords
- alarm
- fall
- person
- monitored
- data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 29
- 230000003287 optical effect Effects 0.000 claims abstract description 34
- 238000012544 monitoring process Methods 0.000 claims abstract description 9
- 238000001514 detection method Methods 0.000 claims description 36
- 230000005540 biological transmission Effects 0.000 claims description 8
- 230000001960 triggered effect Effects 0.000 claims description 7
- 238000011156 evaluation Methods 0.000 claims description 5
- 230000003068 static effect Effects 0.000 claims description 3
- 230000001133 acceleration Effects 0.000 claims description 2
- 230000003213 activating effect Effects 0.000 claims 2
- 230000004913 activation Effects 0.000 claims 1
- 238000007726 management method Methods 0.000 claims 1
- 230000001131 transforming effect Effects 0.000 claims 1
- 238000004422 calculation algorithm Methods 0.000 description 11
- 230000036541 health Effects 0.000 description 8
- 238000012545 processing Methods 0.000 description 7
- 238000004458 analytical method Methods 0.000 description 6
- 238000004891 communication Methods 0.000 description 3
- 238000010191 image analysis Methods 0.000 description 3
- 241001465754 Metazoa Species 0.000 description 2
- 230000002457 bidirectional effect Effects 0.000 description 2
- 230000002159 abnormal effect Effects 0.000 description 1
- 230000009471 action Effects 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000008901 benefit Effects 0.000 description 1
- 238000007635 classification algorithm Methods 0.000 description 1
- 230000000295 complement effect Effects 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000009429 distress Effects 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
- 239000000203 mixture Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003449 preventive effect Effects 0.000 description 1
- 238000000513 principal component analysis Methods 0.000 description 1
- 238000001454 recorded image Methods 0.000 description 1
- 230000002459 sustained effect Effects 0.000 description 1
- 238000004804 winding Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G08—SIGNALLING
- G08B—SIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
- G08B21/00—Alarms responsive to a single specified undesired or abnormal condition and not otherwise provided for
- G08B21/02—Alarms for ensuring the safety of persons
- G08B21/04—Alarms for ensuring the safety of persons responsive to non-activity, e.g. of elderly persons
-
- G—PHYSICS
- G08—SIGNALLING
- G08B—SIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
- G08B21/00—Alarms responsive to a single specified undesired or abnormal condition and not otherwise provided for
- G08B21/02—Alarms for ensuring the safety of persons
- G08B21/04—Alarms for ensuring the safety of persons responsive to non-activity, e.g. of elderly persons
- G08B21/0407—Alarms for ensuring the safety of persons responsive to non-activity, e.g. of elderly persons based on behaviour analysis
- G08B21/043—Alarms for ensuring the safety of persons responsive to non-activity, e.g. of elderly persons based on behaviour analysis detecting an emergency event, e.g. a fall
-
- G—PHYSICS
- G08—SIGNALLING
- G08B—SIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
- G08B21/00—Alarms responsive to a single specified undesired or abnormal condition and not otherwise provided for
- G08B21/02—Alarms for ensuring the safety of persons
- G08B21/04—Alarms for ensuring the safety of persons responsive to non-activity, e.g. of elderly persons
- G08B21/0438—Sensor means for detecting
- G08B21/0476—Cameras to detect unsafe condition, e.g. video cameras
-
- G—PHYSICS
- G08—SIGNALLING
- G08B—SIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
- G08B13/00—Burglar, theft or intruder alarms
- G08B13/18—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength
- G08B13/189—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems
- G08B13/194—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems using image scanning and comparing systems
- G08B13/196—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems using image scanning and comparing systems using television cameras
- G08B13/19678—User interface
- G08B13/19686—Interfaces masking personal details for privacy, e.g. blurring faces, vehicle license plates
Definitions
- the claimed invention relates to a method and system for fall detection, detection of particular emergencies and transmission of an alarm in the home environment. More particularly, the invention relates to a method and system for fall detection by means of a passive optical sensor and passing a multi-level alarm in the form of alienated image data.
- US Pat. No. 7,248,172 B2 has proposed a method and system for fall detection, which includes that the person to be monitored carries a device for detecting falls or for triggering an alarm on the body. This alarm can be transmitted in the form of a radio transmission to a rescue center.
- Another class of methods for fall detection and detection of special emergencies are optically based methods.
- a permanent optical monitoring of the person is performed as well as an optical fall detection / emergency detection, the classification using common classification algorithms such as Bayes classifiers, neural networks or principal component analysis of a recorded Event and triggers an alarm upon detection of a fall / distress and sends it to a higher-level instance.
- US20060145874A1 relates to an optically based method, wherein the image processing algorithms in particular determine the percentage of a person lying on the ground, the position of the person, as well as the size of the person. In addition, the speed and acceleration of a falling person is determined.
- US20030058341A1 relates to an optically based method in which an object is released from a scene, this is detected and analyzed. In doing so, certain properties of the object are analyzed on the basis of comparison values with specific events.
- an event classification using geometric primitives is possible, as described in Chia-Wen Lin, Zhi-Hong Ling, Yeng-Cheng Chang and Chung J. Kuo "Compressed-Domain Case Incident Detection for Intelligent Homecare", submitted to Journal of VLSI Signal Processing.
- US 20050146605 A1 relates to an optical monitoring method with geometric primitives and subsequent event analysis using event discriminators, whereby an alarm can be triggered depending on the detected event. Procedures that involve permanent video surveillance of a person, however, interfere with the personal rights of the person and can therefore lead to acceptance problems. Offenbarune of the invention
- a passive optical sensor such as a video or. Infrared camera monitors. Furthermore, an evaluation of the recorded images with respect to the detection of a fall or a special emergency situation, but the image data are initially transmitted only alienated to an authorized receiver triggering an alarm upon detection of a fall. In this way, the right to one's own image of the monitored person is preserved, since not permanently images of this person, but only alienated images are transmitted upon detection of a fall.
- a return channel is first activated, which serves to verify the detected situation.
- the supervised person is enabled to classify a triggered alarm as a false alarm.
- the alienation of the image data is preferably carried out by means of geometric primitives.
- geometric primitives For example, those present in a captured image Each replaced by one or more geometric primitives.
- each person present in the captured image is treated and displayed as at least one geometric primitive.
- the fall detection is determined on the basis of the geometric primitive or for each person.
- the authorized recipient may initiate further rescue measures, such as request rescue services or instruct medical personnel to take care of the fallen person.
- the essence of the invention lies in a method and system for detecting a particular emergency especially a fall detection, wherein initially alienated and possibly encrypted image data of this person and / or the room to an authorized during the permanent optical monitoring of a person only in emergency or fall detection Instance are triggered by triggering an alarm. Only after a return channel has been formed and if the monitored person fails to classify the alarm as a false alarm will an unaltered image of the situation or of the person be transmitted to an authorized entity.
- This method and system has the advantage that it does not restrict the physical freedom of action of a person to be monitored, for which the person is comfortable and the person's right to his own image as an outflow of the general personality rights and also inexpensive and easy to handle for medical personnel and / or relatives is.
- FIG. 1 shows a system for detecting a special emergency or fall detection with a server and a universal standard gateway and an array of optical sensors
- Figure 2 shows a system for detecting a particular emergency or fall detection with a health box and a health platform and an array of optical sensors
- Figure 3 shows a system for detecting a special emergency or fall detection with a LogicHealth box and a health platform wherein the algorithms for video analysis and fall detection are implemented in an optical sensor.
- Figure 1 shows an embodiment of the system.
- images of a person 118 to be monitored are permanently recorded by means of a number of optical sensors 11 which have individual camera sensors 11.1 to 11.1.
- the camera sensors 11.1 to 11.n can be arranged in a space to be monitored such that the person to be monitored is detected by at least one camera sensor at all times.
- the optical sensors 11 are connected to a server 13 via an interface 19.
- a logic 12 in which the algorithms for decomposing the images recorded by means of the sensors 11 into geometric primitives as well as algorithms for fall detection or for detecting a special emergency situation are implemented.
- an alienated image which by means of a device and method for encryption and authentication, which may include a firewall 15, via a TCP / IP interface 17 and the Internet 113 to an authorized circle of people 117 if necessary ., transmitted via a universal standard gateway 114, wherein the transmission can be wireless 115 or wired 116 bidirectional.
- an alarm is triggered which the monitored person 118 can invalidate (classify as a false alarm).
- a return channel is established via a data or telephone connection between the monitored room and the authorized receiver. If, after a certain period of time, an alarm is not classified as a false alarm, then an unaltered transmission of the picture or a sequence of pictures to the authorized group of people takes place.
- the wired 116 or wireless 115 transmission is also encrypted. Another possibility for forming a return channel is additional given by an acoustic device 14 and a sound card integrated into the system 18.
- the server 13 which is executed in this embodiment as a PC with additional logic, more sensors 112 or user menus 111 may be connected via the interface 19.
- the logic 12 in the embodiment of Figure 1 includes the software for performing video image analysis, crash detection analysis and encryption, and data communication.
- FIG. 2 shows a further embodiment.
- the server 13 of Figure 1 is implemented as a so-called Healthbox 26, i. a device for monitoring the health of a person.
- the image processing and the data processing of all health sensors and the data encryption takes place in a Logic HealthBox 25, which is embedded in the Healthbox 26.
- the authentication and routing of the information takes place in an external HealthPlatform 27, which can be a system for monitoring a large number of persons.
- HealthBox 26 and HealthPlatform 27 are interconnected via the Internet 113.
- the forwarding of the alarm or the alienated and unrecognized in a further stage images is encrypted either wirelessly 115, for example.
- smartphones or wired 116 for example, to a telephone or external computer to an authorized circle 117 people.
- the optical sensors 11.1 to 1 ln are over an interface 19 connected to the Healthbox 26.
- the camera sensors only record the pictures.
- the server of the first embodiment of FIG. 1 with the further devices is implemented in the form of the healthbox 26 in this second embodiment.
- the health platform 27 of this second embodiment also has the functionality of the Universal Standard Gateway 114 of the first embodiment. As for the other elements of this embodiment, reference is made to FIG.
- the logic HealthBox 25 in the embodiment of Figure 2 includes the software for performing the video image analysis, the fall detection analysis, the Data processing of all sensors, especially health sensors and encryption as well as data communication.
- FIG. 3 shows a third embodiment of the system.
- the optical sensors are in the form of a master 30 and a plurality of slaves 41, 42.
- the optical sensor operating as master 30 has a camera part 31, an internal interface 32, a smart camera logic 33 and an interface 34.
- the software and the associated evaluation unit which implements the algorithms for video analysis and emergency detection, are integrated in the SmartKamera logic 33 of the master optical sensor.
- the smart camera logic 33 is internally connected to the camera part 31 via the interface 32.
- the SmartKamera logic 33 is further connected via the interface 34 with the other operating as a slave optical sensors 41, 42 of the system and the interface 19 with the Healthbox 26.
- the smart camera logic 33 in the embodiment of Figure 3 includes the software for performing the video image analysis and the fall detection analysis.
- the logic HealthBox 35 has the remaining functions contained in the logic HealthBox 25 of the embodiment of FIG. This includes the software for the data processing of all sensors, in particular health sensors and the encryption as well as the data communication. The invention will be explained in more detail below with reference to functions and properties.
- the optical sensors are preferably to be placed in the room in such a way that optimum coverage of the space to be monitored, even in winding rooms, is achieved.
- a space area is to be covered by more than one optical sensor.
- an active energy management is implemented in this case, so that only optical sensors are in active mode that are currently required.
- the optical sensors are designed such that they are robust to variable light conditions such as reflections and backlight. In addition, these can be designed in the form of infrared sensors to detect falls and emergencies even in the dark can.
- the fall detection algorithms are implemented in such a way that they can detect any types of crashes, such as collapse, heavy fall, and sustained fall. These types of falls are well defined in advance and are part of a library of activated fall detection algorithms ("patterns.") Emergencies are identified primarily by algorithms that evaluate the static state of a person relative to the home environment and, in particular, detect an abnormal static condition ,
- the recognition algorithms are implemented in such a way that they are robust with respect to other persons or animals in the room as well as with respect to the positions of the optical sensors and obscuring the person to be monitored by furniture and / or other persons or animals. Likewise, they are able to classify rapid movements of objects not as collapses.
- the recognition algorithms include methods of extracting a person from the background and vectorizing and decomposing the image of a person into geometric primitives and determining the location of them in space either in the server or in the optical sensor. Fall detection uses the dynamics of a fall. It analyzes how fast the position of these geometric primitives changes within a sequence of images.
- the image processing can be done centrally in a dedicated Healthbox and be connected to a computer cloud to minimize maintenance and to easily expand the evaluation patterns.
- the system is designed so that within a certain period all persons of an authorized group of persons are informed. This authorized group of persons only has access to personal data and status information about the person to be monitored.
- the system is designed in such a way that the authorized group of persons can be supplemented by additional persons or reduced by certain persons.
- the system will provide the authorized circle of people with the time of the fall, the place of the fall, a still image and possibly a video.
- the system displays at various places when video or video material leaves the room to be monitored.
- the recording of image or video data of the person to be monitored preferably takes place only for a period which is absolutely necessary for the functionality of the fall detection system.
- the security level of the telecommunication is preferably scalable, so that between different security levels of the system from weakly protected (low end security level) to strongly protected (high end security level) can be selected.
- the system may additionally contain suitable administration, billing, routing and customer administration functionalities on the platform.
- the system is preferably designed so that it can be easily maintained from the outside by a device for the automatic upgrade of firmware and drivers as well as the algorithms.
- the system is preferably designed to be scalable so that additional optical and non-optical sensors can be added or removed to the system to complement the functional properties and performance of the system.
Landscapes
- Health & Medical Sciences (AREA)
- General Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- Gerontology & Geriatric Medicine (AREA)
- Business, Economics & Management (AREA)
- Emergency Management (AREA)
- Physics & Mathematics (AREA)
- Psychology (AREA)
- Social Psychology (AREA)
- Psychiatry (AREA)
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Alarm Systems (AREA)
Abstract
Description
Claims
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
EP12712975.7A EP2681722B1 (en) | 2011-03-04 | 2012-02-29 | Method and system for detecting a fall and issuing an alarm |
Applications Claiming Priority (4)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US201161449131P | 2011-03-04 | 2011-03-04 | |
EP11156968 | 2011-03-04 | ||
EP12712975.7A EP2681722B1 (en) | 2011-03-04 | 2012-02-29 | Method and system for detecting a fall and issuing an alarm |
PCT/EP2012/053443 WO2012119903A1 (en) | 2011-03-04 | 2012-02-29 | Method and system for detecting a fall and issuing an alarm |
Publications (2)
Publication Number | Publication Date |
---|---|
EP2681722A1 true EP2681722A1 (en) | 2014-01-08 |
EP2681722B1 EP2681722B1 (en) | 2017-12-20 |
Family
ID=44455253
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
EP12712975.7A Active EP2681722B1 (en) | 2011-03-04 | 2012-02-29 | Method and system for detecting a fall and issuing an alarm |
Country Status (3)
Country | Link |
---|---|
EP (1) | EP2681722B1 (en) |
DK (1) | DK2681722T3 (en) |
WO (1) | WO2012119903A1 (en) |
Families Citing this family (8)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
EP3025316B1 (en) * | 2012-10-29 | 2024-08-28 | Markus Schori | Security system having integrated emergency calling function |
BE1021528B1 (en) | 2013-02-01 | 2015-12-08 | Familyeye Bvba | FALL DETECTION SYSTEM AND METHOD FOR DETECTING A FALL OF A MONITORED PERSON |
CN107749143B (en) * | 2017-10-30 | 2023-09-19 | 安徽工业大学 | WiFi signal-based system and method for detecting falling of personnel in through-wall room |
CN111753587B (en) * | 2019-03-28 | 2023-09-29 | 杭州海康威视数字技术股份有限公司 | Ground falling detection method and device |
CN111104932A (en) * | 2020-02-03 | 2020-05-05 | 北京都是科技有限公司 | Tumble detection system and method and image processor |
CN114067436B (en) * | 2021-11-17 | 2024-03-05 | 山东大学 | Fall detection method and system based on wearable sensor and video monitoring |
FR3136094A1 (en) * | 2022-05-25 | 2023-12-01 | Inetum | Fall detection method by image analysis |
CN117116004A (en) * | 2023-10-25 | 2023-11-24 | 深圳市微克科技有限公司 | Tumble alarm method, system and medium of intelligent wearable device |
Family Cites Families (14)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
SE517900C2 (en) * | 1999-12-23 | 2002-07-30 | Wespot Ab | Methods, monitoring system and monitoring unit for monitoring a monitoring site |
US20050146605A1 (en) | 2000-10-24 | 2005-07-07 | Lipton Alan J. | Video surveillance system employing video primitives |
US9892606B2 (en) * | 2001-11-15 | 2018-02-13 | Avigilon Fortress Corporation | Video surveillance system employing video primitives |
US7110569B2 (en) | 2001-09-27 | 2006-09-19 | Koninklijke Philips Electronics N.V. | Video based detection of fall-down and other events |
SE0203483D0 (en) | 2002-11-21 | 2002-11-21 | Wespot Ab | Method and device for fall detection |
WO2004062496A1 (en) * | 2003-01-10 | 2004-07-29 | Honeywell International Inc. | System and method for automatically generating an alert message with supplemental information |
JP4789511B2 (en) * | 2004-06-04 | 2011-10-12 | キヤノン株式会社 | Status monitoring device and status monitoring system |
US7248172B2 (en) | 2005-03-22 | 2007-07-24 | Freescale Semiconductor, Inc. | System and method for human body fall detection |
US7613324B2 (en) * | 2005-06-24 | 2009-11-03 | ObjectVideo, Inc | Detection of change in posture in video |
US7714728B2 (en) * | 2006-01-07 | 2010-05-11 | Gt Angel, Llc | Using RFID to prevent or detect falls, wandering, bed egress and medication errors |
TW200745996A (en) * | 2006-05-24 | 2007-12-16 | Objectvideo Inc | Intelligent imagery-based sensor |
GB0620620D0 (en) * | 2006-10-17 | 2006-11-29 | Imp Innovations Ltd | Pervasive sensing |
GB2444107A (en) * | 2006-11-22 | 2008-05-28 | Karen Gail Lambert | Resident monitoring system with remote communications and video processing. |
US9277878B2 (en) * | 2009-02-26 | 2016-03-08 | Tko Enterprises, Inc. | Image processing sensor systems |
-
2012
- 2012-02-29 EP EP12712975.7A patent/EP2681722B1/en active Active
- 2012-02-29 WO PCT/EP2012/053443 patent/WO2012119903A1/en active Application Filing
- 2012-02-29 DK DK12712975.7T patent/DK2681722T3/en active
Non-Patent Citations (1)
Title |
---|
See references of WO2012119903A1 * |
Also Published As
Publication number | Publication date |
---|---|
EP2681722B1 (en) | 2017-12-20 |
DK2681722T3 (en) | 2018-03-05 |
WO2012119903A1 (en) | 2012-09-13 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
EP2681722B1 (en) | Method and system for detecting a fall and issuing an alarm | |
EP2220627B1 (en) | Monitoring system having status detection module, method for self-monitoring of an observer and computer program | |
DE102014221527B4 (en) | Method and device for visual accident detail reporting | |
DE102013209055A1 (en) | Data source identification, data collection and data storage for traffic events | |
CN110795963A (en) | Monitoring method, device and equipment based on face recognition | |
EP3561779A1 (en) | System for detecting personal accident data in a vehicle | |
CN113971782B (en) | Comprehensive monitoring information management method and system | |
DE102017106674A1 (en) | Method and device | |
DE102022102479A1 (en) | PRIVACY ENSURING SYSTEMS AND PROCEDURES IN AN AUTONOMOUS VEHICLE | |
DE102019126208A1 (en) | Emergency system for a vehicle | |
EP3312811A1 (en) | Protection against danger | |
WO2020083732A1 (en) | Authentication module for sensor data | |
EP3404570B1 (en) | Method and system for behavioral authentication | |
DE102014003783A1 (en) | Safety device for a motor vehicle and associated operating method | |
DE102014200567A1 (en) | Method for the automatic transmission of image and / or audio data in connection with an accident of a vehicle | |
DE102020128517A1 (en) | Vehicle to vehicle security | |
AT510352A1 (en) | CAMERA FOR THE UNDERTAKING OF PERSONAL DATA | |
DE112017007247T5 (en) | IMAGING DEVICE | |
DE102015010727A1 (en) | Device for emergency monitoring of an occupant of a vehicle | |
EP2603905B1 (en) | Method and device for detecting and verifying attempts to manipulate a self-service terminal | |
EP3616858A2 (en) | Method and device for documenting a status of an autonomous robot | |
DE102017211555A1 (en) | Method for monitoring at least one occupant of a motor vehicle, wherein the method is used in particular for monitoring and detecting possible dangerous situations for at least one occupant | |
DE102017219292A1 (en) | METHOD AND DEVICE FOR DETECTING EVENT-RELATED DATA RELATING TO A VEHICLE | |
EP3404569A1 (en) | Method, system and computer program product for authenticating a user based on behaviour | |
DE112021002509T5 (en) | Authentication system, authentication device, authentication method, and program |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PUAI | Public reference made under article 153(3) epc to a published international application that has entered the european phase |
Free format text: ORIGINAL CODE: 0009012 |
|
17P | Request for examination filed |
Effective date: 20130926 |
|
AK | Designated contracting states |
Kind code of ref document: A1 Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR |
|
DAX | Request for extension of the european patent (deleted) | ||
17Q | First examination report despatched |
Effective date: 20160129 |
|
GRAP | Despatch of communication of intention to grant a patent |
Free format text: ORIGINAL CODE: EPIDOSNIGR1 |
|
INTG | Intention to grant announced |
Effective date: 20170714 |
|
GRAS | Grant fee paid |
Free format text: ORIGINAL CODE: EPIDOSNIGR3 |
|
GRAA | (expected) grant |
Free format text: ORIGINAL CODE: 0009210 |
|
AK | Designated contracting states |
Kind code of ref document: B1 Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR |
|
REG | Reference to a national code |
Ref country code: GB Ref legal event code: FG4D Free format text: NOT ENGLISH |
|
REG | Reference to a national code |
Ref country code: CH Ref legal event code: EP |
|
REG | Reference to a national code |
Ref country code: IE Ref legal event code: FG4D Free format text: LANGUAGE OF EP DOCUMENT: GERMAN |
|
REG | Reference to a national code |
Ref country code: AT Ref legal event code: REF Ref document number: 957026 Country of ref document: AT Kind code of ref document: T Effective date: 20180115 |
|
REG | Reference to a national code |
Ref country code: DE Ref legal event code: R096 Ref document number: 502012011860 Country of ref document: DE |
|
REG | Reference to a national code |
Ref country code: FR Ref legal event code: PLFP Year of fee payment: 7 |
|
REG | Reference to a national code |
Ref country code: DK Ref legal event code: T3 Effective date: 20180228 |
|
REG | Reference to a national code |
Ref country code: NL Ref legal event code: FP |
|
REG | Reference to a national code |
Ref country code: SE Ref legal event code: TRGR |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: NO Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180320 Ref country code: LT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20171220 |
|
REG | Reference to a national code |
Ref country code: LT Ref legal event code: MG4D |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: GR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180321 Ref country code: BG Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180320 Ref country code: LV Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20171220 Ref country code: RS Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20171220 Ref country code: HR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20171220 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: SK Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20171220 Ref country code: CZ Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20171220 Ref country code: CY Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20171220 Ref country code: EE Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20171220 Ref country code: ES Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20171220 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: SM Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20171220 Ref country code: IS Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180420 Ref country code: RO Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20171220 Ref country code: PL Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20171220 |
|
REG | Reference to a national code |
Ref country code: CH Ref legal event code: PL |
|
REG | Reference to a national code |
Ref country code: DE Ref legal event code: R097 Ref document number: 502012011860 Country of ref document: DE |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: MT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20171220 Ref country code: MC Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20171220 |
|
PLBE | No opposition filed within time limit |
Free format text: ORIGINAL CODE: 0009261 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT |
|
26N | No opposition filed |
Effective date: 20180921 |
|
REG | Reference to a national code |
Ref country code: IE Ref legal event code: MM4A |
|
REG | Reference to a national code |
Ref country code: BE Ref legal event code: MM Effective date: 20180228 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: LU Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20180228 Ref country code: LI Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20180228 Ref country code: CH Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20180228 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: IE Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20180228 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: SI Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20171220 Ref country code: BE Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20180228 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: TR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20171220 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: HU Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT; INVALID AB INITIO Effective date: 20120229 Ref country code: PT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20171220 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: MK Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20171220 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: AL Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20171220 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: GB Payment date: 20231212 Year of fee payment: 13 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: FI Payment date: 20231212 Year of fee payment: 13 Ref country code: DK Payment date: 20231212 Year of fee payment: 13 Ref country code: SE Payment date: 20231219 Year of fee payment: 13 Ref country code: FR Payment date: 20231213 Year of fee payment: 13 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: NL Payment date: 20240220 Year of fee payment: 13 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: AT Payment date: 20231213 Year of fee payment: 13 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: DE Payment date: 20231212 Year of fee payment: 13 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: IT Payment date: 20240229 Year of fee payment: 13 |