EP2998184A1 - System and method for automatically locating and recording a beacon - Google Patents

System and method for automatically locating and recording a beacon Download PDF

Info

Publication number
EP2998184A1
EP2998184A1 EP14380028.2A EP14380028A EP2998184A1 EP 2998184 A1 EP2998184 A1 EP 2998184A1 EP 14380028 A EP14380028 A EP 14380028A EP 2998184 A1 EP2998184 A1 EP 2998184A1
Authority
EP
European Patent Office
Prior art keywords
track
balise
beacon
camera
images
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
EP14380028.2A
Other languages
German (de)
French (fr)
Other versions
EP2998184B1 (en
Inventor
David Celestino Salmeron
Jose Luis Munoz Munoz
Maria Saiz Munoz
Rocio Sanchiz Redondo
Faustino Frechiall Daza
Jaime Rodriguez Urbieta
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Siemens Rail Automation SA
Original Assignee
Siemens Rail Automation SA
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Siemens Rail Automation SA filed Critical Siemens Rail Automation SA
Priority to PT143800282T priority Critical patent/PT2998184T/en
Priority to EP14380028.2A priority patent/EP2998184B1/en
Priority to ES14380028T priority patent/ES2780623T3/en
Priority to PCT/EP2015/060496 priority patent/WO2016041645A1/en
Publication of EP2998184A1 publication Critical patent/EP2998184A1/en
Application granted granted Critical
Publication of EP2998184B1 publication Critical patent/EP2998184B1/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • BPERFORMING OPERATIONS; TRANSPORTING
    • B61RAILWAYS
    • B61LGUIDING RAILWAY TRAFFIC; ENSURING THE SAFETY OF RAILWAY TRAFFIC
    • B61L3/00Devices along the route for controlling devices on the vehicle or train, e.g. to release brake or to operate a warning signal
    • B61L3/02Devices along the route for controlling devices on the vehicle or train, e.g. to release brake or to operate a warning signal at selected places along the route, e.g. intermittent control simultaneous mechanical and electrical control
    • B61L3/08Devices along the route for controlling devices on the vehicle or train, e.g. to release brake or to operate a warning signal at selected places along the route, e.g. intermittent control simultaneous mechanical and electrical control controlling electrically
    • B61L3/12Devices along the route for controlling devices on the vehicle or train, e.g. to release brake or to operate a warning signal at selected places along the route, e.g. intermittent control simultaneous mechanical and electrical control controlling electrically using magnetic or electrostatic induction; using radio waves
    • B61L3/121Devices along the route for controlling devices on the vehicle or train, e.g. to release brake or to operate a warning signal at selected places along the route, e.g. intermittent control simultaneous mechanical and electrical control controlling electrically using magnetic or electrostatic induction; using radio waves using magnetic induction
    • BPERFORMING OPERATIONS; TRANSPORTING
    • B61RAILWAYS
    • B61LGUIDING RAILWAY TRAFFIC; ENSURING THE SAFETY OF RAILWAY TRAFFIC
    • B61L23/00Control, warning or like safety means along the route or between vehicles or trains
    • B61L23/04Control, warning or like safety means along the route or between vehicles or trains for monitoring the mechanical state of the route
    • B61L23/041Obstacle detection
    • BPERFORMING OPERATIONS; TRANSPORTING
    • B61RAILWAYS
    • B61LGUIDING RAILWAY TRAFFIC; ENSURING THE SAFETY OF RAILWAY TRAFFIC
    • B61L25/00Recording or indicating positions or identities of vehicles or trains or setting of track apparatus
    • B61L25/02Indicating or recording positions or identities of vehicles or trains
    • B61L25/025Absolute localisation, e.g. providing geodetic coordinates
    • BPERFORMING OPERATIONS; TRANSPORTING
    • B61RAILWAYS
    • B61LGUIDING RAILWAY TRAFFIC; ENSURING THE SAFETY OF RAILWAY TRAFFIC
    • B61L2205/00Communication or navigation systems for railway traffic
    • B61L2205/04Satellite based navigation systems, e.g. global positioning system [GPS]

Definitions

  • the present invention concerns a system and a method for automatically locating and recording beacons installed at points along a route followed by a guided vehicle.
  • the present invention is directed to the detection, localization and recording of beacons or balises installed on the route or way followed by the guided vehicle and which are configured for exchanging data with the guided vehicle by means of an electromagnetic signal each time the guided vehicle passes near, for example above/over, said balise or beacon.
  • said balise is an Eurobalise, i.e. a balise which complies with the European Train Control System, and is installed between rails of a railway followed by the guided vehicle.
  • Guided vehicle refers to public transport means such as buses, trolleybuses, streetcars, subways, trains or train units, etc., as well as load transporting means such as, for example, overhead traveling cranes, freight trains, for which safety is a very important factor and which are guided along a route or railway by guiding means, for example at least one rail, in particular by two rails between which beacons/balises are placed.
  • public transport means such as buses, trolleybuses, streetcars, subways, trains or train units, etc.
  • load transporting means such as, for example, overhead traveling cranes, freight trains, for which safety is a very important factor and which are guided along a route or railway by guiding means, for example at least one rail, in particular by two rails between which beacons/balises are placed.
  • balises are recorded using a manual procedure, wherein at least one operator has to be physically present on the track or route followed by the guided vehicle in order to record each balise by means of a portable device that has to be carried to the railway track, placed on top of the balise and then, via induction, used to upload the balise data into the portable device.
  • This task is time consuming and might be risky for the operator.
  • An objective of the present invention is to propose a method and system for automatically locating and recording balises or beacons installed at points along a route followed by a guided vehicle.
  • the present invention proposes to make use of an autonomous system, e.g. a robot, for automatically carrying out the detection, the localization and the recording of each beacon/balise installed at points along a route followed by a guided vehicle.
  • an autonomous system e.g. a robot
  • the invention concerns an autonomous system for automatically detecting, locating and recording at least one beacon or balise installed at a point along a route designed for a guided vehicle, e.g. a train or metro, said system comprising:
  • the present invention also concerns a method for automatically detecting, locating and recording at least one beacon or balise installed at a point along a route designed for a guided vehicle like a train or metro, said method comprising:
  • the present invention refers thus to an autonomous system, i.e. a system that does not require to be driven by an operator.
  • the processing unit according to the invention is configured for driving the autonomous system along the track by controlling the moving means, for detecting, locating and recording balises identified by means of the camera, free of any operator input.
  • the autonomous system may comprise at least two cameras, a first camera pointing forward for taking images of the track in front of the autonomous system and identifying approaching balises, and a second camera pointing perpendicularly to the track, i.e. to the ground, used for a precise location of the balise.
  • FIG. 1 shows a preferred embodiment of an autonomous system 1 according to the invention, wherein the autonomous system 1 is in particular a robot.
  • Said autonomous system 1 comprises a frame or body 11 in, or on, which is installed at least one camera 12 configured for taking images of a route or track 2, said track 2 comprising guiding means for guiding a guided vehicle along a path defines by said track 2, said guiding means being for example one or several rails configured for guiding the guided vehicle, and in particular the autonomous system 1 according to the invention.
  • the autonomous system 1 according to the invention is thus configured for using the same guiding means provided by the track 2 as those used by the guided vehicle, e.g. said rails 21, in order to move on the same path as the guided vehicle along the track 2.
  • One or several balises 3 are installed at points along the track 2 and are configured for exchanging data with the guided vehicle when the latter passes at proximity, e.g. above, said balise 3.
  • the autonomous system 1 comprises moving means for autonomously moving its body 11 along the track 2.
  • Said moving means may comprise a connection to a power source and/or a power source, a motor 10 for converting the power of the power source into mechanical energy capable of moving the body 11, and therefore the autonomous system 1.
  • the motor converts the power of the power source, e.g. electric power, into a mechanical energy that makes a shaft rotating, and wherein the rotation of said shaft drives at least one wheel 13 into rotation.
  • the moving means of the autonomous system 1 according to the invention are configured for making use of an identical type of interaction with the track 2 as the guided vehicle has with said track 2 for moving the body 11, and therefore the autonomous system 1.
  • the autonomous system 1 comprises wheel 13, that are configured for being supported by the same rails 21.
  • the moving means according to the invention are in particular configured for cooperating with the guiding means of the track 2 that are configured for guiding the guided vehicle along the path defined by said track 2.
  • the wheels 13 of the moving means are configured for being supported and guided by the rails 21 of the track 2.
  • the present invention is not restricted to autonomous systems 1 using the guiding means of the track 2 for being guided along said track 2, but is also open to autonomous systems 1 like a drone capable of identifying the track 2 by means of the camera 12, and to follow the path described by said track 2.
  • the moving means according to the invention are in particular controlled by a processing unit 14 of the autonomous system 1 according to the invention.
  • said processing unit 14 is connected to the camera 12 for processing each image taken by the camera 12 during the displacement of the autonomous system 1 along the track 2.
  • the processing unit 14 preferentially comprises communication means for communicating with a remote communication device installed for instance in a remote command center.
  • the camera 12 is notably configured for taking images of track sections in front of the body 11 (hereafter also called downstream section) compared to its direction of displacement (indicated by the arrow A in the particular case of Fig. 1 ).
  • the processing unit 14 is connected to storing means 15 which may comprise a database for storing data and/or uploading data and/or modifying data already recorded in said storing means 15 and/or database.
  • said camera 12 is configured for capturing in real time at least one image of a downstream section of said track 2, said downstream section being a track portion extending from a first point of said track 2 located downstream of said body 11 to a second point of said track 2 located downstream of said body 11, the distance separating the camera 12 from the second point being greater than the distance separating the camera 12 from said first point.
  • the processing unit 14 is able to automatically analyze each image taken by each camera 12, to identify and locate said downstream section therein and to determine in real time if a balise 3 is located in said downstream section, notably between said first point and second point.
  • the processing unit 14 is able to digitize each image captured by the camera 12 for transforming said captured image into a format that can be used for localizing the balise 3.
  • the processing unit 14 is configured for detecting if a balise 3 is comprised in the image acquired by the camera and optionally for detecting and identifying the track 2.
  • the processing unit 14 uses in particular an object recognition algorithm.
  • the identification of the track 2 in each image may help detecting balises 3, since the position of a balise 3 compared to the track position might be used as a parameter for searching balises in the images acquired by the camera, said parameter being for example used by the object recognition algorithm.
  • said camera 12 might be configured and installed for taking images of portions of track that are located under the body 11 of the autonomous system 1.
  • said camera 12 may point perpendicularly to the track or ground (optical axis of the camera perpendicular to the track).
  • This configuration of the camera 12 may help in determining the position of the balise 3 compared to the position of the body 11. Indeed, it is then easier to determine the center of the balise 3 by making at least one part of the balise coinciding with the optical axis of the camera 12.
  • the autonomous system 1 may comprise two cameras, a first camera pointing forward as previously described, and a second camera having its optical axis pointing perpendicularly to the track 2 as described above, so that the first camera might do a first approximation of the location and position of the balise and is used for controlling the approach of the autonomous system 1 toward the balise 3, and the second camera is used for the precise determination of the location/position of the balise under the body 11 of the autonomous system 1.
  • the images of both cameras might be processed by means of object recognition algorithm in order to determine balise characteristics and position.
  • first camera will refer to a camera pointing forwards
  • second camera will refer to a camera pointing perpendicularly to the track as previously described.
  • the autonomous system 1 might comprise both first and second cameras, or alternatively only one of them.
  • the processing unit 14 uses said object recognition algorithm for identifying a presence or absence of balise 3 in each image captured by the camera 12, for example by the first and/or second camera.
  • the recognition algorithm uses geometric recognition techniques applied to each image or to a part of each image (for example only to the part comprising the track, or only to the part comprised between two rails 21 of a track 2) for identifying for example the balise 3, and in particular the downstream section of the track 2.
  • the object recognition algorithm is in particular capable of first searching for a track 2 (e.g.
  • the object recognition algorithm preferentially uses automatic learning techniques such as boosting for identifying objects, like the track 2 or the balise 3, in the images taken by the camera 12. Techniques such as boosting for identifying objects in images are well known by the skilled man and do not need further explanations.
  • the recognition algorithm is able to make a correlation or matching between a set of pixels of the image captured by said camera and a set of features of the balise 3, and preferentially also of the track 2, wherein said features might be previously saved in the database of the autonomous system 1 according to the invention.
  • said features define for example specific geometric shapes of the balise 3, and preferentially also of the track 2, said features facilitating the track/balise identification by means of the object recognition algorithm by comparing features of the images with the features recorded for the balise/track in said database.
  • said method according to the invention may in particular include a learning phase intended to create said database.
  • the learning phase may comprise an acquisition of images by the camera of the autonomous system 1 when the latter is moving on a track 2 comprising beacons/balises installed at known points along the track 2. Said images acquired during the learning phase might be used as "learning" images (i.e. typical images representing a track comprising balises) for the detection by the object recognition algorithm of the balise and/or the track during the learning phase.
  • track/balise features like specific geometric shapes of the balise 3 and/or of the track 2, might be automatically stored in the database by the processing unit 14.
  • said database is used by the object recognition algorithm for the identification in real time of balise/track features in images captured by the camera.
  • said database might be upgradable and adaptive in that, once said learning phase is completed, it can be updated in real time using new images acquired by said camera during subsequent runs of the autonomous system 1 along the track 2 in order to complete the database and to improve the identification process of balises 3 by means of the object recognition algorithm.
  • the processing unit 14 may carry out at least one of the following steps:
  • the present invention proposes a system and a method for automatically detecting beacons at points along a route of a guided vehicle, wherein a processing unit is used for detecting beacons in images taken by a camera on-board said system.

Landscapes

  • Engineering & Computer Science (AREA)
  • Mechanical Engineering (AREA)
  • Control Of Position, Course, Altitude, Or Attitude Of Moving Bodies (AREA)
  • Image Analysis (AREA)
  • Traffic Control Systems (AREA)
  • Navigation (AREA)

Abstract

System and method for automatically locating and recording a beacon.
The present invention concerns a system and a method for automatically detecting, locating and recording at least one beacon (3) installed at a point along a track (2) designed for a guided vehicle, the system (1) comprising:
- a body (11);
- moving means for autonomously moving said body (11) along said track (2);
- at least one camera (12) installed on said body (11) and configured for taking images of the track (2);
- a processing unit (14) for driving the moving means (14), analyzing the images and detecting in said images beacon (3) installed at points along the track (2);
- recording means (15) for recording information related to each detected beacon (3).

Description

  • The present invention concerns a system and a method for automatically locating and recording beacons installed at points along a route followed by a guided vehicle.
  • The present invention is directed to the detection, localization and recording of beacons or balises installed on the route or way followed by the guided vehicle and which are configured for exchanging data with the guided vehicle by means of an electromagnetic signal each time the guided vehicle passes near, for example above/over, said balise or beacon. In particular, said balise is an Eurobalise, i.e. a balise which complies with the European Train Control System, and is installed between rails of a railway followed by the guided vehicle. "Guided vehicle" according to the present invention refers to public transport means such as buses, trolleybuses, streetcars, subways, trains or train units, etc., as well as load transporting means such as, for example, overhead traveling cranes, freight trains, for which safety is a very important factor and which are guided along a route or railway by guiding means, for example at least one rail, in particular by two rails between which beacons/balises are placed.
  • Nowadays, balises are recorded using a manual procedure, wherein at least one operator has to be physically present on the track or route followed by the guided vehicle in order to record each balise by means of a portable device that has to be carried to the railway track, placed on top of the balise and then, via induction, used to upload the balise data into the portable device. This task is time consuming and might be risky for the operator.
  • An objective of the present invention is to propose a method and system for automatically locating and recording balises or beacons installed at points along a route followed by a guided vehicle.
  • The present invention proposes to make use of an autonomous system, e.g. a robot, for automatically carrying out the detection, the localization and the recording of each beacon/balise installed at points along a route followed by a guided vehicle.
  • More precisely, the invention concerns an autonomous system for automatically detecting, locating and recording at least one beacon or balise installed at a point along a route designed for a guided vehicle, e.g. a train or metro, said system comprising:
    • a body;
    • moving means for autonomously/independently moving said body along said route;
    • at least one camera installed on said body and configured for taking images of the route;
    • a processing unit for driving the moving means, in particular in a manner free of any.operator input, and analyzing the images and detecting from the analysis of said images the location of each beacon;
    • storing/recording means for storing/recording information related to each detected beacon, like its position and/or reference number and/or type.
  • The present invention also concerns a method for automatically detecting, locating and recording at least one beacon or balise installed at a point along a route designed for a guided vehicle like a train or metro, said method comprising:
    • moving an autonomous system, like a robot, along said route, wherein the autonomous system is capable of autonomously/independently moving along said route, notably by means of moving means controlled by a processing unit;
    • taking images of said route by means of at least one camera on-board the system;
    • detecting and locating each beacon/balise installed at a point along said route by analyzing the images of said route taken by the camera when the system is moving along said route;
    • recording/storing information related to each detected beacon, like its position and/or reference number and/or type, in storing/recording means of the system.
  • The present invention refers thus to an autonomous system, i.e. a system that does not require to be driven by an operator. The processing unit according to the invention is configured for driving the autonomous system along the track by controlling the moving means, for detecting, locating and recording balises identified by means of the camera, free of any operator input. Of course, it might be possible for an operator to remotely take the lead and drive the system according to the invention if needed. Optionally, the autonomous system may comprise at least two cameras, a first camera pointing forward for taking images of the track in front of the autonomous system and identifying approaching balises, and a second camera pointing perpendicularly to the track, i.e. to the ground, used for a precise location of the balise.
  • Further aspects of the present invention will be better understood through the following drawing:
  • Figure 1
    schematic representation of an autonomous system according to the invention.
  • Figure 1 shows a preferred embodiment of an autonomous system 1 according to the invention, wherein the autonomous system 1 is in particular a robot. Said autonomous system 1 comprises a frame or body 11 in, or on, which is installed at least one camera 12 configured for taking images of a route or track 2, said track 2 comprising guiding means for guiding a guided vehicle along a path defines by said track 2, said guiding means being for example one or several rails configured for guiding the guided vehicle, and in particular the autonomous system 1 according to the invention. Preferentially, the autonomous system 1 according to the invention is thus configured for using the same guiding means provided by the track 2 as those used by the guided vehicle, e.g. said rails 21, in order to move on the same path as the guided vehicle along the track 2. One or several balises 3 are installed at points along the track 2 and are configured for exchanging data with the guided vehicle when the latter passes at proximity, e.g. above, said balise 3.
  • The autonomous system 1 comprises moving means for autonomously moving its body 11 along the track 2. Said moving means may comprise a connection to a power source and/or a power source, a motor 10 for converting the power of the power source into mechanical energy capable of moving the body 11, and therefore the autonomous system 1. Preferentially, the motor converts the power of the power source, e.g. electric power, into a mechanical energy that makes a shaft rotating, and wherein the rotation of said shaft drives at least one wheel 13 into rotation. Preferentially, the moving means of the autonomous system 1 according to the invention are configured for making use of an identical type of interaction with the track 2 as the guided vehicle has with said track 2 for moving the body 11, and therefore the autonomous system 1. For example, if the guided vehicle is a train comprising wheels that are supported by rails 21, then preferentially, the autonomous system 1 comprises wheel 13, that are configured for being supported by the same rails 21. Additionally, the moving means according to the invention are in particular configured for cooperating with the guiding means of the track 2 that are configured for guiding the guided vehicle along the path defined by said track 2. For example, the wheels 13 of the moving means are configured for being supported and guided by the rails 21 of the track 2. Of course, the present invention is not restricted to autonomous systems 1 using the guiding means of the track 2 for being guided along said track 2, but is also open to autonomous systems 1 like a drone capable of identifying the track 2 by means of the camera 12, and to follow the path described by said track 2.
  • The moving means according to the invention are in particular controlled by a processing unit 14 of the autonomous system 1 according to the invention. In particular said processing unit 14 is connected to the camera 12 for processing each image taken by the camera 12 during the displacement of the autonomous system 1 along the track 2. The processing unit 14 preferentially comprises communication means for communicating with a remote communication device installed for instance in a remote command center. The camera 12 is notably configured for taking images of track sections in front of the body 11 (hereafter also called downstream section) compared to its direction of displacement (indicated by the arrow A in the particular case of Fig. 1). Preferentially, the processing unit 14 is connected to storing means 15 which may comprise a database for storing data and/or uploading data and/or modifying data already recorded in said storing means 15 and/or database.
  • Preferentially, said camera 12 is configured for capturing in real time at least one image of a downstream section of said track 2, said downstream section being a track portion extending from a first point of said track 2 located downstream of said body 11 to a second point of said track 2 located downstream of said body 11, the distance separating the camera 12 from the second point being greater than the distance separating the camera 12 from said first point. The processing unit 14 is able to automatically analyze each image taken by each camera 12, to identify and locate said downstream section therein and to determine in real time if a balise 3 is located in said downstream section, notably between said first point and second point.
  • Preferentially, the processing unit 14 is able to digitize each image captured by the camera 12 for transforming said captured image into a format that can be used for localizing the balise 3. The processing unit 14 is configured for detecting if a balise 3 is comprised in the image acquired by the camera and optionally for detecting and identifying the track 2. For this purpose, the processing unit 14 uses in particular an object recognition algorithm. Advantageously, the identification of the track 2 in each image may help detecting balises 3, since the position of a balise 3 compared to the track position might be used as a parameter for searching balises in the images acquired by the camera, said parameter being for example used by the object recognition algorithm.
  • Alternatively, said camera 12 might be configured and installed for taking images of portions of track that are located under the body 11 of the autonomous system 1. In other words, said camera 12 may point perpendicularly to the track or ground (optical axis of the camera perpendicular to the track). This configuration of the camera 12 may help in determining the position of the balise 3 compared to the position of the body 11. Indeed, it is then easier to determine the center of the balise 3 by making at least one part of the balise coinciding with the optical axis of the camera 12. In another preferred embodiment, the autonomous system 1 may comprise two cameras, a first camera pointing forward as previously described, and a second camera having its optical axis pointing perpendicularly to the track 2 as described above, so that the first camera might do a first approximation of the location and position of the balise and is used for controlling the approach of the autonomous system 1 toward the balise 3, and the second camera is used for the precise determination of the location/position of the balise under the body 11 of the autonomous system 1. The images of both cameras might be processed by means of object recognition algorithm in order to determine balise characteristics and position. In the following text, "first camera" will refer to a camera pointing forwards and "second camera" will refer to a camera pointing perpendicularly to the track as previously described. The autonomous system 1 might comprise both first and second cameras, or alternatively only one of them.
  • Preferably, the processing unit 14 uses said object recognition algorithm for identifying a presence or absence of balise 3 in each image captured by the camera 12, for example by the first and/or second camera. In particular, the recognition algorithm uses geometric recognition techniques applied to each image or to a part of each image (for example only to the part comprising the track, or only to the part comprised between two rails 21 of a track 2) for identifying for example the balise 3, and in particular the downstream section of the track 2. For example, the object recognition algorithm is in particular capable of first searching for a track 2 (e.g. a track downstream section) in the image acquired by the camera, preferentially by the first camera, and second, once a track 2 or track downstream section has been identified in said image, searching for a balise 3 only on an area of the image wherein the track 2 or track downstream section has been identified. The object recognition algorithm preferentially uses automatic learning techniques such as boosting for identifying objects, like the track 2 or the balise 3, in the images taken by the camera 12. Techniques such as boosting for identifying objects in images are well known by the skilled man and do not need further explanations.
  • Preferably, and in order to identify a balise 3 in an image, an in particular in order to recognize said track 2 in the image acquired by the camera, e.g. by the first camera and/or the second camera, the recognition algorithm is able to make a correlation or matching between a set of pixels of the image captured by said camera and a set of features of the balise 3, and preferentially also of the track 2, wherein said features might be previously saved in the database of the autonomous system 1 according to the invention. Preferentially, said features define for example specific geometric shapes of the balise 3, and preferentially also of the track 2, said features facilitating the track/balise identification by means of the object recognition algorithm by comparing features of the images with the features recorded for the balise/track in said database.
  • Preferably, said method according to the invention may in particular include a learning phase intended to create said database. The learning phase may comprise an acquisition of images by the camera of the autonomous system 1 when the latter is moving on a track 2 comprising beacons/balises installed at known points along the track 2. Said images acquired during the learning phase might be used as "learning" images (i.e. typical images representing a track comprising balises) for the detection by the object recognition algorithm of the balise and/or the track during the learning phase. During said learning phase, track/balise features, like specific geometric shapes of the balise 3 and/or of the track 2, might be automatically stored in the database by the processing unit 14. In particular, once said learning phase is completed, then said database is used by the object recognition algorithm for the identification in real time of balise/track features in images captured by the camera. Advantageously, said database might be upgradable and adaptive in that, once said learning phase is completed, it can be updated in real time using new images acquired by said camera during subsequent runs of the autonomous system 1 along the track 2 in order to complete the database and to improve the identification process of balises 3 by means of the object recognition algorithm.
  • Preferentially, each time a balise 3 is identified by the object recognition algorithm in an image taken by the camera 12, in particular by the first camera and/or the second camera, then the processing unit 14 may carry out at least one of the following steps:
    • decreasing the moving speed of the body 1, i.e. of the autonomous system 1, by controlling the moving means, e.g. the motor, so that the speed of the autonomous system 1 according to the invention when passing above/close to a balise 3 is smaller than its speed when no balise is detected in order to improve the determination of the location of the balise 3. For example, the images taken by the first camera are used by the processing unit for controlling the approach of the autonomous system 1 towards the balise 3, and once the autonomous system 1 passes above the balise 3, the second camera and/or a device 16 (see below) might be used for determining the exact location of the balise, wherein the processing unit is able to determine for example the time T corresponding to an alignment of the optical axis of the second camera with a part of the balise (e.g. an edge) and/or the alignment of a receiving loop of the device 16 with a transmitting loop of the balise 3, and to deduct from the position of the autonomous system 1 at said time T the position of the balise, wherein said balise position might then be recorded in the database and/or transmitted to a remote control center;
    • controlling a device 16 for communicating with the balise 3, said device 16 comprising for example an emitter and a receiver and being installed on the body 11 so that in particular at least its emitter and receiver face the balise 3 when the body passes over/above the balise 3. Preferentially, the emitter is configured for remotely powering the balise 3, in particular by means of radiant energy. Said emitter comprises for example an antenna comprising an emitting loop for radiating energy, in particular radio frequency energy, the balise being then powered by said radiated energy and able to transmit, in return, an electromagnetic signal. The receiver comprises an antenna incorporating at least one receiving loop for picking up the electromagnetic signal produced by the balise 3, more precisely the electromagnetic signal produced by a transmitting loop of the balise 3, the receiver comprising thus a receiving loop configured for picking up the electromagnetic signal sent by the balise 3 in response to its powering by the emitter and for delivering a signal SR to the processing unit 14, said signal SR being current induced by the electromagnetic signal in said receiving loop, wherein said current provides a measure of the amplitude of the electromagnetic signal in function of the position of the receiving loop (e.g. in function of the position of the center of the receiving loop) compared to the position of the balise 3, more precisely compared to the position of the center of the transmitting loop of the balise 3. In the context of the present invention, the center of the balise 3 refers to the center of its transmitting loop, and centers of receiving or transmitting loops are notably geometric centers. The processing unit 14 is capable of processing the signal SR delivered by the receiving loop in order to collect information sent by the balise 3 in response to its powering and is able to send information to the balise 3, wherein said information may comprise the position of the balise 3 and/or a reference number. The processing unit is capable to determine the time T at which the center of the receiving loop coincides with the center of the transmitting loop from the signal SR. Such technique of balise center determination is known by the persons skilled in the art and does not need further explanations;
    • controlling a geolocation system for determining the position of each balise 3. Said geolocation system being preferentially installed in or on the body 11. Said geolocation system may make use of the camera installed on or in the body 11 and preferentially configured for having its optical axis pointing perpendicularly to the track 2, i.e. of said second camera, or of the device 16, for determining the position of the balise 3. For example, the position of the optical axis of the second camera might be used for determining the position of the balise on the track 2. Indeed, the processing unit 14 is able to identify said balise 3 in images taken by the second camera by means of said object recognition algorithm and to determine when the second camera optical axis points to or coincides with a part of the balise 3 from analyzing said images. For example, the database may comprise pre-recorded images of the balise, wherein data regarding the distance between the balise center and said part is recorded and used by the processing system for determining the position of the center of the balise 3 at the time T from the position of said part at said time T compared to the position of the autonomous system at said time T, wherein the position of the autonomous system at said time T is determined by means of the geolocation system. By position of the autonomous system, it has to be understood the position of a reference point (for example its geographic coordinates and/or the distance separating said reference point from a fixed reference as described below) of the autonomous system, and therefore, the determination of the position of the balise center compared to the position of the autonomous system means the determination of the position of the balise center compared to the position of the reference point. For example, said reference point might be the projection of the optical axis of the second camera on a horizontal plane. Said geolocation system is in particular configured for determining the position of the reference point of the autonomous system, e.g. the position of the optical axis of said second camera, compared to the fixed reference, the fixed reference being in particular a fixed position on a guided vehicle network, for instance a starting point on the track 2, by using techniques such as odometry and/or Global Positioning System techniques. The same applies mutatis mutandis for the determination of the position of the center of the receiving loop compared to said fixed reference, wherein the center of the receiving loop might be chosen as reference point. Once the position of the balise 3 is determined, then the processing unit 14 may write information related to the balise 3 position on a memory of the balise by means of the device 16 for communicating and/or may write said information in said database, and/or may compare an information related to the balise position provided by the balise 3 to the determined position, and/or may automatically communicate said position to a remote control center.
  • To summarize, the present invention proposes a system and a method for automatically detecting beacons at points along a route of a guided vehicle, wherein a processing unit is used for detecting beacons in images taken by a camera on-board said system.

Claims (15)

  1. System (1) for automatically detecting, locating and recording at least one beacon (3) installed at a point along a track (2) designed for a guided vehicle, the system comprising:
    - a body (11);
    - moving means for autonomously moving said body (11) along said track (2);
    - at least one camera (12) installed on said body (11) and configured for taking images of the track (2) ;
    - a processing unit (14) for driving the moving means (14), analyzing the images, detecting in said images beacon (3) installed at points along the track (2), and determining the position of each beacon (3) ;
    - recording means (15) for recording information related to each detected beacon (3).
  2. System (1) according to claim 1, configured for cooperating with guiding means of the track (2) designed for guiding the guided vehicle in order to be guided along said track (2).
  3. System according to claim 1 or 2, wherein the processing unit (14) comprises communication means.
  4. System according to one of the claims 1-3, wherein the recording means (15) includes a database.
  5. System according to one of the claims 1-4, wherein the processing unit (14) is configured for identifying at least one portion of track (2) in each image.
  6. System according to one of the claims 1-5, wherein the processing unit (14) comprises an object recognition algorithm.
  7. System according to one of the claims 4 to 6, wherein the database comprises information related to features of the balise (3) and/or the track (2).
  8. System according to one of the claims 1 to 7, comprising a device (16) for communicating and exchanging information with the balise (3).
  9. System according to one of the claims 1 to 8, comprising a geolocation system for determining the position of the balise (3).
  10. Method for automatically detecting, locating and recording at least one beacon (3) installed at a point along a track (2) designed for guiding a guided vehicle, the method comprising:
    - moving an system (1) along said track (2), wherein the system (1) is capable of autonomously moving along said track (2);
    - taking images of said track (2) by means of at least one camera (12) on-board the system (1);
    - detecting and locating beacons (3) installed at a point along said track (2) by analyzing, by means of a processing unit (14) of the system (1), the images of said track (2) taken by the camera (12) when the system (1) is moving along said track (2);
    - recording information related to each detected beacon (3) in recording means (15) of the system (1).
  11. Method according to claim 10, comprising controlling a device (16) for communicating with the beacon (3) by means of the processing unit (14).
  12. Method according to claim 10 or 11, comprising controlling a geolocation system for determining the position of each beacon (3).
  13. Method according to one of the claims 10-12, comprising using an object recognition algorithm for detecting at least one part of the track (2) and/or the beacon (3) in the images taken by the camera (12).
  14. Method according to one of the claims 10-13, comprising a learning phase for automatically collecting information related to the track (2) and/or beacon and storing said information in a database.
  15. Method according to claim 14, wherein the object recognition algorithm uses said information stored in the database for identifying the track (2) and/or beacon (3) in the images acquired by the camera (12).
EP14380028.2A 2014-09-18 2014-09-18 System and method for automatically locating and recording a beacon Active EP2998184B1 (en)

Priority Applications (4)

Application Number Priority Date Filing Date Title
PT143800282T PT2998184T (en) 2014-09-18 2014-09-18 System and method for automatically locating and recording a beacon
EP14380028.2A EP2998184B1 (en) 2014-09-18 2014-09-18 System and method for automatically locating and recording a beacon
ES14380028T ES2780623T3 (en) 2014-09-18 2014-09-18 System and method to automatically locate and register a beacon
PCT/EP2015/060496 WO2016041645A1 (en) 2014-09-18 2015-05-12 System and method for automatically locating and recording a beacon

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
EP14380028.2A EP2998184B1 (en) 2014-09-18 2014-09-18 System and method for automatically locating and recording a beacon

Publications (2)

Publication Number Publication Date
EP2998184A1 true EP2998184A1 (en) 2016-03-23
EP2998184B1 EP2998184B1 (en) 2020-01-08

Family

ID=51795591

Family Applications (1)

Application Number Title Priority Date Filing Date
EP14380028.2A Active EP2998184B1 (en) 2014-09-18 2014-09-18 System and method for automatically locating and recording a beacon

Country Status (4)

Country Link
EP (1) EP2998184B1 (en)
ES (1) ES2780623T3 (en)
PT (1) PT2998184T (en)
WO (1) WO2016041645A1 (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111923965A (en) * 2019-05-13 2020-11-13 株洲中车时代电气股份有限公司 Automatic driving method, system and equipment for locomotive
CN112829799A (en) * 2021-02-02 2021-05-25 中铁第四勘察设计院集团有限公司 Signal responder displacement system of automatic vehicle base and control method thereof

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110641522A (en) * 2019-09-30 2020-01-03 中铁第四勘察设计院集团有限公司 Train positioning system of high-speed magnetic levitation track traffic
CN113022643B (en) * 2021-03-23 2022-09-27 上海电气泰雷兹交通自动化***有限公司 Fault-oriented safety transponder detection system and method
US20240067242A1 (en) * 2022-08-31 2024-02-29 Siemens Mobility, Inc. System and method for managing operations of a train relative to a fouling mark
EP4385852A1 (en) * 2022-12-16 2024-06-19 Siemens Mobility GmbH Method for detecting balises and railway vehicle system

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DE102006062549A1 (en) * 2006-12-29 2008-07-03 Deutsches Zentrum für Luft- und Raumfahrt e.V. Rail line's track-sided infrastructure positioning method, involves determining installation position of infrastructure depending on actual position of rail-bound vehicle i.e. train, and installation parameter, and storing position
DE102011106345A1 (en) * 2011-06-08 2012-12-13 Deutsches Zentrum für Luft- und Raumfahrt e.V. Device for validating stretch-side infrastructure elements of railway infrastructure of track, used in rail vehicle, has evaluation unit to determine correlation of data from signal aspect and adapted by infrastructure elements position

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DE10244127A1 (en) * 2002-09-27 2004-04-08 Siemens Ag Sensor system for route monitoring for an autonomous mobile unit, method and computer program with program code means and computer program product for monitoring a route for an autonomous mobile unit
US9810533B2 (en) * 2011-04-27 2017-11-07 Trimble Inc. Railway track monitoring

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DE102006062549A1 (en) * 2006-12-29 2008-07-03 Deutsches Zentrum für Luft- und Raumfahrt e.V. Rail line's track-sided infrastructure positioning method, involves determining installation position of infrastructure depending on actual position of rail-bound vehicle i.e. train, and installation parameter, and storing position
DE102011106345A1 (en) * 2011-06-08 2012-12-13 Deutsches Zentrum für Luft- und Raumfahrt e.V. Device for validating stretch-side infrastructure elements of railway infrastructure of track, used in rail vehicle, has evaluation unit to determine correlation of data from signal aspect and adapted by infrastructure elements position

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
BENEDIKT WENZEL, ET AL.: "SAT.VALID -A New Data Validation Tool for Communication Based Train Control Systems SAT.VALID -A NEW DATA VALIDATION TOOL FOR COMMUNICATION BASED TRAIN CONTROL SYSTEMS (SUCH AS ETCS)", 11 September 2012 (2012-09-11), London, pages 1 - 11, XP055171160, Retrieved from the Internet <URL:http://www.irse.org/knowledge/publicdocuments/2.03 Wenzel - Data validation tool for communications based train control.pdf> [retrieved on 20150220] *

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111923965A (en) * 2019-05-13 2020-11-13 株洲中车时代电气股份有限公司 Automatic driving method, system and equipment for locomotive
CN111923965B (en) * 2019-05-13 2022-02-11 株洲中车时代电气股份有限公司 Automatic driving method, system and equipment for locomotive
CN112829799A (en) * 2021-02-02 2021-05-25 中铁第四勘察设计院集团有限公司 Signal responder displacement system of automatic vehicle base and control method thereof

Also Published As

Publication number Publication date
EP2998184B1 (en) 2020-01-08
WO2016041645A1 (en) 2016-03-24
PT2998184T (en) 2020-03-02
ES2780623T3 (en) 2020-08-26

Similar Documents

Publication Publication Date Title
EP2998184B1 (en) System and method for automatically locating and recording a beacon
JP3421768B2 (en) Autonomous vehicle route guidance method, autonomous vehicle route guidance device, and autonomous vehicle equipped with route guidance device
CN109552367B (en) Train running protection system and method based on combination of communication and active identification
US10940861B2 (en) Method and system for automatically controlling a following vehicle with a front vehicle
RU2357886C2 (en) Method and system for locomotives control
US10136106B2 (en) Train asset tracking based on captured images
US8466777B2 (en) Target object detection server
CN108196552A (en) A kind of GPS vision navigation systems of intelligent carriage
US20200034637A1 (en) Real-Time Track Asset Recognition and Position Determination
US20200363825A1 (en) Pickup system
CN106155066B (en) Carrier capable of detecting road surface obstacle and carrying method
EP3333043B1 (en) Rail inspection system and method
JP2021101613A (en) On-board apparatus for automatic train operation system
CN109415071A (en) For the method, apparatus and rail vehicle of the signal identification in rail traffic, particularly railway traffic, especially rolling stock
JP3448088B2 (en) Obstacle detection system
CN105593103A (en) Device and method for rail-side monitoring of position of stabled rail-based vehicle
US10587845B2 (en) Information processing system
CN111079525B (en) Image processing method, device, system and storage medium
US20240034374A1 (en) Method for Operating a Rail Vehicle and Arrangement Comprising a Rail Vehicle
CN117279815A (en) Method for generating a trajectory of an at least partially autonomous motor vehicle, computer program product, computer-readable storage medium and auxiliary system
US11017553B2 (en) Information processing system
CN113985867A (en) Carrier control method, system, electronic device, medium and carrier
JP7227936B2 (en) Drone guidance method and device
JP2021107994A (en) Autonomously traveling device and autonomous-travel managing system
JPH10241092A (en) Vehicle number plate recognizing device

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

AX Request for extension of the european patent

Extension state: BA ME

17P Request for examination filed

Effective date: 20160819

RBV Designated contracting states (corrected)

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: EXAMINATION IS IN PROGRESS

17Q First examination report despatched

Effective date: 20170330

GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: GRANT OF PATENT IS INTENDED

INTG Intention to grant announced

Effective date: 20191002

RIN1 Information on inventor provided before grant (corrected)

Inventor name: CELESTINO SALMERON, DAVID

Inventor name: SANCHIZ REDONDO, ROCIO

Inventor name: RODRIGUEZ URBIETA, JAIME

Inventor name: SAIZ MUNOZ, MARIA

Inventor name: MUNOZ MUNOZ, JOSE LUIS

Inventor name: FRECHIALL DAZA, FAUSTINO

GRAS Grant fee paid

Free format text: ORIGINAL CODE: EPIDOSNIGR3

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE PATENT HAS BEEN GRANTED

AK Designated contracting states

Kind code of ref document: B1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

REG Reference to a national code

Ref country code: GB

Ref legal event code: FG4D

REG Reference to a national code

Ref country code: CH

Ref legal event code: EP

REG Reference to a national code

Ref country code: DE

Ref legal event code: R096

Ref document number: 602014059623

Country of ref document: DE

REG Reference to a national code

Ref country code: IE

Ref legal event code: FG4D

REG Reference to a national code

Ref country code: AT

Ref legal event code: REF

Ref document number: 1222339

Country of ref document: AT

Kind code of ref document: T

Effective date: 20200215

REG Reference to a national code

Ref country code: PT

Ref legal event code: SC4A

Ref document number: 2998184

Country of ref document: PT

Date of ref document: 20200302

Kind code of ref document: T

Free format text: AVAILABILITY OF NATIONAL TRANSLATION

Effective date: 20200219

REG Reference to a national code

Ref country code: NO

Ref legal event code: T2

Effective date: 20200108

REG Reference to a national code

Ref country code: NL

Ref legal event code: MP

Effective date: 20200108

REG Reference to a national code

Ref country code: LT

Ref legal event code: MG4D

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: RS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200108

Ref country code: FI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200108

Ref country code: NL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200108

Ref country code: LT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200108

REG Reference to a national code

Ref country code: ES

Ref legal event code: FG2A

Ref document number: 2780623

Country of ref document: ES

Kind code of ref document: T3

Effective date: 20200826

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200508

Ref country code: SE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200108

Ref country code: LV

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200108

Ref country code: BG

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200408

Ref country code: HR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200108

REG Reference to a national code

Ref country code: DE

Ref legal event code: R097

Ref document number: 602014059623

Country of ref document: DE

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: RO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200108

Ref country code: CZ

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200108

Ref country code: EE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200108

Ref country code: SM

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200108

Ref country code: DK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200108

Ref country code: SK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200108

PLBE No opposition filed within time limit

Free format text: ORIGINAL CODE: 0009261

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT

26N No opposition filed

Effective date: 20201009

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200108

Ref country code: PL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200108

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MC

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200108

REG Reference to a national code

Ref country code: CH

Ref legal event code: PL

REG Reference to a national code

Ref country code: BE

Ref legal event code: MM

Effective date: 20200930

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LU

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20200918

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: BE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20200930

Ref country code: CH

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20200930

Ref country code: LI

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20200930

Ref country code: IE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20200918

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200108

Ref country code: CY

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200108

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200108

Ref country code: AL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200108

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: GR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200108

REG Reference to a national code

Ref country code: AT

Ref legal event code: UEP

Ref document number: 1222339

Country of ref document: AT

Kind code of ref document: T

Effective date: 20200108

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: TR

Payment date: 20230915

Year of fee payment: 10

Ref country code: NO

Payment date: 20230906

Year of fee payment: 10

Ref country code: IT

Payment date: 20230920

Year of fee payment: 10

Ref country code: AT

Payment date: 20230814

Year of fee payment: 10

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: PT

Payment date: 20230823

Year of fee payment: 10

Ref country code: FR

Payment date: 20230918

Year of fee payment: 10

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: GB

Payment date: 20231009

Year of fee payment: 10

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: ES

Payment date: 20231218

Year of fee payment: 10

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: DE

Payment date: 20231120

Year of fee payment: 10