NO344442B1

NO344442B1 - A drone, system and method for detection and identification of missing persons or objects

Info

Publication number: NO344442B1
Application number: NO20180465A
Authority: NO
Inventors: Lars Christian Danby Engø
Original assignee: Ondur As
Priority date: 2018-04-05
Filing date: 2018-04-05
Publication date: 2019-12-09
Also published as: NO20180465A1

Description

The present invention relates to a device, method and system for finding and identify the location of a person, animal or object.

Searching for persons, specifically in an outdoor environment represents many types of challenges for the searchers. Area to be searched may be vast, and resources may be limited. Further complications may be added if the person is hidden by obstacles such as shielding vegetation or even buried in an avalanche or landslide. These problems are difficult to overcome.

To overcome such problems it has been deployed airborne equipment such as helicopters carrying heat sensitive cameras to search for people or animals. Helicopters are extremely expensive to deploy, and availability is very limited.

In an avalanche or landslide situations improved search success is achieved when the buried person carries a distress signal beacon device that can transmit a beacon such that a searching party can pick up such signals. Searching in an avalanche or landslide with traditional search techniques is a slow process, and an outcome where missing persons is found too late to be saved happens frequently.

JP 2018042059 A describes a system using a drone for searching after persons in an avalanche or the like.

US 2017357263 A1 and WO 2018053513 A1 describes systems wherein a drone is used for searching after missing persons.

US 2016340006 A1 describesa drone is used for searching and saving persons in water.

The problems therefor comprise both the use of too much time and lack of resources to identify exact location of a missing person or object.

If a positive identification of location of the missing person or object is achieved, further problems may be related to communicating the findings to other resources in the search party.

It is the aim of the present invention to provide a solution which will reduce or eliminate one or more of the problems described above.

The present invention is based on providing a low cost easy to operate aerial vehicle comprising one or more advanced features for detection and identification of missing persons or objects, and guiding means for aiding search and rescue resources to an accurate position of the missing person or object.

It shall be understood that the embodiments only describe the principle of the invention, and that there may be additional ways to implement the present invention. It is the associated claims that shall define the protection scope of the present invention.

Additional features and advantages of the present disclosure are described in, and will be apparent from, the following brief description of the drawings, the drawings, and the more detailed description of the embodiments.

Brief description of the drawings:

Fig.1 – Conceptual scenario illustration

Fig.2 – Drone

Fig.3 – Sensor device

Fig.4 – Heat emission from person overview photo

Fig.5 – Drone identifying person in woodland

Fig.6A-C – Search patterns

Fig.7 - Method flow chart

Fig.8 - System description

The embodiments described herein describe the search, identification of target and guiding of a search party in cases where a for example a missing person is to be found in various scenarios. It shall be understood that the invention also comprise a system and method for finding objects in similar environments.

The present invention comprises using an unmanned aerial vehicle, UAV, such as a drone.

The UAV is constructed for being controlled by a remote controlling facility, comprising communication means for communicating data from the UAV to a computer resource being part of the remote controlling facility. The computing means may comprise input and output means for inputting control commands, and for outputting images and/or data from the UAV. The computing means may further comprise communication means for communicating with other resources such as search and rescue teams and/or broadcast services.

The UAV encompass one or more of camera, heat sensitive camera, night vision camera, distress signal receiver, IR camera, homing device, laser emitting device, loudspeaker, and light emitting device.

The UAV further encompass flight controlling features for flying according to one of a predefined flight search path pattern, predefined signaling movement pattern, or following a homing signal emitted from a searched object or other.

Search pattern can be either manually controlled, or preprogrammed according to search pattern, for example one of, but not excluding: creeping line, expanding, sector based as illustrated in figure 6A-C.

The drone described in the present invention is a central element of the invention. It should however be understood that other sensor assemblies able to be maneuvered over a search area and having the ability to be maneuvered according to predefined pattern can be used.

In figure 1, a drone 1 is used for surveying an area in a search for a person 2 carrying a beacon sending device 3. The beacon sending device 3 may be a specialized tracking device such as an avalanche tracker worn by a mountaineer, but could also be a cellphone or the like. The sensor assembly 4 mounted to the drone 1 may comprise a number of sensor features such as, but not limited to: avalanche tracker, cellphone signal analyzer, heat sensitive camera with image analyzing module, and camera with AI (Artificial Intelligence) module for object recognition.

The drone 1 may comprise a control module for controlling the drone flying pattern in various phases of the search, such as pre search phase, searching phase, and post searching phase. The control module may also comprise output for controlling audio visual devices. The drone 1 or senor assembly 4 may further comprise audio visual output devices which, when activated by the control module, can transmit to the user audio signals and/or search status according to a preset setup. The physical implementation of the control module may be distributed, in the sense that remote computer resources may solve some of the control module features, and computing resources arranged in the drone or the sensor assembly may solve other features.

Sensor assembly 4, as illustrated in figure 3, which is mounted to a drone 1, as illustrated in figure 2, may comprise a radio signal detector 6, a camera assembly 7, a laser 5, signaling light 8 and loudspeakers 9. The radio signal detector may for example be tuned to receive one of, but not limited to: avalanche/emergency transmitter, mobile phone, GPS transmitting device or other.

The sensor assembly 4 may be integrated in the drone chassis, either partially or fully.

In the case a mobile phone such as a smart phone is used as a beacon transmitter, or a smart version of an avalanche transmitter, it is within the scope of the invention to comprise wearables communicating with the sensor assembly 4 via a communication line through the mobile phone or avalanche transmitter. Such wearables can for example measure a person’s heart frequency and body temperature and other. This information may be used for adapting response action adequate for the search objects status.

The sensor assembly 4 may comprise a battery for powering the sensor assembly components, or the sensor assembly may be powered by a separate power source or by the same power source that fuels the motors of the drone 1.

The control module comprises one or more position detection modules. The position detection modules use input from the modules of the sensor assembly to calculate an exact position, relative the drone, of a detected and identified object. When a drone is employed in a search for a missing person or object, the control module is initiated with search instructions defining what type of sensors is to be employed in the search.

In figure 1 a scenario is described wherein the search is performed by a drone 1 according to present invention in a search for a person 2 being victim of a snow avalanche or landslide. The victim may wear an avalanche transmitter, which is activated when the bearer is hit by an avalanche. The rescue team which employs the drone 1 will upon arrival to the disaster area initiate one or more of a required search pattern, exemplified but not limited to the illustrations in figure 6A - 6C. Search patterns are not restricted to the illustrated patterns, but these are included for illustrating examples of search patterns possible to implement. Typical search pattern over an avalanche area would be to start at the bottom edge of the avalanche area, and perform a creeping line search pattern as shown in figure 6A. The control module of the drone is therefore programmed/given input parameters such as for example of one or more of start line/point, end line/point, search width, altitude and slope/topography. It is further necessary to define which sensor type is to be used, and what identification features/behavior patterns should be employed if object is found/not found. Typically, in an avalanche search, wherein the victim is believed to carry an avalanche transmitter, is to activate a listening device for the transmitted emergency signal, and activate a homing feature able to recognize and localize the transmitted distress signal from the avalanche transmitter carried by the victim. Typically, the control module is programmed to initiate the search from a lowest corner of the planned search area, and width and length of search area is inputted to the control module. If a laser pointer 5 is comprised in the sensor assembly, the control module may be instructed to point the laser pointer light 10 onto the found object when such object is found. In one example case the object would be the person wearing the avalanche transmitter having fallen victim for the avalanche, and the emergency frequency detector 6 detects the distress signal sent from the victim.

Alternatively or in combination with the laser light pointing feature, the control module may instruct the drone to perform a specific flying pattern if the object is found. One pattern may for example be to flying in tight small circles above the location of the found object. Rescue personnel can then easily move into the area and start evacuation of the victim. A further feature to identify found object could be, but not limited to: a sound warning from a sound speaker 9 and/or light signal illuminating a lamp 8 on the sensor assembly or drone. The lamp 8 may be a led lamp of different color, signaling status of the search. For example, blinking green light for “operation in progress”, blinking red light mean for “distress signal is identified”, and steady red light for “object is located”. Many other light combinations may be employed. The lamp 8 may also for example be a rotary warning light as commonly known from emergency service vehicles. Color code or frequency of blinking/rotation may be used to discriminate between different operating modes of the drone.

The control module of the drone may be remotely controlled, and connected computing resources may wirelessly communicate controlling signals to the drone while under operation. If for example a search in an avalanche or landslide is required to continue after one person is found, this can be communicated to the drone without the need to terminate the search already in progress. This is further illustrated in the flow chart in figure 7.

A further use scenario is illustrated in figure 4 and 5, wherein a search after a person/object 11 is performed using a heat sensitive camera 7. The search may start at for example a last known sighting, and for example be searched in an expanding square search pattern as illustrated in figure 6B. The image provided by the heat sensitive camera 7 may be analyzed by computer resources included locally in the sensor device/control module, or images may be transmitted to computing resources remote from the drone. In the case analysis is performed remote from the control module, result when object is identified will trigger a sequence of transmitting coordinates to the drone of location for the found object. The drone can, when object is identified, start process of identifying and for example illuminate the found person/object by a laser light pointer. The use of heat sensitive camera may be substituted by normal camera where the images are processed through an AI process wherein the system is trained to detect objects such as persons.

A video camera (not shown) may be provided in the sensor assembly. Video streaming may thereby be provided for easy and quick oversight of the search area. The video camera may be a low definition camera able to provide live video streaming over low bandwidth transmission lines.

Different communication protocols may be supported for supporting required amount of data necessary to be communicated. For example will a heat sensitive camera require more bandwidth than a low definition black and white camera.

In any of the search scenarios, it is within the scope of the present invention to use a mobile phone signal as a homing beacon wherein the sensor assembly/control module comprise a mobile phone base station, or the phone may be triangulated by nearby antenna masts, and the position is communicated wirelessly to the control module, and position is pinpointed by the drone as discussed above.

Another possibility is that the search object/person has a GPS tracker able to identify own location, and transmit this to the drone. GPS-tracking may be an application on a smartphone activated either by the owner. It is also foreseen that a smartphone tracking feature may be activated remotely by the drone.

The concept of triangulation may be used in a remotely accessible area where a mobile phone base station triangulating process cannot be performed, but instead of using existing infrastructure, the system may comprise one or more mobile communication nodes. Setting up communication nodes around the search area, for example 3 nodes, and using a communication device on the drone able to communicate/send beacon/signals to/from the nodes which enables the system to triangulate the drone position, and hence any persons or objects found.

The method would then be to arrange the nodes in the terrain at known positions before search is initiated. The triangulation may be performed either in the drone communication module, any of the nodes, or a communication control central being in communication with any of the nodes and/or the drone. Drone communication module is not bound to being a mobile phone, but could be any type of communication device, and communication protocol is not limited to a mobile phone protocol or frequencies. For example a device normally used for emergency communication/radio or avalanche transceivers may be employed.

If a search in a different scenario is initiated to find a specific type of car, a camera connected to an analyzing module comprising AI trained to recognize a pattern of the missing car type may be employed.

The present invention shall also be considered to encompass a miniature embodiment of the above discussed devices and systems. A light weight system is recognized as typically a size configuration for a complete system setup that can easily be carried by man to a search scene. Typically such a system has limited functionality, and may for example be limited to fly a predefined search pattern defined by the starting point, and transmit live video only to a home base 200 being defined by the launch site. A control central may be a processing unit carried by an operator. The control central may for example be implemented as an App on a smartphone, and communicating with the drone over a Bluetooth or Wi-Fi protocol.

The light weight version may also comprise a laser beam device, being controllable by the operator, and a feature letting the operator manually control the flight pattern and operate the laser beam to point at interesting search elements in the search area when video signal reveal interesting search results.

In some user scenario the home base 200 controlled launch site will be used and controlled by the rescue team 105.

A system implementation is illustrated in figure 8.

Figure 8 illustrates a system embodiment of the invention wherein the control module in the drone 1 comprises a wireless communication unit able to communicate a location to a rescue party 105 enabling the rescue party 5 to arrive at speed to the correct location. The wireless communication unit may also be able to transmit 103 sensor readings from the sensor assembly 4 or optional wearable worn by the searched person 2, 3, 11, in order for the searching party to be able to take intelligent decisions, such as send for emergency transport 107. Sensor readings may be related to body temperature of found person and other. The communication unit may further be able to communicate with a cloud or wide area network 100, and through this communicate 102 with a server service 101, the rescue teams 105, the transport 107 or a local alarm station 104. This can typically be an emergency service able to react to distress signals, and which may communicate 102 with appropriate control rescue teams 105 and emergency transportation 107.

Communication transfer medium 102, 103, 106 may be one of, wireless LAN or WAN, Bluetooth, WIFI, mobile network, radio communication, or other communication medium.

A further system feature may comprise a local alarm station 104 provided on site, for example at selected mountain locations. Each drone may regularly fly over a defined area mapping persons in various locations, and or at preset intervals communicate 103 with a local alarm station 104 to identify presence and no-distress signal. When an emergency situation is detected, the local alarm station 104 may be programmed to provide a list of persons out of danger, and who’s in a danger zone.

The invention can also be understood to comprise:

A first embodiment of a drone 1 adapted for searching after persons, animals or object in an outdoor environment, the drone comprising: a control module, the control module comprising processing means for defining a search area on the ground, a search mode selector comprising one or more search modes in automatic and/or manual search modes, and means for maneuvering the drone according to a predefined searching flight pattern when drone is activated in one of one or more automatic search modes, wherein the control module further comprise a homing device (6) for receiving a distress signal or emergency beacon from a search object (3) and wherein the control module (4) further comprise a predefined unique flight pattern for signaling position above target

A second embodiment of a drone 1 according to the first embodiment of the drone 1, wherein the control module further comprise a communication module able to receive flight maneuver commands from a control central and use these received flight maneuver commands to maneuver the drone when drone is activated in a manual search mode.

A third embodiment of a drone according to the first or second embodiment of the drone, wherein the outdoor environment is an avalanche or landslide disaster area.

A fourth embodiment of a drone according to the third embodiment of the drone, wherein the drone further comprise a distress signal receiver for receiving and analyzing the incoming direction of the distress signal or emergency beacon, and use the incoming direction to maneuver the drone towards the source of the distress signal or emergency beacon.

A fifth embodiment of a drone according to any of the first to fourth embodiment of the drone, wherein the drone further comprise a camera module comprising one or more cameras for monitoring by images or live video streaming the search area.

A sixth embodiment of a drone according to the fifth embodiment of the drone, wherein control module further comprise a communication device for transmitting data from camera to a remote controlling unit.

An seventh embodiment of a drone according to any of the fifth or sixth embodiment of the drone, wherein one of the cameras being a heat sensitive camera for identifying heat generating objects in the search area.

A eighth embodiment of a drone according to the seventh embodiment of the drone, wherein control module or remote controlling unit generates maneuver commands for maneuvering the drone to the heat generating object in the search area.

A ninth embodiment of a drone according to any of the fourth to seventh embodiment of the drone, wherein control module or remote controlling unit generates maneuver commands for maneuvering the drone to a target found using AI trained to identify search object.

A tenth embodiment of a drone according to any of the first to ninth embodiment of the drone, wherein drone further comprise a light beam device, the light beam being controlled to point towards an identified target, or control module defined direction.

A eleventh embodiment of a drone according to the tenth embodiment of the drone, wherein the light beam device is one or more of a colored laser light, led light, infrared light or UV light.

A twelfth embodiment of a drone according to any of the first to eleventh embodiment of the drone, wherein the drone further comprise a warning light bulb having one or more light pattern to signalize operation mode of drone.

A thirteenth embodiment of a drone according to any of the first to twelfth embodiment of the drone, wherein the drone further comprise one or more audio speakers having one or more sound pattern to signalize operation mode of drone.

A first system embodiment for persons, animals or objects search in an outdoor environment comprising:

drone (1) according to any of the first to thirteenth embodiment of the drone,

home base (200) for launching the drone (1),

controller central for initiating a search and defining a search area, and

the home base (200) further comprising a communication module for communicating data to and/or from the drone (1).

A second system embodiment according to the first system embodiment, wherein the controller central is comprised in the home base (200).

A third system embodiment according to the first or second system embodiment, wherein the system further comprise a remote server (101), and communication means for communicating sensor data from the drone (1) to the remote server (101) and control and maneuver commands from the remote server (101) to the drone (1).

A third system embodiment according to any of the first to third system embodiment, wherein drone or home base comprise communication means for communicating position data to a mobile rescue team (105, 107).

A first method embodiment for operation of a drone adapted for searching after persons, animals or object in an outdoor environment, wherein the drone according to any of the first to thirteenth embodiment of the drone is used, the method comprising the following steps:

at a launch site selecting search mode for the drone (1) by selecting one of automatic search patterns or manual search,

when automatic search pattern is selected, defining relative or absolute geographical boundaries for the search,

launching the drone (1) from a home base (200) for a search task.

A second method embodiment according to the first method embodiment, wherein the method further comprise when manual search pattern is selected, to use a remote maneuver control from the home base (200) or a remote server (101) to maneuver the drone over the required search area.

A third method embodiment according to any of the first or second method embodiment, wherein the method further comprising the step:

when an identified search object is found, the drone initiating the found persons, animals or object maneuver routine, and

the drone communicating found object.

A fourth method embodiment according to any of the first to third method embodiment, wherein method further comprising the drone identifying found persons, animals or object by lighting a light and pointing the light at the identified target.

A fifth method embodiment according to any of the first to fourth method embodiment, wherein the method further comprising the drone communicating photo or video stream of the found persons, animals or object to a home base (200) and or a remote server (101).

A sixth method embodiment according to any of the first to fifth method embodiment, wherein the method further comprising activating sound or light emitting device for identifying search mode and status of drone (1).

Claims

1. A drone (1) adapted for searching after persons, animals or object in an outdoor environment, the drone comprising:

a control module (4), the control module (4) comprising processing means for defining a search area on the ground, a search mode selector comprising one or more search modes in automatic and/or manual search modes, and means for maneuvering the drone according to a predefined searching flight pattern when drone is activated in one of one or more automatic search modes, the drone being c h a r a c t e r i z e d b y

the control module further comprise a homing device (6) for receiving a distress signal or emergency beacon from a search object (3) and wherein the control module (4) further comprise a predefined unique flight pattern for signaling position above target.

2. Drone (1) according to claim 1, wherein the control module (4) further comprise a communication module able to receive flight maneuver commands from a control central (101) and use these received flight maneuver commands to maneuver the drone when drone is activated in a manual search mode.

3. Drone (1) according to any of the previous claims, wherein the outdoor environment is an avalanche or landslide disaster area.

4. Drone (1) according to claim 3, wherein the drone (1) further comprise a distress signal receiver (6) for receiving and analyzing the incoming direction of the distress signal or emergency beacon, and use the incoming direction to maneuver the drone towards the source of the distress signal or emergency beacon.

5. Drone (1) according to any of the previous claims, wherein the drone further comprise a camera module (7) comprising one or more cameras for monitoring by images or live video streaming the search area.

6. Drone (1) according to claim 5, wherein the control module (4) further comprise a communication device for transmitting (102, 103, 106) data from camera to a remote controlling unit.

7. Drone (1) according to claim 5 or 6, wherein one of the cameras (7) being a heat sensitive camera for identifying heat generating objects in the search area.

8. Drone (1) according to claim 7, wherein control module (4) or remote controlling unit generates maneuver commands for maneuvering the drone to the heat generating object (11) in the search area.

9. Drone (1) according to any of the previous claims 4 to 7, wherein control module (4) or remote controlling unit generates maneuver commands for maneuvering the drone to a target found using AI trained to identify search object.

10. Drone (1) according to any of the previous claims wherein drone further comprise a light beam device (8), the light beam (10) being controlled to point towards an identified target (11), or control module defined direction.

11. Drone (1) according claim 10 wherein the light beam device (5) is one or more of a colored laser light, led light, infrared light or UV light.

12. Drone (1) according to any of the previous claims wherein the drone (1) further comprise a warning light bulb (8) having one or more light pattern to signalize operation mode of drone.

13. Drone (1) according to any of the previous claims wherein the drone (1) further comprise one or more audio speakers (9) having one or more sound pattern to signalize operation mode of drone.

14. System for persons, animals or objects search in an outdoor environment wherein the system is c h a r a c t e r i z e d b y comprising:

drone (1) according to any of claim 1 to 13,

home base (200) for launching the drone (1),

controller central for initiating a search and defining a search area, and

15. System according to claim 14, wherein the controller central is comprised in the home base (200).

16. System according to claim 14 or 15, wherein the system further comprise a remote server (101), and communication means for communicating sensor data from the drone (1) to the remote server (101) and control and maneuver commands from the remote server (101) to the drone (1).

17. System according to any of claim 14 to 16, wherein drone or home base comprise communication means for communicating position data to a mobile rescue team (105, 107).

18. Method for operation of a drone adapted for searching after persons, animals or object in an outdoor environment, wherein the drone according to any of claim 1 to 13 is used, the method being c h a r a c t e r i z e d b y comprising the following steps:

launching the drone (1) from a home base (200) for a search task.

19. Method according to claim 18, wherein the method further comprise when manual search pattern is selected, to use a remote maneuver control from the home base (200) or a remote server (101) to maneuver the drone over the required search area.

20. Method according to any of claim 18 or 19, the method further comprising the step: when an identified search object is found, the drone initiating the found persons, animals or object maneuver routine, and

the drone communicating found object.

21. Method according to any of claim 18 or 20, the method further comprising the drone identifying found persons, animals or object by lighting a light and pointing the light at the identified target.

22. Method according to any of claim 18 or 21, the method further comprising the drone communicating photo or video stream of the found persons, animals or object to a home base (200) and or a remote server (101).

23. Method according to any of claim 18 or 22, the method further comprising activating sound or light emitting device for identifying search mode and status of drone (1).