EP4062423A1 - Interactive modular empathic wearable system of detection - Google Patents

Interactive modular empathic wearable system of detection

Info

Publication number
EP4062423A1
EP4062423A1 EP20841785.7A EP20841785A EP4062423A1 EP 4062423 A1 EP4062423 A1 EP 4062423A1 EP 20841785 A EP20841785 A EP 20841785A EP 4062423 A1 EP4062423 A1 EP 4062423A1
Authority
EP
European Patent Office
Prior art keywords
audio
detection system
subsystem
stream
wearable detection
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
EP20841785.7A
Other languages
German (de)
French (fr)
Inventor
Mirco MANCIULLI
Gabriele CECCHETTI
Anna Lina RUSCELLI
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Scuola Superiore di Studi Universitari e di Perfezionamento SantAnna
Original Assignee
Scuola Superiore di Studi Universitari e di Perfezionamento SantAnna
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from IT202019000003344U external-priority patent/IT201900003344U1/en
Application filed by Scuola Superiore di Studi Universitari e di Perfezionamento SantAnna filed Critical Scuola Superiore di Studi Universitari e di Perfezionamento SantAnna
Publication of EP4062423A1 publication Critical patent/EP4062423A1/en
Pending legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H40/00—ICT specially adapted for the management or administration of healthcare resources or facilities; ICT specially adapted for the management or operation of medical equipment or devices
    • G16H40/60—ICT specially adapted for the management or administration of healthcare resources or facilities; ICT specially adapted for the management or operation of medical equipment or devices for the operation of medical equipment or devices
    • G16H40/63—ICT specially adapted for the management or administration of healthcare resources or facilities; ICT specially adapted for the management or operation of medical equipment or devices for the operation of medical equipment or devices for local operation

Definitions

  • the object of the present invention is an interactive wearable detection system for monitoring, tele-assistance, and/or telementoring in medical and health care fields. It is based on the integrated detection and transmission of biometric data of a patient, of the video stream on the patient's state, and the interaction of the audio stream between a local operator and a remote mentor, then it is particularly suitable for the integrations of components/elements suitably equipped with innovative and sustainable take-over technologies.
  • the invention provides a system based on a simple and cheap structure to implement, as it can use hardware components available on the market and non-proprietary software.
  • Such cameras are integrated with a microphone for audio transmission, but they are not generally equipped with a headset for receiving communications (bi-directional stream).
  • the action cameras are equipped with local storage devices (microSD card or SSD) and transmission devices via/through a Wi-Fi signal send to a location in its range.
  • wearable devices for POV shots transmission have been created that can be worn on the chest, but even these are not equipped with audio reception.
  • the existing wearable A/V (Audio/Video) devices are used in the amateur and recreational sector, in the professional sector to record events, and/or for medical procedures aimed at making educational videos and/or to send the image shooting of the operator's action to a passive public.
  • Telemedicine systems currently consist of devices aimed at a single purpose (eg electrocardiogram) and use a dedicated connection.
  • the first limit of state-of-the-art devices is that they do not allow interaction between the device wearer and the remote receiver.
  • the transmission when possible, thanks to a Wi-Fi connection, is limited by the range characterizing such technology (usually around 100 meters in the open field and less than 30 meters indoors).
  • a further limitation of the traditional systems is low fit, as they: - require specific supports; have a significant weight; when placed directly in contact with the wearer's head, they show overheating problems due to both the battery and the wireless transmission device.
  • these devices are equipped with a small integrated battery, then they are characterized by very limited energy autonomy.
  • Sensor systems for detecting biometric data are limited by being single devices that require a specific data transmission system and a specific software application on the receiver side.
  • said sensor systems are not integrated with the display of the remote operator/user, which is an extremely important feature for medical evaluation, nor do they allow the interaction between the operator and the remote doctor: then such systems are simply one-way data transmission.
  • the present invention differs from the cited document Dl: US2018 / 227658A1 (concerning the characterization and processing of environmental noise) and from the cited document D2: US9743001B1 (concerning an OIS + ES stabilization method of videos and images), since the aforementioned patents describe cameras of wearable typology (action cams) that do not allow the interaction between the wearer and said remote receiver, that are equipped with hooks on a helmet and/or fastening devices, then requiring specific supports and presenting poor wearability, that are characterized by not negligible weight and poor quality of the video stream due to the absence of video stabilization systems, which is affected by the well-known jittering occurring, for example, in the case of fast variation of the frame on all axes.
  • a video stream stabilization is needed, which can be performed/carried out in a specific hardware/software interaction using/through video post-processing.
  • These cameras are equipped with a microphone capable of audio transmission, but they are not equipped with a headset for receiving communications (bi-directional stream). Also, they are equipped with local storage devices (microSD card or SSD) and transmission devices via/through a Wi-Fi signal send to a location in its range. The transmission, when possible and taking place via/through a Wi-Fi connection, is limited by its distance range. Furthermore, sensor systems of biometric data detection are limited by being a set of individual devices, each of which requires a specific data transmission system and a specific software application on the receiver side.
  • the interactive wearable sensing system consists of four subsystems: a wearable monitoring subsystem , a coordination and control subsystem a data collection and transmission subsystem and a sensors subsystem.
  • the wearable monitoring subsystem consists of a wearable electronic device aimed at the acquisition of two-dimensional images in sequence, at predetermined capture rates according to the subjective perspective of the wearer, and that generates a video signal in digitized form.
  • the acquired data are sent to an image stabilization device that uses generated data by an inertial measurement unit (IMU) to drastically reduce image jittering due to the wearer's movements.
  • IMU inertial measurement unit
  • the stabilized data are sent to a hardware or software component that encodes them in a standardized compressed format (by way of example and not limiting the codecs H.264 or H.265).
  • the acquired video stream can be integrated with at least one audio track acquired through the microphone on the device or linked to it through a wired/wireless connection.
  • the resulting audio/video stream can be stored by a component for recording in digital format (DVR) directly connected to said wearable electronic device.
  • DVR digital format
  • the acquisition system with stabilization and compression is wired connected to the data transmission unit, which consists of a transceiver apparatus that provides for the transmission of multimedia audio/video using wireless technology on a public network (for example 3G / 4G / 5G type, or later) or private (public or private hot-spots, for example, Wi-Fi or Li-Fi).
  • All the components of the said wearable monitoring subsystem are powered by at least one battery, possibly light and which allows adequate autonomy for the use (for example made with LiFeP0 4 technology).
  • said wearable monitoring subsystem is equipped with an earphone or a headphone that reproduces the audio sent by the controlling subsystem , received through the transceiver component, and appropriately decoded.
  • the coordination and control subsystem allows the interaction with one or more wearable monitoring subsystems through a transceiver device connected to a private or public network.
  • Said data collection and transmission system consists of a data collection apparatus and a transceiver of video data, audio data, and information collected by a sensors subsystem through a wireless technology on a public network (for example 3G / 4G / 5G type, or later) or private (public or private hot-spots, for example, Wi-Fi or Li-Fi).
  • a public network for example 3G / 4G / 5G type, or later
  • private public or private hot-spots, for example, Wi-Fi or Li-Fi.
  • Said sensors subsystem consists of a set of sensors placed on support easily wearable by the user, connected via wireless to the data collection and transmission subsystem.
  • the information of the sensors is sent to said remote operator/user simultaneously with the audio and video streams.
  • said sensors subsystem includes an oximeter, a sweat sensor, a wrist blood pressure meter, and a wrist ECG, also interfacing digital phonendoscopes for the transmission of the audio file acquired.
  • Other sensors compatible with the system interface specifications can be added subsequently.
  • the proposed wearable monitoring subsystem allows you to perform various adjusting interventions, including:
  • - creation of an integrated communication system aimed at video transmission and two-way audio communication - creation of a transmission system aimed at communication via a private network (public or private hot-spots, for example, Wi-Fi or Li- Fi) or public (e.g. 3G, 4G, 5G type or later) when available, and guaranteeing maximum reliability and continuity, suitable for any sector (professional, sports, recreational, and health); - creation of an eventual additional transmission device, able to increase the range of the system even in areas affected by poor network signal availability;
  • a private network public or private hot-spots, for example, Wi-Fi or Li- Fi
  • public e.g. 3G, 4G, 5G type or later
  • the coordination and control subsystem can select the specific wearable monitoring subsystem to interact with;
  • the device of the present invention presents a further advantage given by the following improvements to the state of the art:
  • the quality of the video stream improved by about 50% compared to the non-stabilized one (or similarly, it eliminates 50% of the jittering effect or the continuous frame variation);
  • the elements of said wearable monitoring subsystem consist of: - at least one image stream acquisition component (a), i.e. a camera;
  • the elements of said data collection and transmission subsystem consist of:
  • - at least one component for the processing of the acquired images and audio streams which includes: - a Small Board Computer (SBC), for example, a Raspberry board or similar;
  • SBC Small Board Computer
  • an image processing software aimed at coordinating the data coming from said image stream acquisition component (a) and the data coming from said inertial measurement sensor (b), which corrects the video basing on the latter and aimed at stabilizing said video stream;
  • - software that compresses the audio and video streams in a standard format through audio (for example MP3, AAC) and video (for example H.264, H.265) codecs;
  • At least one electronic telecommunication device connected to said component for the processing of the acquired images and audio streams (d), aimed at the transmission of the said video stream and the communication of the two-way audio to said remote operator/user.
  • Said electronic telecommunication device (e) can be based on one or more communication technologies such as Wi-Fi, Li-Fi, 3G, 4G, 5G type or later, also combined, and the communication can be made safe also by current state of art encryption techniques supported by a transceiver system (j);
  • a local storage and archiving system (g), preferably consisting of solid-state memory or similar, capable of withstanding the typical stresses of a wearable device over time;
  • a wireless system suitable for connection with said sensors subsystem; - at least a first power supply component (i), or a battery capable of providing the energy needed for the operation.
  • the elements of said coordination and control subsystem consist of:
  • transceiver system (j) aimed at communicating with said electronic telecommunication device (e), and supporting adequate data transmission protocols and adequate security and encryption technologies;
  • bidirectional audio stream (c) including a workstation, or a mobile device with dedicated software to: selection of the controlled operator to interact with; visualization of the video stream; bidirectional audio management.
  • the elements of the sensors subsystem consist of: - a wearable device (m) equipped with sensors as an oximeter, a sweat sensor, a wrist ECG, a wrist blood pressure monitor, and a digital stethoscope;
  • n a wireless transmission system (n) able to electrically connect the sensors remotely and send the biometric data to the patient;
  • the invention is essentially conceived for some application areas/situations, including potential wearers in fast and continuous movement and requiring:
  • - SPORT a wearable system for referees and assistants to transmit the subjective of the same and receive eventual communications (for example from the VAR);
  • CIVIL PROTECTION system wearable by first responders who can transmit the subjective view and other details of the action area, receiving information from remote engineers or health professionals on the behavior and/or information/instructions to be performed;
  • SURGERY system wearable by the surgeon who can transmit his subjective view of the operating area to a remote mentor, receiving information/instructions on the surgical act;
  • INDUSTRIAL MAINTENANCE system wearable by local maintenance technicians who can cooperate with a remote expert by transmitting their subjective view and receiving technical information.
  • the device of the present inventions could be particularly relevant in case of maintenance or assistance interventions in extreme conditions and on complex technologies, such as off-shore platforms, boats, and industrial plants where expert technicians are not available;
  • REMOTE TUTORING the system can be particularly useful for didactic purposes, for example in case when an instructor, or a teacher, has to lead a practical exercise under non-critical conditions for self-learning of an experimental technique. The system allows you to adapt the characteristics and parameters of operations to the different use scenarios.
  • a more advanced stabilization can be set, while in surgery, an increase of the video resolution can be needed, and again, in the health emergency, the system can be configured to maximize the audio communication and the transmission stability.
  • Said wearable monitoring subsystem worn by an operator, allows acquiring the sequence of images constituting the video stream from the operator's view through said image stream acquisition component (a).
  • the acquired images are stabilized by said component for the processing of the acquired images and audio streams (d) using the information obtained from said inertial measurement sensor (b) integral with said image stream acquisition component (a).
  • the image stream after being encoded and compressed in a standard format, is integrated with the audio stream obtained from said device dedicated to interaction (k) through said bidirectional audio stream (c) and by said civil satellite positioning and localization system (f), thus forming an audio/video multimedia stream.
  • Such a multimedia audio/video stream can be optionally locally stored in the said wearable monitoring subsystem in said local storage and archiving system (g).
  • Said audio/video multimedia stream is subsequently sent to said remote operator/user through said electronic telecommunication device (e), preferably wireless, as based on Wi-Fi, Li-Fi, or other 3G, 4G, 5G technology or later.
  • said electronic telecommunication device e
  • e preferably wireless, as based on Wi-Fi, Li-Fi, or other 3G, 4G, 5G technology or later.
  • the devices constituting said wearable monitoring subsystem are powered by a first power supply component (i).
  • the operator/user placed at the coordination and control subsystem receives the multimedia audio/video stream transmitted by said wearable monitoring subsystem through said transceiver system (j) and reproduces such stream on the said system dedicated to the management, control, coordination, interaction and assistance (I).
  • Said data collection and transmission subsystem acquire the biometric data from said sensors subsystem via said wireless transmission system (n) connected to said wearable device (m), where said sensors subsystem is powered by said second power supply component (p).

Landscapes

  • Health & Medical Sciences (AREA)
  • Engineering & Computer Science (AREA)
  • Biomedical Technology (AREA)
  • Business, Economics & Management (AREA)
  • General Business, Economics & Management (AREA)
  • Epidemiology (AREA)
  • General Health & Medical Sciences (AREA)
  • Medical Informatics (AREA)
  • Primary Health Care (AREA)
  • Public Health (AREA)
  • Measuring And Recording Apparatus For Diagnosis (AREA)

Abstract

The document of the present invention describes a device suitable for monitoring, tele-assistance, and/or telementoring for the medical and health care sector, and its method of operation. It is applicable and suitable for the integration of components/elements suitably equipped with innovative and sustainable technologies, and consisting of four subsystems: a wearable monitoring subsystem, a coordination, and control subsystem, a data collection and transmission subsystem, and a sensors subsystem.

Description

"INTERACTIVE MODULAR EMPATHIC WEARABLE SYSTEM
OF DETECTION"
The object of the present invention is an interactive wearable detection system for monitoring, tele-assistance, and/or telementoring in medical and health care fields. It is based on the integrated detection and transmission of biometric data of a patient, of the video stream on the patient's state, and the interaction of the audio stream between a local operator and a remote mentor, then it is particularly suitable for the integrations of components/elements suitably equipped with innovative and sustainable take-over technologies.
The invention provides a system based on a simple and cheap structure to implement, as it can use hardware components available on the market and non-proprietary software.
STATE OF ART The state of the art of the sector is represented by the action cameras and, in particular, by all wearable cameras, typically used with hooks on a helmet and/or with fasteners.
Such cameras are integrated with a microphone for audio transmission, but they are not generally equipped with a headset for receiving communications (bi-directional stream). Currently, the action cameras are equipped with local storage devices (microSD card or SSD) and transmission devices via/through a Wi-Fi signal send to a location in its range.
For example, in the sports sector, wearable devices for POV shots transmission have been created that can be worn on the chest, but even these are not equipped with audio reception.
The existing wearable A/V (Audio/Video) devices are used in the amateur and recreational sector, in the professional sector to record events, and/or for medical procedures aimed at making educational videos and/or to send the image shooting of the operator's action to a passive public.
Telemedicine systems currently consist of devices aimed at a single purpose (eg electrocardiogram) and use a dedicated connection.
The first limit of state-of-the-art devices is that they do not allow interaction between the device wearer and the remote receiver.
The transmission, when possible, thanks to a Wi-Fi connection, is limited by the range characterizing such technology (usually around 100 meters in the open field and less than 30 meters indoors).
A further limitation of the traditional systems is low fit, as they: - require specific supports; have a significant weight; when placed directly in contact with the wearer's head, they show overheating problems due to both the battery and the wireless transmission device.
For this reason, they could be uncomfortable or annoying for the wearer. Usually, users consider the proximity of a radio wave transmitter to the head as negative.
Furthermore, these devices are equipped with a small integrated battery, then they are characterized by very limited energy autonomy.
Finally, the state of the art systems presents a poor quality of the video stream due to the absence of video stabilization, which is affected by the well-known jittering that occurs, for example, in the case of fast variation of the frame on all axes.
In such systems, it is, therefore, necessary the video stream stabilization which can be performed/carried out in a specific hardware/software interaction through/using video post-processing.
Sensor systems for detecting biometric data are limited by being single devices that require a specific data transmission system and a specific software application on the receiver side.
Furthermore, said sensor systems are not integrated with the display of the remote operator/user, which is an extremely important feature for medical evaluation, nor do they allow the interaction between the operator and the remote doctor: then such systems are simply one-way data transmission.
Both in the case of video systems and biometric data detection systems, the data streams are separate and could require the simultaneous use of different technologies that may not be compatible.
Given the cons above described, the use of these systems in the professional and/or health sectors, considered in the current state of the art, is poorly reliable and/or difficult to apply.
More in-depth, the present invention differs from the cited document Dl: US2018 / 227658A1 (concerning the characterization and processing of environmental noise) and from the cited document D2: US9743001B1 (concerning an OIS + ES stabilization method of videos and images), since the aforementioned patents describe cameras of wearable typology (action cams) that do not allow the interaction between the wearer and said remote receiver, that are equipped with hooks on a helmet and/or fastening devices, then requiring specific supports and presenting poor wearability, that are characterized by not negligible weight and poor quality of the video stream due to the absence of video stabilization systems, which is affected by the well-known jittering occurring, for example, in the case of fast variation of the frame on all axes. In such systems, a video stream stabilization is needed, which can be performed/carried out in a specific hardware/software interaction using/through video post-processing.
Furthermore, when placed directly in contact with the wearer's head, they show overheating problems due to both the battery and the wireless transmission device.
These cameras are equipped with a microphone capable of audio transmission, but they are not equipped with a headset for receiving communications (bi-directional stream). Also, they are equipped with local storage devices (microSD card or SSD) and transmission devices via/through a Wi-Fi signal send to a location in its range. The transmission, when possible and taking place via/through a Wi-Fi connection, is limited by its distance range. Furthermore, sensor systems of biometric data detection are limited by being a set of individual devices, each of which requires a specific data transmission system and a specific software application on the receiver side.
Therefore, the data streams are separate and could require the simultaneous use of different technologies that may not be compatible. The solution here proposed does not represent an obvious alternative of what proposed in the document D1 and the document D2, since it is objectively more novel and efficient in its use, even if it would be chosen (by the expert in the art) based on practical considerations and of the specific circumstances, it has never been proposed and/or studied for an alternative usage/employment, claimed by this invention, that could be more efficient. A qualified person, faced with the problem posed in the invention, would not use any of the solutions currently available in the market and would implement it in the D1 or D2 device, since even chosen, it was never designed/proposed to evaluate one peculiar feature of inventive evaluation for the reasons explained above. GENERAL DESCRIPTION OF THE INVENTION
Considering the configuration of the proposed invention, the interactive wearable sensing system consists of four subsystems: a wearable monitoring subsystem , a coordination and control subsystem a data collection and transmission subsystem and a sensors subsystem. The wearable monitoring subsystem consists of a wearable electronic device aimed at the acquisition of two-dimensional images in sequence, at predetermined capture rates according to the subjective perspective of the wearer, and that generates a video signal in digitized form.
The acquired data are sent to an image stabilization device that uses generated data by an inertial measurement unit (IMU) to drastically reduce image jittering due to the wearer's movements. The stabilized data are sent to a hardware or software component that encodes them in a standardized compressed format (by way of example and not limiting the codecs H.264 or H.265).
The acquired video stream can be integrated with at least one audio track acquired through the microphone on the device or linked to it through a wired/wireless connection.
The resulting audio/video stream can be stored by a component for recording in digital format (DVR) directly connected to said wearable electronic device. The acquisition system with stabilization and compression is wired connected to the data transmission unit, which consists of a transceiver apparatus that provides for the transmission of multimedia audio/video using wireless technology on a public network (for example 3G / 4G / 5G type, or later) or private (public or private hot-spots, for example, Wi-Fi or Li-Fi).
All the components of the said wearable monitoring subsystem are powered by at least one battery, possibly light and which allows adequate autonomy for the use (for example made with LiFeP04 technology). To allow complete interactivity, said wearable monitoring subsystem is equipped with an earphone or a headphone that reproduces the audio sent by the controlling subsystem , received through the transceiver component, and appropriately decoded.
The coordination and control subsystem allows the interaction with one or more wearable monitoring subsystems through a transceiver device connected to a private or public network.
It receives the audio/video stream sent by said wearable monitoring subsystem , decodes it, and allows video viewing and interaction with the operator that uses said wearable monitoring subsystem through two- way audio on a remote workstation or a mobile device using an application/software or a browser.
Said data collection and transmission system consists of a data collection apparatus and a transceiver of video data, audio data, and information collected by a sensors subsystem through a wireless technology on a public network (for example 3G / 4G / 5G type, or later) or private (public or private hot-spots, for example, Wi-Fi or Li-Fi).
Said sensors subsystem consists of a set of sensors placed on support easily wearable by the user, connected via wireless to the data collection and transmission subsystem.
The information of the sensors is sent to said remote operator/user simultaneously with the audio and video streams. In particular, said sensors subsystem includes an oximeter, a sweat sensor, a wrist blood pressure meter, and a wrist ECG, also interfacing digital phonendoscopes for the transmission of the audio file acquired. Other sensors compatible with the system interface specifications can be added subsequently.
ADVANTAGES OF THE INVENTION
The proposed wearable monitoring subsystem allows you to perform various adjusting interventions, including:
- video acquisition with onboard digital stabilization of the video stream to eliminate the need for post-processing;
- optimization of the wearability aimed at eliminating weight, supports, and overheating problems;
- creation of an integrated communication system aimed at video transmission and two-way audio communication; - creation of a transmission system aimed at communication via a private network (public or private hot-spots, for example, Wi-Fi or Li- Fi) or public (e.g. 3G, 4G, 5G type or later) when available, and guaranteeing maximum reliability and continuity, suitable for any sector (professional, sports, recreational, and health); - creation of an eventual additional transmission device, able to increase the range of the system even in areas affected by poor network signal availability;
- realization of a video storage system using a mass memory that can be used or excluded;
- the coordination and control subsystem can select the specific wearable monitoring subsystem to interact with;
- possibility of integration of different sensors typologies according to the operational needs of the specific case.
Moreover, the device of the present invention presents a further advantage given by the following improvements to the state of the art:
- the quality of the video stream improved by about 50% compared to the non-stabilized one (or similarly, it eliminates 50% of the jittering effect or the continuous frame variation);
- bidirectional audio stream, including the possibility of receiving indications and transmitting audio/video;
- wearability, allowing/enabling at the same time the acquisition of the exact subjective perspective;
- reliability, due to the architecture of the transmission system and the greater energy autonomy; - possibility of control, coordination, and interaction with multiple wearable monitoring subsystem ;
- possibility of using a single communication channel for sending and receiving audio, video, and information data from biometric sensors. These and other purposes of the present invention are achieved using/through a state of art operating platform to realize a single system capable to satisfy all the needs related to the described purposes.
In the current state of art, there is not available, and it has never been proposed, a device for the activities above described equal or similar to the one proposed by the present invention which guarantees the benefit of not using: servers/software with management and maintenance costs, specific IT skills, use of protection, encryption, backup and recovery systems.
DETAILED DESCRIPTION OF THE INVENTION The characteristics and advantages of the present invention will become more evident thanks to the following detailed description of its real configuration, illustrated by way of non-limiting example in the attached drawings using a logical representation of the functional structure, whose components are represented in a block diagram. The system consists of four subsystems described above:
- a wearable monitoring subsystem sketched in Fig. 1; - a data collection and transmission subsystem sketched in Fig. 2;
- a coordination and control subsystem sketched in Fig. 3;
- a sensors subsystem sketched in Fig. 4.
The elements of said wearable monitoring subsystem consist of: - at least one image stream acquisition component (a), i.e. a camera;
- at least one IMU sensor, or an inertial measurement sensor (b) integral with said image stream acquisition component (a);
- at least one device dedicated to the interaction (k) through a bidirectional audio stream (c), consisting for example of a headset with a microphone.
The elements of said data collection and transmission subsystem consist of:
- at least one component for the processing of the acquired images and audio streams (d), which includes: - a Small Board Computer (SBC), for example, a Raspberry board or similar;
- an image processing software, aimed at coordinating the data coming from said image stream acquisition component (a) and the data coming from said inertial measurement sensor (b), which corrects the video basing on the latter and aimed at stabilizing said video stream; - software that compresses the audio and video streams in a standard format through audio (for example MP3, AAC) and video (for example H.264, H.265) codecs;
- a communication software;
- software for decoding and reproducing the audio stream;
- at least one electronic telecommunication device (e) connected to said component for the processing of the acquired images and audio streams (d), aimed at the transmission of the said video stream and the communication of the two-way audio to said remote operator/user.
Said electronic telecommunication device (e) can be based on one or more communication technologies such as Wi-Fi, Li-Fi, 3G, 4G, 5G type or later, also combined, and the communication can be made safe also by current state of art encryption techniques supported by a transceiver system (j);
- a civil satellite positioning and localization system (f);
- a local storage and archiving system (g), preferably consisting of solid-state memory or similar, capable of withstanding the typical stresses of a wearable device over time;
- a wireless system (h), suitable for connection with said sensors subsystem; - at least a first power supply component (i), or a battery capable of providing the energy needed for the operation.
The elements of said coordination and control subsystem consist of:
- said transceiver system (j), aimed at communicating with said electronic telecommunication device (e), and supporting adequate data transmission protocols and adequate security and encryption technologies;
- at least one said device dedicated to the interaction (k) through said bidirectional audio stream (c);
- at least one system dedicated to the management, control, coordination, interaction, and assistance (I), aimed at the visualization of the video stream received by said transceiver system
(j), and the interactive communication with said remote/user operator through the use of said device dedicated to the interaction
(k) through said bidirectional audio stream (c), including a workstation, or a mobile device with dedicated software to: selection of the controlled operator to interact with; visualization of the video stream; bidirectional audio management.
The elements of the sensors subsystem consist of: - a wearable device (m) equipped with sensors as an oximeter, a sweat sensor, a wrist ECG, a wrist blood pressure monitor, and a digital stethoscope;
- a wireless transmission system (n) able to electrically connect the sensors remotely and send the biometric data to the patient;
- a means of remote connection (o) through said wireless transmission system (n) with said data collection and transmission subsystem)
- a second power supply component (p), or a battery to provide the energy needed for the operations.
The invention is essentially conceived for some application areas/situations, including potential wearers in fast and continuous movement and requiring:
- to show the subjective video to remote supervisors/mentors/assistants;
- to exchange two-way audio to a remote assistant.
These sectors are, without claiming to be exhaustive:
- SPORT : a wearable system for referees and assistants to transmit the subjective of the same and receive eventual communications (for example from the VAR);
- HEALTH EMERGENCY: system for wearable by the operators of the emergency services and medical emergency services, who can transmit their subjective view of the intervention area of the patient, receiving indications/ requests for specific information/instructions from the doctors in the operations center;
CIVIL PROTECTION: system wearable by first responders who can transmit the subjective view and other details of the action area, receiving information from remote engineers or health professionals on the behavior and/or information/instructions to be performed; SURGERY: system wearable by the surgeon who can transmit his subjective view of the operating area to a remote mentor, receiving information/instructions on the surgical act;
INDUSTRIAL MAINTENANCE: system wearable by local maintenance technicians who can cooperate with a remote expert by transmitting their subjective view and receiving technical information. The device of the present inventions could be particularly relevant in case of maintenance or assistance interventions in extreme conditions and on complex technologies, such as off-shore platforms, boats, and industrial plants where expert technicians are not available;
REMOTE TUTORING: the system can be particularly useful for didactic purposes, for example in case when an instructor, or a teacher, has to lead a practical exercise under non-critical conditions for self-learning of an experimental technique. The system allows you to adapt the characteristics and parameters of operations to the different use scenarios.
More specifically, in the sports sector, a more advanced stabilization can be set, while in surgery, an increase of the video resolution can be needed, and again, in the health emergency, the system can be configured to maximize the audio communication and the transmission stability.
PROCEDURE AND METHOD OF DETECTION CONSTITUTING THE DEVICE OF THIS INVENTION Said wearable monitoring subsystem , worn by an operator, allows acquiring the sequence of images constituting the video stream from the operator's view through said image stream acquisition component (a). The acquired images are stabilized by said component for the processing of the acquired images and audio streams (d) using the information obtained from said inertial measurement sensor (b) integral with said image stream acquisition component (a).
The image stream, after being encoded and compressed in a standard format, is integrated with the audio stream obtained from said device dedicated to interaction (k) through said bidirectional audio stream (c) and by said civil satellite positioning and localization system (f), thus forming an audio/video multimedia stream. Such a multimedia audio/video stream can be optionally locally stored in the said wearable monitoring subsystem in said local storage and archiving system (g).
Said audio/video multimedia stream is subsequently sent to said remote operator/user through said electronic telecommunication device (e), preferably wireless, as based on Wi-Fi, Li-Fi, or other 3G, 4G, 5G technology or later.
The devices constituting said wearable monitoring subsystem are powered by a first power supply component (i). The operator/user placed at the coordination and control subsystem receives the multimedia audio/video stream transmitted by said wearable monitoring subsystem through said transceiver system (j) and reproduces such stream on the said system dedicated to the management, control, coordination, interaction and assistance (I). Said data collection and transmission subsystem acquire the biometric data from said sensors subsystem via said wireless transmission system (n) connected to said wearable device (m), where said sensors subsystem is powered by said second power supply component (p).

Claims

1) Interactive wearable detection system, characterized in that it includes: a wearable monitoring subsystem, consisting of:
- at least one image stream acquisition component (a);
- at least one inertial measurement sensor (b) coupled with said image stream acquisition component (a);
- at least one device dedicated to the interaction (k) through a bidirectional audio stream (c); a data collection and transmission subsystem, consisting of:
- at least one component for the processing of the acquired images and audio streams (d), which includes:
- an SBC;
- an image processing software;
- a software that compresses the audio and video streams;
- a communication software;
- a software that decodes and reproduces the audio stream;
- at least one electronic telecommunication device (e) connected to said component for the processing of the acquired images and audio streams (d), in which said electronic telecommunication device (e) is supported by a transceiver system (j); - a civil satellite positioning and localization system (f);
- a local storage and archiving system (g);
- at least a first power supply component (i); a wireless system (h) suitable for connection with a sensors subsystem , in which: said sensors subsystem consists of:
- a wearable device (m) equipped with sensors including an oximeter, a sweat sensor, an ECG device, a wrist blood pressure measuring device, and a digital stethoscope;
- a wireless transmission system (n);
- a means of remote connection (o) through said wireless transmission system (n) with said data collection and transmission subsystem)
- a second power supply component (p); a coordination and control subsystem, consisting of:
- said transceiver system (j) in communication with said electronic telecommunication device (e);
- at least one said device dedicated to the interaction (k) through said bidirectional audio stream (c);
- at least one system dedicated to the management, control, coordination, interaction and assistance (I), including a workstation. 2) Interactive wearable detection system according to claim 1, characterized in that:
- said device dedicated to the interaction (k) through said bidirectional audio stream (c) consists of a headset with a microphone. 3) Interactive wearable detection system according to claim 1, characterized in that:
- said image stream acquisition component (a) is a camera.
4) Interactive wearable detection system according to claim 1, characterized in that: - said first power supply component (i) is a battery.
5) Interactive wearable detection system according to claim 4, characterized in that:
- said first power supply component (i) is a LiFePCM type.
6) Interactive wearable detection system according to claim 1, characterized in that:
- said second power supply component (p) is a battery.
7) Interactive wearable detection system according to claim 6, characterized in that:
- said second power supply component (p) is a LiFePCH type. 8) Interactive wearable detection system according to claim 1, characterized in that: - said electronic telecommunication device (e) is of wireless type.
9) Interactive wearable detection system according to claim 8, characterized in that:
- said electronic telecommunication device (e) is a Wi-Fi type. 10) Interactive wearable detection system according to claim 8, characterized in that:
- said electronic telecommunication device (e) is a 3G or 4G or 5G type or later.
11) Interactive wearable detection system according to claim 1, characterized in that:
- said electronic telecommunication device (e) is a Li-Fi type.
12) Procedure and method for monitoring an interactive wearable detection system according to claim 1, characterized in that: the image stream is acquired through said image acquisition component (a) coupled to said inertial measurement sensor (b) and stabilized by said component for the processing of the acquired images and audio streams (d) forming an audio/video multimedia stream obtained from said device dedicated to interaction (k) through said bidirectional audio stream (c) and from said civil satellite positioning and localization system (f) stored on said local storage and archiving system (g), wherein: said audio/video multimedia stream is sent to an operator/user via said electronic telecommunication device (e) supported by said transceiver system (j) and by reproducing said audio/video multimedia stream on said system dedicated to the the management, control, coordination, interaction and assistance (I).
EP20841785.7A 2019-11-22 2020-11-17 Interactive modular empathic wearable system of detection Pending EP4062423A1 (en)

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
IT202019000003344U IT201900003344U1 (en) 2019-11-22 2019-11-22 Interactive detection system
IT202000000691 2020-01-16
PCT/IT2020/050283 WO2021100067A1 (en) 2019-11-22 2020-11-17 Interactive modular empathic wearable system of detection

Publications (1)

Publication Number Publication Date
EP4062423A1 true EP4062423A1 (en) 2022-09-28

Family

ID=74183476

Family Applications (1)

Application Number Title Priority Date Filing Date
EP20841785.7A Pending EP4062423A1 (en) 2019-11-22 2020-11-17 Interactive modular empathic wearable system of detection

Country Status (2)

Country Link
EP (1) EP4062423A1 (en)
WO (1) WO2021100067A1 (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
PL443466A1 (en) * 2023-01-13 2024-07-15 Alfaresq Spółka Z Ograniczoną Odpowiedzialnością Integrated, multimedia information exchange system, especially in the emergency medical system

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR100789705B1 (en) * 2007-02-05 2008-01-03 동진메디칼 주식회사 Wireless auscultation system using multi-function digital stethoscope
US9743001B1 (en) 2016-02-19 2017-08-22 Fotonation Limited Method of stabilizing a sequence of images
US10405081B2 (en) 2017-02-08 2019-09-03 Bragi GmbH Intelligent wireless headset system

Also Published As

Publication number Publication date
WO2021100067A1 (en) 2021-05-27

Similar Documents

Publication Publication Date Title
CN103845113B (en) Wireless operation magnifying glass and its Application way, device and system
US20080146277A1 (en) Personal healthcare assistant
US20110021317A1 (en) System and method for displaying anonymously annotated physical exercise data
WO2006029322A3 (en) Tele-presence system that allows for remote monitoring/observation and review of a patient and their medical records
KR101321423B1 (en) Medical monitoring system by the smart phone
US20110270631A1 (en) Remote healthcare data-gathering and viewing system and method
US8045685B2 (en) Movable monitoring equipment and remote monitoring and directing system
Daou et al. Patient vital signs monitoring via android application
CN204496498U (en) A kind of Telemedicine System
WO2015181836A2 (en) Apparatus for mobile communication of bio sensor signals
KR101580559B1 (en) Medical image and information real time interaction transfer and remote assist system
CN105812009A (en) Device, terminal, system and method for child safety care
EP3119283A1 (en) Method and apparatus for exercise monitoring combining exercise monitoring and visual data with wireless wearable devices
US20200027568A1 (en) Physician House Call Portal
CN109875505A (en) A kind of integrated operation room remote medical consultation with specialists method based on virtual reality technology
WO2021100067A1 (en) Interactive modular empathic wearable system of detection
CN107315919B (en) Remote medical reproduction system
WO2009013518A1 (en) Computer system
CN110335658A (en) A kind of rehabilitation training system and recovery training method based on IMU technology
EP2988656A1 (en) System for remote diagnosis of a stroke
CN202619843U (en) Synchronous high-definition coding and decoding transmission equipment for output signal of medical system
IT201900003344U1 (en) Interactive detection system
CN209962705U (en) AR telemedicine monitoring mechanism
Toscano et al. Wearable biosensor monitor to support autonomous crew health and readiness to perform
RU105778U1 (en) WIRELESS REMOTE MONITORING SYSTEM FOR PATIENTS

Legal Events

Date Code Title Description
STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: UNKNOWN

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE INTERNATIONAL PUBLICATION HAS BEEN MADE

PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: REQUEST FOR EXAMINATION WAS MADE

17P Request for examination filed

Effective date: 20220609

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

DAV Request for validation of the european patent (deleted)
DAX Request for extension of the european patent (deleted)
RIN1 Information on inventor provided before grant (corrected)

Inventor name: RUSCELLI, ANNA LINA

Inventor name: CECCHETTI, GABRIELE

Inventor name: CASTOLDI, PIERO

Inventor name: MANCIULLI, MIRCO