US20190246910A1 - Imaging-Type Heart Rate Monitoring Device and Method Thereof - Google Patents

Imaging-Type Heart Rate Monitoring Device and Method Thereof Download PDF

Info

Publication number
US20190246910A1
US20190246910A1 US16/039,776 US201816039776A US2019246910A1 US 20190246910 A1 US20190246910 A1 US 20190246910A1 US 201816039776 A US201816039776 A US 201816039776A US 2019246910 A1 US2019246910 A1 US 2019246910A1
Authority
US
United States
Prior art keywords
information
transform
heart rate
image
unit
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Abandoned
Application number
US16/039,776
Inventor
Bing-Fei Wu
Po-Wei Huang
Tsong-Yang Tsou
Tzu-Min LIN
Meng-Liang Chung
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
National Yang Ming Chiao Tung University NYCU
Original Assignee
National Chiao Tung University NCTU
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by National Chiao Tung University NCTU filed Critical National Chiao Tung University NCTU
Priority to US16/039,776 priority Critical patent/US20190246910A1/en
Assigned to NATIONAL CHIAO TUNG UNIVERSITY reassignment NATIONAL CHIAO TUNG UNIVERSITY ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: CHUNG, MENG-LIANG, HUANG, PO-WEI, LIN, TZU-MIN, TSOU, TSONG-YANG, WU, BING-FEI
Publication of US20190246910A1 publication Critical patent/US20190246910A1/en
Abandoned legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G06F3/015—Input arrangements based on nervous system activity detection, e.g. brain waves [EEG] detection, electromyograms [EMG] detection, electrodermal response detection
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/0002—Remote monitoring of patients using telemetry, e.g. transmission of vital signals via a communication network
    • A61B5/0004—Remote monitoring of patients using telemetry, e.g. transmission of vital signals via a communication network characterised by the type of physiological signal transmitted
    • A61B5/0013—Medical image data
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/02—Detecting, measuring or recording pulse, heart rate, blood pressure or blood flow; Combined pulse/heart-rate/blood pressure determination; Evaluating a cardiovascular condition not otherwise provided for, e.g. using combinations of techniques provided for in this group with electrocardiography or electroauscultation; Heart catheters for measuring blood pressure
    • A61B5/021—Measuring pressure in heart or blood vessels
    • A61B5/02108—Measuring pressure in heart or blood vessels from analysis of pulse wave characteristics
    • A61B5/02125—Measuring pressure in heart or blood vessels from analysis of pulse wave characteristics of pulse wave propagation time
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/02—Detecting, measuring or recording pulse, heart rate, blood pressure or blood flow; Combined pulse/heart-rate/blood pressure determination; Evaluating a cardiovascular condition not otherwise provided for, e.g. using combinations of techniques provided for in this group with electrocardiography or electroauscultation; Heart catheters for measuring blood pressure
    • A61B5/024—Detecting, measuring or recording pulse rate or heart rate
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/02—Detecting, measuring or recording pulse, heart rate, blood pressure or blood flow; Combined pulse/heart-rate/blood pressure determination; Evaluating a cardiovascular condition not otherwise provided for, e.g. using combinations of techniques provided for in this group with electrocardiography or electroauscultation; Heart catheters for measuring blood pressure
    • A61B5/024—Detecting, measuring or recording pulse rate or heart rate
    • A61B5/02416—Detecting, measuring or recording pulse rate or heart rate using photoplethysmograph signals, e.g. generated by infrared radiation
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/72—Signal processing specially adapted for physiological signals or for diagnostic purposes
    • A61B5/7203—Signal processing specially adapted for physiological signals or for diagnostic purposes for noise prevention, reduction or removal
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/72—Signal processing specially adapted for physiological signals or for diagnostic purposes
    • A61B5/7235—Details of waveform analysis
    • A61B5/7253—Details of waveform analysis characterised by using transforms
    • A61B5/7257—Details of waveform analysis characterised by using transforms using Fourier transforms
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/72—Signal processing specially adapted for physiological signals or for diagnostic purposes
    • A61B5/7235—Details of waveform analysis
    • A61B5/7253—Details of waveform analysis characterised by using transforms
    • A61B5/726—Details of waveform analysis characterised by using transforms using Wavelet transforms
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/72—Signal processing specially adapted for physiological signals or for diagnostic purposes
    • A61B5/7235—Details of waveform analysis
    • A61B5/7264—Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/72—Signal processing specially adapted for physiological signals or for diagnostic purposes
    • A61B5/7271—Specific aspects of physiological measurement analysis
    • A61B5/7275—Determining trends in physiological measurement data; Predicting development of a medical condition based on physiological measurements, e.g. determining a risk factor
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/72—Signal processing specially adapted for physiological signals or for diagnostic purposes
    • A61B5/7271—Specific aspects of physiological measurement analysis
    • A61B5/7278—Artificial waveform generation or derivation, e.g. synthesising signals from measured signals
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/74—Details of notification to user or communication with user or patient ; user input means
    • A61B5/7475—User input or interface means, e.g. keyboard, pointing device, joystick
    • A61B5/748—Selection of a region of interest, e.g. using a graphics tablet
    • A61B5/7485—Automatic selection of region of interest
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/017—Gesture based interaction, e.g. based on a set of recognized hand gestures
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
    • G06F3/0304—Detection arrangements using opto-electronic means
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/42—Global feature extraction by analysis of the whole pattern, e.g. using frequency domain transformations or autocorrelation
    • G06V10/431—Frequency domain transformation; Autocorrelation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/52—Scale-space analysis, e.g. wavelet analysis
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/174—Facial expression recognition
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B2576/00—Medical imaging apparatus involving image processing or analysis
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/0059—Measuring for diagnostic purposes; Identification of persons using light, e.g. diagnosis by transillumination, diascopy, fluorescence
    • A61B5/0077—Devices for viewing the surface of the body, e.g. camera, magnifying lens
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/02—Detecting, measuring or recording pulse, heart rate, blood pressure or blood flow; Combined pulse/heart-rate/blood pressure determination; Evaluating a cardiovascular condition not otherwise provided for, e.g. using combinations of techniques provided for in this group with electrocardiography or electroauscultation; Heart catheters for measuring blood pressure
    • A61B5/024—Detecting, measuring or recording pulse rate or heart rate
    • A61B5/02405—Determining heart rate variability
    • A61B5/0402—
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/72—Signal processing specially adapted for physiological signals or for diagnostic purposes
    • A61B5/7203—Signal processing specially adapted for physiological signals or for diagnostic purposes for noise prevention, reduction or removal
    • A61B5/7207—Signal processing specially adapted for physiological signals or for diagnostic purposes for noise prevention, reduction or removal of noise induced by motion artifacts
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/72—Signal processing specially adapted for physiological signals or for diagnostic purposes
    • A61B5/7235—Details of waveform analysis
    • A61B5/725—Details of waveform analysis using specific filters therefor, e.g. Kalman or adaptive filters
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/14—Vascular patterns
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/15—Biometric patterns based on physiological signals, e.g. heartbeat, blood flow

Definitions

  • the present invention provides an imaging-type heart rate monitoring device and method thereof. Through extracting the human image information and time-frequency analysis, filtering the residual noise, rebuilding the continuous wavelet and further Fourier transform, to obtain the HR value with lesser noise and higher accuracy.
  • the embodiment of the present invention provides an imaging-type physiological information monitoring device and method, comprising: acquiring at least a human image information of at least a human skin area; tracking at least a target area in the human image information; extracting the target image of the target area; transforming the human image information through a first frequency domain, to obtain a first frequency domain transformed information; filtering the first frequency domain transformed information, to obtain a residual noise filtered information; rebuilding the residual noise filtered information by the first frequency domain transform, to obtain a first frequency domain rebuilt information; transforming the first frequency domain rebuilt information by a second frequency domain transform, to obtain a second frequency domain transformed information; and obtaining a physiological information by the second frequency domain transformed information.
  • the physiological information of the invention comprises generating a heart rate information, generating a heart rate variability information, and generating a respiratory information.
  • the first frequency domain transformed information is a time-frequency analysis, which includes but not limited to continuous wavelet transform or Short Time Fourier Transform
  • the first frequency domain rebuilt information is an inverse time-frequency analysis, which includes but not limited to continuous wavelet transform rebuilt information or inverse Short Time Fourier Transform rebuilt information.
  • the second frequency domain transform includes but not limited to Fourier transform.
  • the embodiment of the present invention provides an imaging-type physiological information monitoring device, comprising: an image acquisition unit to acquire at least a human image information of at least a human skin area; an image treatment unit to electrically connect the image acquisition unit, in order to receive the human image information transmitted from the image acquisition unit; a continuous wavelet transform unit to electrically connect the image treatment unit, in order to conduct the continuous wavelet transform to the human image information to obtain a continuous wavelet transform information, or rebuild the continuous wavelet transform information to obtain a continuous wavelet transform rebuilt information; a Fourier transform unit to electrically connect the image treatment unit, to receive the continuous wavelet transform rebuilt information, and employ a Fourier transform unit to conduct Fourier transform to obtain a Fourier transform information; a storage unit to electrically connect the image treatment unit, to store human image information; and an output unit to electrically connect the image treatment unit, to output the heart rate value; where the image treatment unit filters the residual noise in the continuous wavelet transform information, to obtain a residual noise filtered information, and the image treatment unit obtain the physiological information in accordance with
  • the image treatment unit extracts a target image in the target area.
  • FIG. 1 illustrates a flowchart of image-type heart beat measurement method of the present invention
  • FIG. 3 schematically illustrates the use of an image-type heart beat monitoring device of the present invention.
  • the specific embodiments are used to describe the imaging-type physiological information monitoring device and method of the present invention.
  • the people who are familiar to this art can understand the advantages and efficacies of the present invention easily from the content disclosed in this article.
  • the present invention can also be implemented or applied by other different embodiments. Every detail in this article can also be modified and changed based on different viewpoints and applications without violating the spirit of the present invention.
  • the figures in the present invention are only brief description, and they are not drawn in actual dimension to reflect the actual size.
  • the following description of preferred embodiment describes the viewpoint of the present invention in more detail, which will not limit the scope of the present invention by any viewpoint.
  • FIG. 1 illustrates a flowchart of image-type heart beat measurement method of the present invention.
  • the imaging-type physiological information monitoring method comprises the following steps:
  • Step S 104 in FIG. 1 in which tracking at least a target area in the human image information. It has to describe that this target area is the exposed skin area of human, which includes but not limited to the face, hand or neck of human.
  • Step S 110 of FIG. 1 in which filtering the first frequency domain transformed information, in order to obtain a residual noise filtered information.
  • Step S 112 of FIG. 1 in which rebuilding the residual noise filtered information by the first frequency domain transform, in order to obtain a first frequency domain rebuilt information.
  • Step S 114 of FIG. 1 in which transforming the first frequency domain rebuilt information by a second frequency domain transform, in order to obtain a second frequency domain transformed information.
  • the second frequency domain transform is a Fourier transform.
  • the second frequency domain transformed information includes but limited to Fast Fourier transform information. That is, a first frequency domain rebuilt information is transformed by a second frequency domain transform, to obtain a second frequency domain transformed information.
  • Step S 116 of FIG. 1 in which obtaining a physiological information by the second frequency domain transformed information.
  • the physiological information includes but not limited to heart rate (HR), heart rate variability, or respiratory.
  • HR heart rate
  • the physiological information of the invention comprises generating a heart rate information, generating a heart rate variability information, and generating a respiratory information.
  • FIG. 2 illustrates an imaging-type heart beat monitoring device of the present invention.
  • FIG. 2 shows an imaging-type heart rate monitoring device 200 , comprising: an image treatment unit 202 , an image acquisition unit 204 , a continuous wavelet transform unit 206 , a Fourier transform unit 208 , an output unit 210 , and a storage unit 212 .
  • the function of the image acquisition unit 204 is to acquire at least a human image information of at least a human skin area.
  • the function of the time-frequency analysis unit 206 in FIG. 2 is to conduct the continuous wavelet transform for the human image information d 1 to obtain a continuous wavelet transform information d 2 , or rebuild the continuous wavelet transform information S 2 , to obtain a continuous wavelet transform rebuilt information d 3 . It has to describe that rebuild the continuous wavelet transform information S 2 is an inverse continuous wavelet transform step.
  • the continuous wavelet transform unit 206 is electrically connected to the image treatment unit 202 .
  • time-frequency analysis comprises the continuous wavelet transform, or the short time Fourier transform.
  • the function of the storage unit 212 in FIG. 2 is to store the human image information d 1 .
  • the storage unit 212 is electrically connected to the image treatment unit 202 .
  • FIG. 3 schematically illustrates the use of an imaging-type heart beat monitoring device of the present invention.
  • the image acquisition unit 204 takes the photo image of human body 302 .
  • the image acquisition unit 204 tracks at least a target area T 1 in the human image information d 1 .
  • the image treatment unit 202 in FIG. 3 is used to extract a target image dT in the target area T 1 .

Landscapes

  • Health & Medical Sciences (AREA)
  • Engineering & Computer Science (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Physics & Mathematics (AREA)
  • General Health & Medical Sciences (AREA)
  • Biomedical Technology (AREA)
  • Molecular Biology (AREA)
  • Veterinary Medicine (AREA)
  • Public Health (AREA)
  • Heart & Thoracic Surgery (AREA)
  • Medical Informatics (AREA)
  • Pathology (AREA)
  • Animal Behavior & Ethology (AREA)
  • Biophysics (AREA)
  • Surgery (AREA)
  • Physiology (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Artificial Intelligence (AREA)
  • Signal Processing (AREA)
  • Cardiology (AREA)
  • Psychiatry (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • Multimedia (AREA)
  • Mathematical Physics (AREA)
  • Neurosurgery (AREA)
  • Neurology (AREA)
  • Dermatology (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Radiology & Medical Imaging (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Vascular Medicine (AREA)
  • Evolutionary Computation (AREA)
  • Fuzzy Systems (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • Measuring And Recording Apparatus For Diagnosis (AREA)
  • Measuring Pulse, Heart Rate, Blood Pressure Or Blood Flow (AREA)
  • Image Analysis (AREA)

Abstract

The present invention provides an imaging-type heart rate monitoring device and method thereof. It acquires the human image information tracks the target area in the human image information, and extracts the target image in the target area. It transforms the human image information through the first frequency domain transform. It filters the residual noise of the continuous wavelet transform information. It rebuilds the residual noise filtered information by the continuous wavelet transform. It transforms the continuous wavelet transform rebuilt information by the Fourier transform. It obtains a heart rate value through the second frequency domain transformed information.

Description

    BACKGROUND OF THE INVENTION 1. Field of the Invention
  • The present invention relates to an imaging-type physiological information monitoring device and method thereof, particularly to a device and method for an imaging-type heart rate monitoring without the need to contact human body directly.
    • 2. Description of the Prior Art
  • The heart rate (HR) can be regarded as an important physiological characteristic information in humans. It also can be regarded as an important parameter for assessing the human health. Furthermore, it is an important factor for identifying the heart health state and other clinical diagnosis risk prediction. The daily measurement will contribute to the monitoring of human health, and the prevention of relevant diseases, particularly to the prediction for the body function degradation and disease risk of aging people.
  • At present, the Electrocardiogram (EKG/ECG) is the most popular and most accurate method for measuring the heart rate. However, because the electrocardiogram is a contact HR measurement, it is necessary to employ the sensors to contact with the human skin, which might often cause the inconvenience of operation and the peeling or sticking.
  • The conventional non-contact HR measurement includes electrocardiography (EGG), Doppler radar, microwave Doppler radar, photoplethysmography (PPG), remote photoplethysmography (rPPG) and thermal imaging etc. However, the cost of the above-mentioned methods will very high, and the relevant information only can be obtained by the professional operation method.
  • On the other hand, in recent years, the family medical diagnostic measurement and remote health state monitoring have seen increased attention. However, at present, it is still at the starting stage, thus it is quite urgent to find a robust low-cost, convenient-operation, and real time non-contact HR measurement method.
  • In addition, the development of wearable devices has already been grown vigorously now, and has become the mainstream of HR measurement technology in sports. However, the wearable devices still have some uncomfortable problems to the users, such as the perspiration after exercise, skin allergic phenomenon by contacting wearable device, or physical discomfort by the constraint of wearable device. Therefore, in order to solve the problems faced by wearable devices, there are a lot of non-contact HR measurement studies in recent years.
  • In the past few years, with the development of computer vision technology, the photoplethysmography (PPG) had been utilized to treat the video frequency adopted from human skin, to realize the HP measurement. However, these methods often adopt the linear wave filter to reduce the residual noise of video frequency, which is unable to reduce the residual noise located at the same frequency range of HR signal, and cause the decrease of HR extraction accuracy.
  • In the HR monitoring of conventional non-contact photoplethysmography (PPG), the best signal in three optical channels of image is extracted for analysis mainly, in order to obtain the HR value. Even with the obtained HR value, there is still some residual noise with short time duration but high energy, which will cause the decrease of HR extraction accuracy. In addition, the human face contacts the surrounding environment directly, and has great influence from the ambient temperature, which will cause the decrease of human face HR extraction accuracy and stability. Thus, the abovementioned non-contact HR measurement method still have several drawbacks, such as high noise at the same HR frequency range, great influence of ambient temperature, and lack for judging the authenticity of separated signal etc.
  • In view of this, in order to meet the abovementioned demand, the present invention provides an imaging-type heart rate monitoring device and method thereof. Through extracting the human image information and time-frequency analysis, filtering the residual noise, rebuilding the continuous wavelet and further Fourier transform, to obtain the HR value with lesser noise and higher accuracy.
    • SUMMARY OF THE INVENTION
  • The embodiment of the present invention provides an imaging-type physiological information monitoring device and method thereof. Through the time-frequency analysis and the filtering of residual noise, dissolve the problem of residual noise with short time duration but high energy generated from present non-contact imaging-type physiological information monitoring, and increase the accuracy of measurement.
  • In order to achieve the abovementioned purpose, the embodiment of the present invention provides an imaging-type physiological information monitoring device and method, comprising: acquiring at least a human image information of at least a human skin area; tracking at least a target area in the human image information; extracting the target image of the target area; transforming the human image information through a first frequency domain, to obtain a first frequency domain transformed information; filtering the first frequency domain transformed information, to obtain a residual noise filtered information; rebuilding the residual noise filtered information by the first frequency domain transform, to obtain a first frequency domain rebuilt information; transforming the first frequency domain rebuilt information by a second frequency domain transform, to obtain a second frequency domain transformed information; and obtaining a physiological information by the second frequency domain transformed information.
  • The embodiment of the present invention is after acquiring at least a human image information of at least a human skin area, further comprises tracking at least a target area in the human image information.
  • The embodiment of the present invention is after tracking at least a target area in the human image information, further comprises extracting the target image of the target area.
  • In the embodiment of the present invention, the first frequency domain transform is a time-frequency analysis, which includes but not limited to continuous wavelet transform or Short Time Fourier Transform.
  • In the embodiment of the present invention, the physiological information of the invention comprises generating a heart rate information, generating a heart rate variability information, and generating a respiratory information.
  • In the embodiment of the present invention, the first frequency domain transformed information is a time-frequency analysis, which includes but not limited to continuous wavelet transform or Short Time Fourier Transform, and the first frequency domain rebuilt information is an inverse time-frequency analysis, which includes but not limited to continuous wavelet transform rebuilt information or inverse Short Time Fourier Transform rebuilt information.
  • In the embodiment of the present invention, the second frequency domain transform includes but not limited to Fourier transform.
  • In the embodiment of the present invention, the second frequency domain transformed information includes but not limited to Fourier transformed information.
  • The embodiment of the present invention provides an imaging-type physiological information monitoring device, comprising: an image acquisition unit to acquire at least a human image information of at least a human skin area; an image treatment unit to electrically connect the image acquisition unit, in order to receive the human image information transmitted from the image acquisition unit; a continuous wavelet transform unit to electrically connect the image treatment unit, in order to conduct the continuous wavelet transform to the human image information to obtain a continuous wavelet transform information, or rebuild the continuous wavelet transform information to obtain a continuous wavelet transform rebuilt information; a Fourier transform unit to electrically connect the image treatment unit, to receive the continuous wavelet transform rebuilt information, and employ a Fourier transform unit to conduct Fourier transform to obtain a Fourier transform information; a storage unit to electrically connect the image treatment unit, to store human image information; and an output unit to electrically connect the image treatment unit, to output the heart rate value; where the image treatment unit filters the residual noise in the continuous wavelet transform information, to obtain a residual noise filtered information, and the image treatment unit obtain the physiological information in accordance with the Fourier transform information.
  • In the embodiment of the present invention, the image acquisition unit tracks at least a target area in the human image information.
  • In the embodiment of the present invention, the image treatment unit extracts a target image in the target area.
  • In order to further understand the features and technological content of the present invention, please refer to the following detailed description and attached figures of the present invention. Nevertheless, the attached figures are used for reference and description, which are not used for limiting the present invention.
    • BRIEF DESCRIPTION OF THE DRAWINGS
  • The foregoing aspects and many of the attendant advantages of this invention will become more readily appreciated as the same becomes better understood by reference to the following detailed description, when taken in conjunction with the accompanying drawings, wherein:
  • FIG. 1 illustrates a flowchart of image-type heart beat measurement method of the present invention;
  • FIG. 2 illustrates an image-type heart beat monitoring device of the present invention; and
  • FIG. 3 schematically illustrates the use of an image-type heart beat monitoring device of the present invention.
    •  DESCRIPTION OF THE PREFERRED EMBODIMENT
  • In the following context, the specific embodiments are used to describe the imaging-type physiological information monitoring device and method of the present invention. The people who are familiar to this art can understand the advantages and efficacies of the present invention easily from the content disclosed in this article. The present invention can also be implemented or applied by other different embodiments. Every detail in this article can also be modified and changed based on different viewpoints and applications without violating the spirit of the present invention. In addition, the figures in the present invention are only brief description, and they are not drawn in actual dimension to reflect the actual size. The following description of preferred embodiment describes the viewpoint of the present invention in more detail, which will not limit the scope of the present invention by any viewpoint.
  • Please refer to FIG. 1. FIG. 1 illustrates a flowchart of image-type heart beat measurement method of the present invention. The imaging-type physiological information monitoring method comprises the following steps:
  • Continuously, please refer to Step S102 in FIG. 1, in which acquiring at least a human image information of at least a human skin area.
  • Then, please refer to Step S102 FIG. 1, and further including Step S104 and Step S106.
  • Again, please refer to Step S104 in FIG. 1, in which tracking at least a target area in the human image information. It has to describe that this target area is the exposed skin area of human, which includes but not limited to the face, hand or neck of human.
  • Then, refer to Step S106 in FIG. 1, in which extracting the target image of the target area.
  • Further refer to Step S108 in FIG. 1, in which transforming the human image information through a first frequency domain, in order to obtain a first frequency domain transformed information, where the first frequency domain transform is a time-frequency analysis, which includes but not limited to continuous wavelet transform or Short Time Fourier Transform.-That is the human image information is transformed through the time-frequency analysis to obtain a time-frequency transform information.
  • In Step S110 of FIG. 1, in which filtering the first frequency domain transformed information, in order to obtain a residual noise filtered information.
  • Further in Step S112 of FIG. 1, in which rebuilding the residual noise filtered information by the first frequency domain transform, in order to obtain a first frequency domain rebuilt information.
  • And in Step S114 of FIG. 1, in which transforming the first frequency domain rebuilt information by a second frequency domain transform, in order to obtain a second frequency domain transformed information. Where the second frequency domain transform is a Fourier transform. The second frequency domain transformed information includes but limited to Fast Fourier transform information. That is, a first frequency domain rebuilt information is transformed by a second frequency domain transform, to obtain a second frequency domain transformed information.
  • Further in Step S116 of FIG. 1, in which obtaining a physiological information by the second frequency domain transformed information.
  • The physiological information includes but not limited to heart rate (HR), heart rate variability, or respiratory. Normally, the physiological information of the invention comprises generating a heart rate information, generating a heart rate variability information, and generating a respiratory information.
  • In another embodiment, please refer to FIG. 2. FIG. 2 illustrates an imaging-type heart beat monitoring device of the present invention.
  • FIG. 2 shows an imaging-type heart rate monitoring device 200, comprising: an image treatment unit 202, an image acquisition unit 204, a continuous wavelet transform unit 206, a Fourier transform unit 208, an output unit 210, and a storage unit 212. Where the function of the image acquisition unit 204 is to acquire at least a human image information of at least a human skin area.
  • The function of the image treatment unit 202 in FIG. 2 is to receive the human image information d1 transmitted from the image acquisition unit 204, and the image treatment unit 202 is electrically connected to the image acquisition unit 204.
  • The function of the time-frequency analysis unit 206 in FIG. 2 is to conduct the continuous wavelet transform for the human image information d1 to obtain a continuous wavelet transform information d2, or rebuild the continuous wavelet transform information S2, to obtain a continuous wavelet transform rebuilt information d3. It has to describe that rebuild the continuous wavelet transform information S2 is an inverse continuous wavelet transform step. The continuous wavelet transform unit 206 is electrically connected to the image treatment unit 202. In fact, time-frequency analysis comprises the continuous wavelet transform, or the short time Fourier transform.
  • The function of the frequency transform unit 208 in FIG. 2 is to receive a continuous wavelet transform rebuilt information, in which the frequency transform is conducted by a Fourier transform unit 208, to obtain a Fourier transform information d4. The frequency transform unit 208 is electrically connected to the image treatment unit 202. In fact, frequency transform unit 208 comprises the Fourier transform, or fast Fourier transform.
  • The output unit 210 in FIG. 2 is electrically connected to the image treatment unit 202, its function is to output the heart rate value HR, The image treatment unit 202 filters the residual noise in the continuous wavelet transform information d1, to obtain a residual noise filtered information d5. The image treatment unit 202 obtains the heart rate value HR in accordance with the Fourier transform information d4. The output unit 210 is electrically connected to the image treatment unit 202. Also, “HR” for the output unit 210 in FIG. 2 is shown as the heart rate (HR).
  • The function of the storage unit 212 in FIG. 2 is to store the human image information d1. The storage unit 212 is electrically connected to the image treatment unit 202.
  • Please refer to FIG. 2 and FIG. 3. FIG. 3 schematically illustrates the use of an imaging-type heart beat monitoring device of the present invention. As shown in FIG. 3, the image acquisition unit 204 takes the photo image of human body 302. The image acquisition unit 204 tracks at least a target area T1 in the human image information d1. The image treatment unit 202 in FIG. 3 is used to extract a target image dT in the target area T1.
  • It is understood that various other modifications will be apparent to and can be readily made by those skilled in the art without departing from the scope and spirit of the invention. Accordingly, it is not intended that the scope of the claims appended hereto be limited to the description as set forth herein, but rather that the claims be construed as encompassing all the features of patentable novelty that reside in the present invention, including all features that would be treated as equivalents thereof by those skilled in the art to which the invention pertains.

Claims (16)

What is claimed is:
1. An imaging-type heart rate monitoring method, comprising:
acquiring at least a human image information of at least a human skin area;
transforming said human image information through a first frequency domain transform in order to obtain a first frequency domain transformed information;
filtering said first frequency domain transformed information in order to obtain a residual noise filtered information;
rebuilding said residual noise filtered information by said first frequency domain transform in order to obtain said first frequency domain rebuilt information;
transforming said first frequency domain rebuilt information by a second frequency domain transform in order to obtain a second frequency domain transformed information; and
obtaining a physiological information by said second frequency domain transformed information.
2. The method according to claim 1, wherein physiological information comprises generate a heart rate information.
3. The method according to claim 1, wherein physiological information comprises generate a heart rate variability information.
4. The method according to claim 1, wherein physiological information comprises generate a respiratory information.
5. The imaging-type heart rate monitoring method according to claim 1, wherein after the step of acquiring at least a human image information of at least a human skin area, further comprising tracking at least a target area in the human image information.
6. The imaging-type heart rate monitoring method according to claim 5, wherein after the step of tracking at least a target area in the human image information, further comprising extracting the target image of the target area.
7. The imaging-type heart rate monitoring method according to claim 6, wherein the first frequency domain transform is a continuous wavelet transform.
8. The imaging-type heart rate monitoring method according to claim 7, wherein the first frequency domain transform information is a continuous wavelet transform information, the first frequency domain rebuilt information is a continuous wavelet transform rebuilt information.
9. The package structure according to claim 6, wherein the second frequency domain transform is a Fourier transform.
10. The imaging-type heart rate monitoring method according to claim 9, wherein the second frequency domain transform information is a Fourier transform information.
11. An imaging-type heart rate monitoring device, comprising:
an image acquisition unit, said image acquisition unit for acquiring at least a human image information of at least a human skin area;
an image treatment unit, said image treatment unit for electrically connecting said image acquisition unit in order to receive said human image information transmitted from said image acquisition unit;
a time-frequency transform unit, said time-frequency transform unit for electrically connecting said image treatment unit in order to conduct said time-frequency transform to said human image information to obtain a time-frequency transform information, or rebuild said time-frequency transform information to obtain a time-frequency transform rebuilt information;
a frequency transform unit, said Fourier transform unit for electrically connect said image treatment unit in order to receive said time-frequency analysis rebuilt information, and employ a Fourier transform unit in order to conduct Fourier transform to obtain a Fourier transform information;
a storage unit, said storage unit for electrically connect said image treatment unit in order to store a human image information; and
an output unit to electrically connect said image treatment unit in order to output a physiological information;
wherein, said image treatment unit filters a residual noise in said time-frequency analysis, to obtain said residual noise filtered information, and said image treatment unit obtain said physiological information in accordance with said Fourier transform information.
12. The imaging-type heart rate monitoring device according to claim 11, wherein said time-frequency transform unit comprises a continuous wavelet transform unit.
13. The imaging-type heart rate monitoring device according to claim 11, wherein said time-frequency transform unit comprises a short time Fourier transform unit.
14. The imaging-type heart rate monitoring device according to claim 11, wherein said frequency transform unit comprises a fast Fourier transform unit.
15. The imaging-type heart rate monitoring device according to claim 11, wherein said image acquisition unit tracks at least a target area in said human image information.
16. The imaging-type heart rate monitoring device according to claim 11, wherein said image treatment unit extracts a target image in said target area.
US16/039,776 2018-02-13 2018-07-19 Imaging-Type Heart Rate Monitoring Device and Method Thereof Abandoned US20190246910A1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US16/039,776 US20190246910A1 (en) 2018-02-13 2018-07-19 Imaging-Type Heart Rate Monitoring Device and Method Thereof

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US201862629945P 2018-02-13 2018-02-13
US16/039,776 US20190246910A1 (en) 2018-02-13 2018-07-19 Imaging-Type Heart Rate Monitoring Device and Method Thereof

Publications (1)

Publication Number Publication Date
US20190246910A1 true US20190246910A1 (en) 2019-08-15

Family

ID=67540627

Family Applications (3)

Application Number Title Priority Date Filing Date
US16/039,776 Abandoned US20190246910A1 (en) 2018-02-13 2018-07-19 Imaging-Type Heart Rate Monitoring Device and Method Thereof
US16/040,660 Active 2038-11-01 US10694959B2 (en) 2018-02-13 2018-07-20 Image based blood pressure monitoring method
US16/040,721 Active 2038-10-10 US10779739B2 (en) 2018-02-13 2018-07-20 Contactless-type sport training monitor method

Family Applications After (2)

Application Number Title Priority Date Filing Date
US16/040,660 Active 2038-11-01 US10694959B2 (en) 2018-02-13 2018-07-20 Image based blood pressure monitoring method
US16/040,721 Active 2038-10-10 US10779739B2 (en) 2018-02-13 2018-07-20 Contactless-type sport training monitor method

Country Status (1)

Country Link
US (3) US20190246910A1 (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113592741A (en) * 2021-08-04 2021-11-02 西北工业大学 Digital image processing method

Families Citing this family (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP3640951A1 (en) * 2018-10-15 2020-04-22 Siemens Healthcare GmbH Evaluating a condition of a person
US11948401B2 (en) 2019-08-17 2024-04-02 Nightingale.ai Corp. AI-based physical function assessment system
CN111297350B (en) * 2020-02-27 2021-08-31 福州大学 Three-heart beat multi-model comprehensive decision-making electrocardiogram feature classification method integrating source end influence
CN112200162B (en) * 2020-12-03 2021-02-23 中国科学院自动化研究所 Non-contact heart rate measuring method, system and device based on end-to-end network
CN113255585B (en) * 2021-06-23 2021-11-19 之江实验室 Face video heart rate estimation method based on color space learning

Family Cites Families (16)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6475153B1 (en) * 2000-05-10 2002-11-05 Motorola Inc. Method for obtaining blood pressure data from optical sensor
US8360986B2 (en) * 2006-06-30 2013-01-29 University Of Louisville Research Foundation, Inc. Non-contact and passive measurement of arterial pulse through thermal IR imaging, and analysis of thermal IR imagery
US9770227B2 (en) * 2011-06-13 2017-09-26 Konica Minolta, Inc. Ultrasound diagnostic apparatus and ultrasound measurement method using same
US20130331669A1 (en) * 2012-06-11 2013-12-12 Raytheon Company Multi-spectral imaging system and method for remote biometric measurement of human physiological parameters
US8768438B2 (en) * 2012-06-25 2014-07-01 Xerox Corporation Determining cardiac arrhythmia from a video of a subject being monitored for cardiac function
US9600711B2 (en) * 2012-08-29 2017-03-21 Conduent Business Services, Llc Method and system for automatically recognizing facial expressions via algorithmic periocular localization
US9743848B2 (en) * 2015-06-25 2017-08-29 Whoop, Inc. Heart rate variability with sleep detection
WO2015078735A1 (en) * 2013-11-27 2015-06-04 Koninklijke Philips N.V. Device and method for obtaining pulse transit time and/or pulse wave velocity information of a subject
US20150257653A1 (en) * 2014-03-14 2015-09-17 Elwha Llc Device, system, and method for determining blood pressure in a mammalian subject
US20170245768A1 (en) * 2014-09-05 2017-08-31 Lakeland Ventures Development LLC Method and apparatus for the continous estimation of human blood pressure using video images
KR20160044267A (en) * 2014-10-15 2016-04-25 삼성전자주식회사 Apparatus and method for acquiring bio- information
US20160213303A1 (en) * 2015-01-22 2016-07-28 Elwha LLC, a limited liability company of the State of Delaware Devices and methods for remote hydration measurement
US10149958B1 (en) * 2015-07-17 2018-12-11 Bao Tran Systems and methods for computer assisted operation
WO2017121834A1 (en) * 2016-01-15 2017-07-20 Koninklijke Philips N.V. Device, system and method for generating a photoplethysmographic image carrying vital sign information of a subject
RU2648029C2 (en) * 2016-08-10 2018-03-21 Самсунг Электроникс Ко., Лтд. Device and method of blood pressure measurement
WO2018112613A1 (en) * 2016-12-19 2018-06-28 Nuralogix Corporation System and method for contactless blood pressure determination

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113592741A (en) * 2021-08-04 2021-11-02 西北工业大学 Digital image processing method

Also Published As

Publication number Publication date
US20190246921A1 (en) 2019-08-15
US20190246917A1 (en) 2019-08-15
US10779739B2 (en) 2020-09-22
US10694959B2 (en) 2020-06-30

Similar Documents

Publication Publication Date Title
US20190246910A1 (en) Imaging-Type Heart Rate Monitoring Device and Method Thereof
Shi et al. Differences of heart rate variability between happiness and sadness emotion states: a pilot study
Panerai et al. Transfer function analysis of dynamic cerebral autoregulation: a CARNet white paper 2022 update
Soni et al. Breath rate variability: a novel measure to study the meditation effects
Alafeef et al. Smartphone-based respiratory rate estimation using photoplethysmographic imaging and discrete wavelet transform
US20140288446A1 (en) Apparatus for Monitoring Physiological Condition
Goshvarpour et al. Affective visual stimuli: Characterization of the picture sequences impacts by means of nonlinear approaches
Dey et al. Analysis of photoplethysmographic signals modified by reversible watermarking technique using prediction-error in wireless telecardiology
Abadi et al. User-centric affective video tagging from MEG and peripheral physiological responses
CN110402104B (en) Blood pressure measurement device, method, and program
Dey et al. Lifting wavelet transformation based blind watermarking technique of photoplethysmographic signals in wireless telecardiology
Fernandez Rojas et al. A systematic review of neurophysiological sensing for the assessment of acute pain
KR102243012B1 (en) Estimation method of blood vessel elasticity and arrhythmia using skin image
Qayyum et al. Assessment of physiological states from contactless face video: a sparse representation approach
JP2008173160A (en) Method of analyzing fluctuation of heart rate and method of determining health state using the same
Zhang et al. Recent progress of optical imaging approaches for noncontact physiological signal measurement: A review
Talukdar et al. Evaluating visual photoplethysmography method
Garção et al. A novel approach to automatic seizure detection using computer vision and independent component analysis
Costa et al. Blood pressure estimation from photoplethysmography by considering intra-and inter-subject variabilities: guidelines for a fair assessment
JP6244724B2 (en) Frequency domain analysis transformation of renal blood flow Doppler signal to determine stress level
KIDO et al. Development and evaluation of a smartphone application for self-estimation of daily mental stress level
Ab Hamid et al. Methods of extracting feature from photoplethysmogram waveform for non-invasive diagnostic applications
Bousefsaf et al. Remote assessment of physiological parameters by non-contact technologies to quantify and detect mental stress states
Talukdar et al. Evaluation of a camera-based monitoring solution against regulated medical devices to measure heart rate, respiratory rate, oxygen saturation, and blood pressure
CN112218575A (en) Device for determining stress and/or pain level

Legal Events

Date Code Title Description
AS Assignment

Owner name: NATIONAL CHIAO TUNG UNIVERSITY, TAIWAN

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:WU, BING-FEI;HUANG, PO-WEI;TSOU, TSONG-YANG;AND OTHERS;REEL/FRAME:047267/0885

Effective date: 20180711

STPP Information on status: patent application and granting procedure in general

Free format text: DOCKETED NEW CASE - READY FOR EXAMINATION

STPP Information on status: patent application and granting procedure in general

Free format text: NON FINAL ACTION MAILED

STCB Information on status: application discontinuation

Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION