CN113842128B - Non-contact heart rate detection device based on multiple filtering and mixed amplification - Google Patents
Non-contact heart rate detection device based on multiple filtering and mixed amplification Download PDFInfo
- Publication number
- CN113842128B CN113842128B CN202111147201.0A CN202111147201A CN113842128B CN 113842128 B CN113842128 B CN 113842128B CN 202111147201 A CN202111147201 A CN 202111147201A CN 113842128 B CN113842128 B CN 113842128B
- Authority
- CN
- China
- Prior art keywords
- face
- amplification
- color
- heart rate
- motion
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/0059—Measuring for diagnostic purposes; Identification of persons using light, e.g. diagnosis by transillumination, diascopy, fluorescence
- A61B5/0077—Devices for viewing the surface of the body, e.g. camera, magnifying lens
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/02—Detecting, measuring or recording pulse, heart rate, blood pressure or blood flow; Combined pulse/heart-rate/blood pressure determination; Evaluating a cardiovascular condition not otherwise provided for, e.g. using combinations of techniques provided for in this group with electrocardiography or electroauscultation; Heart catheters for measuring blood pressure
- A61B5/024—Detecting, measuring or recording pulse rate or heart rate
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/72—Signal processing specially adapted for physiological signals or for diagnostic purposes
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/72—Signal processing specially adapted for physiological signals or for diagnostic purposes
- A61B5/7203—Signal processing specially adapted for physiological signals or for diagnostic purposes for noise prevention, reduction or removal
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/72—Signal processing specially adapted for physiological signals or for diagnostic purposes
- A61B5/7235—Details of waveform analysis
- A61B5/7253—Details of waveform analysis characterised by using transforms
- A61B5/7257—Details of waveform analysis characterised by using transforms using Fourier transforms
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/72—Signal processing specially adapted for physiological signals or for diagnostic purposes
- A61B5/7235—Details of waveform analysis
- A61B5/7264—Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems
Landscapes
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- Animal Behavior & Ethology (AREA)
- Biomedical Technology (AREA)
- Heart & Thoracic Surgery (AREA)
- Medical Informatics (AREA)
- Molecular Biology (AREA)
- Surgery (AREA)
- Pathology (AREA)
- Biophysics (AREA)
- Public Health (AREA)
- Veterinary Medicine (AREA)
- Artificial Intelligence (AREA)
- Physiology (AREA)
- Signal Processing (AREA)
- Psychiatry (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Cardiology (AREA)
- Mathematical Physics (AREA)
- Evolutionary Computation (AREA)
- Fuzzy Systems (AREA)
- Measuring And Recording Apparatus For Diagnosis (AREA)
Abstract
The invention discloses a non-contact heart rate detection device based on multiple filtering and mixed amplification, which comprises: the camera module is used for carrying out static shooting of a second threshold number of seconds on the face of the tested person within a first threshold distance; the multi-task neural network face detection module is used for simultaneously detecting a face region and a face key point, removing false and repeated face candidate frames and extracting a final face region; the mixed amplification video processing module is used for carrying out mixed amplification video processing on a final face area by adopting a parallel structure and adopting Euler motion amplification and color amplification modes, and extracting the color and motion information of a face video image; the multiple filter module adopts a serial structure for filtering color and motion information, eliminates electromyographic signal noise and high-frequency noise interference, and improves the effective information of the picture sequence after mixed amplification processing; and the calculation module is used for carrying out signal processing and fast Fourier transformation on the processed picture sequence, and completing heart rate calculation according to the obtained frequency domain diagram.
Description
Technical Field
The invention relates to the field of non-contact heart rate detection, in particular to a non-contact heart rate detection device based on multiple filtering and mixed amplification.
Background
Heart rate is a basic physiological sign parameter, is an objective measurable index for representing the sign of human vital activity, has wide application in health test, motion evaluation, psychological evaluation and other aspects, and can be used for judging the severity and criticality of the sick and wounded in disaster rescue scenes. The traditional detection method is mainly based on contact means at present, for example: the electrocardiosignals are obtained by sticking the electrode plates at the chest of the patient. The contact type physiological parameter detection method is still a gold standard method for clinical detection at present because of good accuracy and stability.
With the diversified development of social demands, the rapid progress of detection technology and the increase of detection demands of specific scenes, the defects and limitations of contact detection are gradually revealed, and the following two aspects are mainly shown:
(1) Effective contact of the detection device with the patient is not possible in special situations. When the skin of a patient is burned or ulcerated in a large area, it is difficult to continue using the contact type measuring instrument.
(2) The contact measurement method requires a high degree of patient compliance. Related instruments are required to be worn before the physiological parameters are detected, and a fixed posture is kept, otherwise, larger deviation occurs.
Disclosure of Invention
The invention provides a non-contact heart rate detection device based on multiple filtering and mixed amplification, which is based on a visual image processing means, extracts human pulse information, can detect the heart rate of a user only through simple portable equipment such as a mobile phone camera, a network camera and the like, is quick and convenient, has great application potential, and is described in detail below:
in a first aspect, a non-contact heart rate detection device based on multiple filtering and hybrid amplification, the device comprising:
the camera module is used for carrying out static shooting of a second threshold number of seconds on the face of the tested person within a first threshold distance;
the multi-task neural network face detection module comprises three cascaded networks and is used for simultaneously detecting a face region and a face key point, removing false and repeated face candidate frames and extracting a final face region;
the mixed amplification video processing module adopts a parallel structure, applies Euler motion amplification and color amplification modes to carry out mixed amplification video processing on a final face region, and extracts the color and motion information of a face video image;
the multiple filter module adopts a serial structure for filtering color and motion information, eliminates electromyographic signal noise and high-frequency noise interference, and improves the effective information of the picture sequence after mixed amplification processing;
and the calculation module is used for carrying out signal processing and fast Fourier transformation on the processed picture sequence, and completing heart rate calculation according to the obtained frequency domain diagram.
In a second aspect, a non-contact heart rate detection device based on multiple filtering and hybrid amplification, the device comprising: a processor and a memory, the memory having stored therein program instructions that the processor invokes the program instructions stored in the memory to cause the apparatus to perform the steps of:
carrying out static shooting of a second threshold number of seconds on the face of the tested person within a first threshold distance; simultaneously detecting a face region and a face key point, removing false and repeated face candidate frames, and extracting a final face region;
carrying out mixed amplification video processing on the final face area by adopting a parallel structure and adopting an Euler motion amplification mode and a color amplification mode, and extracting the color and motion information of a face video image;
filtering the color and motion information by adopting a serial structure, eliminating electromyographic signal noise and high-frequency noise interference, and improving the effective information of the picture sequence after the mixed amplification treatment; and performing signal processing and fast Fourier transformation on the processed picture sequence, and completing heart rate calculation according to the obtained frequency domain diagram.
In a third aspect, a computer readable storage medium stores a computer program comprising program instructions which, when executed by a processor, cause the processor to perform the steps in the apparatus of the second aspect.
The technical scheme provided by the invention has the beneficial effects that:
1. aiming at the detection requirement of the non-contact heart rate, the invention provides a non-contact heart rate measuring and calculating frame based on multiple filtering and mixed amplification, fully utilizes the color and motion information in video information, can effectively remove electromyographic signal interference, and realizes non-contact accurate detection of the human heart rate based on a common visual camera;
2. according to the invention, the face with the highest human exposure rate is selected as a detection area, and a region-of-interest detection method based on a multi-task network is constructed, so that accurate detection of heart rate parameters can be completed even if the face is blocked by a face mask and the like;
3. the invention combines Euler motion amplification and color amplification methods, constructs a mixed amplification video processing frame, can fully capture micro motion and color change signals of the face in the video, and improves the accuracy and reliability of sampling heart rate signals;
4. according to the invention, through the penta-Butterworth filter and the smooth wavelet transformation, a novel multiple filter is constructed, the myoelectric signal noise and the high-frequency noise are better eliminated, smoother signals are obtained, and the accuracy and the robustness of heart rate acquisition are improved;
5. this non-contact heart rate detects only can detect user's heart rate through the camera, does not need extra equipment, can be applied to the remote detection field, and is more simple convenient.
Drawings
FIG. 1 is a schematic structural diagram of a non-contact heart rate detection device based on multiple filtering and mixed amplification;
FIG. 2 is a schematic diagram of a face detection module of the multi-task neural network;
FIG. 3 is a schematic diagram of a hybrid amplified video processing module;
FIG. 4 is a schematic diagram of another structure of a non-contact heart rate detection device based on multiple filtering and mixed amplification;
FIG. 5 is a schematic diagram of the operation of a non-contact heart rate detection device based on multiple filtering and hybrid amplification;
FIG. 6 is a flow chart of a multitasking network;
FIG. 7 is a schematic diagram of a Laplacian pyramid and a Gaussian pyramid;
FIG. 8 is a schematic diagram of an original input versus Euler motion amplified output image sequence;
wherein (a) is an original input image sequence; (b) amplifying the image sequence for euler motion.
FIG. 9 is a gain diagram of a one to five-order Butterworth low pass filter;
FIG. 10 is a frequency domain plot of Euler exercise enlarging heart rate;
FIG. 11 is a schematic diagram showing a consistency analysis of heart rate results in a resting state;
fig. 12 is a schematic diagram of another structure of a non-contact heart rate detection device based on multiple filtering and hybrid amplification.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, embodiments of the present invention will be described in further detail below.
Example 1
The embodiment of the invention designs a non-contact heart rate detection device based on multiple filtering and mixed amplification, and referring to fig. 1, the device comprises:
the camera module 1 is used for carrying out static shooting of a second threshold number of seconds on the face of the tested person within a first threshold distance;
the multi-task neural network face detection module 2 comprises three cascaded networks and is used for simultaneously detecting a face region and a face key point, removing false and repeated face candidate frames and extracting a final face region;
the mixed amplification video processing module 3 adopts a parallel structure and adopts Euler motion amplification and color amplification modes to carry out mixed amplification video processing on the final face area, and extracts the color and motion information of the face video image;
the multiple filter module 4 adopts a serial structure for filtering color and motion information, eliminates electromyographic signal noise and high-frequency noise interference, and improves the effective information of the picture sequence after mixed amplification processing;
and the calculation module 5 is used for carrying out signal processing and fast Fourier transformation on the processed picture sequence and completing heart rate calculation according to the obtained frequency domain diagram.
The embodiment of the invention realizes the extraction of the human pulse information based on the visual image processing means by the mutual coordination of the components, and can detect the heart rate of a user only through simple portable equipment such as a mobile phone camera, a network camera and the like, thereby being quick and convenient and having great application potential.
In one embodiment, the aforementioned multi-task neural network face detection module 2 in fig. 1 is developed based on a deep learning network, see fig. 2, and includes: a face region suggestion network 21, an optimization network 22, and an output network 23, which are cascaded in sequence.
The face region suggestion network 21 generates a candidate window for performing preliminary face region selection on the video image processed by the pyramid to generate candidate frames of a plurality of faces and suspected faces;
the optimization network 22 includes: the full-connection layer is used for screening the candidate frames through face frame regression and face key point positioning;
and an output network 23 for outputting the final face area.
The face which is blocked can be identified through the multi-task neural network model formed by the three cascaded networks, and the heart rate non-contact detection is realized.
Referring to fig. 3, the above-described hybrid amplified video processing module 3 in fig. 1 includes:
a motion amplifying and color amplifying unit 31, configured to simultaneously perform motion amplifying and color amplifying on a group of face videos, and adopt a parallel mode;
and a weight coefficient superposition unit 32, configured to perform weight coefficient superposition on the video after motion and color amplification.
In one embodiment, the motion amplifying and color amplifying unit 31 in fig. 3 includes: and amplifying the motion video by using a Laplacian pyramid image sequence, and amplifying the color video by using a Gaussian pyramid image sequence.
In one embodiment, the multiple filter module 4 includes: butterworth filter unit and stationary wavelet transform unit.
The butterworth filter unit is preferably 5-order, and the stationary wavelet transformation unit is preferably 4-layer.
Referring to fig. 4, the apparatus further includes:
and the consistency analysis evaluation module 6 is used for reflecting the scattering trend and the consistency limit through graphs and judging the consistency degree of the two measurement results.
In summary, the embodiment of the invention provides a non-contact heart rate measuring and calculating frame based on multiple filtering and mixed amplification, which fully utilizes color and motion information in video information, can effectively remove electromyographic signal interference and realizes non-contact accurate detection of human heart rate based on a common visual camera.
Example 2
The scheme of example 1 is further described below in conjunction with specific examples, devices, calculation formulas, and fig. 5-11, and is described in detail below:
(1) The device does not need any additional light source or optical filter, and a common network camera is used as a video shooting device to store the video shot by the camera;
(2) Performing multi-task network face detection, extracting the detected face, and removing background noise interference;
(3) Performing mixed amplification video processing on the extracted face region, and fully extracting color and motion information of a face video image;
(4) The multiple filters are utilized to effectively eliminate electromyographic signal noise and high-frequency noise interference, and the effective information of the picture sequence is improved;
(5) And performing signal processing and fast Fourier transformation on the processed picture sequence, and completing heart rate calculation according to the obtained frequency domain diagram.
1. Camera video shooting
The device is suitable for driving-free camera equipment of various UVC (USB Video Class) protocols, and the embodiment of the invention takes a network camera supporting C922pro as an example. The camera is opposite to the face of the tested person, and keeps relatively static operation for 30s within a distance of 1m, so that video shooting is completed.
2. Multitasking neural network face detection
And developing a multi-task network based on the deep learning network, and simultaneously detecting a face region and key points of the face, so as to keep the robustness to the light, angle and expression change in the natural environment. The model mainly adopts three cascaded networks, such as the workflow of the multitasking network shown in fig. 6:
1) Performing image pyramid processing;
2) The P-Net (suggested network) network rapidly generates candidate windows;
3) The R-Net (optimized network) network selects candidate windows;
4) The O-Net (output network) network generates a face bounding box and corresponding keypoints.
The P-Net network is a face region suggestion network, and the preliminary face region selection is performed after the pictures are subjected to pyramid processing. And candidate frames of a plurality of faces and suspected faces are quickly generated, so that subsequent image processing is facilitated.
Wherein, the R-Net network is added with a full connection layer, which is more accurate for face selection. The network can delete a large number of wrong input images, and outputs the face images with certain credibility through frame regression and face key point positioning.
The O-Net network has better performance and can accurately identify the face part. And finally outputting the face frame selection area and 5 face key points.
In practical application, the embodiment of the invention selects a multitasking neural network model to construct heart rate non-contact detection in view of cost, algorithm light weight, and capability of recognizing the blocked face and portability.
3. Hybrid amplified video processing
The invention innovatively adopts a mixed amplification video processing frame, comprehensively utilizes the Euler motion amplification and color amplification modes, namely simultaneously carries out motion amplification and color amplification on a group of face videos, and completes the process in parallel. Then, the two groups of amplified videos are overlapped according to the weight coefficient, the weight coefficient of motion amplification is w, the weight coefficient of color amplification is 1-w, and experiments prove that the method takes w=0.7. The motion amplification is based on the Laplacian pyramid image sequence, image boundary information can be well stored, and when a human face generates tiny vibration which cannot be observed by naked eyes, the method is used for performing motion amplification on the image. The color amplification is based on a Gaussian pyramid image sequence, and when the color of the face changes slightly due to pulse waves, the color change of the face can be amplified.
The mixed amplification video processing framework can fully capture the tiny motions and color changes of the face video, and the information complementarity of the two is high. Meanwhile, a simple series mode is not adopted, a coefficient weight superposition means is used, the weight is obtained through experimental setting, and the accuracy of the heart rate signal is effectively improved.
A schematic of the laplacian pyramid and gaussian pyramid is shown in fig. 7. When the image is up-sampled, zero value columns and zero value rows are respectively inserted at the right side and the lower side of each pixel, so that a new image with zero values in even number rows and even number columns is obtained. Therefore, a "distortion" phenomenon occurs when the picture is up-sampled. The acquisition of the laplacian pyramid L1 image is an image up-sampled by the gaussian pyramid G1 image minus the gaussian pyramid G2. Similarly, the L0 image is acquired in this manner.
When the human body breathes and heart beats, the blood volume in the capillary vessel of the face can be changed, the absorption and reflection of light can be changed, the color of the face can be changed with naked eyes, and the change is effectively amplified by using color amplification; the face is slightly vibrated which is difficult to catch by naked eyes, and Euler motion is used for effectively amplifying the motion; the heart rate value can be extracted more accurately by combining Euler motion amplification and color amplification. An enlarged image of the euler movement is shown in fig. 8.
4. Multiple filtering noise reduction link
In order to overcome the artifact interference of human myoelectric signals and high-frequency noise on heart rate signals, the embodiment of the invention also constructs a multiple filtering noise reduction link, and the maximum suppression of noise signals is realized by integrating a 5-order Butterworth filter and 4-layer stable wavelet transformation. According to experimental results, the multiple filters adopt a serial structure, and the signals are sequentially subjected to Butterworth and stationary wavelet filtering.
Firstly, inputting a signal into a Butterworth filter for noise reduction treatment, wherein a frequency response curve in a passband of the Butterworth filter is maximally flat, no ripple exists, and the frequency response curve gradually drops to zero in a blocking band. As shown in fig. 9, the attenuation rate of the first-order butterworth filter is 6dB per frequency multiplication, 20dB per ten times frequency (all the first-order low-pass filters have the same normalized frequency response). The attenuation rate of the second order butterworth filter is 12dB per frequency multiplication, and so on. The filter is a monotonic function and the amplitude versus frequency curve remains the same shape of the filter regardless of the filter order. The higher the filter order, the faster the amplitude decay rate in the blocking band. According to the experimental parameter tuning procedure, the embodiment of the invention uses a 5 th order butterworth filter.
After passing through the Butterworth filter, the myoelectric signal noise is eliminated, and the high-frequency noise is removed after smooth wavelet transformation. The method inherits and develops the concept of short-time Fourier transform localization, overcomes the defects that the size of a window does not change along with frequency and the like, and can provide a time-frequency window which changes along with frequency. The method is mainly characterized in that characteristics of certain aspects of the problems can be fully highlighted through transformation, time frequency localization analysis can be achieved, multi-scale refinement is gradually carried out on signals through telescopic translational operation, finally, the time subdivision at high frequency is finally achieved, the frequency subdivision at low frequency is finally achieved, and the requirement of time frequency signal analysis can be automatically met.
The wavelet transform decomposes information into two parts, low frequency information, which is a slowly changing part, and high frequency information, which is a rapidly changing part, which is a small part of the total information. The above is a first layer decomposition, and the high-frequency information part is re-decomposed into two parts on the basis of the first layer: low frequency and high frequency. The third layer is to decompose the high-frequency information decomposed by the second layer into low frequency and high frequency, and so on to obtain a smooth signal. According to the experimental parameter tuning process, the embodiment of the invention uses a 4-layer stationary wavelet transform.
5. Fast fourier transform heart rate calculation
And performing fast Fourier transform on the heart rate signal subjected to signal processing and noise reduction. The heart rate frequency interval is 1-2 Hz. Finally, the step of obtaining the product,
the heart rate frequency domain diagram obtained through band-pass filtering is shown in fig. 10.
Wherein, heart rate calculation formula is:
V hr ＝60*f hr
wherein V is hr -heart rate value; f (f) hr -frequency values corresponding to peaks in the frequency domain map.
In a specific implementation, the peak point of the frequency domain diagram 10 is extracted, and it can be seen from fig. 10 that the peak of the frequency domain diagram appears at 1.3Hz, and according to the calculation result of the formula, the heart rate value should be 78bpm.
6. Bland-Altman consistency analysis evaluation
The Bland-Altman consistency analysis and evaluation are proposed by Bland J M and Altman D G in 1986 together, make great contribution to medical statistics and biological statistics, and are widely used in various fields. The method comprises the steps of firstly calculating the mean value and standard deviation of the difference between an experimental value and a comparison value, determining a consistency limit, and then reflecting the scattering trend and the consistency limit through a graph so as to judge the consistency degree of two measurement results. The method adopts a contact finger-clip instrument for comparison experiments.
Using a real environment test, 20 groups of 1 minute videos were taken together, and the heart rate versus gold standard was Kangtai CMS50D finger-clipped pulse oximeter.As a 95% confidence interval, it is determined whether the true value and the measured value have a limit of agreement. />Is the average value of the difference between the true value and the measured value, S d Is the standard deviation of the difference between the two. FIG. 7 shows a Bland-Altman analysis corresponding to the experimental data.
As can be seen from fig. 11, only one point of the heart rate is outside the coincidence boundary, and more than 95% of the data is within the coincidence boundary. Experimental results show that the detection device has good consistency with the traditional contact gold standard detection method.
Example 3
A non-contact heart rate detection device based on multiple filtering and hybrid amplification, see fig. 12, the device comprising: a processor 7 and a memory 8, the memory 8 storing program instructions, the processor 7 calling the program instructions stored in the memory 8 to cause the apparatus to perform the steps of:
carrying out static shooting of a second threshold number of seconds on the face of the tested person within a first threshold distance; detecting the face region and the face key points, removing false and repeated face candidate frames, and extracting a final face region;
the parallel structure is used for carrying out mixed amplification video processing on the final face area by using an Euler motion amplification mode and a color amplification mode, and extracting the color and motion information of the face video image;
the serial structure is used for filtering color and motion information, eliminating electromyographic signal noise and high-frequency noise interference, and improving the effective information of the picture sequence after mixed amplification processing; and performing signal processing and fast Fourier transformation on the processed picture sequence, and completing heart rate calculation according to the obtained frequency domain diagram.
In one embodiment, the face region and the face key point are detected, the face region is extracted based on deep learning network development, and the network comprises: the face area proposal network, the optimization network and the output network are sequentially cascaded.
Preferably, the face region suggestion network generates a candidate window, which is used for performing preliminary face region selection on the video image processed by the pyramid to generate candidate frames of a plurality of faces and suspected faces;
the optimization network comprises a full connection layer and is used for screening candidate frames through face frame regression and face key point positioning;
and the output network is used for outputting the final face area.
The method for performing mixed amplification video processing on the final face area by using the Euler motion amplification and color amplification modes by adopting the parallel structure specifically comprises the following steps:
simultaneously carrying out motion amplification and color amplification on a group of face videos, and adopting a parallel mode; and (5) carrying out weight coefficient superposition on the video after the motion and color amplification.
Wherein, the motion amplification and the color amplification specifically are: and amplifying the motion video by using a Laplacian pyramid image sequence, and amplifying the color video by using a Gaussian pyramid image sequence.
In one embodiment, filtering of the color and motion information is achieved by a butterworth filter unit and a stationary wavelet transform unit, eliminating electromyographic signal noise and high frequency noise interference.
It should be noted that, the device descriptions in the above embodiments correspond to the method descriptions in the embodiments, and the embodiments of the present invention are not described herein in detail.
The execution main bodies of the processor 7 and the memory 8 may be devices with calculation functions, such as a computer, a singlechip, a microcontroller, etc., and in particular implementation, the execution main bodies are not limited, and are selected according to the needs in practical application.
Data signals are transmitted between the memory 8 and the processor 7 via the bus 9, which is not described in detail in the embodiment of the present invention.
Example 4
Based on the same inventive concept, an embodiment of the present invention further provides a computer readable storage medium, where the storage medium includes a stored program, and the apparatus in which the storage medium is controlled to execute the steps in the foregoing embodiment 3 when the program runs.
The computer readable storage medium includes, but is not limited to, flash memory, hard disk, solid state disk, and the like.
It should be noted that the readable storage medium descriptions in the above embodiments correspond to the device descriptions in the embodiments, and the embodiments of the present invention are not described herein.
In the above embodiments, it may be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When loaded and executed on a computer, produces a flow or function in accordance with embodiments of the invention, in whole or in part.
The computer may be a general purpose computer, a special purpose computer, a computer network, or other programmable apparatus. The computer instructions may be stored in or transmitted across a computer-readable storage medium. Computer readable storage media can be any available media that can be accessed by a computer or data storage devices, such as servers, data centers, etc., that contain an integration of one or more available media. The usable medium may be a magnetic medium or a semiconductor medium, or the like.
The embodiment of the invention does not limit the types of other devices except the types of the devices, so long as the devices can complete the functions.
Those skilled in the art will appreciate that the drawings are schematic representations of only one preferred embodiment, and that the above-described embodiment numbers are merely for illustration purposes and do not represent advantages or disadvantages of the embodiments.
The foregoing description of the preferred embodiments of the invention is not intended to limit the invention to the precise form disclosed, and any such modifications, equivalents, and alternatives falling within the spirit and scope of the invention are intended to be included within the scope of the invention.
Claims (6)
1. A non-contact heart rate detection device based on multiple filtering and hybrid amplification, the device comprising:
the camera module is used for carrying out static shooting of a second threshold number of seconds on the face of the tested person within a first threshold distance;
the multi-task neural network face detection module comprises three cascaded networks and is used for simultaneously detecting a face region and a face key point, removing false and repeated face candidate frames and extracting a final face region;
the mixed amplification video processing module is used for carrying out mixed amplification video processing on a final face area by adopting a parallel structure and adopting Euler motion amplification and color amplification modes, and extracting the color and motion information of a face video image;
the multiple filter module adopts a serial structure for filtering color and motion information, eliminates electromyographic signal noise and high-frequency noise interference, and improves the effective information of the picture sequence after mixed amplification processing;
the calculation module is used for carrying out signal processing and fast Fourier transformation on the processed picture sequence, and completing heart rate calculation according to the obtained frequency domain diagram;
the multi-task neural network face detection module is developed based on a deep learning network, and comprises: a face region suggestion network, an optimization network and an output network which are sequentially cascaded;
the face region suggestion network generates a candidate window which is used for carrying out preliminary face region selection on the video image processed by the pyramid to generate candidate frames of a plurality of faces and suspected faces;
the optimization network comprises a full connection layer and is used for screening candidate frames through face frame regression and face key point positioning;
the output network is used for outputting a final face area;
the hybrid amplified video processing module includes:
the motion amplifying and color amplifying unit is used for simultaneously carrying out motion amplifying and color amplifying on a group of face videos and adopts a parallel mode;
and the weight coefficient superposition unit is used for carrying out weight coefficient superposition on the video after the motion and color amplification.
2. The non-contact heart rate detection device based on multiple filtering and mixed amplification as claimed in claim 1, wherein said motion amplification, color amplification unit comprises:
and amplifying the motion video by using a Laplacian pyramid image sequence, and amplifying the color video by using a Gaussian pyramid image sequence.
3. The non-contact heart rate detection device based on multiple filtering and hybrid amplification of claim 1, wherein the multiple filter module comprises: butterworth filter unit and stationary wavelet transform unit.
4. A non-contact heart rate detection apparatus based on multiple filtering and hybrid amplification as claimed in any one of claims 1 to 3, further comprising:
and the consistency analysis evaluation module is used for reflecting the scattering trend and the consistency limit through graphs and judging the consistency degree of the two measurement results.
5. A non-contact heart rate detection device based on multiple filtering and hybrid amplification, the device comprising: a processor and a memory, the memory having stored therein program instructions that the processor invokes the program instructions stored in the memory to cause the apparatus to perform the steps of:
carrying out static shooting of a second threshold number of seconds on the face of the tested person within a first threshold distance; simultaneously detecting a face region and a face key point, removing false and repeated face candidate frames, and extracting a final face region; carrying out mixed amplification video processing on the final face area by adopting a parallel structure and adopting an Euler motion amplification mode and a color amplification mode, and extracting the color and motion information of a face video image;
filtering the color and motion information by adopting a serial structure, eliminating electromyographic signal noise and high-frequency noise interference, and improving the effective information of the picture sequence after the mixed amplification treatment; performing signal processing and fast Fourier transformation on the processed picture sequence, and completing heart rate calculation according to the obtained frequency domain diagram;
the detection of the face region and the face key points is based on deep learning network development and comprises the following steps: a face region suggestion network, an optimization network and an output network which are sequentially cascaded;
the face region suggestion network generates a candidate window which is used for carrying out preliminary face region selection on the video image processed by the pyramid to generate candidate frames of a plurality of faces and suspected faces;
the optimization network comprises a full connection layer and is used for screening candidate frames through face frame regression and face key point positioning;
the output network is used for outputting a final face area;
the hybrid amplified video processing includes: simultaneously carrying out motion amplification and color amplification on a group of face videos, and adopting a parallel mode; and (5) carrying out weight coefficient superposition on the video after the motion and color amplification.
6. A computer readable storage medium, characterized in that the computer readable storage medium stores a computer program comprising program instructions which, when executed by a processor, cause the processor to perform the steps of claim 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111147201.0A CN113842128B (en) | 2021-09-29 | 2021-09-29 | Non-contact heart rate detection device based on multiple filtering and mixed amplification |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111147201.0A CN113842128B (en) | 2021-09-29 | 2021-09-29 | Non-contact heart rate detection device based on multiple filtering and mixed amplification |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113842128A CN113842128A (en) | 2021-12-28 |
| CN113842128B true CN113842128B (en) | 2023-09-26 |
Family
ID=78976934
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111147201.0A Active CN113842128B (en) | 2021-09-29 | 2021-09-29 | Non-contact heart rate detection device based on multiple filtering and mixed amplification |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113842128B (en) |
Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8965090B1 (en) * | 2014-07-06 | 2015-02-24 | ARC Devices, Ltd | Non-touch optical detection of vital signs |
| CN107529646A (en) * | 2017-05-02 | 2018-01-02 | 广东工业大学 | A kind of non-contact type method for measuring heart rate and device based on Euler's image zoom |
| CN108021892A (en) * | 2017-12-06 | 2018-05-11 | 上海师范大学 | A kind of human face in-vivo detection method based on extremely short video |
| CN109044314A (en) * | 2018-06-15 | 2018-12-21 | 西安电子科技大学 | A kind of contactless rhythm of the heart method based on Euler's video amplifier |
| CN109793506A (en) * | 2019-01-18 | 2019-05-24 | 合肥工业大学 | A kind of contactless radial artery Wave shape extracting method |
| WO2021017307A1 (en) * | 2019-07-31 | 2021-02-04 | 平安科技（深圳）有限公司 | Non-contact heart rate measurement method, system, device, and storage medium |
| WO2021084488A1 (en) * | 2019-10-31 | 2021-05-06 | Facense Ltd. | Smartglasses for detecting physiological parameters |
| CN112861588A (en) * | 2019-11-28 | 2021-05-28 | 杭州萤石软件有限公司 | Living body detection method and device |
| CN112929622A (en) * | 2021-02-05 | 2021-06-08 | 浙江大学 | Euler video color amplification method based on deep learning |
| CN112949349A (en) * | 2019-12-09 | 2021-06-11 | 南宁莲现健康科技有限公司 | Method and system for displaying pulse condition waveform in real time based on face video |
| CN113378798A (en) * | 2021-07-16 | 2021-09-10 | 福建师范大学 | Pulse video pulse extraction method and system based on video micro-motion amplification |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10742904B2 (en) * | 2018-05-25 | 2020-08-11 | Fotonation Limited | Multispectral image processing system for face detection |
-
2021
- 2021-09-29 CN CN202111147201.0A patent/CN113842128B/en active Active
Patent Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8965090B1 (en) * | 2014-07-06 | 2015-02-24 | ARC Devices, Ltd | Non-touch optical detection of vital signs |
| CN107529646A (en) * | 2017-05-02 | 2018-01-02 | 广东工业大学 | A kind of non-contact type method for measuring heart rate and device based on Euler's image zoom |
| CN108021892A (en) * | 2017-12-06 | 2018-05-11 | 上海师范大学 | A kind of human face in-vivo detection method based on extremely short video |
| CN109044314A (en) * | 2018-06-15 | 2018-12-21 | 西安电子科技大学 | A kind of contactless rhythm of the heart method based on Euler's video amplifier |
| CN109793506A (en) * | 2019-01-18 | 2019-05-24 | 合肥工业大学 | A kind of contactless radial artery Wave shape extracting method |
| WO2021017307A1 (en) * | 2019-07-31 | 2021-02-04 | 平安科技（深圳）有限公司 | Non-contact heart rate measurement method, system, device, and storage medium |
| WO2021084488A1 (en) * | 2019-10-31 | 2021-05-06 | Facense Ltd. | Smartglasses for detecting physiological parameters |
| CN112861588A (en) * | 2019-11-28 | 2021-05-28 | 杭州萤石软件有限公司 | Living body detection method and device |
| CN112949349A (en) * | 2019-12-09 | 2021-06-11 | 南宁莲现健康科技有限公司 | Method and system for displaying pulse condition waveform in real time based on face video |
| CN112929622A (en) * | 2021-02-05 | 2021-06-08 | 浙江大学 | Euler video color amplification method based on deep learning |
| CN113378798A (en) * | 2021-07-16 | 2021-09-10 | 福建师范大学 | Pulse video pulse extraction method and system based on video micro-motion amplification |
Non-Patent Citations (2)
| Title |
|---|
| 基于欧拉视角的视频放大方法研究;吴秀;《中国博士学位论文全文数据库》;全文 * |
| 基于视频分析的非接触式心率检测***研究;陈怡然;《中国优秀硕士学位论文全文数据库》;全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113842128A (en) | 2021-12-28 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109247923B (en) | Non-contact type pulse real-time estimation method and device based on video | |
| EP2979631A1 (en) | Blood flow index calculation method, blood flow index calculation program and blood flow index calculation device | |
| CN112949349B (en) | Method and system for displaying pulse condition waveform in real time based on face video | |
| CN112200162A (en) | Non-contact heart rate measuring method, system and device based on end-to-end network | |
| Chambino | Android-based implementation of Eulerian Video Magnification for vital signs monitoring | |
| CN105997035A (en) | Non-contact type heart rate measurement method and system | |
| Liu et al. | Detecting pulse wave from unstable facial videos recorded from consumer-level cameras: A disturbance-adaptive orthogonal matching pursuit | |
| Luke et al. | Motion artifact removal and feature extraction from PPG signals using efficient signal processing algorithms | |
| Barki et al. | Mental stress detection using a wearable in-ear plethysmography | |
| Bousefsaf et al. | Estimation of blood pressure waveform from facial video using a deep U-shaped network and the wavelet representation of imaging photoplethysmographic signals | |
| CN113842128B (en) | Non-contact heart rate detection device based on multiple filtering and mixed amplification | |
| CN112716468A (en) | Non-contact heart rate measuring method and device based on three-dimensional convolution network | |
| Karmuse et al. | A robust rPPG approach for continuous heart rate measurement based on face | |
| CN116028809A (en) | Continuous blood pressure measurement model training and measuring method and device | |
| CN116138756A (en) | Non-contact heart rate detection method, system and medium based on face feature point detection | |
| KR102598219B1 (en) | Method and apparatus for restoring remote photoplethysmography | |
| Cajas et al. | Modeling of motion artifacts on PPG signals for heart-monitoring using wearable devices | |
| Ayesha | ReViSe: An end-to-end framework for remote measurement of vital signs | |
| Pansare et al. | Heart Rate Measurement from Face and Wrist Video | |
| Jayaprabha et al. | Blood flow, vein and nerves detector using an NIR sensor with RLS estimation for embedded signal processing | |
| Bondareva et al. | Segmentation-free heart pathology detection using deep learning | |
| Devaki et al. | Smartphone-based diagnostic model for hypertension using features from photoplethysmogram | |
| Raju et al. | DNN Based Blood Glucose Level Estimation Using PPG Characteristic Features of Smartphone Videos | |
| Yang et al. | A novel video magnification method for revealing subtle signals based on adaptive Laplacian pyramid | |
| Jia et al. | CEBPM: A cloud-edge collaborative noncontact blood pressure estimation model |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB03 | Change of inventor or designer information |
Inventor after: Wu Nan Inventor after: An Na Inventor after: Shi Aoli Inventor after: Chai Rongxuan Inventor before: Chai Rongxuan Inventor before: An Na Inventor before: Wu Nan Inventor before: Shi Aoli |
|
| CB03 | Change of inventor or designer information | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |