CN113657319A - Method for recognizing non-interference sleep action behaviors based on image recognition technology - Google Patents
Method for recognizing non-interference sleep action behaviors based on image recognition technology Download PDFInfo
- Publication number
- CN113657319A CN113657319A CN202110969526.0A CN202110969526A CN113657319A CN 113657319 A CN113657319 A CN 113657319A CN 202110969526 A CN202110969526 A CN 202110969526A CN 113657319 A CN113657319 A CN 113657319A
- Authority
- CN
- China
- Prior art keywords
- sleep
- action
- image
- behavior
- gaussian
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000007958 sleep Effects 0.000 title claims abstract description 121
- 230000006399 behavior Effects 0.000 title claims abstract description 120
- 230000009471 action Effects 0.000 title claims abstract description 118
- 238000000034 method Methods 0.000 title claims abstract description 59
- 238000005516 engineering process Methods 0.000 title claims abstract description 24
- 239000000203 mixture Substances 0.000 claims abstract description 16
- 238000012545 processing Methods 0.000 claims abstract description 16
- 230000008030 elimination Effects 0.000 claims abstract description 12
- 238000003379 elimination reaction Methods 0.000 claims abstract description 12
- 238000012544 monitoring process Methods 0.000 claims abstract description 9
- 238000009826 distribution Methods 0.000 claims description 64
- 230000033001 locomotion Effects 0.000 claims description 22
- 238000001914 filtration Methods 0.000 claims description 13
- 230000003860 sleep quality Effects 0.000 claims description 12
- 230000006870 function Effects 0.000 claims description 11
- 230000009466 transformation Effects 0.000 claims description 9
- 238000012360 testing method Methods 0.000 claims description 5
- 230000008859 change Effects 0.000 claims description 4
- 206010062519 Poor quality sleep Diseases 0.000 claims description 3
- 238000002474 experimental method Methods 0.000 claims description 3
- 238000000605 extraction Methods 0.000 claims description 3
- 239000012634 fragment Substances 0.000 claims description 3
- 239000011159 matrix material Substances 0.000 claims description 3
- 239000000758 substrate Substances 0.000 claims description 3
- 238000013461 design Methods 0.000 abstract description 5
- 238000004519 manufacturing process Methods 0.000 abstract description 4
- 238000001514 detection method Methods 0.000 description 5
- 238000010586 diagram Methods 0.000 description 3
- 230000000694 effects Effects 0.000 description 3
- 230000003068 static effect Effects 0.000 description 3
- 230000009286 beneficial effect Effects 0.000 description 2
- 230000007547 defect Effects 0.000 description 2
- 230000002452 interceptive effect Effects 0.000 description 2
- 206010052904 Musculoskeletal stiffness Diseases 0.000 description 1
- 208000004210 Pressure Ulcer Diseases 0.000 description 1
- 208000034972 Sudden Infant Death Diseases 0.000 description 1
- 206010042440 Sudden infant death syndrome Diseases 0.000 description 1
- 230000003044 adaptive effect Effects 0.000 description 1
- QVGXLLKOCUKJST-UHFFFAOYSA-N atomic oxygen Chemical compound [O] QVGXLLKOCUKJST-UHFFFAOYSA-N 0.000 description 1
- 230000003542 behavioural effect Effects 0.000 description 1
- 230000008901 benefit Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 239000008280 blood Substances 0.000 description 1
- 210000004369 blood Anatomy 0.000 description 1
- 230000017531 blood circulation Effects 0.000 description 1
- 230000036760 body temperature Effects 0.000 description 1
- 210000004556 brain Anatomy 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 230000006835 compression Effects 0.000 description 1
- 238000007906 compression Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000002349 favourable effect Effects 0.000 description 1
- 125000001475 halogen functional group Chemical group 0.000 description 1
- 230000036541 health Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000009434 installation Methods 0.000 description 1
- 230000001788 irregular Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012806 monitoring device Methods 0.000 description 1
- 210000003205 muscle Anatomy 0.000 description 1
- 230000007830 nerve conduction Effects 0.000 description 1
- 230000035764 nutrition Effects 0.000 description 1
- 235000016709 nutrition Nutrition 0.000 description 1
- 229910052760 oxygen Inorganic materials 0.000 description 1
- 239000001301 oxygen Substances 0.000 description 1
- 238000007781 pre-processing Methods 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 238000011084 recovery Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 230000029058 respiratory gaseous exchange Effects 0.000 description 1
- 239000000523 sample Substances 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 208000019116 sleep disease Diseases 0.000 description 1
- 230000003997 social interaction Effects 0.000 description 1
- 210000004872 soft tissue Anatomy 0.000 description 1
- 230000007306 turnover Effects 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/22—Matching criteria, e.g. proximity measures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/40—Image enhancement or restoration using histogram techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/70—Denoising; Smoothing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20024—Filtering details
- G06T2207/20032—Median filtering
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Physics & Mathematics (AREA)
- Artificial Intelligence (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Bioinformatics & Computational Biology (AREA)
- General Engineering & Computer Science (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Measurement Of The Respiration, Hearing Ability, Form, And Blood Characteristics Of Living Organisms (AREA)
Abstract
The invention provides an identification method of non-interference sleep action behaviors based on an image identification technology, which comprises the following steps: collecting human sleep behavior monitoring data: a subject lies down on a bed, and an infrared camera system or an infrared thermal imager system is adopted to collect sleep action behaviors; sleep action behavior image processing: carrying out noise elimination and contrast enhancement on the infrared image or the thermal image in two ranges of a frequency domain and a space domain; identification of sleep action behaviors: the Gaussian mixture model and the frame difference method are combined, the background model is accurately established by the Gaussian mixture model, and then the foreground contour is clearly extracted by the frame difference method, so that the accurate identification of the sleep action is completed. On the premise of not influencing the sleeping comfort of a testee, the invention accurately identifies the sleeping action by a non-contact method, and provides technical support for evaluating the sleeping quality of a human body and realizing intelligent design and customized production of a bedding or mattress system.
Description
Technical Field
The invention relates to the technical field of smart home, in particular to an identification method of non-interference sleep action behaviors based on an image identification technology.
Background
Good sleep is an important guarantee for the recovery of energy and physical strength of people, and the sleep quality has important influence on social interaction and working life of people. In sleeping, the body of a person is not kept still, the person turns over about 20 times per night on average, the sleep action behaviors with large and small sizes can reach hundreds of times, and the posture adjustment and action behaviors are not only beneficial to blood circulation, nerve conduction and nutrition supply, but also beneficial to avoiding long-time compression of muscles and soft tissues and preventing muscle stiffness. However, the action is too frequent, which may cause intermittent sleep and affect the sleep quality. Therefore, good sleep requires an appropriate frequency of sleep behaviors. The sleep behavior of the person can be monitored and identified, the sleep quality of the person can be reflected to a certain degree, and the method has important significance for evaluating the sleep comfort and diagnosing the sleep problem. Meanwhile, the defect that the existing bed cannot autonomously react to the change of the sleeping condition of a person is overcome, the support performance is adaptively adjusted according to the sleeping behavior of the person, the sleeping comfort is dynamically improved, and the intelligent development direction of the bed is provided. On the premise of not interfering sleep, the sleep behavior is monitored and identified, and the method has important significance for realizing the self-adaptive adjustment function of the intelligent bed. Especially for the old, the infant and other people who are difficult to turn over, the bed is adaptive to adjustment, is favorable for reducing the risk of bedsore and sudden infant death syndrome, and has more significance. In addition, due to the fact that the temperatures of different parts of a human body are different, the thermal infrared imager can be used for collecting temperature distribution images of the human body, the influence of the temperatures on sleeping behaviors can be reflected to a certain degree while the sleeping behaviors of the human body are monitored and identified through the thermal images, and the method has important significance for realizing self-adaptive adjustment of the temperature and humidity environment of the bed. Currently, the monitoring and recognition methods for sleep behaviors mainly include a PSG (Polysomnography) detection method, a pressure sensor detection method, and the like. The PSG method requires direct contact of the brain electrode and several monitoring devices such as blood oxygen, respiration and pulse sensors with the human body, and is not only complicated in operation but also strong in constraint, which seriously interferes with the sleep behavior of the user and is not suitable for daily use. The pressure sensor detection method requires a pressure sensing element to be built in the mattress, is expensive in manufacturing cost, and can affect the comfort of the mattress, thereby affecting the sleeping behavior.
Disclosure of Invention
In order to solve the problems, the invention aims to provide an identification method of non-interference sleep action behaviors based on an image identification technology, which accurately identifies the sleep action behaviors by a non-contact method on the premise of not influencing the sleep comfort of a testee, and provides technical support for evaluating the sleep quality of a human body and realizing intelligent design and customized production of a bed or mattress system.
In order to achieve the purpose, the technical scheme of the invention is realized as follows:
a method for identifying non-interference sleep action behaviors based on an image identification technology comprises the following steps:
s1, collecting human sleep behavior monitoring data: a subject lies down on a bed, and an infrared camera system or an infrared thermal imager system is adopted to collect sleep action behaviors;
s2, sleep movement behavior image processing: carrying out noise elimination and contrast enhancement on the infrared image or the thermal image in two ranges of a frequency domain and a space domain;
s3, recognition of sleep action behaviors: the Gaussian mixture model and the frame difference method are combined, the background model is accurately established by the Gaussian mixture model, and then the foreground contour is clearly extracted by the frame difference method, so that the accurate identification of the sleep action is completed.
Further, the noise elimination of the S2 sleep motion behavior image processing adopts a median filtering elimination method, specifically: one-way multilevel median filter subwindow definition: setting the size of a filtering window of the multistage median filter to be (2N +1) × (2N +1), (i, j) to represent an image pixel positioned at the center point of the filtering window;
the following sub-windows are defined:
horizontal direction sub-window:
W1(i,j)={(i+n,j);-N≤n≤N} (1)
vertical direction sub-window:
W2(i,j)={(i,j+n);-N≤n≤N} (2)
45 ° directional sub-window:
W3(i,j)={(i+n,j+n);-N≤n≤N} (3)
135 ° directional sub-window:
W4(i,j)={(i+n,j-n);-N≤n≤N} (4)
setting four front sub-windows W1(i,j),W2(i,j),W3(i,j),W4The median value of (i, j) is:
Mk(i,j)=mediam[f(m,n),(m,n)∈Wk(i,j),k=1,2,3,4] (5)
the following two equations represent the maximum and minimum values of the median, respectively:
Mmax(i,j)=max1≤k≤4Mk(i,j) (6)
Mmin(i,j)=max1≤k≤4Mk(i,j) (7)
the output of the unidirectional multilevel median filter is defined as:
MMLM(i,j)=mediam[Mmax(i,j),Mmin(i,j),f(i,j)] (8)。
further, the contrast enhancement in the S2 sleep action behavior image processing adopts a histogram equalization technique to enhance the image, specifically: assume that the original image is f (x, y) and the gray scale distribution thereof is in the range [ r ]max,rmin]In the above, g (x, y) represents an equalized image whose gradation distribution is in the range [ s ] after equalizationmax,smin]The gray level before equalization is r, the gray level after equalization is s, the probability distribution of the gray level before histogram equalization is phi (r), the probability distribution of the gray level after histogram equalization is phi(s), and s ═ t (r) is a gray level transformation function; first, the gray scale of the original image is normalized to obtain the gray scale range [ r ] of the original imagemax,rmin]Compressed to the range [0, 1]]Above, so there are:
the gray scale transformation function is:
wherein, T (r) is density distribution of r, the gray scale of a new image is calculated as s by using a gray scale transformation function, and when the gray scale s of the image is uniformly distributed on an interval [0, 1], histogram equalization is performed; the actual gray value of the equalized pixel is obtained by the following formula because
Therefore, the first and second electrodes are formed on the substrate,
s′=(s′max-s′min)×s+s′min (12)
the formula takes a discretization form, and the formula is a frequency of a gray level k, which is
Available in the band (11)
(14) The formula is the histogram equalization formula obtained finally, and if the formula is expanded to the gray scale range of [0, 255], the formula has
Further, the specific steps of accurately establishing the background model by the gaussian mixture model in the recognition of the sleep action behavior of S3 are as follows:
(1) describing the pixel value of the current pixel point by using K Gaussian distributions, and respectively modeling each pixel by using the K Gaussian distributions; the probability of the current pixel point is:
k is the number of Gaussian distributions which are set at the beginning; omegai,tIs the weight occupied by the ith Gaussian distribution; mu.si,tIs the expectation of the ith gaussian distribution, i.e. the mean;is the covariance matrix of the ith gaussian model;is the variance value of the ith gaussian distribution.
(2) Updating the model: for each pixel point x in the current frame imagetMatching with the K Gaussian distribution models already existing, if the expected value mu of a certain Gaussian distributioni,tAnd if the difference value between the pixel value of the current pixel point and the pixel value of the current pixel point is less than 3 times of the standard difference value, the pixel point is considered to belong to the pixel point in the background model, namely, the following conditions are met:
|xt-μi,t-1|<3σi,t-1 (18)
if the gaussian distribution model satisfying the above condition is updated as follows:
ωi,t=(1-αi)ωi,t-1+αi (19)
μi,t=(1-βi)μi,t-1+βixt (20)
in the formula of alphaiA learning rate of a Gaussian model, whose value is determined by experiments, 0 ≦ αi≤1,βiFor the parameter learning rate, available (α)i/ωi,t) And (4) showing.
Further, the specific steps of clearly extracting the foreground contour by using a frame difference method in the recognition of the sleep action behavior of S3 are as follows:
(1) first, a region A in which a change has occurred between two adjacent frames is detectedcIncluding the area A covered by the moving object in the previous framei-1And the motion region itself a of the current frameiAnd performing frame difference processing on the two adjacent frames of images, namely calculating the absolute value of the difference of the gray values of corresponding pixels between the two adjacent frames:
in the formula: ri(x, y) is the motion extraction result, fi(x, y) is the input i frame image, 2.6 x sigmai,t-1For a set threshold, 2.6 a confidence parameter, σi,t-1Is the standard deviation of the ith Gaussian distribution at time t-1, AbgRepresenting a background region of the current frame;
(2) the region A divided in (1)cIn each pixel point Xt(x, y) and normalized according toMatching the first N Gaussian distributions in descending order, then:
wherein i is more than 0 and less than or equal to N; mu.si,t-1The mean value of the ith Gaussian distribution at the t-1 moment; when the matching is successful, the pixel point X ist(x, y) belongs to the motion region A of the current framei。
Further, the method further includes step S4, classifying the sleep action behavior: according to the action duration Δ t, the sleep action behaviors are classified into four categories:
Δ t < 5s is class I, i.e., an action with a duration of less than 5 s;
delta t is more than or equal to 5s and less than 10s, and is the II-type action, namely the action with the duration of 5 s-10 s;
delta t is more than or equal to 10s and less than 15s, and is the type III action, namely the action with the duration between 10s and 15 s;
and delta t is more than or equal to 15s, namely the action behavior with the duration of more than 15 s.
Further, the method further includes step S5 of evaluating the sleep action behavior: evaluating the sleep action behaviors based on the classification of the sleep action behaviors all night, counting 9 parameter data of the sleep action behaviors, such as the total times of the sleep action behaviors, the total duration time of the sleep action behaviors, the duration time of each action behavior, the occurrence times of each action behavior, the ratio of the total time without action to the total times of action, the occurrence times of a real estate fragment exceeding 30 minutes, the total times of action in the 1 st hour, the total times of action in the 2 nd hour, the total times of action in the 5 th hour and the like, and analyzing and evaluating the sleep quality by using indexes; too frequent an action or too few immobility segments indicate poor sleep quality.
Furthermore, the bedding comprises a bedstead and a mattress, the testee performs sleep test on the mattress within a set time interval, the infrared camera system or the infrared thermal imager system comprises an infrared camera or an infrared thermal imager, a video recorder and a display screen, the infrared camera or the infrared thermal imager is used for monitoring the action behavior of the testee in sleep in real time, the video recorder is connected with the infrared camera and is used for performing video recording on the sleep action behavior, the video recorder is connected with the infrared thermal imager and is used for performing video recording on the human body temperature distribution and the sleep action behavior in sleep, and the display screen is connected with the video recorder and is used for displaying video pictures in real time.
Has the advantages that: (1) the image recognition technology used by the invention does not influence the comfort level of the testee, has the characteristics of high recognition efficiency, high recognition speed and the like, and overcomes the human body discomfort caused by pasting electrode plates and wearing probes in the polysomnography detection method; the defects of high price and poor real-time performance of the pressure sensor are overcome. According to the invention, the thermal image of the interface temperature distribution is acquired by using the thermal infrared imager, so that the condition that the accuracy rate is reduced due to the shielding of the bedding when the sleep behavior is identified is avoided.
(2) The invention overcomes the limitation that the prior sleep behavior test technology can not effectively judge the sleep action behavior, accurately and efficiently identifies the sleep behavior without influencing the sleep comfort of a subject, and provides technical support for evaluating the sleep quality of a human body and realizing the intelligent design and customized production of a bed or mattress system.
(3) The method combines median filtering and histogram equalization technologies to perform noise elimination and image enhancement processing on the image, and the interference elimination effect is obvious, so that the accuracy of image identification is guaranteed. By combining the Gaussian mixture model and the frame difference method, the slow and tiny sleep action behaviors can be effectively distinguished, the efficiency and the accuracy of the sleep action behavior identification are greatly improved, the sleep action behaviors are quickly and accurately identified, and the method is more feasible than other algorithms. (4) The invention overcomes the limitation that the prior sleep behavior test technology can only analyze the sleep behavior mode, not only can detect the sleep behavior, but also can establish the quantitative relation between the sleep behavior mode and the mattress performance, and provides theoretical basis for further research on the self-adaptive sleep action behavior and the design of an intelligent bed; and provides a new method and a new idea for realizing intelligent design of a bed or a mattress system, and has important significance for improving the sleep quality and the human health.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate an embodiment of the invention and, together with the description, serve to explain the invention and not to limit the invention. In the drawings:
fig. 1 is a schematic structural diagram of a bed and an infrared camera system or a thermal infrared imager system in the method for identifying non-interfering sleep behavior based on an image identification technology according to the embodiment of the present invention;
fig. 2 is a diagram illustrating comparison effects before and after image processing in the method for identifying non-interference sleep behavior based on image identification technology according to the embodiment of the present invention;
fig. 3 is a schematic diagram illustrating the recognition of a sleep action behavior in the method for recognizing a non-interference sleep action behavior based on an image recognition technology according to the embodiment of the present invention.
Description of reference numerals:
1-a mattress, 2-a bed frame, 3-an infrared camera or an infrared thermal imager, 4-a display screen and 5-a video recorder.
Detailed Description
It should be noted that the embodiments and features of the embodiments may be combined with each other without conflict.
The present invention will be described in detail below with reference to the embodiments with reference to the attached drawings.
Example 1
Referring to FIGS. 1-3: a method for identifying non-interference sleep action behaviors based on an image identification technology comprises the following steps:
s1, collecting human sleep behavior monitoring data: a subject lies down on a bed, and an infrared camera system or an infrared thermal imager system is adopted to collect sleep action behaviors; the bedding comprises a bedstead 2 and a mattress 1, the testee carries out sleep test on the mattress 1 within a set time interval, the infrared camera system or the infrared thermal imager system comprises an infrared camera or an infrared thermal imager 3, a video recorder 5 and a display screen 4, the infrared camera or the infrared thermal imager 3 is used for monitoring the action behavior of the testee in real time when the testee is sleeping, the video recorder 5 is connected with the infrared camera or the infrared thermal imager 3 and is used for carrying out video recording on the sleep action behavior, and the display screen 4 is connected with the video recorder 5 and is used for displaying video pictures in real time.
Considering the influence of light rays at night on the sleep of people, an infrared camera or an infrared thermal imager with the wavelength of more than 900nm and the sampling rate of 50f/s is selected. The installation position of the infrared camera or the thermal infrared imager is on the wall at the tail part of the mattress, the image covers the whole mattress, the visual range is large, the mattress is not easy to be shielded by obstacles, and the irregular movement of people cannot exceed the acquisition range of the camera. When the sleep movement behaviors are collected, the indoor temperature (23 +/-2) DEG C, the relative humidity (50 +/-5)%, and the environment is quiet; video recordings were collected from 23:00 a day to 7:00 a day.
In addition, in specific implementation, the infrared camera system or the thermal infrared imager system of the embodiment may also be disposed in a bedroom of the subject under the condition of obtaining the consent of the subject, and the subject may complete the sleep behavior monitoring data acquisition at home, so that the system is not limited by a field and has a wide application range.
S2, sleep movement behavior image processing: carrying out noise elimination and contrast enhancement on the infrared image or the thermal image in two ranges of a frequency domain and a space domain;
it should be noted that the infrared image collected by the infrared camera or the infrared thermal imager has the phenomena of smaller gray scale range, low contrast, darker image than the visible light image, halo, and the like. Therefore, an appropriate method is selected to carry out comprehensive preprocessing on the image so as to reduce image noise, enhance contrast, make the image clearer and facilitate later image processing.
In a specific implementation, the noise elimination of the S2 sleep motion behavior image processing adopts a median filtering elimination method, and a one-way multilevel median filter sub-window is defined: setting the size of a filtering window of the multistage median filter to be (2N +1) × (2N +1), (i, j) to represent an image pixel positioned at the center point of the filtering window;
the following sub-windows are defined:
horizontal direction sub-window:
W1(i,j)={(i+n,j);-N≤n≤N} (1)
vertical direction sub-window:
W2(i,j)={(i,j+n);-N≤n≤N} (2)
45 ° directional sub-window:
W3(i,j)={(i+n,j+n);-N≤n≤N} (3)
135 ° directional sub-window:
W4(i,j)={(i+n,j-n);-N≤n≤N} (4)
setting four front sub-windows W1(i,j),W2(i,j),W3(i,j),W4The median value of (i, j) is:
Mk(i,j)=mediam[f(m,n),(m,n)∈Wk(i,j),k=1,2,3,4] (5)
the following two equations represent the maximum and minimum values of the median, respectively:
Mmax(i,j)=max1≤k≤4Mk(i,j) (6)
Mmin(i,j)=max1≤k≤4Mk(i,j) (7)
the output of the unidirectional multilevel median filter is defined as:
MMLM(i,j)=mediam[Mmax(i,j),Mmin(i,j),f(i,j)] (8)。
it should be noted that the median filtering belongs to a nonlinear filter, which can better protect the detail information of the image while filtering. A square template slides in an image, the central point of the square template is aligned with the current pixel, the gray values of all the pixels in the square template are sorted from large to small, or from small to large, the pixel with the gray value at the middle position is found out, and the gray value of the pixel at the middle position is used for replacing the gray value of the current pixel. Due to the environment, the influence of equipment and noise pollution in the image transmission and conversion process, various noises are mixed in the images collected by the infrared camera or the infrared thermal imager, and the image quality is degraded. The median filter is good at filtering long tail superimposed noise and superimposed white noise, is most effective for interference generated by filter pulse and image scanning noise, and can protect target boundary from becoming fuzzy, because it can effectively weaken signal burr and can not damage ramp and step signal.
In a specific implementation, the contrast enhancement in the S2 sleep motion behavior image processing adopts a histogram equalization technique to enhance an image, specifically: assume that the original image is f (x, y) and the gray scale distribution thereof is in the range [ r ]max,rmin]In the above, g (x, y) represents an equalized image whose gradation distribution is in the range [ s ] after equalizationmax,smin]The gray level before equalization is r, the gray level after equalization is s, the probability distribution of the gray level before histogram equalization is phi (r), the probability distribution of the gray level after histogram equalization is phi(s), and s ═ t (r) is a gray level transformation function; first, the gray scale of the original image is normalized to obtain the gray scale range [ r ] of the original imagemax,rmin]Compressed to the range [0, 1]]Above, so there are:
the gray scale transformation function is:
wherein, T (r) is density distribution of r, the gray scale of a new image is calculated as s by using a gray scale transformation function, and when the gray scale s of the image is uniformly distributed on an interval [0, 1], histogram equalization is performed; the actual gray value of the equalized pixel is obtained by the following formula because
Therefore, the first and second electrodes are formed on the substrate,
s′=(s′max-s′min)×s+s′min (12)
the formula takes a discretization form, and the formula is a frequency of a gray level k, which is
Available in the band (11)
(14) The formula is the histogram equalization formula obtained finally, and if the formula is expanded to the gray scale range of [0, 255], the formula has
Referring to fig. 2, the present embodiment enhances the contrast of an image using a histogram equalization technique. The histogram equalization technology changes the gray level probability distribution of the image, so that the gray level distribution of the image is in a larger range interval, the excessively concentrated pixel points on the gray level are dispersed to the gray levels on two sides as much as possible, and simultaneously the less gray levels of the pixel points are mutually close, so that the gray level distribution of the whole image is more uniform, the contrast of the image is enhanced, and the identification degree of the image is higher.
S3, recognition of sleep action behaviors: it should be noted that when a moving object appears in a monitored scene, two consecutive frames are obviously different, so that the two frames are subtracted to obtain an absolute value of the luminance difference between the two frames, then an appropriate threshold is set, the obtained luminance difference between the two frames is compared with the set threshold, if the luminance difference is greater than the threshold, a moving pixel is determined, and if the luminance difference is less than the threshold, a background pixel is determined. According to the method for analyzing the motion characteristics of the image sequence, whether a moving target exists in the image sequence is determined, and then the sleep motion behavior is analyzed.
When the sleep action behavior is collected, the camera is fixed, the background is static and unchangeable, but when the influence of shaking, moving and the like of an object with a shielding function is received, a certain pixel point on a background image does not belong to background content at a certain moment. The Gaussian mixture model analyzes the distribution of pixel points at the same position on a time sequence, the distribution can be described by a limited number of Gaussian distributions, a plurality of states of the background are respectively modeled, and parameters of the state model are updated according to the state to which the pixel points belong, so that the problem of background modeling under a non-static background can be solved. And reading N frames of adjacent continuous images once by using the Gaussian mixture model, and performing iterative modeling on each pixel point of each frame.
Based on the method, a Gaussian mixture model and a frame difference method are combined, a background model is accurately established by using the Gaussian mixture model, and then a foreground contour is clearly extracted by using the frame difference method, so that the accurate identification of the sleep action is completed;
the specific steps of accurately establishing the background model by the Gaussian mixture model in the recognition of the S3 sleep action behavior are as follows:
(1) describing the pixel value of the current pixel point by using K Gaussian distributions, and respectively modeling each pixel by using the K Gaussian distributions; the probability of the current pixel point is:
k is the number of Gaussian distributions which are set at the beginning; omegai,tIs the weight occupied by the ith Gaussian distribution; mu.si,tIs the expectation of the ith gaussian distribution, i.e. the mean;is the covariance matrix of the ith gaussian model;is the variance value of the ith gaussian distribution.
(2) Updating the model: for each pixel point x in the current frame imagetMatching with the K Gaussian distribution models already existing, if the expected value mu of a certain Gaussian distributioni,tAnd if the difference value between the pixel value of the current pixel point and the pixel value of the current pixel point is less than 3 times of the standard difference value, the pixel point is considered to belong to the pixel point in the background model, namely, the following conditions are met:
|xt-μi,t-1|<3σi,t-1 (18)
if the gaussian distribution model satisfying the above condition is updated as follows:
ωi,t=(1-αi)ωi,t-1+αi (19)
μi,t=(1-βi)μi,t-1+βixt (20)
in the formula of alphaiA learning rate of a Gaussian model, whose value is determined by experiments, 0 ≦ αi≤1,βiFor the parameter learning rate, available (α)i/ωi,t) And (4) showing.
The specific steps of clearly extracting the foreground contour by using a frame difference method in the identification of the sleep action behavior of the S3 are as follows:
(1) first, a region A in which a change has occurred between two adjacent frames is detectedcIncluding the area A covered by the moving object in the previous framei-1And the motion region itself a of the current frameiAnd performing frame difference processing on the two adjacent frames of images, namely calculating the absolute value of the difference of the gray values of corresponding pixels between the two adjacent frames:
in the formula: ri(x, y) is the motion extraction result, fi(x, y) is the input i frame image, 2.6 x sigmai,t-1For a set threshold, 2.6 a confidence parameter, σi,t-1Is the standard deviation of the ith Gaussian distribution at time t-1, AbgRepresenting a background region of the current frame;
(2) the region A divided in (1)cIn each pixel point Xt(x, y) and normalized according toMatching the first N Gaussian distributions in descending order, then:
wherein i is more than 0 and less than or equal to N; mu.si,t-1The mean value of the ith Gaussian distribution at the t-1 moment; when the matching is successful, the pixel point X ist(x, y) belongs to the motion region A of the current framei。
Referring to fig. 3: according to the embodiment, the sleep motion behavior can be completely detected by establishing an efficient Gaussian mixture model and combining a frame difference method, and the slow and tiny sleep motion behavior can be effectively distinguished.
It can be understood that the sleep action behavior image collected by the infrared camera or the thermal infrared imager has obvious motion, clear foreground contour and definite background, and the frame difference method is a method for detecting a moving target more effectively and quickly under a static background. However, the position of the moving target between adjacent frames changes slowly, the frame difference method is difficult to detect, the mixed Gaussian model is very effective for establishing the background of the image, the mixed Gaussian model analyzes the distribution of pixel points at the same position on a time sequence, and the distribution can be described by a limited number of Gaussian distributions. Therefore, in the embodiment, the background model is accurately established by using the Gaussian mixture model, and the foreground contour is clearly extracted by using the frame difference method, so that the embodiment has a good detection effect on the moving target including the sleep action behavior.
In a specific example, the method further includes step S4, classifying the sleep action behavior: according to the action duration Δ t, the sleep action behaviors are classified into four categories:
Δ t < 5s is class I, i.e., an action with a duration of less than 5 s;
delta t is more than or equal to 5s and less than 10s, and is the II-type action, namely the action with the duration of 5 s-10 s;
delta t is more than or equal to 10s and less than 15s, and is the type III action, namely the action with the duration between 10s and 15 s;
and delta t is more than or equal to 15s, namely the action behavior with the duration of more than 15 s.
Data for sleep action behavioral events are obtained in time series, as shown in Table 1
In a specific example, the method further includes step S5, evaluating the sleep action behavior: evaluating the sleep action behaviors based on the classification of the sleep action behaviors all night, counting 9 parameter data of the sleep action behaviors, such as the total times of the sleep action behaviors, the total duration time of the sleep action behaviors, the duration time of each action behavior, the occurrence times of each action behavior, the ratio of the total time without action to the total times of action, the occurrence times of a real estate fragment exceeding 30 minutes, the total times of action in the 1 st hour, the total times of action in the 2 nd hour, the total times of action in the 5 th hour and the like, and analyzing and evaluating the sleep quality by using indexes; too frequent an action or too few immobility segments indicate poor sleep quality.
The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents, improvements and the like that fall within the spirit and principle of the present invention are intended to be included therein.
Claims (8)
1. A method for recognizing non-interference sleep action behaviors based on an image recognition technology is characterized by comprising the following steps of:
s1, collecting human sleep behavior monitoring data: a subject lies down on a bed, and an infrared camera system or an infrared thermal imager system is adopted to collect sleep action behaviors;
s2, sleep movement behavior image processing: carrying out noise elimination and contrast enhancement on the infrared image or the thermal image in two ranges of a frequency domain and a space domain;
s3, recognition of sleep action behaviors: the Gaussian mixture model and the frame difference method are combined, the background model is accurately established by the Gaussian mixture model, and then the foreground contour is clearly extracted by the frame difference method, so that the accurate identification of the sleep action is completed.
2. The method for recognizing the non-interference sleep behavior based on the image recognition technology as claimed in claim 1, wherein the noise elimination of the S2 sleep behavior image processing adopts a median filtering elimination method, specifically: one-way multilevel median filter subwindow definition: setting the size of a filtering window of the multistage median filter to be (2N +1) × (2N +1), (i, j) to represent an image pixel positioned at the center point of the filtering window;
the following sub-windows are defined:
horizontal direction sub-window:
W1(i,j)={(i+n,j);-N≤n≤N} (1)
vertical direction sub-window:
W2(i,j)={(i,j+n);-N≤n≤N} (2)
45 ° directional sub-window:
W3(i,j)={(i+n,j+n);-N≤n≤N} (3)
135 ° directional sub-window:
W4(i,j)={(i+n,j-n);-N≤n≤N} (4)
setting four front sub-windows W1(i,j),W2(i,j),W3(i,j),W4The median value of (i, j) is:
Mk(i,j)=mediam[f(m,n),(m,n)∈Wk(i,j),k=1,2,3,4] (5)
the following two equations represent the maximum and minimum values of the median, respectively:
Mmax(i,j)=max1≤k≤4Mk(i,j) (6)
Mmin(i,j)=max1≤k≤4Mk(i,j) (7)
the output of the unidirectional multilevel median filter is defined as:
MMLM(i,j)=mediam[Mmax(i,j),Mmin(i,j),f(i,j)] (8)。
3. the method for identifying non-interference sleep behavior based on image identification technology as claimed in claim 1, wherein the contrast enhancement in the image processing of the S2 sleep behavior adopts histogram equalization technology to enhance the image, specifically: assume that the original image is f (x, y) and the gray scale distribution thereof is in the range [ r ]max,rmin]In the above, g (x, y) represents an equalized image whose gradation distribution is in the range [ s ] after equalizationmax,smin]The gray level before equalization is r, the gray level after equalization is s, the probability distribution of the gray level before histogram equalization is phi (r), the probability distribution of the gray level after histogram equalization is phi(s), and s ═ t (r) is a gray level transformation function; first, the gray scale of the original image is normalized to obtain the gray scale range [ r ] of the original imagemax,rmin]Compressed to the range [0, 1]]Above, so there are:
the gray scale transformation function is:
wherein, T (r) is density distribution of r, the gray scale of a new image is calculated as s by using a gray scale transformation function, and when the gray scale s of the image is uniformly distributed on an interval [0, 1], histogram equalization is performed; the actual gray value of the equalized pixel is obtained by the following formula because
Therefore, the first and second electrodes are formed on the substrate,
s′=(s′max-s′min)×s+s′min (12)
the formula takes a discretization form, and the formula is a frequency of a gray level k, which is
Available in the band (11)
(14) The formula is the histogram equalization formula obtained finally, and if the formula is expanded to the gray scale range of [0, 255], the formula has
4. The method for identifying the non-interference sleep behavior based on the image identification technology as claimed in claim 1, wherein the specific steps of accurately establishing the background model by the Gaussian mixture model in the identification of the S3 sleep behavior are as follows:
(1) describing the pixel value of the current pixel point by using K Gaussian distributions, and respectively modeling each pixel by using the K Gaussian distributions; the probability of the current pixel point is:
k is the number of Gaussian distributions which are set at the beginning; omegai,tIs the weight occupied by the ith Gaussian distribution; mu.si,tIs the expectation of the ith gaussian distribution, i.e. the mean;is the covariance matrix of the ith gaussian model;is the variance value of the ith gaussian distribution.
(2) Updating the model: for each pixel point x in the current frame imagetMatching with the K Gaussian distribution models already existing, if the expected value mu of a certain Gaussian distributioni,tAnd if the difference value between the pixel value of the current pixel point and the pixel value of the current pixel point is less than 3 times of the standard difference value, the pixel point is considered to belong to the pixel point in the background model, namely, the following conditions are met:
|xt-μi,t-1|<3σi,t-1 (18)
if the gaussian distribution model satisfying the above condition is updated as follows:
ωi,t=(1-αi)ωi,t-1+αi (19)
μi,t=(1-βi)μi,t-1+βixt (20)
in the formula of alphaiA learning rate of a Gaussian model, whose value is determined by experiments, 0 ≦ αi≤1,βiFor the parameter learning rate, available (α)i/ωi,t) And (4) showing.
5. The method for recognizing the non-interference sleep behavior based on the image recognition technology as claimed in claim 4, wherein the specific steps of clearly extracting the foreground contour by using a frame difference method in the recognition of the sleep behavior of S3 are as follows:
(1) first, a region A in which a change has occurred between two adjacent frames is detectedcIncluding the area A covered by the moving object in the previous framei-1And the motion region itself a of the current frameiAnd performing frame difference processing on the two adjacent frames of images, namely calculating the absolute value of the difference of the gray values of corresponding pixels between the two adjacent frames:
in the formula: ri(x, y) is the motion extraction result, fi(x, y) is the input i frame image, 2.6 x sigmai,t-1For a set threshold, 2.6 a confidence parameter, σi,t-1Is the standard deviation of the ith Gaussian distribution at time t-1, AbgRepresenting a background region of the current frame;
(2) the region A divided in (1)cIn each pixel point Xt(x, y) and normalized according toMatching the first N Gaussian distributions in descending order, then:
wherein i is more than 0 and less than or equal to N; mu.si,t-1The mean value of the ith Gaussian distribution at the t-1 moment; when the matching is successful, the pixel point X ist(x, y) belongs to the motion region A of the current framei。
6. The method for identifying non-interference sleep behavior based on image recognition technology as claimed in claim 1, further comprising step S4, classifying sleep behavior: according to the action duration Δ t, the sleep action behaviors are classified into four categories:
Δ t < 5s is class I, i.e., an action with a duration of less than 5 s;
delta t is more than or equal to 5s and less than 10s, and is the II-type action, namely the action with the duration of 5 s-10 s;
delta t is more than or equal to 10s and less than 15s, and is the type III action, namely the action with the duration between 10s and 15 s;
and the action behavior of which the delta t is more than or equal to 15s is the 1V type, namely the action behavior with the duration of more than 15 s.
7. The method for identifying non-interference sleep behavior based on image recognition technology as claimed in claim 6, further comprising step S5 of evaluating sleep behavior: evaluating the sleep action behaviors based on the classification of the sleep action behaviors all night, counting 9 parameter data of the sleep action behaviors, such as the total times of the sleep action behaviors, the total duration time of the sleep action behaviors, the duration time of each action behavior, the occurrence times of each action behavior, the ratio of the total time without action to the total times of action, the occurrence times of a real estate fragment exceeding 30 minutes, the total times of action in the 1 st hour, the total times of action in the 2 nd hour, the total times of action in the 5 th hour and the like, and analyzing and evaluating the sleep quality by using indexes; too frequent an action or too few immobility segments indicate poor sleep quality.
8. The method for recognizing the non-interference sleep behavior based on the image recognition technology as claimed in claim 1, characterized in that the bed comprises a bed frame (2) and a mattress (1), the subject performs a sleep test on the mattress (1) within a set time interval, the infrared camera system or the thermal infrared imager system comprises an infrared camera or a thermal infrared imager (3), a video recorder (5) and a display screen (4), the infrared camera or the thermal infrared imager (3) is used for monitoring the action behavior of the testee in real time when the testee sleeps, the video recorder (5) is connected with the infrared camera (3) and is used for carrying out video recording on the sleep action behavior, the video recorder (5) is connected with the thermal infrared imager (3) and is used for carrying out video recording on the temperature distribution of the human body and the sleeping action during sleeping, the display screen (4) is connected with the video recorder (5) and is used for displaying video pictures in real time.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202110969526.0A CN113657319A (en) | 2021-08-23 | 2021-08-23 | Method for recognizing non-interference sleep action behaviors based on image recognition technology |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202110969526.0A CN113657319A (en) | 2021-08-23 | 2021-08-23 | Method for recognizing non-interference sleep action behaviors based on image recognition technology |
Publications (1)
Publication Number | Publication Date |
---|---|
CN113657319A true CN113657319A (en) | 2021-11-16 |
Family
ID=78492064
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN202110969526.0A Pending CN113657319A (en) | 2021-08-23 | 2021-08-23 | Method for recognizing non-interference sleep action behaviors based on image recognition technology |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN113657319A (en) |
Cited By (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN114333065A (en) * | 2021-12-31 | 2022-04-12 | 济南博观智能科技有限公司 | Behavior identification method, system and related device applied to monitoring video |
CN116386120A (en) * | 2023-05-24 | 2023-07-04 | 杭州企智互联科技有限公司 | Noninductive monitoring management system |
Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN107895379A (en) * | 2017-10-24 | 2018-04-10 | 天津大学 | The innovatory algorithm of foreground extraction in a kind of video monitoring |
CN107967684A (en) * | 2017-12-11 | 2018-04-27 | 深圳大学 | Contactless sleep-respiratory detection method and device |
CN108765466A (en) * | 2018-05-31 | 2018-11-06 | 重庆科技学院 | A kind of intelligent video monitoring system and method based on IP Camera |
-
2021
- 2021-08-23 CN CN202110969526.0A patent/CN113657319A/en active Pending
Patent Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN107895379A (en) * | 2017-10-24 | 2018-04-10 | 天津大学 | The innovatory algorithm of foreground extraction in a kind of video monitoring |
CN107967684A (en) * | 2017-12-11 | 2018-04-27 | 深圳大学 | Contactless sleep-respiratory detection method and device |
CN108765466A (en) * | 2018-05-31 | 2018-11-06 | 重庆科技学院 | A kind of intelligent video monitoring system and method based on IP Camera |
Non-Patent Citations (2)
Title |
---|
支一飞: "基于图像技术的可自动调节高度睡枕***研究", 中国优秀硕士学位论文全文数据库, 15 November 2014 (2014-11-15), pages 1 - 3 * |
潘广贞等: "基于混合高斯模型和帧差法的吸烟检测算法", 计算机工程与设计, 16 May 2015 (2015-05-16), pages 1 - 1 * |
Cited By (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN114333065A (en) * | 2021-12-31 | 2022-04-12 | 济南博观智能科技有限公司 | Behavior identification method, system and related device applied to monitoring video |
CN116386120A (en) * | 2023-05-24 | 2023-07-04 | 杭州企智互联科技有限公司 | Noninductive monitoring management system |
CN116386120B (en) * | 2023-05-24 | 2023-08-18 | 杭州企智互联科技有限公司 | A noninductive control management system for wisdom campus dormitory |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
Niu et al. | Rhythmnet: End-to-end heart rate estimation from face via spatial-temporal representation | |
CN112528960B (en) | Smoking behavior detection method based on human body posture estimation and image classification | |
CN106778695B (en) | Multi-person rapid heart rate detection method based on video | |
CN112587153B (en) | End-to-end non-contact atrial fibrillation automatic detection system and method based on vPPG signal | |
CN113657319A (en) | Method for recognizing non-interference sleep action behaviors based on image recognition technology | |
CN112163564B (en) | Tumble prejudging method based on human body key point behavior identification and LSTM (least Square TM) | |
Chen et al. | Remote recognition of in-bed postures using a thermopile array sensor with machine learning | |
CN111488850B (en) | Neural network-based old people falling detection method | |
JP6952299B2 (en) | Sleep depth judgment system, sleep depth judgment device and sleep depth judgment method | |
CN108937905B (en) | Non-contact heart rate detection method based on signal fitting | |
DE102015208144A1 (en) | Increase the accuracy of a physiological signal, obtained from a video of a patient | |
CN113111865B (en) | Fall behavior detection method and system based on deep learning | |
CN114596448A (en) | Meat duck health management method and management system thereof | |
CN111863232A (en) | Remote disease intelligent diagnosis system based on block chain and medical image | |
WO2021120007A1 (en) | Infrared image sequence-based sleep quality evaluation system and method | |
CN111476196A (en) | Facial action-based nursing demand identification method for old disabled people | |
CN114067435A (en) | Sleep behavior detection method and system based on pseudo-3D convolutional network and attention mechanism | |
CN113392765A (en) | Tumble detection method and system based on machine vision | |
CN114898261A (en) | Sleep quality assessment method and system based on fusion of video and physiological data | |
CN210956179U (en) | Monitoring system for intelligent medical community service | |
Tao et al. | 3D convolutional neural network for home monitoring using low resolution thermal-sensor array | |
CN117809379B (en) | Intelligent humanoid recognition alarm system and method based on monitoring camera | |
CN112752541A (en) | Deriving information about a person's sleep state and awake state from a sequence of video frames | |
CN115857595B (en) | Functional environment adjusting method, system and device based on user mood | |
CN115019233B (en) | Mental retardation judging method based on gesture detection |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination |