AU2021106380A4 - An novel method -intelligent safe home system for the elderly people. - Google Patents

Abstract

It is critical today for older persons to receive care and support in order to live a safe life free of anxieties and anxiety. The harassment of elderly people at home is exacerbated by their relatives' lack of knowledge of emerging behavioral tendencies. We have developed a feasible home security equipment for the elderly to use in their homes. Using open-source hardware to assist with cameras and networks, we created a smart home protection system that includes pedestrian surveillance, facial recognition, and fall warning. To recognize moving objects, we use the KNN model context subtraction method based on the open source OpenCV library and combine it with hog-svm to construct a pedestrian tracking module. The trained vggnet-16 neural network model is utilised to extract facial characteristics, and then a face recognition module is built that may be used for international alarm intrusion. Based on the original openpose, the initial caffer model was modified to the mobile net model for human motion recognition. At 18 key locations, information on the location of 6 key points on the body trunk was gathered, and the role of fall detection was realised by integrating the SVM classifier. The elderly man's residence and fall details would be a first-time input to the old man's family members, who could entirely ensure the old man's safety by integrating the GSM module. According to our experiment, the fall behavior recognition performance of face recognition is strong; the face recognition rate can reach 85 percent, the fall behavior recognition rate can reach more than 90 percent, and the fall false alarm rate is less than 10% for strangers and elders. As a result, the proposed method should be put into practice.

Description

TITLE OF THE INVENTION An Novel method -Intelligent Safe Home System for the Elderly People.

FIELD OF THE INVENTION

[001]. The current disclosure is strongly connected to an Intelligent Safe Home System for Protecting the Elderly from Foreign Interference, Unintentional Falling, Illness, and Other Abnormal Circumstances.

BACKGROUND OF THE INVENTION

[002]. The fundamental goal of this idea is to create an intelligent, safe home system for the elderly that is built on openCV. This invention consists of five major components, including a real-time monitoring module, a tracking module for human body recognition, a face recognition module, a behaviour recognition module, a warning module, and so on. The monitoring module collects the real-time image stream in real time, and the image is processed using openCV. Three modules for recognition activities, such as human body recognition, face recognition, and behaviour recognition, are supplied to the processed image, and the living conditions of the elderly at home are rated based on the target feature information. The device is designed to capture abnormal information, such as foreign interference, elderly people falling down, elderly people being unwell, and other abnormal conditions. The abnormal information is then conveyed to the senior family via network contact. This technology allows family members of the elderly to get real-time information about the elderly at any time.

SUMMARY OF THE INVENTION

[003]. It is critical today for older persons to receive care and support in order to live a safe life free of anxieties and anxiety. The harassment of elderly people at home is exacerbated by their relatives' lack of knowledge of emerging behavioural tendencies. We have developed a feasible home security equipment that may be installed in the homes of the elderly. Using open-source hardware to assist with cameras and networks, we created a smart home protection system that includes pedestrian surveillance, facial recognition, and fall warning.

[004]. Figure 1 depicts the full system's operating principle. Video handles the acquisition of real-time video data and sends it to the computer for processing, after which the computer sends the processed images to the measurement and recognition modules for applicable face identification, pedestrian detection, and activity detection. The results of identification and analysis are fed back into the system. The machine analyses the results and, if any danger is detected, sends dangerous information to the proper cell phones.

[005]. The device is made up of five major modules: a preprocessing picture capture module, a pedestrian surveillance module, a face recognition module, a gesture recognition module, and an alert module. To recognise moving objects, we use the KNN model context subtraction method based on the open source OpenCV library and combine it with hog-svm to construct a pedestrian tracking module.

[006]. After discriminating, the machine regulates the GSM module to provide dangerous information to the family cell phone. If the elderly are identified, the visual information for behaviour judgement will be sent to the behaviour recognition module. The recognition module is now solely used to determine the risky acts of falling. If it is determined that the elderly person would fall, the risky information will be communicated to the elderly person's relatives via GSM. The device was tested and proven to be accurate up to 85 percent of the time.

DETAILED DESCRIPTION OF THE INVENTION

[0071. The elderly, on the other hand, continue to suffer from a variety of injuries at home. As people get older, their bodies can become weaker and weaker. When confronted with the invasion and decline of outsiders, the elderly also show little opposition. When they are in trouble, they may not even call out for help or ask for it. They may also miss out on the best chance of rescue and ultimately perish because no one saves them. As a result, we require a smart home security system that can monitor geriatric activities at home in real time and provide urgent assistance. It is possible to simplify and improve the monitoring duty by including a smart home device into the home. Furthermore, when outsiders enter or the elderly collapse, the device can track the elderly at home and provide fast feedback information, making it more reliable. Furthermore, the initiative is less expensive and safer than transporting the elderly to a nursing home and asking for assistance.

[0081. With the growing popularity of the Internet of Things, the smart home device is no longer remote. A minimal number of devices and open source programmes can be used to build a viable smart home system at home. Based on open source Raspberry Pi technology and supplemented by cameras and wireless networks, we created a smart home security system for the elderly. Tensor Flow builds a neural network to process images using the open source OpenCV library, and the face recognition and body posture recognition modules are produced. Then, conduct the following activities that are urgently required by the elderly: stranger intrusion alarm and elderly fall alarm.

[009]. Combined with the APP cell phone, the details about the senior's home and collapse can be transmitted back to the elderly family for the first time, entirely ensuring the safety of the system's elderly, and then I canfinally deliver the potential system design enhancement.

[0010]. Proposed System Design: Figure 1 depicts the full system's operating principle. Video handles the acquisition of real-time video data and sends it to the computer for processing, after which the computer sends the processed images to the measurement and recognition modules for applicable face identification, pedestrian detection, and activity detection. The results of identification and analysis are fed back into the system. The machine analyses the results and, if any danger is detected, sends dangerous information to the proper cell phones. The gadget is made up of five major modules: a preprocessing picture capture module, a pedestrian surveillance module, a face recognition module, a gesture recognition module, and an alert module.

[0011]. Image acquisition and preprocessing: The first step is to place the monitor in critical places of the house to collect real-time video data. Because the gathered video information is large, the calculation amount will be too large to ensure acceptable real time performance if all of the data is processed in the second phase, thus the key frames of the obtained video information will be removed. The key's frame image should be processed. The critical frame picture acquisition frequency in this innovation is 15 photos per second. Third, because different modules have different image processing needs, we execute different preprocessing on key frame images. Here, we duplicate the main frame into two images.

[0012]. Some of them have a greater resolution, and to improve image contrast, run them through the sharpening filter. It is mostly employed in the face recognition module, as this module requires a greater resolution to adequately recognise face details. The other image can be resized and its resolution reduced appropriately, effectively reducing the human tracking and behaviour recognition module processing while maintaining its recognition capacity.

[0013]. Fourth, the image will be processed in accordance with the software flow, and the processed frame image will be passed to other modules for processing. Preprocessing, primarily through sharp filtering and context deletion. The sharpening filter can be used to improve image accuracy. The Laplacian operator is principally employed in this invention to explain the edges, contour lines, and facial area image information and boost the rate of face identification.

[0014]. The human body tracking module requires all moving body monitoring and identification. Background subtraction is a technique that is extensively employed in modem technologies for identifying motion targets. It can successfully track moving objects and determine whether a space contains a moving object. So, background subtraction is used in this innovation to eliminate the foreground image. Typical names include adaptive mixed Gaussian model, KNN model, and so on. Subtraction of context The KNN model is used to extract moving objects from images.

[0015]. Pedestrian detection and tracking module: To detect and identify human targets, the device employs a directional gradient histogram-support vector machine (HOG-SVM) human body identification algorithm. The algorithm incorporates HOG and SVM. The fundamental principle is to execute human body recognition by monitoring the contrast between the contour and the context. Different individuals wearing different garments may have different appearances, but the contours of the human body are identical. Based on this HOG-SVM, the location of the human body may be precisely described and monitored. This innovation is built on OpenCV for human body identification, which uses the trained HOG-SVM model in the openCV library, considerably reducing growth complexity.

[0016]. Face recognition module: The face recognition module is divided into four sections: face image acquisition and identification, face image preprocessing, face image feature extraction, matching and recognition.

[0017]. Image collection can utilise regular cameras, install numerous cameras at home in a fixed position to cover all regions, take photographs to collect images, and use the pedestrian detection and tracking module to identify and capture human body position. In the previous stage, we must remove the face from the preprocessed image. So we must first train the face detector before using it to detect the face in the image.

[0018]. The Viola-Jones Face Detector is being used here. This method is based on the shape of a sliding window. A fixed size window is utilised to move the range into our image in order to determine if the area of the window contains human faces. If the image has multiple faces, choose the window with the best chance of cutting out the face. Finally, the acquired face image is sent to the next level of processing. Face image preprocessing: The acquired image is preprocessed, which includes things like image enhancement, de-drying, sharpening, and so on.

[0019]. In this innovation, the mean filtering approach is employed to eliminate noise from the image. Mean filtering effectively removes picture noise, reduces the effect of light on the image, softens image sharp points, and retains precise characteristics. Face picture feature extraction: Convolutional Neural Networks (CNN) are used to extract face features. While conventional image processing algorithms can extract facial features to some extent, the accuracy of extracting facial features in face recognition is still far from realistic application. Face characteristics retrieved by the CNN approach can accurately portray the face by including crucial points of information such as eye, ear, nose, mouth, and even eyebrow characteristics and position.

[0020]. Some research focusing on in-depth learning has also surpassed human comprehension rates. The neural network employed in this invention is vggnet-16. The vggnet neural network architecture is a convolutional neural network proposed by Oxford University's computer vision department. Its performance is exceptional. There are numerous and relatively developed Vggnet-based studies and implementations. The

VGGFace face database is utilised to train vggnet-16, and the face recognition accuracy rate exceeds 95%. The trained model is utilised for feature extraction. Matching and recognition: When a human enters the camera's range of view, the machine automatically snaps photos.

[0021]. Image by face field extracts and preprocesses the acquired image. The preprocessed face is placed into the qualified Vggnet-16 model for facial feature extraction. The measured face eigenvalues are balanced and compared to those in the database. In this innovation, the Euclidean distance is used to determine matching recognition.

[0022]. Human pose Recognition Module: The module for identifying human motion's major job is to recognise human behaviours and actions, as well as to recognise and warn against unsafe behaviours and activities. They are fragile and prone to falling for older folks, especially if the family only has one elderly member. After the old guy collapsed, he was unable to contact the police. The primary purpose of this module is to raise behavioural awareness of geriatric falls.

[0023]. When the outcome is recognised as a fall, the device control warning module sends essential information to the senior's relatives' mobile phones, allowing the elderly to be rescued as soon as possible. In this innovation, an openpose training model based on openCV is used to enter human photos into the openpose model, extract the key position of the human body, and then input the key position to the SVM classifier for evaluation. Openpose, developed by Carnegie Mellon University's computer lab, can easily and correctly determine major points of the human skeleton from two-dimensional photographs. Openpose will detect the location information of 18 key human body points and use the key human body information points in conjunction with the SVM classifier to determine whether or not individuals are descending.

[0024]. Carnegie Mellon University's OpenPost Human Attitude Recognition Project is an open source library focused on convolutional neural networks and supervised learning, as well as the caffe platform (CMU). The Openpose algorithm is a posture algorithm based on a human bottom-up skeleton. First, vggnet-19 in the main body network extracts the F function of the original image, and then two branches are utilised to regress the S position of the joint points and the L direction of the pixels in the skeleton. The resulting branch network topology is multi-stage iteration.

[0025]. Every step computes a loss function and connects the node's location S and direction L of the pixels in the branch 1 skeleton to the original picture characteristics collected for training by vggnet-19 in the following stage.

[0026]. For the whole stage, the process can be expressed as St = pt(F, St-1, Lt-1), M t 2and L' = O(FSt-1,Lt-1),Mt 2, here pt and t represent respectively the position S of the T stage and the convolutional neural network of the skeletal direction L.

Two loss functions are introduced in the process of regressing the position S of the joint points and the direction L of the pixels in the skeleton. The L2 loss function is used between the correct value and the predicted value as f, = W(p).S(p) - SJ| and fj = Z 1 ZW(p).IL (p) - L71, Sj* and L* separately represent the real labeled values of PCM and PAFs in the image. Because the position P in the original image is not marked, W(P) = 0.

[0027]. The overall objective function can be derived as ft = Ei(ft + f). The open pose recognition rate is quite high during use, but the quantity of calculation is very enormous, which can only be achieved under good hardware circumstances to provide good real-time. As a result, the open attitude must be altered. The original opening is USES vgg-19, and the trained coffee model is wide and around 200 M in size.

[0028]. In this case, the size of the mobileNet model utilised by openCV is only 7.8MB, which significantly reduces the size of the model and the measurement number in the event of a minor loss of recognition degree. As a result, mobile enet-thin achieves 4.2fps quicker than vgg-19 model fps on the 2.8GHz Quad-core 7 CPU performance, which is expected to perform better on GPU and be more important in real life. In terms of recognition accuracy, the mobilenet training model achieves 81.3 percent, which is lower than the vgg-19's 94.2 percent due to computation decrease.

[0029]. The concept is modified in this invention based on an open source kit. The original author can modify the model to run openpose on the CPU, which can run openpose on mobile devices but has worse efficiency. During the usage phase, the high resolution of the measurement quantity is discovered to be huge, and the real-time output is low. The resolution is too low, the real-time output is adequate, but the rate of recognition declines rapidly, the human location cannot be computed, and false alarms are common.

[0030]. With a resolution of 368x368, it provides exceptional real-time performance and high dependability. There are five movements: side stand, front stand, front open arms, side squat, side crouch, and so on. Openpose can correctly determine the skeleton's position, whether front or side, squat or lie down. The fall recognition module will determine whether or not the elderly fell based on the relative changes in skeleton position.

[0031]. Openpose largely measures the human body's 18 primary positions, whereas assessing the body's state requires only six primary trunk positions, including the nose, spine, left hip, right hip, left knee, and right knee. Six point data may be trained using the SVM classifier, and the generated model can be used to determine whether to fall. Because the SVM classifier is based on the statistical principle of small samples, even small samples can yield good results, and because a shortage of training samples is a significant issue in the field of fall detection at the moment, employing the SVM classifier can help offset this issue.

[0032]. Additionally, because the SVM kernel function is capable of handling large data sets, it can readily manage the high-dimensional joint features generated by the previous openpose without impairing the classifier's accuracy or generalisation potential due to its high latitude. The video's prospective state is classified into two categories here: regular state and fallen state.

[0033]. In the normal condition, elders walk and sit; in the fall state, elders lie on the ground. When training an SVM classifier, we choose a Gauss kernel function, set the kernel function parameter (gamma) to 0.05, and set the penalty parameter to 1. After training, the SVM classifier's accuracy for joint state categorization was 93.25 percent.

[0034]. Alarm module: This innovation utilises the GPRS module as an alarm module, enables serial communication control of the GPRS module, and enables brief message transmission. When the system determines that the data created by the above-mentioned face recognition and attitude recognition modules is harmful, the system transmits the necessary alert information to the specified mobile phone number via the GSM module to activate the alarm function.

[0035]. Work flow of the system: The camera captures video data in real time and delivers it to the computer for processing. The gadget transmits the processed images to the necessary face recognition, pedestrian detection, and behaviour detection modules for operation recognition. The machine receives the result of the identification's interpretation. The results are analysed by the machine. When a hazard occurs, it sends dangerous information to the appropriate mobile phone by regulating the GSM module. Fig. 2 illustrates the flow chart.

[0036]. The camera collects and processes image data that is based on openCV. The processed picture information is sent to the pedestrian detection and tracking module to determine whether there is an individual in the image information. If there is a male in the pedestrian information image, it is transmitted to the facial recognition module for discrimination. If the facial recognition module determines that the person is a stranger, it sends an alarm to the machine.

[0037]. After discriminating, the machine regulates the GSM module to provide dangerous information to the family cell phone. If the elderly are identified, the visual information for behaviour judgement will be sent to the behaviour recognition module. The recognition module is now solely used to determine the risky acts of falling. If it is determined that the elderly person would fall, the risky information will be communicated to the elderly person's relatives via GSM. The device was tested and proven to be accurate up to 85 percent of the time.

[0038]. Based on openCV, we successfully established an intelligent, secure home system for the elderly in this invention. The complete system is comprised of five major modules: real-time monitoring module, tracking module for human body recognition, face recognition module, behaviour recognition module, warning module, and so on. The monitoring module collects the real-time image stream in real time, and the image is processed using openCV.

[0039]. Three modules of human body recognition, face recognition, and behaviour recognition are communicated to the processed image for recognition activities, and the living conditions of the elderly at home are rated based on the target feature information. Once the gadget is built to collect abnormal information, such as foreign interference, elderly people falling down, elderly people becoming unwell, and other abnormal conditions, the abnormal information would be conveyed to the elderly family via network contact. Simultaneously, the GSM module can be upgraded and outfitted with mobile applications like Android. The client can also be used by the elderly members of the family at any time to access real-time information about the elderly.

Claims (6)

1. CLAIMS: We Claim: 1. We claim that our invention, Safe Home System for Protecting the Elderly People, will be effective in protecting elders from foreign meddling, unintentional falls, illness, and other unusual circumstances.
2. 2. The fundamental goal of this invention, as stated in 1, is to create an intelligent, secure home system for the elderly based on OpenCV.
3. 3. The innovation, according to us, consists of five primary modules: real-time monitoring module, tracking module for human body recognition, face recognition module, behaviour recognition module, warning module, and so on.
4. 4. As we claimed in 1 and 3, the monitoring module collects the real-time image stream in real time, and the image is processed using openCV.
5. 5. We claimed in 1, the device in this invention is designed to collect abnormal information, such as foreign interference, elderly people falling down, elderly people being unwell, and other abnormal conditions, and the abnormal information would be conveyed to the senior family via network contact.
6. 6. This technology allows family members of the elderly to get real-time information about the elderly at any time.