CN114469000A

CN114469000A - Human body falling reduction and shock prevention intelligent monitoring method and system based on multi-sensor data reinforcement learning

Info

Publication number: CN114469000A
Application number: CN202210102329.3A
Authority: CN
Inventors: 余海燕; 黄学达; 胡邓; 谢昊飞; 蔡宇翔; 尹彦臻; 邓兴媛; 范国慷; 周杰
Original assignee: Chongqing University of Post and Telecommunications
Current assignee: Chongqing University of Post and Telecommunications
Priority date: 2022-01-27
Filing date: 2022-01-27
Publication date: 2022-05-13
Anticipated expiration: 2042-01-27
Also published as: CN114469000B

Abstract

The invention belongs to the field of sensor data processing, and particularly relates to a human body falling and shock prevention intelligent monitoring method and system based on multi-sensor data reinforcement learning, which comprises the following steps: monitoring the monitored object in real time, and acquiring sensing data of the monitored object by adopting a plurality of sensors; obtaining a human body falling related state according to the perception data; inputting the relevant state information into a semi-observation Markov decision process model to obtain an optimal execution strategy; executing corresponding actions by the strategy executing arm system, wherein the actions comprise selecting an executing arm with the maximum protection effect from executing arms of candidate parts, activating the executing arm and outputting an inflation command to achieve the effects of reducing falling and preventing vibration; after the action of the execution arm is finished, the sensor sensing data is collected again, the relevant state of the falling of the human body is obtained, and then the next stage decision support process is carried out; the intelligent air bag intervention mode based on reinforcement learning can reduce the injury risk of falling down for patients with different falling types in a targeted manner.

Description

Human body falling reduction and shock prevention intelligent monitoring method and system based on multi-sensor data reinforcement learning

Technical Field

The invention belongs to the field of sensor data processing, and particularly relates to a human body falling reduction and shock prevention intelligent monitoring method and system based on multi-sensor data reinforcement learning.

Background

Health technological innovation in the process of population aging has become a hot spot in relevant theoretical and industrial circles. The aging of the population begins in the western world of the 20 th century, which is unprecedented in human history. In china, the number of elderly people over the age of 60 years is always on the rise between 2010 and 2050, and the degree of aging of the population is increasing. In the case of unintended injury to the elderly, falls serve as a significant cause of death in the elderly, since the elderly are older and have reduced reaction and balance abilities and reduced muscle strength. In which the craniocerebral trauma associated with falls is also four times more hospitalized by the patient than the community population and is generally less well-being. Therefore, falling is an important risk event often faced in the health care of the elderly, and high-quality and efficient strategic measures are urgently needed to be taken to individualize prevention and fine management.

At present, for the problem of fall monitoring of old people, a plurality of scholars propose related solutions; if a three-dimensional acceleration value of a human body is acquired by using a three-axis acceleration sensor, space positioning is carried out after abnormal data appear, and finally the falling abnormal position is sent in a wireless communication mode; the mobile alarm system has the advantages that the three-axis acceleration sensor is used for detecting the activity information of the old, the single chip microcomputer is used for collecting and storing data, the wavelet means is used for analyzing the data, when the old falls down, the terminal is automatically positioned, and the accurate position is automatically remotely alarmed in a short message mode. Almeida et al propose a cane with a gyroscope that is used to measure the angular rate of activity of the elderly to determine whether the elderly have fallen. However, the walking stick may fall down due to the reason that the old person falls down unintentionally, and therefore, a certain false alarm exists when the old person falls down according to the judgment of the falling of the walking stick. Bourke AK et al propose a fall detection device wearable on the chest, which is composed of two orthogonal gyroscopes and is used to measure the angular velocity and the change of the human body angle to judge whether the elderly falls.

However, it can be seen from the existing research technology that the core devices for monitoring the falling of the old people all adopt a three-axis acceleration sensor; the sensor is easily influenced by environmental factors and self factors of the old people in the process of collecting the information of the old people, so that the collected information has errors, and the monitoring is difficult; therefore, a solution for realizing effective early warning, timely response, reduction of collision in the falling process and reduction of injury caused by falling when a human body falls is urgently needed.

Disclosure of Invention

In order to solve the problems in the prior art, the invention provides a human body falling and shock prevention intelligent monitoring method based on multi-sensor data reinforcement learning, which comprises the following steps:

s1: setting a state transition probability and an observation probability;

s2: monitoring the monitored object in real time, and sensing the state data of the monitored object through a plurality of sensors;

s3: processing the multi-sensor sensing data according to the observation probability to obtain relevant state information of human body falling;

s4: inputting relevant state information of human body falling, observation probability and state transition probability into a semi-observation Markov decision process model to obtain an optimal decision of a monitored object at the current moment;

s5: inputting the optimal decision into an arm system, and executing a corresponding command by the arm system according to the optimal decision, wherein the command comprises an execution arm with the maximum protection effect (return value) selected from the execution arms of the candidate parts, activating the execution arm and outputting an inflation command;

s6: inflating the intelligent airbag according to an inflation command;

s7: after the action of the executing arm is finished, sensing the state data of the monitored object again by adopting a multi-sensor, and obtaining the relevant state of the falling of the human body according to the data sensed secondarily; the report value is updated according to the human fall-related status, and the process returns to step S2.

Preferably, the process of setting the state transition probability and the observation probability includes: the observation probability calculation process comprises the steps of obtaining historical monitoring data of a monitored object, and carrying out statistical normalization processing on the historical monitoring data by adopting a statistical method to obtain state transition probability; the observation probability calculation process includes dividing observation categories of the monitoring object into 9 categories, including: no signal, waist, left upper limb, right upper limb, head, left lower limb, right lower limb, spine sensor signal, and death signal; all observation sets correspond to a symbol of Ω ═ o_N,o_W,o_LA,o_RA,o_H,o_LL,o_RL,o_S,o_DIn which o is_NDenotes no signal, o_WRepresenting the waist sensor signal, o_LARepresenting the left upper limb sensor signal, o_RARepresenting the right upper limb sensor signal, o_HRepresenting head sensor signals, o_LLRepresenting left lower limb sensor signal, o_RLRepresenting the right lower limb sensor signal, o_SRepresenting spinal sensor signals, o_DIndicating a death signal; and acquiring historical monitoring data of the monitored object, and performing statistical normalization processing on the historical monitoring data by adopting a statistical method to obtain the observation probability of each observation category.

Preferably, the multi-sensor includes three hall acceleration sensors for measuring acceleration information of the x-axis, the y-axis, and the z-axis of the monitored object, and three angle measurement sensors for measuring attitude angles of the acceleration of the x-axis, the y-axis, and the z-axis.

Preferably, the obtaining of the fall-related status information of the human body includes:

s31: calculating the attitude information of the monitored object according to the information acquired by the sensor; the posture information comprises four categories of normal walking, high risk of falling, falling and death;

s32: determining a target of state transition of the monitored object at the current moment by adopting the state transition probability according to the posture information of the monitored object;

s33: and determining the state target of the monitored object at the current moment by adopting the observation probability according to the attitude information of the monitored object.

Further, the process of calculating the posture information of the monitored object includes:

step 1: establishing a space rectangular coordinate system by taking the front side of the monitored object as an x axis, the front left side as a y axis and the vertical direction as a z axis; setting an acceleration amplitude threshold value, an x-axis acceleration attitude angle threshold value, a y-axis acceleration attitude angle threshold value and a downward acceleration threshold value;

step 2: smoothing the acquired acceleration of the x axis, the y axis and the z axis;

and step 3: calculating the acceleration amplitude of the monitored object according to the smoothed data;

and 4, step 4: comparing the calculated acceleration amplitude with a set acceleration amplitude threshold, if the calculated acceleration amplitude is greater than the set threshold, executing the step 5, otherwise, re-acquiring the attitude information of the monitored object;

and 5: comparing the x-axis acceleration attitude angle threshold value and the y-axis acceleration attitude angle threshold value with the x-axis acceleration attitude angle threshold value and the y-axis acceleration attitude angle threshold value respectively; if the x-axis acceleration attitude angle is larger than the set x-axis acceleration attitude angle threshold or the y-axis acceleration attitude angle is larger than the y-axis acceleration attitude angle threshold, executing the step 6, otherwise, sensing the attitude information of the monitored object again;

step 6: and calculating the acceleration in the vertical direction according to the z-axis acceleration attitude angle, comparing the acceleration in the vertical direction with a set downward acceleration threshold, if the acceleration in the vertical direction is smaller than the downward acceleration threshold, generating signal data when the monitored object falls down, and otherwise, acquiring the attitude information of the monitored object again.

Further, the set acceleration amplitude threshold is 1.9, the x-axis acceleration attitude angle threshold and the y-axis acceleration attitude angle threshold are both 65 degrees, and the downward acceleration threshold is 0.6.

Preferably, the process of obtaining the optimal decision by using the semi-observation markov decision process model includes:

step 1: initializing half-observation Markov decision process model parameters, and using the initialized parameters and data input into a model for seven-tuple < S, A, P, omega, O, R, gamma > representation, wherein S represents a group of state sets, A represents a group of action sets, P represents a transition matrix between states, omega represents a group of observation sets, O represents an observation probability, R is a return function, and gamma is a discount factor;

step 2: setting and determining time intervals and moments, and making a decision at each moment; the set time is T ═ {0, …, T }, where T denotes a timeline;

and step 3: setting an initial belief state of the monitored object, wherein the belief state represents the understanding condition of a decision maker on the current walking state of the monitored object;

and 4, step 4: calculating a return value of the monitored object according to the input data, and calculating the expectation of the model according to the return value;

and 5: updating the initial belief state according to the input data;

step 6: obtaining a Bellman optimal equation according to the updated mind state and the return value;

and 7: and calculating the optimal solution of the Bellman optimal equation, wherein the optimal solution is the optimal decision of the monitoring object at the current moment.

Further, the belief state update formula is as follows:

where O (O | s') represents the observation probability and π(s) represents the initial belief state.

Further, the expression of the bellman optimal equation is as follows:

where s represents a state in the state set of the monitoring object, pi represents a belief state of the monitoring object, r (s, a) represents a return value, a represents an action performed by the arm system, p (s '| s, a) represents a transition probability, O (O | s') represents an observation probability, and V (s ', pi') represents a bellman equation.

A human body falling and shock prevention intelligent monitoring system based on multi-sensor data reinforcement learning comprises a human body acceleration data acquisition assembly, a micro-processing module, an alarm module and a power supply module; the human body acceleration data acquisition assembly, the micro-processing module and the alarm module are all arranged on the PCB;

the human body acceleration data acquisition assembly comprises three Hall acceleration sensors and three angle measurement sensors, the three Hall acceleration sensors are used for measuring acceleration information of an x axis, a y axis and a z axis of a monitored person, the three angle measurement sensors are used for measuring attitude angles of the acceleration of the x axis, the y axis and the z axis and are connected with the micro-processing module through an OR gate;

the micro-processing module is used for receiving and processing data transmitted by the human body acceleration data acquisition assembly in real time and sending signal data to the alarm module;

the alarm module is used for acquiring signal data, processing the sent signals by adopting a semi-observation Markov decision process model, predicting the falling risk of the monitored object, outputting an inflation command according to the predicted falling risk, and inflating an air bag according to the inflation command to protect the monitored object;

the power module is connected with the PCB and used for supplying power to the human body falling-reducing and shockproof intelligent monitoring system.

In order to achieve the above object, the present invention further provides a computer readable storage medium, on which a computer program is stored, wherein the computer program, when executed by a processor, implements any one of the above human fall and shock prevention intelligent monitoring methods based on multi-sensor data reinforcement learning.

In order to achieve the above object, the present invention further provides a human body fall-reducing and earthquake-preventing intelligent monitoring device based on multi-sensor data reinforcement learning, which comprises a processor and a memory; the memory is used for storing a computer program; the processor is connected with the memory and is used for executing the computer program stored in the memory so as to enable the human body falling and earthquake prevention intelligent monitoring device based on multi-sensor data reinforcement learning to execute any one of the human body falling and earthquake prevention intelligent monitoring methods based on multi-sensor data reinforcement learning.

The invention has the following advantages: for the judgment of the falling state, the invention designs a falling algorithm, and under the condition that three-level judgment is established, the falling is judged to take a protective measure; the invention combines the human body falling process with a semi-observation Markov decision process (POMDP) model, predicts the action of an intelligent execution arm, thereby prejudging the human body falling state and executing the intervention of an intelligent air bag; the intelligent air bag intervention mode based on reinforcement learning can reduce the injury risk of falling down for patients with different falling types in a targeted manner.

Drawings

FIG. 1 is a schematic diagram of data collection of human posture sensors for fall-reducing, earthquake-resistant and reinforcement learning of human body

FIG. 2 is a hardware wiring diagram of the human body falling prevention intelligent protection system based on multi-sensor data reinforcement learning disclosed by the embodiment of the invention;

FIG. 3 is a flowchart of a tilt angle threshold determination algorithm disclosed in the present embodiment of the invention;

FIG. 4 is a schematic diagram of a data testing apparatus according to an embodiment of the present invention;

FIG. 5 is a flow chart of execution arm intelligent control based on reinforcement learning in human body fall-reducing and earthquake-resisting;

fig. 6 is a schematic structural diagram of a human body falling prevention intelligent protection system based on multi-sensor data reinforcement learning.

Detailed Description

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

A human body fall and shock prevention intelligent monitoring method based on multi-sensor data reinforcement learning is disclosed, as shown in figure 1, and comprises the following steps: monitoring the monitored object in real time, and acquiring attitude information of the monitored object by adopting a sensor; processing the acquired attitude information to obtain signal data; setting a prediction threshold, comparing the set prediction threshold with the signal data, if the signal data is less than or equal to the set prediction threshold, re-acquiring the attitude information of the monitored object, otherwise, inputting the signal data into a semi-observation Markov decision process model to predict the falling risk of the monitored object; and outputting an inflation command according to the predicted falling risk, and inflating the air bag according to the inflation command to protect the monitored object.

An important link of the invention is based on a specific implementation mode of multi-sensor data reinforcement learning, the human body falling process and a semi-observation Markov decision process (POMDP) model are combined, the action of an intelligent execution arm is predicted, and thus the human body falling state is predicted and the intervention of an intelligent air bag is executed. In the POMDP model, the elements are usually represented by a seven-tuple. The method specifically comprises the following steps:

step S11: a set of finite state sets is represented by S, wherein the states are { normal walking, high-risk falling, falling and death } and are represented as { S1, S2, S3 and S4 };

step S12: a group of limited action sets is represented by a, and the actions which can be executed by the intelligent execution arm are represented by (non-execution, waist, left upper limb, right upper limb, head, left lower limb, right lower limb, spine), and are respectively represented as { a0, a1, a2, A3, a4, a5, a6, a7 }; the signal data is transmitted to the inclination sensor air bag device; the device is inflated to protect the object.

Step S13: the state transition matrix is denoted by P, P_a(s ' | s) ═ P (s ' | s, a) represents the probability that action a taken at time t state s can transition to state s ' at time t + 1;

step S14: r represents the return obtained by the intelligent execution arm in the state of the human body at the next moment after the intelligent execution arm executes a certain action at a certain moment;

step S15: the method is characterized in that an observation result set is represented by omega, observation of a human body is mainly obtained through sensors, the sensors are bound at different parts of the human body { sole, left lower limb, right lower limb, waist, spine, left upper limb, right upper limb and head (face) }, signals in the falling process are tested, and the sensors at different parts can be carried out simultaneously or respectively;

step S16: o represents the conditional observation probability, and when the signals of the sensors are received, only partial characteristics of the human body in walking can be observed, so that the probability that the human body is in a certain state needs to be judged by combining the signals; monitoring the monitored object in real time; acquiring attitude information of a monitored object; processing the attitude information using an algorithm; calculating to obtain signal data of the attitude state; comparing the obtained signal data with a threshold value; and if the signal data is larger than the threshold value and the three-level judgment is established, judging that the monitored object falls down. And if the signal data is smaller than the threshold value and any judgment in the three-level judgment is not true, continuing monitoring. The corresponding part signal is judged as 1 when falling, otherwise, the corresponding part signal is 0. At completion, 256 elements are included, such as {1,1,1,1,1,1, 1} {1,0,0,0,0,0,0 }. The signal set of the sensor can be simplified to 9 elements, i.e., o1{0,0,0,0,0, 0}, o2{1,0,0,0,0,0, 0}, o3{0,1,0,0,0,0,0,0}, o4{0,0,1,0,0,0,0,0}, o5{0,0,0,1,0,0,0,0}, o6{0,0,0,0, 0,1,0,0,0,0}, o7{0,0,0,0,0, 0,1,0,0}, o8{0,0,0,0,0,0, 0,1,0}, o9{0,0,0,0,0,0,0 }.0, 0, 1.

Step S17: let γ denote the discount factor γ ∈ [0,1 ].

Since doctors can only observe partial characteristics of the human body in walking, the doctors need to judge that the human body is in a certain state by combining signals, and 8 sensors are bound to 8 positions at the same time to test signals in the falling process of the human body; or 8 tests were performed separately. After the state of the human body is judged, the human body can obtain a corresponding report value. Therefore, in order to prevent the human body from falling down, the intelligent execution arm is worn on the human body, and corresponding actions are executed when the human body falls down. When falling down every time, in five positions, for effectively judging the position of the intelligent air bag to execute actions, two kinds of constraints are provided: firstly, every time the person falls down, 8 sensors at the positions can sense signals, and modeling analysis of the 8 signals is carried out; and secondly, every time the user falls down, the user can only automatically (by a model algorithm) select one from five (by the air bags at the positions).

The state transition probability matrix is described by taking the example of performing action a1 at time t. The first column in the table represents the state of the human body at time t, the first row represents the state of the human body at time t +1 after the action a1 is performed based on the state at time t, the first row data represents that the state of the human body at time t is normal and no action is performed (a1), the probability that the human body still walks normally at time t +1 is 0.5, the probability of high risk is 0.2, the probability of falling is 0.2, and the probability of death is 0.1. The first column of data indicates that the state of the human body at time t is normal/high risk/fall/death without performing any action (a1), and the probability that the state of the human body at time t +1 is normal is 0.5/0.1/0.01/0 in this order. The data for each row in this table is summed to 1.

TABLE 1 State transition probability matrix for the execution of action a1 at time t

The observation probability matrix is described by taking the execution of action a1 at time t as an example. The first column in the table represents the state of the human body at time t, the first column represents the state of the human body at time t, the first row represents the signal observed by the sensor at time t +1 after the action a1 is executed based on the state at time t, the first row data represents that the state of the human body at time t is normal without any action (a1), and the probability of observing the signals of the sensor at time t + 1. The first column of data shows that the state of the human body at the time t is normal/high-risk/fall/death, no action is performed (a1), and the probability of no signal of the sensor at the time t +1 is 0.2/0.1/0/0 in sequence. The data for each row in this table is summed to 1.

TABLE 2 Observation probability matrix for executing action a1 at time t +1

And performing calculation processing on the acquired detection data, wherein the process is mainly used for cleaning low-quality video images in the source data and eliminating data which are not in accordance with the specification. The method comprises the steps of extracting, converting, loading and other data cleaning of input data so as to realize missing data processing, similar repeated object detection, abnormal data processing, logic error detection, inconsistent data processing and the like.

The database design is shown in table 3, and an example of a portion of the feature data extracted by the tilt sensor and associated data is shown in table 4.

TABLE 3 sensor data sheet

TABLE 4 example feature data part sensing and associated data extraction by Tilt sensor

A hardware link of a human body falling prevention intelligent protection system based on multi-sensor data reinforcement learning is shown in FIG. 2 and comprises: all devices connected with the single chip microcomputer are grounded. The power supply of a driver of the motor driver can not be reversely connected, a 15A fuse is connected in series at the power supply interface, and the voltage is between 6.5 and 27V; if the voltage is over-voltage, the driving module can be burnt out when the power is on; the boost module inputs 7.4V and outputs 12V; if the output is not 12V, the blue regulator on the module can be rotated, and the voltage output can be adjusted. The single chip microcomputer controls IN1, IN2 and ENA1 of the motor driving module to realize the forward and reverse rotation of the motor, and the +5V of the enabling end is recommended to be connected with 3.3V; the ENA1 device enable terminal is connected to high level regardless of positive and negative rotation; when the IN1 inputs a high level and the IN2 inputs a low level, the full-speed forward transmission of the motor is realized, and when the IN1 inputs a high level and the IN2 inputs a low level, the full-speed reverse transmission of the motor is realized; the commutation cannot be performed when the motor has not stopped, otherwise the drive may be damaged; when the driving module is powered off, the motor is not directly or indirectly rotated at a high speed, otherwise, the driving module can be burnt by electromotive force generated by the motor. If the motor needs to be rotated at a high speed when the drive module is powered off in application, a relay (NO and COM ends are connected in series) is connected in series with a motor interface of the driver, and a relay coil and the driver share a power supply, namely when the power supply is powered off, the relay can disconnect the driver from the motor.

As shown in fig. 3, the process of processing the collected pose information includes:

step 1: establishing a space rectangular coordinate system by taking the front side of the monitored object as an x axis, the front left side as a y axis and the vertical direction as a z axis; setting an acceleration amplitude threshold value, an x-axis acceleration attitude angle threshold value, a y-axis acceleration attitude angle threshold value and a downward acceleration threshold value.

When the human body is at rest, the human body is only acted by gravity acceleration in the vertical direction. The x, y, z axis acceleration and attitude angles pitch, roly and raw obtained from the MPU6050 data are collected.

Step 2: and smoothing the acquired acceleration of the x axis, the y axis and the z axis.

In some cases, some extrinsic factors may affect the acceleration picked up by the sensor. In order to reduce the misjudgment possibly caused by noise, five-point multiple sliding average is carried out on the collected acceleration in a certain time window, dirty data with large deviation is removed, and the error caused by enlarging the dirty data when square operation is carried out is prevented.

And 3, step 3: and calculating the acceleration amplitude of the monitored object according to the data subjected to the smoothing processing.

And calculating an acceleration amplitude value (the acceleration amplitude value reflects the violent movement degree of the human body) by using the processed data. Falling belongs to a violent activity in life, when falling occurs, the height of a human body relative to the ground changes rapidly, and the acceleration applied to the human body in the process also changes. This change can be quantified by the acceleration magnitude value and reflects the change in attitude.

And 4, step 4: and comparing the calculated acceleration amplitude with a set acceleration amplitude threshold, executing the step 5 if the calculated acceleration amplitude is larger than the set threshold, and otherwise, re-acquiring the attitude information of the monitored object.

To reduce the effect of dirty data during data acquisition, the algorithm performs five-point multiple smoothing on the x, y, z3 directional axial acceleration data within the time window T. The falling time of the human body is 0.3-0.4 s, so the time window T is about the time of 10 sampling points. And smoothing the acquired data, and calculating the acceleration amplitude according to the distance characteristic value. And setting the threshold value between the falling state and the non-falling state to be 1.9, and if the calculated value is greater than 1.9, entering the next judgment.

And 5: comparing the x-axis acceleration attitude angle threshold value and the y-axis acceleration attitude angle threshold value with the x-axis acceleration attitude angle threshold value and the y-axis acceleration attitude angle threshold value respectively; and if the x-axis acceleration attitude angle is larger than the set x-axis acceleration attitude angle threshold or the y-axis acceleration attitude angle is larger than the y-axis acceleration attitude angle threshold, executing the step 6, otherwise, re-acquiring the attitude information of the monitored object.

A single calculated value cannot mask some disturbing actions such as a violent movement of the human body, and therefore, in order to better judge the fall state, an angle judgment assistance judgment by a downward acceleration and a roll angle and a pitch angle is adopted. When the pitch angle is larger than 65 when the human body falls forwards and backwards, and the roll angle is larger than 65 when the human body falls leftwards and rightwards, the next step of judgment is carried out.

When the human body is still or vertically walking, the downward acceleration is about equal to 9.8m/s, the human body can lose weight when falling down, and the acceleration can be reduced. When the acceleration of the human body falling downwards is less than 0.6m/s, the next step is carried out. If the three-level judgment is met, the carbon dioxide bottle is judged to be in a falling state, the single chip microcomputer outputs a motor forward rotation signal, and the motor breaks the carbon dioxide bottle.

The process of obtaining the optimal decision by adopting the semi-observation Markov decision process model comprises the following steps:

and step 3: setting a belief state of an initial monitored object, wherein the belief state represents the understanding condition of a decision maker on the current walking state of the monitored object;

and 5: updating the initial belief state according to the input data;

Specifically, the decision time and the time line. The time interval and the time are determined according to the sampling frequency of the sensor perception data, and the decision is made at the beginning of each time when the monitored object wears the intelligent device. The time instant in this context is denoted by T ═ {0, …, T }, where T denotes the time line.

A state space. The state of the human body during walking is divided into four categories: normal walking, high risk of falling, death, symbolized as S ═ S_N,s_H,s_C,s_DThe last state (death) is an absorption state. The state space is partially observable, but death is the only state that can be observed completely. As used herein s_tE S represents the state of the human body at time t.

And (4) an action space. Since the mechanical arms in different parts are used for executing the actions, the action space is divided according to different parts, including no action, and the intelligent machine execution arm for waist/left upper limb/right upper limb/head/left lower limb/right lower limb/spine air bag is represented by symbol A ═ a ═ by no action_N,a_W,a_LA,a_RA,a_H,a_LL,a_RL,a_SUsing a herein }_te.A represents the action performed at decision time t.

The transition probability. P(s) for transition probability_t+1|s_t,a_t) Meaning that the human body is in the current state(s)_tE.g. S) performs action (a)_tE.g. A), then the state is shifted to the next moment(s)_t+1E.g., S). These probabilities are obtained statistically from sensor data of fall prevention devices worn by the subjects, and can also be obtained by simulating more than 1000 walks and falls.

And (5) observation and observation probability. At each decision instant, a set of observations (o e Ω) provides some information about the unobserved states that the human body really exists in. The model is mainly observed through a sensor and obtained according to sensor data of falling prevention equipment worn by a subject. According to the signals transmitted back by the sensors at different positions, the observations are divided into 9 types: no signal, { waist, left upper limb, right upper limb, head, left lower limb, right lower limb, spine } sensor signal, represented in turn using two classes as { (0000000), (1000000), (0100000), (0010000), (0001000), (0000100), (0000010), (0000001) }, and one class of observations was death,thus all observation sets correspond to the symbol Ω ═ o_N,o_W,o_LA,o_RA,o_H,o_LL,o_RL,o_S,o_D}. The signals of the sensors are also not completely accurate, there is a probabilistic relationship between the observed and the non-observed states, represented by the probability matrix O, consisting of the probability of observation O (O | s), in the sense of the probability of observing the state O given the patient's true state s and the execution of the action a. And the observation probability matrix data is obtained by statistics according to the sensor data of the falling prevention equipment worn by the subject.

A belief state. Let Π (S) denote the probability of each state in the state space S, which in the case herein is a total of four states, with the probabilities expressed as:

the vector pi is called a belief state and represents the understanding of the current walking state of the human body by a decision maker. The belief state is represented by a set of probability sets, each probability represents the possibility of each state, and the belief state of the human body at the time t can be represented as:

π_t＝(π_t(s_N),π_t(s_H),π_t(s_C),π_t(s_D))

the belief state vector of the human body at the time t is pi ═ (0.3, 0.4, 0.4, 0), which indicates that the human body has a probability of walking normally, a probability of falling to a high risk state of 30% and a probability of falling to 40%.

A reward function. In the POMDP model, the optimal strategy is to maximize the expected return within the decision range T, which is expressed by the mathematical formula:

where γ represents a discount factor, r_tIndicating immediate return at time tReporting the value. The reward depends on the state of the human body and the action taken, and the set of possible rewards is derived from a reward function r (s, a, s '), meaning the reward of performing action a at state s and transitioning to state s'. The reported value is obtained by calculating the statistical probability and medical cost corresponding to the injury caused by various states (normalization processing).

And updating the belief state. Given a new observation o ' epsilon omega, the belief state pi can be updated to pi ' according to Bayesian rules, and for each state S ' epsilon S, the corresponding updated belief state value can be calculated by using the following formula:

wherein, O (O | s ') represents the observation probability, pi(s) represents the initial belief state, and s ' and pi ' represent the state corresponding to the next time and the belief state respectively.

The POMDP model can be re-established by MDP of a continuous state, and the optimal strategy is obtained by solving a Bellman optimal equation:

wherein s represents a state in the state set of the monitoring object, pi represents a belief state of the monitoring object, r (s, a) represents a return value, a represents an action performed by the arm system, p (s '| s, a) represents a transition probability, O (O | s') represents an observation probability, V (s ', pi') represents a bellman equation value corresponding to the next time, gamma is a discount factor, pi represents a time corresponding to the next time, and_jrepresenting the belief state corresponding to state j.

The algorithmic process of the markov decision process based on the observable walking state of the human body part is shown in table 5.

TABLE 5

As shown in fig. 4, a data testing device, in which a processor performs smoothing processing and acceleration amplitude calculation on acquired MPU6050 data to transmit effective information; the vibration sensor module judges the state of the sensor; the communication interface transmits information; and under the falling state, the falling-preventing anti-seismic air field device is inflated for protection.

As shown in fig. 5, the state transition of the monitoring system and the selection of the execution arm of the intelligent device to execute the action include: the intelligent execution arm can execute actions at the positions including the waist, the left upper limb, the right upper limb, the head, the left lower limb, the right lower limb, the spine and the like. The control signal activates the execution arm at a certain position, and the intelligent execution arm executes a certain action at a certain moment. Will be in the new state and receive a corresponding reward.

A human body falling prevention intelligent protection system based on multi-sensor data reinforcement learning comprises a human body acceleration data acquisition assembly, a micro-processing module, an alarm module and a power supply module; the human body acceleration data acquisition assembly, the micro-processing module and the alarm module are all arranged on the PCB;

As shown in fig. 6, the overall architecture of the human body fall prevention intelligent protection system based on multi-sensor data reinforcement learning includes: the power module turns on a power supply; an alarm in the vibration sensor module senses to judge whether the vibration sensor module is in a vibration or inclination state; when the sensor shakes or inclines, the carbon dioxide small steel cylinder is closed; after closing the carbon dioxide small steel cylinder, the motor breaks the carbon dioxide small steel cylinder; after the carbon dioxide small steel cylinder is broken by the motor, the anti-falling anti-seismic buffering inflatable bag and the air bag of the air bubble column start to inflate.

In an embodiment of the present invention, the present invention further includes a computer readable storage medium, on which a computer program is stored, where the computer program is executed by a processor to implement any one of the above-mentioned human fall-resistance and earthquake-prevention intelligent monitoring methods based on multi-sensor data reinforcement learning.

Those of ordinary skill in the art will understand that: all or part of the steps for implementing the above method embodiments may be performed by hardware associated with a computer program. The aforementioned computer program may be stored in a computer-readable storage medium. When executed, the program performs steps comprising the method embodiments described above; and the aforementioned storage medium includes: various media that can store program codes, such as ROM, RAM, magnetic or optical disks.

A human body fall-reducing and shock-proof intelligent monitoring device based on multi-sensor data reinforcement learning comprises a processor and a memory; the memory is used for storing a computer program; the processor is connected with the memory and is used for executing the computer program stored in the memory so as to enable the human body falling and earthquake prevention intelligent monitoring device based on multi-sensor data reinforcement learning to execute any one of the human body falling and earthquake prevention intelligent monitoring methods based on multi-sensor data reinforcement learning.

Specifically, the memory includes: various media that can store program codes, such as ROM, RAM, magnetic disk, U-disk, memory card, or optical disk.

Preferably, the Processor may be a general-purpose Processor, including a Central Processing Unit (CPU), a Network Processor (NP), and the like; the Integrated Circuit may also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other Programmable logic device, discrete Gate or transistor logic device, or discrete hardware components.

The above-mentioned embodiments, which further illustrate the objects, technical solutions and advantages of the present invention, should be understood that the above-mentioned embodiments are only preferred embodiments of the present invention, and should not be construed as limiting the present invention, and any modifications, equivalents, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A human body fall-reducing and shock-proof intelligent monitoring method based on multi-sensor data reinforcement learning is characterized by comprising the following steps:

s1: setting a state transition probability and an observation probability;

s4: inputting relevant state information, observation probability and state transition probability of human body falling into a semi-observation Markov decision process model to obtain an optimal decision of a monitoring object at the current moment;

s6: inflating the intelligent airbag according to an inflation command;

s7: after the action of the executing arm is finished, sensing the state data of the monitored object again by adopting a multi-sensor, and obtaining the relevant state of the falling of the human body according to the data sensed secondarily; the return value is updated according to the human fall-related status, and the process returns to step S2.

2. The human body falling and shock prevention intelligent monitoring method based on multi-sensor data reinforcement learning of claim 1, wherein the process of setting the state transition probability and the observation probability comprises: the observation probability calculation process comprises the steps of obtaining historical monitoring data of a monitored object, and carrying out statistical normalization processing on the historical monitoring data by adopting a statistical method to obtain state transition probability; the observation probability calculation process includes dividing observation categories of the monitoring object into 9 categories, including: no signal, lumbar, left upper limb, right upper limb, head, left lower limb, right lower limb, spine sensor signal, and death signal; all observation sets correspond to a symbol of Ω ═ o_N,o_W,o_LA,o_RA,o_H,o_LL,o_RL,o_S,o_DIn which o is_NDenotes no signal, o_WRepresenting the waist sensor signal, o_LARepresenting the left upper limb sensor signal, o_RARepresenting the right upper limb sensor signal, o_HRepresenting head sensor signals, o_LLIndicating left lower limb sensor signal, o_RLRepresenting the right lower limb sensor signal, o_SRepresenting spinal sensor signals, o_DIndicating a death signal; and acquiring historical monitoring data of the monitored object, and performing statistical normalization processing on the historical monitoring data by adopting a statistical method to obtain the observation probability of each observation category.

3. The human body falling and earthquake prevention intelligent monitoring method based on multi-sensor data reinforcement learning of claim 1 is characterized in that the multi-sensor comprises three Hall acceleration sensors and three angle measurement sensors, wherein the three Hall acceleration sensors are used for measuring acceleration information of an x axis, a y axis and a z axis of a monitored object, and the three angle measurement sensors are used for measuring attitude angles of the acceleration of the x axis, the y axis and the z axis.

4. The intelligent human fall and shock prevention monitoring method based on multi-sensor data reinforcement learning of claim 1, wherein obtaining human fall related status information comprises:

5. The human body falling and shock prevention intelligent monitoring method based on multi-sensor data reinforcement learning as claimed in claim 4, wherein the process of calculating the posture information of the monitored object comprises:

and 5: comparing the x-axis acceleration attitude angle threshold value and the y-axis acceleration attitude angle threshold value with the x-axis acceleration attitude angle threshold value and the y-axis acceleration attitude angle threshold value respectively; if the x-axis acceleration attitude angle is larger than the set x-axis acceleration attitude angle threshold value or the y-axis acceleration attitude angle is larger than the y-axis acceleration attitude angle threshold value, executing the step 6, otherwise, perceiving the attitude information of the monitored object again;

6. The human body falling and shock prevention intelligent monitoring method based on multi-sensor data reinforcement learning of claim 5 is characterized in that an acceleration amplitude threshold value is set to be 1.9, an x-axis acceleration attitude angle threshold value and a y-axis acceleration attitude angle threshold value are both 65 degrees, and a downward acceleration threshold value is 0.6.

7. The intelligent human fall-down and shock-prevention monitoring method based on multi-sensor data reinforcement learning of claim 1, wherein the process of obtaining the optimal decision by using a semi-observation Markov decision process model comprises:

and 5: updating the initial belief state according to the input data;

8. The human body falling and shock prevention intelligent monitoring method based on multi-sensor data reinforcement learning of claim 7 is characterized in that a belief state updating formula is as follows:

9. The human body falling reduction and earthquake prevention intelligent monitoring method based on multi-sensor data reinforcement learning of claim 7, wherein an expression of a Bellman optimal equation is as follows:

10. A human body falling and shock prevention intelligent monitoring system based on multi-sensor data reinforcement learning is characterized by comprising a human body acceleration data acquisition assembly, a micro-processing module, an alarm module and a power supply module; the human body acceleration data acquisition assembly, the micro-processing module and the alarm module are all arranged on the PCB;

the human body acceleration data acquisition assembly comprises: the three Hall acceleration sensors are used for measuring acceleration information of an x axis, a y axis and a z axis of a monitored person, and the three angle measurement sensors are used for measuring attitude angles of the acceleration of the x axis, the y axis and the z axis and are connected with the micro-processing module through an OR gate;