CN113807275A - Household video falling detection method based on privacy protection type GAN feature enhancement - Google Patents

Household video falling detection method based on privacy protection type GAN feature enhancement Download PDF

Info

Publication number
CN113807275A
CN113807275A CN202111113708.4A CN202111113708A CN113807275A CN 113807275 A CN113807275 A CN 113807275A CN 202111113708 A CN202111113708 A CN 202111113708A CN 113807275 A CN113807275 A CN 113807275A
Authority
CN
China
Prior art keywords
video
privacy protection
matrix
compressed sensing
detection method
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202111113708.4A
Other languages
Chinese (zh)
Other versions
CN113807275B (en
Inventor
刘佶鑫
孟茹
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Nanjing University of Posts and Telecommunications
Original Assignee
Nanjing University of Posts and Telecommunications
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Nanjing University of Posts and Telecommunications filed Critical Nanjing University of Posts and Telecommunications
Priority to CN202111113708.4A priority Critical patent/CN113807275B/en
Publication of CN113807275A publication Critical patent/CN113807275A/en
Application granted granted Critical
Publication of CN113807275B publication Critical patent/CN113807275B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
    • G06F17/10—Complex mathematical operations
    • G06F17/16—Matrix or vector computation, e.g. matrix-matrix or matrix-vector multiplication, matrix factorization
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/213—Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods
    • G06F18/2135—Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods based on approximation criteria, e.g. principal component analysis
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N7/00—Computing arrangements based on specific mathematical models
    • G06N7/08—Computing arrangements based on specific mathematical models using chaos models or non-linear system models

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Mathematical Physics (AREA)
  • Evolutionary Computation (AREA)
  • Artificial Intelligence (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Software Systems (AREA)
  • Computing Systems (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Pure & Applied Mathematics (AREA)
  • Mathematical Optimization (AREA)
  • Computational Mathematics (AREA)
  • Evolutionary Biology (AREA)
  • Mathematical Analysis (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Biomedical Technology (AREA)
  • Algebra (AREA)
  • Health & Medical Sciences (AREA)
  • Nonlinear Science (AREA)
  • Databases & Information Systems (AREA)
  • Image Analysis (AREA)

Abstract

The invention relates to a privacy protection type GAN feature enhancement-based household video fall detection method, which mainly comprises the following steps: (1) inputting a training data set of an original video; (2) multilayer compressed sensing processing; (3) extracting a foreground moving target; (4) obtaining a fall classification result: and (3) performing space-time migration compensation on the foreground moving target in the video with the visual privacy protection effect obtained in the step (3) by using a GAN network with an auxiliary classifier, so that the lost features in the video with the visual privacy protection effect are compensated, and a final falling classification result is obtained on the basis. The invention solves the problem that privacy leakage risks exist in the current home video monitoring process facing the health monitoring of the old. The falling detection of the old people at home can be realized, the protection of the personal and family privacy of the old people can be met, and the practical application value is high.

Description

Household video falling detection method based on privacy protection type GAN feature enhancement
Technical Field
The invention belongs to the field of computer vision and pattern recognition, and particularly relates to a privacy protection type GAN feature enhancement-based old people home video fall detection method.
Background
With the increasing seriousness of China and the problem of household empty nestification, the safety monitoring problem of the solitary old people receives wide attention. The fall is the main reason of the injury and death of the old people, the risk of falling is increased along with the increase of the age, and once the fall occurs, the body and the mind of the old people can be greatly and negatively influenced. Along with the concern of the society on the healthy life of the old, various fall detection modes are developed, and fall detection technologies at the present stage are mainly based on wearable equipment, scene sensors and computer vision. Wearable equipment needs to be worn with a user and is charged at regular time, so that great potential safety hazards exist; the scene sensor technology needs to arrange a large number of various sensors in the range of activities of the old, so that the cost is high and the maintenance is difficult; in contrast, the fall detection technology based on computer vision receives more and more attention of researchers due to the advantages of rich monitoring information, a non-contact monitoring mode, a zero electromagnetic interference monitoring environment and the like. The computer vision fall detection technology is a research direction which is widely concerned at present, but has the defect of personal privacy disclosure, because the computer vision technology is based on one or more cameras to take a picture/a detected object in real time, and uploads a video/picture through a network or other communication modes, in the process, the hidden danger of privacy disclosure exists, such as clothes wearing or toilet process and the like. Even if some technical means are taken to deal with the privacy process, the relevant private information of the home decoration or environmental arrangement, such as gold and silver articles or luxury artwork, is exposed.
CN110942009A provides a method for detecting a fall based on a space-time hybrid convolutional network, which adopts a method for detecting a space-time hybrid convolutional network to divide detection into two parts, namely, coordinate regression and classification, and combines with the space-time hybrid convolutional networks of 2D CNN and 3D CNN to obtain high-quality location features and category features at the same time, the features after coordinate regression are all generated by 2D CNN, the classified features are generated by 3D CNN features, then the location features and the category features are fused, model training is performed to obtain a prediction model, and the prediction model is used to detect video stream input to obtain a prediction result, that is, whether a person falls or not. In the processing process, the original video data acquired by the camera is directly processed, meanwhile, the video monitoring is used as a monitoring means for intelligent processing such as target tracking analysis and the like under the condition of not starting all the day, and the sacrifice cost is the complete disclosure of personal privacy for the application effect of timely detecting the dangerous behaviors of the old. It goes without saying that any elderly with autonomous behavioural capabilities will be reluctant to accept such an unattended monitoring mode, even if the actual risk of a critical behaviour's lack of effective monitoring is clear, as long as there is a possibility of privacy breaches in the visual or cognitive level.
With the continuous and intensive research on GAN networks, since GAN networks (GANs) can achieve a "from none to nothing" breakthrough, i.e., clear target images are obtained with noisy data and are visually indistinguishable from the original target images. It is easy to guess: since GAN can be implemented from none to nothing, it should theoretically be possible to implement a change from "having" to "more". This means that GAN network architecture can be used as a new information migration tool to explore the best balance between quality improvement and privacy protection.
Disclosure of Invention
In order to solve the problem of privacy invasion in the current video monitoring-based old people indoor fall detection method, the privacy protection type GAN feature enhancement-based old people home video fall detection method is provided, and a GAN network is used as a new information migration tool to realize fall detection as accurate as possible.
In order to achieve the purpose, the invention is realized by the following technical scheme:
the invention relates to a privacy protection type GAN feature enhancement-based household video fall detection method, which comprises the following steps:
step 1: based on a public falling detection data set, dividing training data into positive and negative sets according to the characteristics of behaviors in the data set, and manually marking corresponding class labels;
step 2: performing multilayer compressed sensing processing on an original video based on the chaotic pseudorandom Bernoulli measurement matrix to generate a video with a visual privacy protection effect on a visual layer;
and step 3: extracting a foreground moving target by using a low-rank sparse decomposition algorithm based on generalized non-convex robust principal component analysis to obtain the foreground moving target of the video with the visual privacy protection effect;
and 4, step 4: and (3) performing space-time migration compensation on the foreground moving target in the video with the visual privacy protection effect obtained in the step (3) by using a GAN network with an auxiliary classifier, so that the lost features in the video with the visual privacy protection effect are compensated, and a final falling classification result is obtained on the basis.
The GAN network with the auxiliary classifier in the step 4 comprises a generator G network and a discriminator D, both of which use a convolutional neural network, and the processing procedure thereof comprises the following steps:
step 4-1: a training process, wherein the foreground moving target characteristics of the original video in the step 1, corresponding category information and the foreground moving target characteristics compensated in the step 4 are input into a network end of a discriminator D, the foreground moving target characteristics of the video with the visual privacy protection effect after multilayer compressed sensing processing corresponding to the original video are input into a network end of a generator G, and a category information one-to-one corresponding relation is formed between the original video and the video with the visual privacy protection effect;
step 4-2: in the testing process, only the characteristics of the video with the visual privacy protection effect after the multilayer compressed sensing processing in the step 2 are input, the corresponding category information of the characteristics of the video with the visual privacy protection effect is output in the discriminator D, then the loss calculation between the output classification and the real classification is increased, and finally the accuracy of the fall detection is obtained, wherein the loss calculation expression is as follows:
LS＝E[log P(S＝real|Xreal)]+E[logP(S＝fake|Xfake_cs)]
LC＝E[log P(C＝cs|Xreal)]+E[logP(C＝cs|Xfake_cs)]
wherein LSRepresenting the corresponding loss function, L, of the real sampleCLoss function representing correct class of sample，XrealRepresenting original video data, S representing original data class label, C representing class label of data after multi-layer compressed sensing processing, Xfake_csRepresenting the data after compensation by the generator G, P (S | X), P (C | X) represent the probability distribution of the original data and the probability distribution on the class label, respectively.
The invention is further improved in that: the low-rank sparse decomposition algorithm in step 3 is represented as:
Figure BDA0003272900590000041
where g (-) is a non-convex, closed, normal lower semicontinuous function, σi(L) is the i-th singular value, S, of the low-rank matrix LjIs the jth element, τ, γ, of the sparse matrix S>0 is a parameter, M is an element of Rm×nFor the data matrix to be processed, L ∈ Rm×nFor a low rank matrix, S ∈ Rm×nIs a sparse matrix;
the invention is further improved in that: the decomposition process of the low-rank sparse decomposition in the step 3 comprises the following steps:
step 3-1: taking noise generated in the multi-layer compression perception processing process in the step 2 as a sparse matrix, and taking the foreground and the background in the step 1 and the step 2 as low-rank matrices to remove the noise;
step 3-2: and (3) taking the background information of the data set in the step (1) and the video in the step (2) as a low-rank matrix, taking the foreground information of the data set in the step (1) and the video in the step (2) as a sparse matrix, and extracting the foreground moving target by using a low-rank sparse decomposition algorithm based on generalized non-convex robust principal component analysis again.
The invention is further improved in that: the multilayer compressed sensing processing of the step 2 specifically comprises the following steps:
step 2-1: pseudo-random numbers are generated using the metson rotation algorithm, expressed as:
Figure BDA0003272900590000051
wherein ,
Figure BDA0003272900590000052
denotes alphak+1The lowest or rightmost r bits of (a),
Figure BDA0003272900590000053
denotes alphakThe uppermost or leftmost w-r bits (k 0,1,2, …),
Figure BDA0003272900590000054
representing a bitwise exclusive-or operation; and | represents a join operation.
Step 2-2: generating a chaotic pseudorandom sequence: after pseudo random numbers are obtained through a Meisen rotation algorithm, the pseudo random numbers are used as original signals, the generated random numbers are substituted into a chaotic model to generate a chaotic pseudo random sequence, and the corresponding chaotic model is as follows:
Figure BDA0003272900590000055
wherein j represents the number of iterations;
step 2-3: constructing a chaotic pseudorandom Bernoulli measurement matrix: carrying out nonlinear transformation on the chaotic pseudorandom sequence obtained in the step 2-2, and then continuously carrying out sign function mapping on the basis to ensure that the mapped sequence meets Bernoulli distribution, thereby constructing a chaotic pseudorandom Bernoulli measurement matrix
Figure BDA0003272900590000061
And is
Figure BDA0003272900590000062
Expressed as:
Figure BDA0003272900590000063
wherein ,
Figure BDA0003272900590000064
to normalize the coefficients, the matrix elements are arranged in columns as a chaotic pseudorandom sequence { beta }0,β1,β2…,βn}；
The mapping function is represented as:
Figure BDA0003272900590000065
step 2-4: multi-layer compressed sensing sampling: partitioning the pseudorandom Bernoulli measurement matrix constructed in the step 2-3, converting the measurement process into performing inner product operation on the same position block at the moment, and performing multiple partitioning and inner product operations to realize multilayer compressed sensing coding and further realize privacy protection in video monitoring, wherein the calculation process of the multilayer compressed sensing coding is as follows:
Figure BDA0003272900590000066
wherein: y represents video data after multi-layer compressed sensing processing, X represents original video data, and n represents the number of layers of multi-layer compressed sensing coding processing.
The invention has the beneficial effects that:
(1) aiming at the defect of a Gaussian random mechanism in multilayer compressed sensing visual shielding coding, the chaotic pseudo-random Bernoulli mechanism is introduced to be used for generating a measurement matrix in the multilayer compressed sensing processing process, so that the application performance of the multilayer compressed sensing coding is improved while the advantage of visual privacy protection is kept;
(2) aiming at the foreground extraction of video time series data, the generalized non-convex robust principal component analysis algorithm is used for replacing typical low-rank sparse decomposition, so that the foreground extraction and extraction quality is improved, and the robustness of data processing is improved;
(3) aiming at the characteristic enhancement requirement of visual privacy protection state data, the invention takes a GAN network architecture with an auxiliary classifier as a new information migration tool to explore the optimal balance of behavior characterization between quality improvement and privacy protection.
Drawings
FIG. 1 is a schematic flow diagram of the process of the present invention;
FIG. 2 is a diagram of the effect of using a chaotic pseudo-random Bernoulli measurement matrix after sequentially performing 1-4 layers of compressed sensing processing;
FIG. 3 is foreground information of data in a visual privacy preserving state extracted using a low-rank sparse decomposition theory based on generalized non-convex robust principal component analysis.
FIG. 4 is a classifier based on a GAN network with an auxiliary classifier
Detailed Description
In the following description, for purposes of explanation, numerous implementation details are set forth in order to provide a thorough understanding of the embodiments of the invention. It should be understood, however, that these implementation details are not to be interpreted as limiting the invention. That is, in some embodiments of the invention, such implementation details are not necessary. In addition, some conventional structures and components are shown in simplified schematic form in the drawings.
The invention discloses a privacy protection type GAN feature enhancement-based household video fall detection method, which takes the household video of the old as an example, and comprises the following steps:
step 1: input training data set of raw video: based on a public falling detection data set, dividing training data into positive and negative sets according to the characteristics of behaviors in the data set, and manually marking corresponding class labels;
step 2: multilayer compressed sensing processing: performing multilayer compressed sensing processing on an original video based on the chaotic pseudorandom Bernoulli measurement matrix to generate a video with a visual privacy protection effect on a visual layer;
the method comprises the following three steps:
in a first step, pseudo random numbers are generated, which are generated using the merson rotation algorithm and are represented as:
Figure BDA0003272900590000081
wherein ,
Figure BDA0003272900590000082
denotes alphak+1The lowest or rightmost r bits of (a),
Figure BDA0003272900590000083
denotes alphakThe uppermost or leftmost w-r bits (k 0,1,2, …),
Figure BDA0003272900590000084
representing a bitwise exclusive-or operation; and | represents a join operation.
The second step is that: after the pseudo random number is obtained through the Meisen rotation algorithm, the pseudo random number is used as an original signal, if the pseudo random number is directly used for multilayer compressed sensing coding, the original signal reconstruction performance is poor, and therefore the pseudo random number generated in the first step can be substituted into the chaos model represented by the formula (2) to generate a chaos pseudo random sequence.
Figure BDA0003272900590000085
Where j represents the number of iterations.
Carrying out nonlinear transformation on the generated chaos pseudo-random sequence to ensure that the transformed chaos sequence obeys uniform distribution, and then carrying out symbolic function mapping on the chaos sequence, wherein the mapping function is expressed as:
Figure BDA0003272900590000091
the mapped sequences satisfy Bernoulli distribution, thereby constructing the chaotic pseudo-random Bernoulli measurement matrix proposed herein
Figure BDA0003272900590000092
And is
Figure BDA0003272900590000093
Expressed as:
Figure BDA0003272900590000094
wherein ,
Figure BDA0003272900590000095
to normalize the coefficients, the matrix elements are arranged in columns as a chaotic pseudorandom sequence { beta }0,β1,β2…,βn}；
The third step: multi-layer compressed sensing sampling: the original video is subjected to multilayer compressed sensing processing, firstly, a measurement matrix needs to be partitioned, and then the measurement process is converted into inner product operation on blocks at the same position. However, because the risk of privacy disclosure still exists in the single-layer visual privacy protection processing, in order to enhance the security of image information, the single-layer compressive sensing sampling can be expanded to the multi-layer compressive sensing sampling, and the calculation process of the multi-layer compressive sensing coding is as follows:
Figure BDA0003272900590000096
wherein: y represents video data after multi-layer compressed sensing processing, X represents original video data, and n represents the number of layers of multi-layer compressed sensing coding processing.
And step 3: and extracting the foreground moving target by using a low-rank sparse decomposition algorithm based on generalized non-convex robust principal component analysis.
After the video data is subjected to multilayer compressed sensing processing, not only can part of information be lost, but also a large amount of noise can be introduced, so that a detected target is difficult to distinguish, the difficulty of subsequent falling behavior identification is increased, therefore, background information and noise need to be removed, and the foreground information of the video data subjected to compressed sensing processing can be effectively obtained by a low-rank sparse decomposition method. Therefore, the foreground moving target is extracted, so that the accuracy rate of the subsequent fall behavior detection can be improved, the peripheral environment information of the detected object is eliminated, and the visual privacy protection is enhanced.
For a surveillance video sequence containing n frames, each frame can be vectorized into a column vector of M dimensions, whereby the entire video sequence can be seen as a data matrix M of size M × n. The background of a matrix formed by the video sequence belongs to a low-rank matrix, the foreground information corresponds to a sparse matrix, and the background information and the foreground information can be separated through low-rank sparse decomposition. Therefore, the low-rank sparse decomposition algorithm is used for reference and improvement, and the method is used for extracting the foreground moving target of the video with the visual privacy protection effect. The conventional low-rank sparse decomposition model is represented as follows:
min r ank(L)+γ||S||0,s.t.M＝L+S (6)
wherein, M is a data matrix to be processed, L is a low-rank matrix, S is a sparse matrix, rank (L) represents a rank function of the low-rank matrix L, | | S | | y0L representing the sparse matrix S0Norm, which represents the sparsity of matrix S, and γ is a compromise parameter.
Since this is a non-convex model and is difficult to solve directly. The method and the device use the generalized non-convex robust principal component analysis algorithm to extract the sparse foreground component in the video with the visual privacy protection effect, and have strong robustness on noise interference and data processing. The generalized non-convex robust principal component analysis adopts a non-convex generalized kernel norm to approximate a rank function, and most importantly, in order to better approximate sparsity, non-convex generalized norm approximation is introduced to replace l representing sparse prior0Norm, ratio l1The norm is more approximate to the sparsity of the matrix, and the low-rank sparse decomposition model is as follows:
Figure BDA0003272900590000101
where g (-) is a non-convex, closed, normal lower semicontinuous function, σi(L) is the i-th singular value, S, of the low-rank matrix LjIs the jth element, τ, γ, of the sparse matrix S>0 is a parameter, M is an element of Rm×nFor the data matrix to be processed, L ∈Rm×nFor a low rank matrix, S ∈ Rm×nIs a sparse matrix;
the video with the visual privacy protection effect obtained through the multi-layer compressed sensing process includes not only foreground information and background information, but also a large amount of noise introduced in the compression process, so that the matrix M of formula (7) is:
M(n)＝L(n)+(E(n)+S(n)) (8)
the decomposition process comprises the following steps:
the first step is as follows: taking noise generated in the multi-layer compression perception processing process in the step 2 as a sparse matrix, and taking the foreground and the background in the step 1 and the step 2 as low-rank matrices to remove the noise;
the second step is that: and (3) taking the background information of the data set in the step (1) and the video in the step (2) as a low-rank matrix, taking the foreground information of the data set in the step (1) and the video in the step (2) as a sparse matrix, and extracting the foreground moving target by using a low-rank sparse decomposition algorithm based on generalized non-convex robust principal component analysis again.
And 4, step 4: and (3) performing space-time migration compensation on the foreground moving target in the video with the visual privacy protection effect obtained in the step (3) by using a GAN network with an auxiliary classifier, so that the lost features in the video with the visual privacy protection effect are compensated, and a final falling classification result is obtained on the basis.
The GAN network with the auxiliary classifier comprises a generator G network and a discriminator D, wherein the generator G network and the discriminator D both use a convolutional neural network, the input of the G network is noise z and a corresponding class label c, the class label c is artificially calibrated, and the generator generates a sample by using the noise z and the class label c, so that the generated sample contains label information. The discriminator D gives the probability distribution of true and false samples and the categories to which the samples belong, and the loss functions of the GAN network with the auxiliary classifier are shown in the formulas (9) and (10).
LS＝E[log P(S＝real|Xreal)]+E[log P(S＝fake|Xfake_cs)] (9)
LC＝E[log P(C＝c|Xreal)]+E[log P(C＝c|Xfake_cs)] (10)
wherein LSRepresenting the corresponding loss function, L, of the real sampleCLoss function, X, representing samples corresponding to the correct classrealRepresenting original video data, S representing a class label of the original data, C representing a class label of the data after multi-layer compressed sensing processing, Xfake_csRepresenting the data after compensation by the generator G, P (S | X), P (C | X) represent the probability distribution of the original data and the probability distribution on the class label, respectively. The training goal of the discriminator is to maximize Ls+LcThe training goal of the generator is to maximize Ls-Lc。
In the processing process of the GAN network with the auxiliary classifier, the foreground information and the label of the visual privacy state are input into the generator, the information with the visual privacy protection effect contains state characteristics, the guidance of the label information is assisted, the data generated by the generator is purposefully gathered to the original video frame data set with the same label, in the process, the original data is used for supplementing part of information missing from the visual privacy state, the compensation of the information with the visual privacy protection effect is achieved, then the original data and the data with the visual privacy protection effect are sent into the discriminator D together, and the true and false probabilities and the category probabilities of the information are output. Obviously, in the early stage of iteration, the difference from the data characteristic is large, and the corresponding loss function is large. The G network and the D network of the GAN network with the auxiliary classifier are both of convolutional neural network structures, loss functions are propagated backwards to the D network and the G network according to a chain derivation rule, parameters such as the weight of the D network and the G network are continuously corrected and updated, the G network can obtain better information compensation from the loss functions, after multiple iterations, approximation is output, and meanwhile, the D network outputs more accurate classification information. The loss functions of the improved GAN network with the auxiliary classifier are shown in equations (11), (12).
LS＝E[log P(S＝real|Xreal)]+E[log P(S＝fake|Xfake_cs)] (11)
LC＝E[log P(C＝cs|Xreal)]+E[log P(C＝cs|Xfake_cs)] (12)
The discriminator D of the GAN network with the auxiliary classifier can output corresponding class information, and then increase the loss calculation of the output classification and the real classification, so that the generated simulation data corresponds to the class to which the simulation data belongs one to one. And considering the time sequence information among the video frames, fusing the classification probability of each frame in the video sequence with the pre-trained weight by pre-training the weight of the video, thereby obtaining the classification result of the video sequence.
The invention can realize the falling detection of the old people at home, can meet the protection of the privacy of the old people and the family, and has higher practical application value.
The above description is only an embodiment of the present invention, and is not intended to limit the present invention. Various modifications and alterations to this invention will become apparent to those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention should be included in the scope of the claims of the present invention.

Claims (8)

1. A household video fall detection method based on privacy protection type GAN feature enhancement is characterized by comprising the following steps: the household video fall detection method comprises the following steps:
step 1: input training data set of raw video: based on a public falling detection data set, dividing training data into positive and negative sets according to the characteristics of behaviors in the data set, and manually marking corresponding class labels;
step 2: multilayer compressed sensing processing: performing multilayer compressed sensing processing on an original video based on the chaotic pseudorandom Bernoulli measurement matrix to generate a video with a visual privacy protection effect on a visual layer;
and step 3: and (3) extraction of foreground moving targets: extracting a foreground moving target by using a low-rank sparse decomposition algorithm based on generalized non-convex robust principal component analysis to obtain the foreground moving target of the video with the visual privacy protection effect;
and 4, step 4: obtaining a fall classification result: and (3) performing space-time migration compensation on the foreground moving target in the video with the visual privacy protection effect obtained in the step (3) by using a GAN network with an auxiliary classifier, so that the lost features in the video with the visual privacy protection effect are compensated, and a final falling classification result is obtained on the basis.
2. The privacy protection type GAN feature enhancement based home video fall detection method as claimed in claim 1, wherein: the low-rank sparse decomposition algorithm in step 3 is represented as:
Figure FDA0003272900580000011
where g (-) is a non-convex, closed, normal lower semicontinuous function, σi(L) is the i-th singular value, S, of the low-rank matrix LjIs the jth element, τ, γ, of the sparse matrix S>0 is a parameter, M is an element of Rm×nFor the data matrix to be processed, L ∈ Rm×nFor a low rank matrix, S ∈ Rm×nIs a sparse matrix;
the decomposition process comprises the following steps:
step 3-1: taking noise generated in the multi-layer compression perception processing process in the step 2 as a sparse matrix, and taking the foreground and the background in the step 1 and the step 2 as low-rank matrices to remove the noise;
step 3-2: and (3) taking the background information of the data set in the step (1) and the video in the step (2) as a low-rank matrix, taking the foreground information of the data set in the step (1) and the video in the step (2) as a sparse matrix, and extracting the foreground moving target by using a low-rank sparse decomposition algorithm based on generalized non-convex robust principal component analysis again.
3. The privacy protection type GAN feature enhancement based home video fall detection method as claimed in claim 1, wherein: the GAN network with the auxiliary classifier in the step 4 comprises a generator G network and a discriminator D, wherein the generator G network and the discriminator D both use a convolutional neural network.
4. The privacy protection type GAN feature enhancement based home video fall detection method as claimed in claim 3, wherein: the processing procedure of the GAN network with the auxiliary classifier in the step 4 comprises the following steps:
step 4-1: a training process, wherein the foreground moving target characteristics of the original video in the step 1, corresponding category information and the foreground moving target characteristics compensated in the step 4 are input into a network end of a discriminator D, the foreground moving target characteristics of the video with the visual privacy protection effect after multilayer compressed sensing processing corresponding to the original video are input into a network end of a generator G, and a category information one-to-one corresponding relation is formed between the original video and the video with the visual privacy protection effect;
step 4-2: in the testing process, only the characteristics of the video with the visual privacy protection effect after the multilayer compressed sensing processing in the step 2 are input, the corresponding category information of the characteristics of the video with the visual privacy protection effect is output in the discriminator D, then the loss calculation between the output classification and the real classification is increased, and finally the accuracy of the fall detection is obtained.
5. The privacy protection type GAN feature enhancement based home video fall detection method as claimed in claim 4, wherein: the loss calculation in step 4-2 is expressed as:
LS＝E[log P(S＝real|Xreal)]+E[log P(S＝fake|Xfake_cs)]
LC＝E[log P(C＝cs|Xreal)]+E[log P(C＝cs|Xfake_cs)]
wherein LSRepresenting the corresponding loss function, L, of the real sampleCLoss function, X, representing samples corresponding to the correct classrealRepresenting original video data, S representing a class label of the original data, C representing a class label of the data after multi-layer compressed sensing processing, Xfake_csRepresenting the data after compensation by the generator G, P (S | X), P (C | X) represent the probability distribution of the original data and the probability distribution on the class label, respectively.
6. The privacy protection type GAN feature enhancement based home video fall detection method as claimed in claim 1, wherein: the multi-layer compressed sensing processing of step 2 specifically includes the following steps:
step 2-1: pseudo-random numbers are generated using the metson rotation algorithm, expressed as:
Figure FDA0003272900580000031
wherein ,
Figure FDA0003272900580000032
denotes alphak+1The lowest or rightmost r bits of (a),
Figure FDA0003272900580000033
denotes alphakThe uppermost or leftmost w-r position (k 0,1, 2.),
Figure FDA0003272900580000034
representing a bitwise exclusive-or operation; and | represents a join operation.
Step 2-2: generating a chaotic pseudorandom sequence: after pseudo random numbers are obtained through a Meisen rotation algorithm, the pseudo random numbers are used as original signals, the generated random numbers are substituted into a chaotic model to generate a chaotic pseudo random sequence, and the corresponding chaotic model is as follows:
Figure FDA0003272900580000041
wherein j represents the number of iterations;
step 2-3: constructing a chaotic pseudorandom Bernoulli measurement matrix: carrying out nonlinear transformation on the chaotic pseudorandom sequence obtained in the step 2-2, and then continuously carrying out sign function mapping on the basis to ensure that the mapped sequence meets Bernoulli distribution, thereby constructing a chaotic pseudorandom Bernoulli measurement matrix
Figure FDA0003272900580000042
And is
Figure FDA0003272900580000043
Expressed as:
Figure FDA0003272900580000044
wherein ,
Figure FDA0003272900580000045
to normalize the coefficients, the matrix elements are arranged in columns to chaos pseudo-random sequences { beta }0，β1，β2...，βn}；
Step 2-4: multi-layer compressed sensing sampling: pseudo-random Bernoulli measurement matrix constructed by steps 2-3
Figure FDA0003272900580000046
And partitioning is carried out, the measurement process is converted into the inner product operation of the same position block, and multi-layer compressed sensing coding is realized through multiple partitioning and inner product operations, so that the privacy protection in video monitoring is realized.
7. The privacy protection type GAN feature enhancement based home video fall detection method as claimed in claim 6, wherein: the mapping function in the sign function mapping in step 2-3 is represented as:
Figure FDA0003272900580000047
8. the privacy protection type GAN feature enhancement based home video fall detection method as claimed in claim 6, wherein: the calculation process of the multilayer compressed sensing coding in the step 2-4 is as follows:
Figure FDA0003272900580000051
wherein: y represents video data after multi-layer compressed sensing processing, X represents original video data, and n represents the number of layers of multi-layer compressed sensing coding processing.
CN202111113708.4A 2021-09-22 2021-09-22 Household video fall detection method based on privacy protection GAN feature enhancement Active CN113807275B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202111113708.4A CN113807275B (en) 2021-09-22 2021-09-22 Household video fall detection method based on privacy protection GAN feature enhancement

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202111113708.4A CN113807275B (en) 2021-09-22 2021-09-22 Household video fall detection method based on privacy protection GAN feature enhancement

Publications (2)

Publication Number Publication Date
CN113807275A true CN113807275A (en) 2021-12-17
CN113807275B CN113807275B (en) 2023-10-24

Family

ID=78940130

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202111113708.4A Active CN113807275B (en) 2021-09-22 2021-09-22 Household video fall detection method based on privacy protection GAN feature enhancement

Country Status (1)

Country Link
CN (1) CN113807275B (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114998805A (en) * 2022-06-14 2022-09-02 南京邮电大学 Visual privacy protection home health behavior video homomorphic monitoring method based on DCGAN temporal-spatial information migration compensation
CN115238827A (en) * 2022-09-16 2022-10-25 支付宝(杭州)信息技术有限公司 Privacy-protecting sample detection system training method and device

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110598606A (en) * 2019-09-02 2019-12-20 南京邮电大学 Indoor falling behavior detection method with visual privacy protection advantage
CN111310647A (en) * 2020-02-12 2020-06-19 北京云住养科技有限公司 Generation method and device for automatic identification falling model
US20210052170A1 (en) * 2019-08-20 2021-02-25 Robin H. Stewart Systems and methods for dynamic biometric detection and response
CN113158882A (en) * 2021-04-19 2021-07-23 南京邮电大学 Bionic compound eye privacy protection intelligent binary modeling method for home video monitoring

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20210052170A1 (en) * 2019-08-20 2021-02-25 Robin H. Stewart Systems and methods for dynamic biometric detection and response
CN110598606A (en) * 2019-09-02 2019-12-20 南京邮电大学 Indoor falling behavior detection method with visual privacy protection advantage
CN111310647A (en) * 2020-02-12 2020-06-19 北京云住养科技有限公司 Generation method and device for automatic identification falling model
CN113158882A (en) * 2021-04-19 2021-07-23 南京邮电大学 Bionic compound eye privacy protection intelligent binary modeling method for home video monitoring

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114998805A (en) * 2022-06-14 2022-09-02 南京邮电大学 Visual privacy protection home health behavior video homomorphic monitoring method based on DCGAN temporal-spatial information migration compensation
CN115238827A (en) * 2022-09-16 2022-10-25 支付宝(杭州)信息技术有限公司 Privacy-protecting sample detection system training method and device
CN115238827B (en) * 2022-09-16 2022-11-25 支付宝(杭州)信息技术有限公司 Privacy-protecting sample detection system training method and device

Also Published As

Publication number Publication date
CN113807275B (en) 2023-10-24

Similar Documents

Publication Publication Date Title
CN108710868B (en) Human body key point detection system and method based on complex scene
Jiang et al. Semisupervised spectral learning with generative adversarial network for hyperspectral anomaly detection
CN109543602B (en) Pedestrian re-identification method based on multi-view image feature decomposition
CN113807275A (en) Household video falling detection method based on privacy protection type GAN feature enhancement
CN112163564B (en) Tumble prejudging method based on human body key point behavior identification and LSTM (least Square TM)
CN108764059A (en) A kind of Human bodys' response method and system based on neural network
Tong et al. Multi-view gait recognition based on a spatial-temporal deep neural network
CN110738154A (en) pedestrian falling detection method based on human body posture estimation
CN110598606B (en) Indoor falling behavior detection method with visual privacy protection advantage
CN110991274B (en) Pedestrian tumbling detection method based on Gaussian mixture model and neural network
CN111160294B (en) Gait recognition method based on graph convolution network
CN113139489B (en) Crowd counting method and system based on background extraction and multi-scale fusion network
CN109886102B (en) Fall-down behavior time-space domain detection method based on depth image
CN113111767A (en) Fall detection method based on deep learning 3D posture assessment
CN107967941A (en) A kind of unmanned plane health monitoring method and system based on intelligent vision reconstruct
CN113469050A (en) Flame detection method based on image subdivision classification
CN112926522B (en) Behavior recognition method based on skeleton gesture and space-time diagram convolution network
CN107863153A (en) A kind of human health characteristic modeling measuring method and platform based on intelligent big data
Bora et al. Human suspicious activity detection system using CNN model for video surveillance
CN113516232B (en) Self-attention mechanism-based wall-penetrating radar human body posture reconstruction method
Islam et al. A deep Spatio-temporal network for vision-based sexual harassment detection
Zhou End-to-end video violence detection with transformer
CN116402811B (en) Fighting behavior identification method and electronic equipment
Lee A study on abnormal behavior detection in CCTV images through the supervised learning model of deep learning
Konstantinidis et al. Skeleton-based action recognition based on deep learning and Grassmannian pyramids

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant