CN111310668B - Gait recognition method based on skeleton information - Google Patents
Gait recognition method based on skeleton information Download PDFInfo
- Publication number
- CN111310668B CN111310668B CN202010100136.5A CN202010100136A CN111310668B CN 111310668 B CN111310668 B CN 111310668B CN 202010100136 A CN202010100136 A CN 202010100136A CN 111310668 B CN111310668 B CN 111310668B
- Authority
- CN
- China
- Prior art keywords
- gait
- node
- sample
- sequence
- dimension
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/103—Static body considered as a whole, e.g. static pedestrian or occupant recognition
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
- G06F18/2415—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on parametric or probabilistic models, e.g. based on likelihood ratio or false acceptance rate versus a false rejection rate
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/047—Probabilistic or stochastic networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/084—Backpropagation, e.g. using gradient descent
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Physics & Mathematics (AREA)
- Evolutionary Computation (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Engineering & Computer Science (AREA)
- Computing Systems (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computational Linguistics (AREA)
- Biophysics (AREA)
- Biomedical Technology (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Probability & Statistics with Applications (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Biology (AREA)
- Human Computer Interaction (AREA)
- Multimedia (AREA)
- Image Analysis (AREA)
Abstract
The invention provides a gait recognition method based on skeleton information, which comprises the following steps: acquiring gait video sequences; performing gesture estimation on the gait video sequence by using OpenPose to obtain a gait key point sequence; constructing a space-time skeleton sequence; inputting the adjacency matrix and gait key point sequences into a multi-scale space-time diagram convolution network for training; after training, testing by using a trained model, extracting gait characteristics, and performing characteristic matching. The invention mainly adopts a human body key point form, introduces a graph convolution neural network aiming at a graph structure, improves a connection mode and a division strategy, combines cross entropy loss and contrast loss by adopting a twin mechanism, fuses shallow layer characteristics, middle layer characteristics and deep layer characteristics of the network, and improves the robustness of gait recognition to a certain extent.
Description
Technical Field
The invention relates to the technical field of pattern recognition, in particular to a gait recognition method based on skeleton information.
Background
Gait recognition is an emerging biological feature recognition technology, aims to identify people by walking gestures in a video sequence, and has the advantages of non-contact, long distance and difficult disguising compared with other biological recognition technologies. The method has more advantages in the fields of security protection and intelligent monitoring, and the gait recognition application still has some problems in the actual complex environment so far.
In recent years, scientific research institutions at home and abroad pay more attention to gait recognition technology. The prior art is mainly divided into the following two types:
1. model-based methods. The method mainly comprises the steps of cutting a body into a plurality of blocks or acquiring body joints, and fitting through the joints or partial motion tracks of body motion. Such methods rely primarily on the static characteristics of each block of the body and the motion trajectories of the joints for modeling, including two-dimensional models and three-dimensional models.
2. Non-model based methods. The method is mainly realized by collecting the appearance of human walking and the gait characteristics such as characteristic parameters, and the like, and a walking gait model is not required to be reconstructed. There are roughly three subjects: gait energy diagram, contour diagram sequence and human key point sequence.
In recent years, with great progress of deep learning technology in various fields of computer vision, a large number of gait recognition methods based on deep learning are emerging. For example: and describing a gait sequence by adopting a gait energy diagram, and training a matching model by using a deep convolutional neural network so as to match the identity of a person. The method has the advantages that when the variation range of the walking view angle of a human body is large, the extracted multi-view angle gait feature representation capability is insufficient, and meanwhile, the robustness to clothes, carried objects and the like is low. Furthermore, the gait feature representation and matching are performed using the pose information. The human body key point coordinates are obtained from the gait video sequence by using an attitude estimation algorithm, the gait key point sequence is trained by using a convolutional neural network and a long-short-time memory network, and meanwhile, manual features are introduced to perform gait recognition. However, the existing gait recognition technology still has the following defects:
1. gait recognition is performed through a gait contour map or a gait energy map, requirements on contour quality and background are high, and the contour map is often extracted incompletely due to the fact that the contour map is greatly influenced by illumination conditions and complex background.
2. The covariates can not be separated from the human body due to the influence of covariates such as clothes, carrying objects and the like, so that the recognition accuracy is reduced.
Disclosure of Invention
According to the technical problem, a gait recognition method based on skeleton information is provided. The invention mainly adopts a human body key point form, introduces a graph convolution neural network aiming at a graph structure, improves a connection mode and a division strategy, combines cross entropy loss and contrast loss by adopting a twin mechanism, fuses shallow layer characteristics, middle layer characteristics and deep layer characteristics of the network, and improves the robustness of gait recognition to a certain extent.
The invention adopts the following technical means:
a gait recognition method based on skeleton information comprises the following steps:
s1, acquiring gait video sequences;
s2, performing gesture estimation on the gait video sequence by using OpenPose to obtain a gait key point sequence;
s3, constructing a space-time skeleton sequence;
s4, inputting the adjacency matrix and gait key point sequences into a multi-scale space-time diagram convolution network for training;
and S5, after training, testing by using a trained model, extracting gait characteristics, and performing characteristic matching.
Further, the step S3 specifically includes:
s31, carrying out natural connection on human bodies in space on the gait key point sequence; at the same time, symmetry is introduced to connect symmetrical joint points (only the symmetrical key points of legs are connected, because symmetry between arms is lost under the condition of carrying objects); connecting the same key points among frames in time;
s32, defining a sampling function, a node v ti Is defined as:
B(v ti )＝{v tj |d(v tj ,v ti )≤D},
wherein B is node v ti Is a neighbor set of (1); v represents a node; d represents a distance; d (v) tj ,v ti ) Representing the shortest path between two nodes, typically taking d=1; thus, the sampling function is defined as: p (v) tj ,v ti )＝v tj ；
S33, selecting a partitioning strategy, namely dividing a neighborhood set into four subsets, namely the node is a first subset, wherein in an asymmetric node, the node is a second subset which is closer to the gravity center than the node, the node is a third subset which is farther from the gravity center than the node, and the symmetric node is defined as a fourth subset, namely:
s34, defining a weight function, dividing a neighborhood set into four subsets, wherein each subset is provided with a digital label, and a mapping function l is adopted ti Mapping each node to its subset labels, the mapping function is defined as: b (v) ti ) -0, … …, K-1, k=4; the weight function is defined as: w (v) ti ,v tj )＝w’(l ti (v tj ))；
S35, extending the space diagram convolution to a space time domain, and defining a space time neighborhood set as: b (v) ti )＝{v qj |d(v tj ,v ti ) K is less than or equal to K, q-t is less than or equal to Γ, B is node v ti Is a neighbor set of (1); v represents a node;k represents a distance; Γ controls the range of the graph that is included in the neighborhood, i.e., the temporal convolution kernel.
Further, the process of training the multi-scale space-time diagram convolutional neural network is specifically as follows:
s41, after selecting samples, randomly selecting one sample from all samples with the same ID as the selected samples as a positive sample, and randomly selecting one sample from all samples with different IDs as a negative sample;
s42, adopting a twin mechanism, inputting the selected sample into the branch 1, and sequentially inputting the positive sample and the negative sample into the branch 2, wherein the branch 1 and the branch 2 share parameters;
s43, classifying the selected sample characteristics in the branch 1 by adopting SoftMax and a cross entropy loss function;
s44, comparing the characteristics of the selected sample and the positive sample with the characteristics of the selected sample and the negative sample by adopting a comparison loss function; samples from the same ID, label 1, samples from different IDs, label 0.
S45, adding two parts of losses, wherein the total loss is as follows:
Loss＝Lid+0.5*[Lc(sample,pos,1)+Lc(sample,neg,0)]，
and (3) performing back propagation to update the network, wherein Lid is cross entropy loss, lc is contrast loss.
Further, in the step S4, when training the multiscale space-time diagram convolutional neural network, the following setting process is further included:
step 1, inputting a gait sequence, wherein the dimension is [3,100,18], 3 is that 3 channels are provided for inputting key point characteristics, X and Y coordinates and confidence coefficient C are respectively provided, 100 is that the time dimension is 100 frames, and 18 is that 18 key points are provided for each frame;
step 2, outputting 64 channels from the first three layers, wherein the convolution kernel size is (9, 3), 9 is the time convolution kernel size, 3 is the space convolution kernel size, and the output dimension is [64,100,18];
step 3, outputting 128 channels by three layers in the middle, wherein the convolution kernel size is (9, 3), the output dimension is [128,50,18], and the convolution step length of the time dimension is 2 in the fourth layer;
step 4, outputting 256 channels from the rear three layers, wherein the convolution kernel size is (9, 3), the output dimension is [256,25,18], and the convolution step length of the time dimension is 2 in the seventh layer;
step 5, carrying out global average pooling, wherein after pooling, the characteristic dimension is changed into 256 dimensions;
step 6, carrying out dimension exchange on the characteristics [64,100,18] output by the first layer, and carrying out average pooling to obtain 18-dimensional characteristics;
step 7, carrying out dimension exchange on the output characteristics [128,50,18] of the fifth layer, and carrying out average pooling to obtain 18-dimensional characteristics;
step 8, expressing gait characteristics in a mode of fusing shallow layer characteristics and deep layer characteristics, and splicing 18-dimensional characteristics of a first layer, 18-dimensional characteristics of a fifth layer and 256-dimensional characteristics of a last layer to obtain 292-dimensional characteristics;
and 9, classifying 292-dimensional features by adopting a softMax classifier.
The invention also provides a testing method of the gait recognition method based on the skeleton information, which comprises the following steps:
step I: inputting a gait key point sequence to be tested;
step II: extracting gait characteristics by using a trained network, and carrying out two-norm normalization on the characteristics;
step III: performing the operations of the step I and the step II on the samples in the sample library, and using the feature vectors to represent the gait sequence of the pedestrian to be searched and the gait sequence of the pedestrian in the search library;
step IV: calculating the distance between the gait sequence of the pedestrian to be searched and the gait sequence of the pedestrian in the search library, namely calculating the distance between the characteristics of the pedestrian to be searched and all the gait sequences of the pedestrian in the search library aiming at one gait sequence of the pedestrian to be searched;
step V: and sorting the similarity of the samples in the search library from small to large according to the calculated distance, wherein the earlier the sample is, the more likely the sample is consistent with the ID of the pedestrian to be searched.
Compared with the prior art, the invention has the following advantages:
1. the gait recognition method provided by the invention adopts a key point sequence to represent the gait aiming at the covariate influence of backpacks, clothes and the like, solves the problem that the gait characteristics represented by the profile diagram and the energy diagram are lower in robustness under the covariate condition, and improves the recognition accuracy under the covariate influence.
2. According to the gait recognition method provided by the invention, aiming at the characteristic symmetry of gait, symmetry is introduced when a space-time skeleton sequence is constructed, and joint point information of human leg symmetry is added into an adjacent matrix, so that the association degree of related nodes is enhanced, meanwhile, noise caused by inaccurate joint estimation is reduced, and the recognition accuracy is improved.
3. Because the deep convolutional neural network extracts high-level features, the high-level semantic information is singly represented, and static features cannot be described, the expression form of gait features is enriched by adopting a multi-scale mode of fusing shallow features, middle features and deep features, and the recognition accuracy is improved.
For the reasons, the method can be widely popularized in the fields of pattern recognition and the like.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings that are required in the embodiments or the description of the prior art will be briefly described, and it is obvious that the drawings in the following description are some embodiments of the present invention, and other drawings may be obtained according to the drawings without inventive effort to a person skilled in the art.
FIG. 1 is a flow chart of the method of the present invention.
Fig. 2 is a schematic diagram of a specific arrangement of a multi-scale space-time convolutional neural network according to the present invention.
FIG. 3 is a schematic diagram of the test method of the present invention.
Detailed Description
In order that those skilled in the art will better understand the present invention, a technical solution in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in which it is apparent that the described embodiments are only some embodiments of the present invention, not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the present invention without making any inventive effort, shall fall within the scope of the present invention.
It should be noted that the terms "first," "second," and the like in the description and the claims of the present invention and the above figures are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged where appropriate such that the embodiments of the invention described herein may be implemented in sequences other than those illustrated or otherwise described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
As shown in fig. 1, the invention provides a gait recognition method based on skeleton information, which comprises the following steps:
s1, acquiring gait video sequences;
s2, performing gesture estimation on the gait video sequence by using OpenPose to obtain a gait key point sequence;
s3, constructing a space-time skeleton sequence;
further, as a preferred embodiment of the present invention, the step S3 specifically includes:
s31, carrying out natural connection on human bodies in space on the gait key point sequence; because the walking gesture of the person has the characteristic of symmetry, symmetry is introduced at the same time to connect symmetrical joint points (only the symmetrical key points of the legs are connected, because the symmetry between arms is lost under the condition of carrying objects); connecting the same key points among frames in time;
s32, defining samplesFunction, a node v ti Is defined as:
B(v ti )＝{v tj |d(v tj ,v ti )≤D},
wherein B is node v ti Is a neighbor set of (1); v represents a node; d represents a distance; d (v) tj ,v ti ) Representing the shortest path between two nodes, typically taking d=1; thus, the sampling function is defined as: p (v) tj ,v ti )＝v tj ；
S33, selecting a partitioning strategy, namely dividing a neighborhood set into four subsets, namely the node is a first subset, wherein in an asymmetric node, the node is a second subset which is closer to the gravity center than the node, the node is a third subset which is farther from the gravity center than the node, and the symmetric node is defined as a fourth subset, namely:
s34, defining a weight function, dividing a neighborhood set into four subsets, wherein each subset is provided with a digital label, and a mapping function l is adopted ti Mapping each node to its subset labels, the mapping function is defined as: b (v) ti ) -0, … …, K-1, k=4; the weight function is defined as: w (v) ti ,v tj )＝w’(l ti (v tj ))；
S35, extending the space diagram convolution to a space time domain, and defining a space time neighborhood set as: b (v) ti )＝{v qj |d(v tj ,v ti ) K is less than or equal to K, q-t is less than or equal to Γ, B is node v ti Is a neighbor set of (1); v represents a node; k represents a distance; Γ controls the range of the graph that is included in the neighborhood, i.e., the temporal convolution kernel.
S4, inputting the adjacency matrix and gait key point sequences into a multi-scale space-time diagram convolution network for training;
further, as a preferred embodiment of the present invention, as shown in fig. 2, the step S4 further includes the following setup procedure before training the multiscale space-time diagram convolutional neural network:
step 1, inputting a gait sequence, wherein the dimension is [3,100,18], 3 is that 3 channels are provided for inputting key point characteristics, X and Y coordinates and confidence coefficient C are respectively provided, 100 is that the time dimension is 100 frames, and 18 is that 18 key points are provided for each frame;
step 2, outputting 64 channels from the first three layers, wherein the convolution kernel size is (9, 3), 9 is the time convolution kernel size, 3 is the space convolution kernel size, and the output dimension is [64,100,18];
step 3, outputting 128 channels by three layers in the middle, wherein the convolution kernel size is (9, 3), the output dimension is [128,50,18], and the convolution step length of the time dimension is 2 in the fourth layer;
step 4, outputting 256 channels from the rear three layers, wherein the convolution kernel size is (9, 3), the output dimension is [256,25,18], and the convolution step length of the time dimension is 2 in the seventh layer;
step 5, carrying out global average pooling, wherein after pooling, the characteristic dimension is changed into 256 dimensions;
step 6, carrying out dimension exchange on the characteristics [64,100,18] output by the first layer, and carrying out average pooling to obtain 18-dimensional characteristics;
step 7, carrying out dimension exchange on the output characteristics [128,50,18] of the fifth layer, and carrying out average pooling to obtain 18-dimensional characteristics;
step 8, because the deep convolutional neural network extracts high-level features, the high-level semantic information is singly represented, and static features cannot be described, gait features are represented by adopting a mode of fusing shallow features and deep features, and 18-dimensional features of a first layer, 18-dimensional features of a fifth layer and 256-dimensional features of a last layer are spliced to become 292-dimensional features;
and 9, classifying 292-dimensional features by adopting a softMax classifier.
In this embodiment, the CASIA-B dataset is used, NM: normal walking conditions, BG: carrier conditions, CL: conditions of wearing the overcoat. The following table shows:
the process of training the multi-scale space-time diagram convolutional neural network is specifically as follows:
in the training stage, the aim is to train the network to extract the characteristics representing pedestrians, so the network trains in a classified mode, and the specific steps are as follows:
s41, after selecting samples, randomly selecting one sample from all samples with the same ID as the selected samples as a positive sample, and randomly selecting one sample from all samples with different IDs as a negative sample;
s42, adopting a twin mechanism, inputting the selected sample into the branch 1, and sequentially inputting the positive sample and the negative sample into the branch 2, wherein the branch 1 and the branch 2 share parameters;
s43, classifying the selected sample characteristics in the branch 1 by adopting SoftMax and a cross entropy loss function;
s44, comparing the characteristics of the selected sample and the positive sample with the characteristics of the selected sample and the negative sample by adopting a comparison loss function; samples from the same ID are labeled 1, samples from different IDs are labeled 0.
S45, adding two parts of losses, wherein the total loss is as follows:
Loss＝Lid+0.5*[Lc(sample,pos,1)+Lc(sample,neg,0)]，
and (3) performing back propagation to update the network, wherein Lid is cross entropy loss, lc is contrast loss.
And S5, after training, testing by using a trained model, extracting gait characteristics, and performing characteristic matching.
As shown in fig. 3, the invention further provides a testing method of the gait recognition method based on the skeleton information, which comprises the following steps:
step I: inputting a gait key point sequence to be tested;
step II: extracting gait characteristics by using a trained network, and carrying out two-norm normalization on the characteristics;
step III: performing the operations of the step I and the step II on the samples in the sample library, and using the feature vectors to represent the gait sequence of the pedestrian to be searched and the gait sequence of the pedestrian in the search library;
step IV: calculating the distance between the gait sequence of the pedestrian to be searched and the gait sequence of the pedestrian in the search library, namely calculating the distance between the characteristics of the pedestrian to be searched and all the gait sequences of the pedestrian in the search library aiming at one gait sequence of the pedestrian to be searched;
step V: and sorting the similarity of the samples in the search library from small to large according to the calculated distance, wherein the earlier the sample is, the more likely the sample is consistent with the ID of the pedestrian to be searched.
Finally, it should be noted that: the above embodiments are only for illustrating the technical solution of the present invention, and are not limiting; although the invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some or all of the technical features thereof can be replaced with equivalents; such modifications and substitutions do not depart from the spirit of the technical solutions according to the embodiments of the present invention.
Claims (2)
1. The gait recognition method based on the skeleton information is characterized by comprising the following steps of:
s1, acquiring gait video sequences;
s2, performing gesture estimation on the gait video sequence by using OpenPose to obtain a gait key point sequence;
s3, constructing a space-time skeleton sequence; the step S3 specifically comprises the following steps:
s31, carrying out natural connection on human bodies in space on the gait key point sequence; meanwhile, symmetry is introduced to connect symmetrical joint points, and only leg symmetrical key points are connected, because symmetry between arms is lost under the condition of carrying objects; connecting the same key points among frames in time;
s32, defining a sampling function, a node v ti Is defined as:
B(v ti )＝{v tj |d(v tj ,v ti )≤D},
wherein the method comprises the steps ofB is node v ti Is a neighbor set of (1); v represents a node; d represents a distance; d (v) tj ,v ti ) Representing the shortest path between two nodes, typically taking d=1; thus, the sampling function is defined as: p (v) tj ,v ti )＝v tj ；
S33, selecting a partitioning strategy, namely dividing a neighborhood set into four subsets, namely the node is a first subset, wherein in an asymmetric node, the node is a second subset which is closer to the gravity center than the node, the node is a third subset which is farther from the gravity center than the node, and the symmetric node is defined as a fourth subset, namely:
s34, defining a weight function, dividing a neighborhood set into four subsets, wherein each subset is provided with a digital label, and a mapping function l is adopted ti Mapping each node to its subset labels, the mapping function is defined as: b (v) ti ) -0, … …, K-1, k=4; the weight function is defined as: w (v) ti ,v tj )＝w’(l ti (v tj ))；
S35, extending the space diagram convolution to a space time domain, and defining a space time neighborhood set as: b (v) ti )＝{v qj |d(v tj ,v ti ) K is less than or equal to K, q-t is less than or equal to Γ, B is node v ti Is a neighbor set of (1); v represents a node; k represents a distance; Γ controls the range of the graph included in the neighborhood, i.e., the temporal convolution kernel;
s4, inputting the adjacency matrix and gait key point sequences into a multi-scale space-time diagram convolution network for training; the process of training the multi-scale space-time diagram convolutional neural network is specifically as follows:
s41, after selecting samples, randomly selecting one sample from all samples with the same ID as the selected samples as a positive sample, and randomly selecting one sample from all samples with different IDs as a negative sample;
s42, adopting a twin mechanism, inputting the selected sample into the branch 1, and sequentially inputting the positive sample and the negative sample into the branch 2, wherein the branch 1 and the branch 2 share parameters;
s43, classifying the selected sample characteristics in the branch 1 by adopting SoftMax and a cross entropy loss function;
s44, comparing the characteristics of the selected sample and the positive sample with the characteristics of the selected sample and the negative sample by adopting a comparison loss function; samples from the same ID, with a 1 tag, samples from different IDs, and a 0 tag;
s45, adding two parts of losses, wherein the total loss is as follows:
loss=lid+0.5 [ Lc (sample, pos, 1) +lc (neg, 0) ], where Lid is the cross entropy Loss, lc is the contrast Loss, and then back propagation is performed to update the network;
the method also comprises the following setting process when training the multi-scale space-time diagram convolutional neural network:
step 1, inputting a gait sequence, wherein the dimension is [3,100,18], 3 is that 3 channels are provided for inputting key point characteristics, X and Y coordinates and confidence coefficient C are respectively provided, 100 is that the time dimension is 100 frames, and 18 is that 18 key points are provided for each frame;
step 2, outputting 64 channels from the first three layers, wherein the convolution kernel size is (9, 3), 9 is the time convolution kernel size, 3 is the space convolution kernel size, and the output dimension is [64,100,18];
step 3, outputting 128 channels by three layers in the middle, wherein the convolution kernel size is (9, 3), the output dimension is [128,50,18], and the convolution step length of the time dimension is 2 in the fourth layer;
step 4, outputting 256 channels from the rear three layers, wherein the convolution kernel size is (9, 3), the output dimension is [256,25,18], and the convolution step length of the time dimension is 2 in the seventh layer;
step 5, carrying out global average pooling, wherein after pooling, the characteristic dimension is changed into 256 dimensions;
step 6, carrying out dimension exchange on the characteristics [64,100,18] output by the first layer, and carrying out average pooling to obtain 18-dimensional characteristics;
step 7, carrying out dimension exchange on the output characteristics [128,50,18] of the fifth layer, and carrying out average pooling to obtain 18-dimensional characteristics;
step 8, expressing gait characteristics in a mode of fusing shallow layer characteristics, middle layer characteristics and deep layer characteristics, and splicing 18-dimensional characteristics of a first layer, 18-dimensional characteristics of a fifth layer and 256-dimensional characteristics of a last layer to obtain 292-dimensional characteristics;
step 9, classifying 292-dimensional features by adopting a softMax classifier;
and S5, after training, testing by using a trained model, extracting gait characteristics, and performing characteristic matching.
2. A method of testing a gait recognition method based on skeletal information as claimed in claim 1, comprising the steps of:
step I: inputting a gait key point sequence to be tested;
step II: extracting gait characteristics by using a trained network, and carrying out two-norm normalization on the characteristics;
step III: performing the operations of the step I and the step II on the samples in the sample library, and using the feature vectors to represent the gait sequence of the pedestrian to be searched and the gait sequence of the pedestrian in the search library;
step IV: calculating the distance between the gait sequence of the pedestrian to be searched and the gait sequence of the pedestrian in the search library, namely calculating the distance between the characteristics of the pedestrian to be searched and all the gait sequences of the pedestrian in the search library aiming at one gait sequence of the pedestrian to be searched;
step V: and sorting the similarity of the samples in the search library from small to large according to the calculated distance, wherein the earlier the sample is, the more likely the sample is consistent with the ID of the pedestrian to be searched.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010100136.5A CN111310668B (en) | 2020-02-18 | 2020-02-18 | Gait recognition method based on skeleton information |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010100136.5A CN111310668B (en) | 2020-02-18 | 2020-02-18 | Gait recognition method based on skeleton information |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111310668A CN111310668A (en) | 2020-06-19 |
| CN111310668B true CN111310668B (en) | 2023-06-23 |
Family
ID=71147331
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010100136.5A Active CN111310668B (en) | 2020-02-18 | 2020-02-18 | Gait recognition method based on skeleton information |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111310668B (en) |
Families Citing this family (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111860291A (en) * | 2020-07-16 | 2020-10-30 | 上海交通大学 | Multi-mode pedestrian identity recognition method and system based on pedestrian appearance and gait information |
| CN112101176B (en) * | 2020-09-09 | 2024-04-05 | 元神科技(杭州)有限公司 | User identity recognition method and system combining user gait information |
| CN112434655B (en) * | 2020-12-07 | 2022-11-08 | 安徽大学 | Gait recognition method based on adaptive confidence map convolution network |
| CN112633222B (en) * | 2020-12-30 | 2023-04-28 | 民航成都电子技术有限责任公司 | Gait recognition method, device, equipment and medium based on countermeasure network |
| CN112906599A (en) * | 2021-03-04 | 2021-06-04 | 杭州海康威视数字技术股份有限公司 | Gait-based personnel identity identification method and device and electronic equipment |
| CN112926522B (en) * | 2021-03-30 | 2023-11-24 | 广东省科学院智能制造研究所 | Behavior recognition method based on skeleton gesture and space-time diagram convolution network |
| CN113191230A (en) * | 2021-04-20 | 2021-07-30 | 内蒙古工业大学 | Gait recognition method based on gait space-time characteristic decomposition |
| CN114052726A (en) * | 2021-11-25 | 2022-02-18 | 湖南中科助英智能科技研究院有限公司 | Thermal infrared human body gait recognition method and device in dark environment |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20090232353A1 (en) * | 2006-11-10 | 2009-09-17 | University Of Maryland | Method and system for markerless motion capture using multiple cameras |
| CN110096950A (en) * | 2019-03-20 | 2019-08-06 | 西北大学 | A kind of multiple features fusion Activity recognition method based on key frame |
-
2020
- 2020-02-18 CN CN202010100136.5A patent/CN111310668B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20090232353A1 (en) * | 2006-11-10 | 2009-09-17 | University Of Maryland | Method and system for markerless motion capture using multiple cameras |
| CN110096950A (en) * | 2019-03-20 | 2019-08-06 | 西北大学 | A kind of multiple features fusion Activity recognition method based on key frame |
Non-Patent Citations (6)
| Title |
|---|
| "Design of Real-Time Measurement System with Vision/IMU for Close-Range Semi-Physical Rendezvous and Docking Simulation";Zhenshen Qu 等;《Proceedings of 2016 IEEE Chinese Guidance, Navigation and Control Conference (IEEE CGNCC2016)》;全文 * |
| "图像和深度图中的动作识别与手势姿态估计";李瑞;《中国博士学位论文全文数据库 信息科技辑》;全文 * |
| "基于图卷积的骨架行为识别";董安 等;《图形图像》;全文 * |
| "基于时空图卷积网络的人体运动状态识别研究";宋宪 等;《新材料、新技术、新工艺在竞技体育的研发与应用（二）》;全文 * |
| "智能监控***中行人重识别方法研究";***;《中国博士学位论文全文数据库 信息科技辑》;全文 * |
| 张向荣 等.《模式识别》.西安：西安电子科技大学出版社,2019,西安：西安电子科技大学出版社. * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111310668A (en) | 2020-06-19 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111310668B (en) | Gait recognition method based on skeleton information | |
| CN109919031B (en) | Human behavior recognition method based on deep neural network | |
| CN108537136B (en) | Pedestrian re-identification method based on attitude normalization image generation | |
| CN111666843B (en) | Pedestrian re-recognition method based on global feature and local feature splicing | |
| CN111259786B (en) | Pedestrian re-identification method based on synchronous enhancement of appearance and motion information of video | |
| CN112307995B (en) | Semi-supervised pedestrian re-identification method based on feature decoupling learning | |
| CN106909938B (en) | Visual angle independence behavior identification method based on deep learning network | |
| CN109614853B (en) | Bilinear pedestrian re-identification network construction method based on body structure division | |
| CN111428658B (en) | Gait recognition method based on modal fusion | |
| CN110598543A (en) | Model training method based on attribute mining and reasoning and pedestrian re-identification method | |
| Ji et al. | Human-centric clothing segmentation via deformable semantic locality-preserving network | |
| CN111160225B (en) | Human body analysis method and device based on deep learning | |
| CN108595558B (en) | Image annotation method based on data equalization strategy and multi-feature fusion | |
| CN111985332B (en) | Gait recognition method of improved loss function based on deep learning | |
| CN113221625A (en) | Method for re-identifying pedestrians by utilizing local features of deep learning | |
| CN112819065A (en) | Unsupervised pedestrian sample mining method and unsupervised pedestrian sample mining system based on multi-clustering information | |
| JP2022082493A (en) | Pedestrian re-identification method for random shielding recovery based on noise channel | |
| CN111582154A (en) | Pedestrian re-identification method based on multitask skeleton posture division component | |
| CN114299542A (en) | Video pedestrian re-identification method based on multi-scale feature fusion | |
| CN113221770A (en) | Cross-domain pedestrian re-identification method and system based on multi-feature hybrid learning | |
| CN109670423A (en) | A kind of image identification system based on deep learning, method and medium | |
| CN116206327A (en) | Image classification method based on online knowledge distillation | |
| CN113869105A (en) | Human behavior recognition method | |
| Özbay et al. | 3D Human Activity Classification with 3D Zernike Moment Based Convolutional, LSTM-Deep Neural Networks. | |
| CN110348395B (en) | Skeleton behavior identification method based on space-time relationship |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |