CN107145826B - Pedestrian re-identification method based on double-constraint metric learning and sample reordering - Google Patents

Pedestrian re-identification method based on double-constraint metric learning and sample reordering Download PDF

Info

Publication number
CN107145826B
CN107145826B CN201710213894.6A CN201710213894A CN107145826B CN 107145826 B CN107145826 B CN 107145826B CN 201710213894 A CN201710213894 A CN 201710213894A CN 107145826 B CN107145826 B CN 107145826B
Authority
CN
China
Prior art keywords
camera
constraint
matrix
candidate
pictures
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201710213894.6A
Other languages
Chinese (zh)
Other versions
CN107145826A (en
Inventor
于慧敏
谢奕
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Zhejiang University ZJU
Original Assignee
Zhejiang University ZJU
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Zhejiang University ZJU filed Critical Zhejiang University ZJU
Priority to CN201710213894.6A priority Critical patent/CN107145826B/en
Publication of CN107145826A publication Critical patent/CN107145826A/en
Application granted granted Critical
Publication of CN107145826B publication Critical patent/CN107145826B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands

Landscapes

  • Engineering & Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Theoretical Computer Science (AREA)
  • Image Analysis (AREA)

Abstract

The invention discloses a pedestrian re-identification method based on double-constraint metric learning and sample reordering, which comprises two stages of training and testing; the training phase comprises the following steps: establishing cross-camera association constraint; establishing a constraint associated with the camera; solving a measurement matrix; the testing phase comprises the following steps: performing feature space projection by using the measurement matrix; calculating Euclidean distances between the query picture and the candidate pictures in the feature space; calculating initial sequence of candidate pictures; selecting front K candidate pictures in a sorting queue; constructing a probability hypergraph by utilizing the relevance of the previous K candidate pictures in the feature space; calculating a reordering result based on the probabilistic hypergraph; and returning the final ordering of the candidate pictures. According to the invention, two kinds of association constraints of the training samples are considered at the same time, so that the feature space obtained by learning is more suitable for pedestrian re-identification, and meanwhile, the re-ordering is carried out by utilizing the association of the candidate pictures, so that a more accurate pedestrian re-identification result is obtained.

Description

Pedestrian re-identification method based on double-constraint metric learning and sample reordering
Technical Field
The invention relates to a method in the technical field of video image processing, in particular to a pedestrian re-identification method based on double-constraint metric learning and sample reordering.
Background
Video monitoring provides a rich information source for safety early warning, investigation and evidence collection, suspect tracking and other works. However, the monitoring range of a single camera is very limited, so that it is impossible to monitor a large or complex scene (e.g. train station, airport, campus, etc.) in all directions. In order to capture more comprehensive and extensive information in a public area, a large number of monitoring cameras are often required to work in concert. The traditional video processing technology is mainly designed for a single camera, and when a pedestrian target moves out of a current video, the direction of the target cannot be judged. Therefore, how to re-identify pedestrians in the monitoring network according to the query picture of the pedestrian target and establish identity association of the pedestrian target under different cameras becomes a core problem in the field of intelligent video monitoring.
For the pedestrian re-identification problem, the traditional method is mainly based on the appearance characteristics of the pedestrian image, such as extracting the characteristics of color, shape, texture and the like, and then the pedestrian re-identification result is obtained according to the characteristic similarity. However, the illumination, the viewing angle difference and the posture change of the pedestrian between different cameras can significantly change the appearance of the same pedestrian, and the satisfactory accuracy of pedestrian re-identification cannot be obtained only by means of similarity matching of the appearance characteristics of the pedestrian pictures. The introduction of measurement learning provides an important means for relieving the influence of cross-camera difference on pedestrian re-identification, and the measurement learning learns a measurement matrix through a training set, so that a pedestrian picture can be projected to a new feature space, the feature distance between the same pedestrian pictures is smaller, and the feature distance between different pedestrian pictures is larger. However, in the existing metric learning algorithm, only cross-camera correlation information between pedestrian pictures of different cameras is considered in the training process, and the correlation between different pedestrian pictures in the same camera is ignored. Meanwhile, the metric learning algorithm is easy to generate an overfitting phenomenon on a training set, and a suboptimal pedestrian re-identification result can be obtained by completely depending on a distance metric matrix obtained by learning to perform similarity sequencing in a testing stage.
Aiming at the defects and shortcomings of the existing pedestrian re-identification method based on metric learning, the dual-constraint metric learning technology provided by the invention can simultaneously consider the associated information of the same camera and the cross camera between training samples in the metric learning process, and learn to obtain a feature space projection matrix with stronger discriminability. In addition, by introducing a reordering technology in the test stage, the method can effectively relieve the influence of the over-fitting phenomenon in metric learning by utilizing the associated information among the candidate pictures, and obtain a candidate picture ordering result which is more stable and accurate than the existing pedestrian re-identification technology.
Disclosure of Invention
The invention provides a pedestrian re-identification method based on double-constraint metric learning and sample reordering to solve the problems in the prior art, so that the accuracy and the stability of the existing pedestrian re-identification method based on metric learning are improved.
In order to achieve the purpose, the invention discloses a pedestrian re-identification method based on double-constraint metric learning and sample reordering, which comprises two stages of training and testing;
the training phase comprises the steps of:
step 1, establishing cross-camera association constraint: forming a cross-camera sample pair by using pedestrian pictures from different cameras in the training set, and establishing a constraint item to ensure that the characteristic distance between the cross-camera positive sample pair is smaller than the characteristic distance between the cross-camera negative sample pair;
step 2, establishing a camera association constraint: forming a same-camera sample pair by using pedestrian pictures from the same camera in the training set, and establishing a constraint item to ensure that the characteristic distance between the same-camera negative sample pair is greater than the characteristic distance between the camera-crossing positive sample pair;
step 3, solving a measurement matrix: obtaining a target function of double-constraint metric learning by combining the two constraint terms in the step 1 and the step 2, solving a semi-positive definite metric matrix M which minimizes the target function to obtain a training result of metric learning, and ending the training stage;
the testing phase comprises the following steps:
and 4, performing characteristic space projection by using the measurement matrix: according to the semi-positive nature of the measurement matrix M, the characteristics of the measurement matrix M are decomposed into M-PTP, utilizing the matrix P to search the feature vector x of the picture in the test stagepAnd feature vectors of candidate sets
Figure BDA0001261699130000021
Projecting the images to a new feature space in a unified manner, wherein N is the total number of the candidate concentrated images in the testing stage;
step 5, calculating Euclidean distances of the query picture and the candidate pictures in the feature space: respectively calculating the Euclidean distance between the query picture and each candidate picture in the new feature space:
Figure BDA0001261699130000022
step 6, calculating the initial sequence of the candidate pictures: sorting the candidate pictures according to the Euclidean distances calculated in the step 5, wherein the candidate pictures with smaller Euclidean distances to the query picture can obtain a more front sorting position;
step 7, selecting the first K candidate pictures in the sorting queue: selecting K candidate pictures with the top ranking from the candidate picture ranking queue obtained in the step 6;
step 8, constructing a probability hypergraph by using the relevance of the previous K candidate pictures in the feature space: taking the query picture and the K candidate pictures as vertexes of the probability hypergraph, generating hyperedges of the probability hypergraph through the relevance between the vertexes, and finally giving corresponding weight to each hyperedge;
step 9, calculating a reordering result based on the probability hypergraph: calculating a Laplace matrix of the probability hypergraph, establishing a target function by combining with the experience loss of the initial label, calculating the ranking score of the candidate pictures according to the target function, and reordering the K candidate pictures from large to small according to the ranking score;
step 10, returning the final ordering of the candidate pictures: and (4) replacing the sorting positions of the previous K pictures in the sorting queue in the step 6 with the re-sorting results of the K candidate pictures in the step 9, and returning the whole candidate set sorting queue as the final result of pedestrian re-identification.
Further: the establishment of the cross-camera association constraint in the step 1 comprises the following steps:
step 1.1, respectively defining training pictures from different cameras as a query set
Figure BDA0001261699130000031
And candidate set
Figure BDA0001261699130000032
Wherein xiAnd yjIs a feature vector of a pedestrian picture, and
Figure BDA0001261699130000033
and
Figure BDA0001261699130000034
the number of the pictures in the search set is n, and the number of the pictures in the candidate set is m;
step 1.2, defining sample pairs (x) composed of pedestrian pictures from different camerasi,yj) Is a cross-camera sample pair; when x isiAnd yjWhen belonging to the same pedestrian, i.e.
Figure BDA0001261699130000035
Scale (x)i,yj) For a cross-camera positive sample pair, and define z ij1 is ═ 1; when in
Figure BDA0001261699130000036
When it comes to (x)i,yj) For pairs of negative samples across the camera, and set zij＝-1；
Step 1.3, constraining any cross-camera positive sample pair (x) in the training seti,yj) Is less than the negative sample pair (x) across the camerai,yk) The distance between:
Figure BDA0001261699130000037
wherein d isM(-) is the mahalanobis distance metric function to be learned, expressed as follows:
Figure BDA0001261699130000038
in the above formula, M is a semi-positive measurement matrix, i.e. the target of measurement learning;
step 1.4, performing equivalent transformation on the constraint in step 1.3, wherein the distance between any cross-camera positive sample pair in the constraint training set is smaller than a threshold ξ, and the distance between any cross-camera negative sample pair in the training set is larger than a threshold ξ, so as to obtain the following loss function:
Figure BDA0001261699130000041
Figure BDA0001261699130000042
wherein
Figure BDA0001261699130000047
Is a logistic regression function; ep(M) is a loss function across the camera positive sample pair, Ed(M) is a loss function of the cross-camera negative sample pairs, ξ takes on all cross-camera sample pairs (x)i,yj) And with camera sample pair (y)j,yk) The average distance of (c).
Further: the establishment of the camera association constraint in the step 2 comprises the following steps:
step 2.1, define candidate set
Figure BDA0001261699130000048
Picture of different pedestrians in middlejAnd ykPairs of constituent samples (y)j,yk) For the same camera negative sample pair, and set label zjk＝-1；
Step 2.2, constraining any cross-camera positive sample pair (x) in the training seti,yj) Is less than the same camera negative sample pair (y)j,yk) The distance between:
Figure BDA0001261699130000043
step 2.3, since step 1.4 already constrains the distances between all pairs of cross-camera positive samples to be less than the threshold ξ, the constraint in step 2.2 is equivalently converted into any pair of same-camera negative samples (y) in the constraint training setj,yk) Greater than ξ, the following loss function is obtained:
Figure BDA0001261699130000044
wherein Es(M) is the loss function of the same camera negative sample pair.
Further: the solving of the measurement matrix in the step 3 specifically comprises the following steps:
step 3.1, jointly considering the loss functions in step 1.4 and step 2.3, obtaining a target function of the double-constraint distance metric learning:
Φ(M)＝Ep(M)+Ed(M)+Es(M)
step 3.2, give weight w to the sample pairs in the objective functionijAnd WjkAnd simplifying the objective function expression in the step 3.1 to obtain:
Figure BDA0001261699130000045
wherein
Figure BDA0001261699130000046
When z isijWhen 1 is equal to wij＝1/NposIn which N isposThe total number of pairs of positive samples across cameras in the training set; when z isijWhen is-1 time wijIs set to be 1/NnegIn which N isnegThe total number of all cross-camera and same-camera negative sample pairs in the training set; at the same time, since there is no same-camera positive sample pair, w will bejkAre uniformly set to 1/Nneg；
Step 3.3, defining the dual constraint metric learning as the following optimization problem:
Figure BDA0001261699130000051
and 3.4, solving the optimization problem in the step 3.3 to obtain a semi-positive definite metric matrix M.
Further: the constructing of the probability hypergraph by using the relevance of the previous K candidate pictures in the feature space in the step 8 specifically comprises the following steps:
step 8.1, first queryMerging the pictures and K candidate pictures to obtain a vertex set of the probability hypergraph
Figure BDA0001261699130000052
Step 8.2, in
Figure BDA0001261699130000057
Each vertex v iniAs central node, by connection viGenerating three super edges by 5, 15 and 25 vertexes which are closest to each other in the projection feature space, and adding the three super edges into a super edge set epsilon of the probability hypergraph, so that the set epsilon contains 3 x (K +1) super edges in total;
step 8.3, for each super edge e in the super edge set epsiloniAssigning a non-negative weight value wh(ei) When the super edge takes the query picture as the central node, a weighted value is distributed to the super edge
Figure BDA0001261699130000053
When the super edge takes the candidate picture as the central node, the weight value is distributed to the super edge
Figure BDA0001261699130000054
Step 8.4, according to
Figure BDA00012616991300000511
The subordination relation between the middle vertex and the epsilon middle transfinite has a structure size of
Figure BDA0001261699130000058
The element of the incidence matrix H is defined as:
Figure BDA0001261699130000055
wherein A (v)i,ej) Representing a vertex viBelonging to the super edge ejIs calculated by the following formula:
Figure BDA0001261699130000056
wherein v isjIs a super edge ejσ is the average distance between all vertices in the projection feature space; final completion probability hypergraph
Figure BDA0001261699130000059
And obtaining a correlation matrix H.
Further: in step 9, a reordering result is calculated based on the probabilistic hypergraph, which specifically includes the following substeps:
step 9.1, based on the correlation matrix H, calculate the degree d (v) of each vertex and the degree δ (e) of each superedge in the probability hypergraph, where d (v) Σe∈εwh(e) H (v, e), and
Figure BDA00012616991300000510
defining a diagonal matrix DvMaking the elements on the diagonal line correspond to the degree of each vertex in the probability hypergraph; defining a diagonal matrix DeMaking the elements on the diagonal line correspond to the degree of each hyper-edge in the probability hyper-graph; defining diagonal matrix W to make its diagonal elements correspond to weight W of each superedgeh(e)；
Step 9.2, utilizing incidence matrix H and vertex degree matrix DvOvercritical matrix DeAnd the Laplace matrix L of the probability hypergraph is calculated together with the hyperedge weight matrix W:
Figure BDA0001261699130000061
wherein I is a size of
Figure BDA0001261699130000064
The identity matrix of (1);
step 9.3, simultaneously considering Laplacian constraint and initial label experience loss of the probability hypergraph by utilizing a normalization framework, and defining an objective function of sample reordering as follows:
Figure BDA0001261699130000062
wherein f represents a reordering score vector needing to be learned, r represents an initial label vector, the label of a query picture in r is set to be 1, the labels of all candidate pictures are set to be 0, and μ >0 is a normalization parameter used for weighing the importance between a first item and a second item in an objective function; the first item in the target function restrains the peaks sharing more hyper-edges in the probability hypergraph to obtain similar reordering scores, and the second item in the target function restrains the reordering scores to be close to the initial label information;
step 9.4, by making the first derivative of the objective function in step 9.3 with respect to f zero, an optimal solution to the reordering problem can be obtained quickly:
Figure BDA0001261699130000063
and 9.5, reordering the K candidate pictures from large to small according to the reordering scores of the candidate pictures in the vector f.
Compared with the prior art, the invention adopting the technical scheme has the following beneficial effects:
1) compared with the existing pedestrian re-identification method based on metric learning, which only considers the cross-camera association constraint of the training samples, the method provided by the invention simultaneously considers the same-camera and cross-camera association information among the training samples in the process of metric learning, so that the learned metric matrix has stronger discriminability;
2) according to the method, the probability hypergraph is constructed by using the associated information among different candidate pictures, the similarity sorting result in the test stage is reordered, the influence of an over-fitting phenomenon in metric learning is effectively relieved, and a more stable and accurate candidate picture sorting result is obtained;
3) according to the invention, only K candidate pictures with the front initial ordering positions are considered during reordering, and compared with the reordering of the whole candidate set, the calculation complexity of probability hypergraph construction is reduced on the basis of ensuring the ordering accuracy, so that the reordering speed is accelerated.
Drawings
FIG. 1 is a schematic overall flow chart of the present invention.
Detailed Description
The technical solution of the present invention will be further described in detail with reference to the following specific examples.
The following examples are carried out on the premise of the technical scheme of the invention, and detailed embodiments and specific operation processes are given, but the scope of the invention is not limited to the following examples.
Examples
In the embodiment of the invention, pedestrian pictures shot by different cameras are processed, a metric matrix is learned through a training set, a query picture of a certain pedestrian target is used in a test stage, and correct matching of the pedestrian target is found in candidate sets shot by different cameras, referring to fig. 1, in the embodiment of the invention, the method comprises two stages of training and testing;
the training phase comprises the steps of:
step 1, establishing cross-camera association constraint: the method comprises the following steps of forming a cross-camera sample pair by using pedestrian pictures from different cameras in a training set, establishing a constraint term to enable the characteristic distance between the cross-camera positive sample pair to be smaller than the characteristic distance between the cross-camera negative sample pair, and specifically comprising the following sub-steps:
step 1.1, respectively defining training pictures from different cameras as a query set
Figure BDA0001261699130000071
And candidate set
Figure BDA0001261699130000072
Wherein xiAnd yjIs a feature vector of a pedestrian picture, and
Figure BDA0001261699130000073
and
Figure BDA0001261699130000074
the number of the pictures in the search set is n, and the number of the pictures in the candidate set is m;
step 1.2, defining sample pairs (x) composed of pedestrian pictures from different camerasi,yj) Is a cross-camera sample pair; when x isiAnd yjWhen belonging to the same pedestrian, i.e.
Figure BDA0001261699130000075
Scale (x)i,yj) For a cross-camera positive sample pair, and define z ij1 is ═ 1; when in
Figure BDA0001261699130000076
When it comes to (x)i,yj) For pairs of negative samples across the camera, and set zij＝-1；
Step 1.3, constraining any cross-camera positive sample pair (x) in the training seti,yj) Is less than the negative sample pair (x) across the camerai,yk) The distance between:
Figure BDA0001261699130000077
wherein d isM(-) is the mahalanobis distance metric function to be learned, expressed as follows:
Figure BDA0001261699130000078
in the above formula, M is a semi-positive measurement matrix, i.e. the target of measurement learning;
step 1.4, performing equivalent transformation on the constraint in the step 1.3, wherein the distance between any cross-camera positive sample pair in the constraint training set is smaller than a threshold value ξ, and any cross-camera negative sample pair in the training set
The distance between is greater than the threshold ξ, the following loss function is obtained:
Figure BDA0001261699130000086
Figure BDA0001261699130000082
wherein
Figure BDA0001261699130000087
Is a logistic regression function; ep(M) is a loss function across the camera positive sample pair, Ed(M) is a loss function of the cross-camera negative sample pairs, ξ takes on all cross-camera sample pairs (x)i,yj) And with camera sample pair (y)j,yk) The average distance of (c).
Step 2, establishing a camera association constraint: the method comprises the following steps of forming a same-camera sample pair by using pedestrian pictures from the same camera in a training set, establishing a constraint term to enable the characteristic distance between the same-camera negative sample pair to be larger than the characteristic distance between a cross-camera positive sample pair, and specifically comprising the following substeps:
step 2.1, define candidate set
Figure BDA0001261699130000088
Picture of different pedestrians in middlejAnd ykPairs of constituent samples (y)j,yk) For the same camera negative sample pair, and set label zjk＝-1；
Step 2.2, constraining any cross-camera positive sample pair (x) in the training seti,yj) Is less than the same camera negative sample pair (y)j,yk) The distance between:
Figure BDA0001261699130000083
step 2.3, since step 1.4 already constrains the distances between all pairs of cross-camera positive samples to be less than the threshold ξ, the constraint in step 2.2 is equivalently converted into any pair of same-camera negative samples (y) in the constraint training setj,yk) Greater than ξ, the following loss function is obtained:
Figure BDA0001261699130000084
step 3, solving a measurement matrix: obtaining a target function of double-constraint metric learning by combining the two constraint terms in the step 1 and the step 2, and solving a semi-positive definite metric matrix M which minimizes the target function to obtain a training result of metric learning, wherein the training result specifically comprises the following substeps:
step 3.1, jointly considering the loss functions in step 1.4 and step 2.3, obtaining a target function of the double-constraint distance metric learning:
Φ(M)＝Ep(M)+Ed(M)+Es(M)
step 3.2, giving weight to the sample pairs in the objective function, and simplifying the objective function expression in step 3.1 to obtain:
Figure BDA0001261699130000085
wherein
Figure BDA0001261699130000091
When z isijWhen 1 is equal to wij=1/NposIn which N isposThe total number of pairs of positive samples across cameras in the training set; when z isijWhen is-1 time wijIs set to be 1/NnegIn which N isnegThe total number of all cross-camera and same-camera negative sample pairs in the training set; at the same time, since there is no same-camera positive sample pair, w will bejkAre uniformly set to 1/Nneg；
Step 3.3, defining the dual constraint metric learning as the following optimization problem:
Figure BDA0001261699130000092
step 3.4, solving the optimization problem in the step 3.3 to obtain a semi-positive definite metric matrix M; in this embodiment, first, a matrix X and a matrix Y are defined, where the matrix X and the matrix Y respectively store a query set
Figure BDA0001261699130000093
Middle n pictures and candidate set
Figure BDA0001261699130000094
Feature vectors of the middle m pictures; then, X and Y are combined into a matrix C ═ X, Y]And use in combination of ciRepresents the ith column of matrix C; by assuming when yjAnd ykZ is the same candidate picturejk0 and wjkThe objective function in step 3.2 can be changed to 0:
Figure BDA0001261699130000095
the gradient of the objective function with respect to the matrix M is found as follows:
Figure BDA0001261699130000096
finally, iteratively solving a measurement matrix M which minimizes the objective function by using a gradient descent method;
the testing phase comprises the following steps:
and 4, performing characteristic space projection by using the measurement matrix: according to the semi-positive nature of the measurement matrix M, the characteristics of the measurement matrix M are decomposed into M-PTP, utilizing the matrix P to search the feature vector x of the picture in the test stagepAnd feature vectors of candidate sets
Figure BDA0001261699130000097
Projecting the images to a new feature space in a unified manner, wherein N is the total number of the candidate concentrated images in the testing stage;
step 5, calculating Euclidean distances of the query picture and the candidate pictures in the feature space: respectively calculating the Euclidean distance between the query picture and each candidate picture in the new feature space:
Figure BDA0001261699130000098
step 6, calculating the initial sequence of the candidate pictures: sorting the candidate pictures according to the Euclidean distances calculated in the step 5, wherein the candidate pictures with smaller Euclidean distances to the query picture can obtain a more front sorting position;
step 7, selecting the first K candidate pictures in the sorting queue: selecting K candidate pictures with the top ranking from the candidate picture ranking queue obtained in the step 6, wherein K is 100 in the embodiment;
step 8, constructing a probability hypergraph by using the relevance of the previous K candidate pictures in the feature space: taking the query picture and the K candidate pictures as vertexes of the probability hypergraph, generating hyperedges of the probability hypergraph through the relevance between the vertexes, and finally giving corresponding weight to each hyperedge; the method specifically comprises the following substeps:
step 8.1, firstly, merging the query picture and K candidate pictures to obtain a vertex set of the probability hypergraph
Figure BDA0001261699130000101
Step 8.2, in
Figure BDA0001261699130000102
Each vertex v iniAs central node, by connection viGenerating three super edges by 5, 15 and 25 vertexes which are closest to each other in the projection feature space, and adding the three super edges into a super edge set epsilon of the probability hypergraph, so that the set epsilon contains 3 x (K +1) super edges in total;
step 8.3, for each super edge e in the super edge set epsiloniAssigning a non-negative weight value wh(ei) When the super edge takes the query picture as the central node, a larger weight value is distributed to the super edge
Figure BDA0001261699130000103
Emphasizing the role of the query picture in reordering; when the super edge takes the candidate picture as the central node, a smaller weight value is distributed to the super edge
Figure BDA0001261699130000104
In this example take
Figure BDA0001261699130000105
Step 8.4, according to
Figure BDA0001261699130000106
The subordination relation between the middle vertex and the epsilon middle transfinite has a structure size of
Figure BDA0001261699130000107
The element of the incidence matrix H is defined as:
Figure BDA0001261699130000108
wherein A (v)i,ej) Representing a vertex viBelonging to the super edge ejIs calculated by the following formula:
Figure BDA0001261699130000109
wherein v isjIs a super edge ejσ is the average distance between all vertices in the projection feature space; final completion probability hypergraph
Figure BDA00012616991300001010
And obtaining a correlation matrix H;
step 9, calculating a reordering result based on the probability hypergraph: calculating a Laplace matrix of the probability hypergraph, establishing a target function by combining with the experience loss of the initial label, calculating the ranking score of the candidate pictures according to the target function, and reordering the K candidate pictures from large to small according to the ranking score; the method specifically comprises the following substeps:
step 9.1, based on the correlation matrix H, calculate the degree d (v) of each vertex and the degree δ (e) of each superedge in the probability hypergraph, where d (v) Σe∈εwh(e) H (v, e), and
Figure BDA00012616991300001011
defining a diagonal matrix DvMaking the elements on the diagonal line correspond to the degree of each vertex in the probability hypergraph; defining a diagonal matrix DeMaking the elements on the diagonal line correspond to the degree of each hyper-edge in the probability hyper-graph; defining diagonal matrix W to make its diagonal elements correspond to weight W of each superedgeh(e)；
Step 9.2, utilizing incidence matrix H and vertex degree matrix DvOvercritical matrix DeAnd the Laplace matrix L of the probability hypergraph is calculated together with the hyperedge weight matrix W:
Figure BDA0001261699130000111
wherein I is a size of
Figure BDA0001261699130000114
The identity matrix of (1);
step 9.3, simultaneously considering Laplacian constraint and initial label experience loss of the probability hypergraph by utilizing a normalization framework, and defining an objective function of sample reordering as follows:
Figure BDA0001261699130000112
wherein f represents a reordering score vector needing to be learned, r represents an initial label vector, the label of a query picture in r is set to be 1, the labels of all candidate pictures are set to be 0, and μ >0 is a normalization parameter used for weighing the importance between a first item and a second item in an objective function; the first item in the target function restrains the peaks sharing more super edges in the hypergraph to obtain similar reordering scores, and the second item in the target function restrains the reordering scores to be close to the initial label information; μ in this example is 0.01;
step 9.4, by making the first derivative of the objective function in step 9.3 with respect to f zero, an optimal solution to the reordering problem can be obtained quickly:
Figure BDA0001261699130000113
step 9.5, reordering the K candidate pictures from large to small according to the reordering scores of the candidate pictures in the vector f;
step 10, returning the final ordering of the candidate pictures: and (4) replacing the sorting positions of the previous K pictures in the sorting queue in the step 6 with the re-sorting results of the K candidate pictures in the step 9, and returning the whole candidate set sorting queue as the final result of pedestrian re-identification.
The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents and improvements made within the spirit and principle of the present invention are intended to be included within the scope of the present invention.

Claims (6)

1. A pedestrian re-identification method based on double-constraint metric learning and sample reordering is characterized by comprising two stages of training and testing;
the training phase comprises the steps of:
step 1, establishing cross-camera association constraint: forming a cross-camera sample pair by using pedestrian pictures from different cameras in the training set, and establishing a constraint item to ensure that the characteristic distance between the cross-camera positive sample pair is smaller than the characteristic distance between the cross-camera negative sample pair;
step 2, establishing a camera association constraint: forming a same-camera sample pair by using pedestrian pictures from the same camera in the training set, and establishing a constraint item to ensure that the characteristic distance between the same-camera negative sample pair is greater than the characteristic distance between the camera-crossing positive sample pair;
step 3, solving a measurement matrix: obtaining a target function of double-constraint metric learning by combining the two constraint terms in the step 1 and the step 2, solving a semi-positive definite metric matrix M which minimizes the target function to obtain a training result of metric learning, and ending the training stage;
the testing phase comprises the following steps:
step 4, utilizing the measurement matrixCarrying out characteristic space projection: according to the semi-positive nature of the measurement matrix M, the characteristics of the measurement matrix M are decomposed into M-PTP, utilizing the matrix P to search the feature vector x of the picture in the test stagepAnd feature vectors of candidate sets
Figure FDA0002286140660000011
Projecting the images to a new feature space in a unified manner, wherein N is the total number of the candidate concentrated images in the testing stage;
step 5, calculating Euclidean distances of the query picture and the candidate pictures in the feature space: respectively calculating the Euclidean distance between the query picture and each candidate picture in the new feature space:
Figure FDA0002286140660000012
step 6, calculating the initial sequence of the candidate pictures: sorting the candidate pictures according to the Euclidean distances calculated in the step 5, wherein the candidate pictures with smaller Euclidean distances to the query picture can obtain a more front sorting position;
step 7, selecting the first K candidate pictures in the sorting queue: selecting K candidate pictures with the top ranking from the candidate picture ranking queue obtained in the step 6;
step 8, constructing a probability hypergraph by using the relevance of the previous K candidate pictures in the feature space: taking the query picture and the K candidate pictures as vertexes of the probability hypergraph, generating hyperedges of the probability hypergraph through the relevance between the vertexes, and finally giving corresponding weight to each hyperedge;
step 9, calculating a reordering result based on the probability hypergraph: calculating a Laplace matrix of the probability hypergraph, establishing a target function by combining with the experience loss of the initial label, calculating the ranking score of the candidate pictures according to the target function, and reordering the K candidate pictures from large to small according to the ranking score;
step 10, returning the final ordering of the candidate pictures: and (4) replacing the sorting positions of the previous K pictures in the sorting queue in the step 6 with the re-sorting results of the K candidate pictures in the step 9, and returning the whole candidate set sorting queue as the final result of pedestrian re-identification.
2. The pedestrian re-identification method based on the dual-constraint metric learning and the sample re-ranking as claimed in claim 1, wherein: the establishment of the cross-camera association constraint in the step 1 comprises the following steps:
step 1.1, respectively defining training pictures from different cameras as a query set
Figure FDA0002286140660000021
And candidate set
Figure FDA0002286140660000022
Wherein xiAnd yjIs a feature vector of a pedestrian picture, and
Figure FDA0002286140660000023
and
Figure FDA0002286140660000024
the number of the pictures in the search set is n, and the number of the pictures in the candidate set is m;
step 1.2, defining sample pairs (x) composed of pedestrian pictures from different camerasi，yj) Is a cross-camera sample pair; when x isiAnd yjWhen belonging to the same pedestrian, i.e.
Figure FDA0002286140660000025
Scale (x)i，yj) For a cross-camera positive sample pair, and define zij1 is ═ 1; when in
Figure FDA0002286140660000026
When it comes to (x)i，yj) For pairs of negative samples across the camera, and set zij＝-1；
Step 1.3, constraining any cross-camera positive sample pair (x) in the training seti，yj) Is less than the negative sample pair (x) across the camerai，yk) The distance between:
Figure FDA0002286140660000027
wherein d isM(-) is the mahalanobis distance metric function to be learned, expressed as follows:
Figure FDA0002286140660000028
in the above formula, M is a semi-positive measurement matrix, i.e. the target of measurement learning;
step 1.4, performing equivalent transformation on the constraint in step 1.3, wherein the distance between any cross-camera positive sample pair in the constraint training set is smaller than a threshold ξ, and the distance between any cross-camera negative sample pair in the training set is larger than a threshold ξ, so as to obtain the following loss function:
Figure FDA0002286140660000029
Figure FDA00022861406600000210
wherein
Figure FDA00022861406600000211
Is a logistic regression function; ep(M) is a loss function across the camera positive sample pair, Ed(M) is a loss function of the cross-camera negative sample pairs, ξ takes on all cross-camera sample pairs (x)i，yj) And with camera sample pair (y)j，yk) The average distance of (c).
3. The pedestrian re-identification method based on the dual-constraint metric learning and the sample re-ranking as claimed in claim 2, wherein: the establishment of the camera association constraint in the step 2 comprises the following steps:
step 2.1, define candidate set
Figure FDA0002286140660000031
Picture of different pedestrians in middlejAnd ykPairs of constituent samples (y)j，yk) For the same camera negative sample pair, and set label zjk＝-1；
Step 2.2, constraining any cross-camera positive sample pair (x) in the training seti，yj) Is less than the same camera negative sample pair (y)j，yk) The distance between:
Figure FDA0002286140660000032
step 2.3, since step 1.4 already constrains the distances between all pairs of cross-camera positive samples to be less than the threshold ξ, the constraint in step 2.2 is equivalently converted into any pair of same-camera negative samples (y) in the constraint training setj，yk) Greater than ξ, the following loss function is obtained:
Figure FDA0002286140660000033
wherein Es(M) is the loss function of the same camera negative sample pair.
4. The pedestrian re-identification method based on the dual-constraint metric learning and the sample re-ranking as claimed in claim 3, wherein: the solving of the measurement matrix in the step 3 specifically comprises the following steps:
step 3.1, jointly considering the loss functions in step 1.4 and step 2.3, obtaining a target function of the double-constraint distance metric learning:
Φ(M)＝Ep(M)+Ed(M)+Es(M)
step 3.2, give weight w to the sample pairs in the objective functionijAnd wjkAnd for step 3.1The target function expression of (2) is simplified to obtain:
Figure FDA0002286140660000034
wherein
Figure FDA0002286140660000035
When z isijWhen 1 is equal to wij＝1/NposIn which N isposThe total number of pairs of positive samples across cameras in the training set; when z isijWhen is-1 time wijIs set to be 1/NnegIn which N isnegThe total number of all cross-camera and same-camera negative sample pairs in the training set; at the same time, since there is no same-camera positive sample pair, w will bejkAre uniformly set to 1/Nneg；
Step 3.3, defining the dual constraint metric learning as the following optimization problem:
Figure FDA0002286140660000041
and 3.4, solving the optimization problem in the step 3.3 to obtain a semi-positive definite metric matrix M.
5. The pedestrian re-identification method based on the dual-constraint metric learning and the sample re-ranking as claimed in claim 1, wherein: the constructing of the probability hypergraph by using the relevance of the previous K candidate pictures in the feature space in the step 8 specifically comprises the following steps:
step 8.1, firstly, merging the query picture and K candidate pictures to obtain a vertex set of the probability hypergraph
Figure FDA0002286140660000042
Step 8.2, in
Figure FDA00022861406600000412
Each vertex v iniAs a central nodePoint, by connecting viGenerating three super edges by 5, 15 and 25 vertexes which are closest to each other in the projection feature space, and adding the three super edges into a super edge set epsilon of the probability hypergraph, so that the set epsilon contains 3 x (K +1) super edges in total;
step 8.3, for each super edge e in the super edge set epsiloniAssigning a non-negative weight value wh(ei) When the super edge takes the query picture as the central node, a weighted value is distributed to the super edge
Figure FDA0002286140660000043
When the super edge takes the candidate picture as the central node, the weight value is distributed to the super edge
Figure FDA0002286140660000044
Step 8.4, according to
Figure FDA0002286140660000045
The subordination relation between the middle vertex and the epsilon middle transfinite has a structure size of
Figure FDA0002286140660000046
The element of the incidence matrix H is defined as:
Figure FDA0002286140660000047
wherein A (v)i，ej) Representing a vertex viBelonging to the super edge ejIs calculated by the following formula:
Figure FDA0002286140660000048
wherein v isjIs a super edge ejσ is the average distance between all vertices in the projection feature space; final completion probability hypergraph
Figure FDA0002286140660000049
And obtaining a correlation matrix H.
6. The pedestrian re-identification method based on the dual-constraint metric learning and the sample re-ranking as claimed in claim 5, wherein: in step 9, a reordering result is calculated based on the probabilistic hypergraph, which specifically includes the following substeps:
step 9.1, calculating the degree d (v) of each vertex and the degree delta (e) of each hyper-edge in the probability hypergraph based on the incidence matrix H, wherein
Figure FDA00022861406600000410
While
Figure FDA00022861406600000411
Defining a diagonal matrix DvMaking the elements on the diagonal line correspond to the degree of each vertex in the probability hypergraph; defining a diagonal matrix DeMaking the elements on the diagonal line correspond to the degree of each hyper-edge in the probability hyper-graph; defining diagonal matrix W to make its diagonal elements correspond to weight W of each superedgeh(e)；
Step 9.2, utilizing incidence matrix H and vertex degree matrix DvOvercritical matrix DeAnd the Laplace matrix L of the probability hypergraph is calculated together with the hyperedge weight matrix W:
Figure FDA0002286140660000051
wherein I is a size of
Figure FDA0002286140660000052
The identity matrix of (1);
step 9.3, simultaneously considering Laplacian constraint and initial label experience loss of the probability hypergraph by utilizing a normalization framework, and defining an objective function of sample reordering as follows:
Figure FDA0002286140660000053
wherein f represents a reordering fraction vector needing to be learned, r represents an initial label vector, the label of a query picture in r is set to be 1, the labels of all candidate pictures are set to be 0, and mu >0 is a normalization parameter used for weighing the importance between a first item and a second item in an objective function; the first item in the target function restrains the peaks sharing more hyper-edges in the probability hypergraph to obtain similar reordering scores, and the second item in the target function restrains the reordering scores to be close to the initial label information;
step 9.4, by making the first derivative of the objective function in step 9.3 with respect to f zero, an optimal solution to the reordering problem can be obtained quickly:
Figure FDA0002286140660000054
and 9.5, reordering the K candidate pictures from large to small according to the reordering scores of the candidate pictures in the vector f.
CN201710213894.6A 2017-04-01 2017-04-01 Pedestrian re-identification method based on double-constraint metric learning and sample reordering Active CN107145826B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201710213894.6A CN107145826B (en) 2017-04-01 2017-04-01 Pedestrian re-identification method based on double-constraint metric learning and sample reordering

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201710213894.6A CN107145826B (en) 2017-04-01 2017-04-01 Pedestrian re-identification method based on double-constraint metric learning and sample reordering

Publications (2)

Publication Number Publication Date
CN107145826A CN107145826A (en) 2017-09-08
CN107145826B true CN107145826B (en) 2020-05-08

Family

ID=59773502

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201710213894.6A Active CN107145826B (en) 2017-04-01 2017-04-01 Pedestrian re-identification method based on double-constraint metric learning and sample reordering

Country Status (1)

Country Link
CN (1) CN107145826B (en)

Families Citing this family (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107729818B (en) * 2017-09-21 2020-09-22 北京航空航天大学 Multi-feature fusion vehicle re-identification method based on deep learning
CN107704824B (en) * 2017-09-30 2020-05-29 北京正安维视科技股份有限公司 Pedestrian re-identification method and equipment based on space constraint
CN108133192A (en) * 2017-12-26 2018-06-08 武汉大学 A kind of pedestrian based on Gauss-Laplace distribution statistics identifies again
CN109002792B (en) * 2018-07-12 2021-07-20 西安电子科技大学 SAR image change detection method based on layered multi-model metric learning
CN109635686B (en) * 2018-11-29 2021-04-23 上海交通大学 Two-stage pedestrian searching method combining human face and appearance
CN109711366B (en) * 2018-12-29 2021-04-23 浙江大学 Pedestrian re-identification method based on group information loss function
CN109784266B (en) * 2019-01-09 2021-12-03 江西理工大学应用科学学院 Handwritten Chinese character recognition algorithm of multi-model hypergraph
CN111291611A (en) * 2019-12-20 2020-06-16 长沙千视通智能科技有限公司 Pedestrian re-identification method and device based on Bayesian query expansion
CN111259786B (en) * 2020-01-14 2022-05-03 浙江大学 Pedestrian re-identification method based on synchronous enhancement of appearance and motion information of video
CN111476168B (en) * 2020-04-08 2022-06-21 山东师范大学 Cross-domain pedestrian re-identification method and system based on three stages
CN112651335B (en) * 2020-12-25 2024-05-07 深圳集智数字科技有限公司 Method, system, equipment and storage medium for identifying fellow persons

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103500345A (en) * 2013-09-29 2014-01-08 华南理工大学 Method for learning person re-identification based on distance measure
CN104268140A (en) * 2014-07-31 2015-01-07 浙江大学 Image retrieval method based on weight learning hypergraphs and multivariate information combination
US9141852B1 (en) * 2013-03-14 2015-09-22 Toyota Jidosha Kabushiki Kaisha Person detection and pose estimation system
US9436895B1 (en) * 2015-04-03 2016-09-06 Mitsubishi Electric Research Laboratories, Inc. Method for determining similarity of objects represented in images
CN105989369A (en) * 2015-02-15 2016-10-05 中国科学院西安光学精密机械研究所 Pedestrian Re-Identification Method Based on Metric Learning

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9396412B2 (en) * 2012-06-21 2016-07-19 Siemens Aktiengesellschaft Machine-learnt person re-identification
US20150206069A1 (en) * 2014-01-17 2015-07-23 Matthew BEERS Machine learning-based patent quality metric

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9141852B1 (en) * 2013-03-14 2015-09-22 Toyota Jidosha Kabushiki Kaisha Person detection and pose estimation system
CN103500345A (en) * 2013-09-29 2014-01-08 华南理工大学 Method for learning person re-identification based on distance measure
CN104268140A (en) * 2014-07-31 2015-01-07 浙江大学 Image retrieval method based on weight learning hypergraphs and multivariate information combination
CN105989369A (en) * 2015-02-15 2016-10-05 中国科学院西安光学精密机械研究所 Pedestrian Re-Identification Method Based on Metric Learning
US9436895B1 (en) * 2015-04-03 2016-09-06 Mitsubishi Electric Research Laboratories, Inc. Method for determining similarity of objects represented in images

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
Learning Visual-Spatial Saliency for Multiple-Shot Person Re-identification;Yi Xie 等;《IEEE Signal Processing Letters》;20151130;第1854-1857页 *
Person Re-identification by Graph-based;Yi Xie 等;《Electronics Letters》;20160818;第1447-1449页 *
距离度量学*** 等;《中国计量大学学报》;20161231;第424-428页 *

Also Published As

Publication number Publication date
CN107145826A (en) 2017-09-08

Similar Documents

Publication Publication Date Title
CN107145826B (en) Pedestrian re-identification method based on double-constraint metric learning and sample reordering
CN107832672B (en) Pedestrian re-identification method for designing multi-loss function by utilizing attitude information
CN107506703B (en) Pedestrian re-identification method based on unsupervised local metric learning and reordering
CN109711366B (en) Pedestrian re-identification method based on group information loss function
CN106127780B (en) A kind of curved surface defect automatic testing method and its device
CN104601964B (en) Pedestrian target tracking and system in non-overlapping across the video camera room of the ken
CN104573614B (en) Apparatus and method for tracking human face
CN108268838B (en) Facial expression recognition method and facial expression recognition system
CN111160297A (en) Pedestrian re-identification method and device based on residual attention mechanism space-time combined model
Jiang et al. Optimizing through learned errors for accurate sports field registration
CN105138998B (en) Pedestrian based on the adaptive sub-space learning algorithm in visual angle recognition methods and system again
CN112036447B (en) Zero-sample target detection system and learnable semantic and fixed semantic fusion method
CN104615986B (en) The method that pedestrian detection is carried out to the video image of scene changes using multi-detector
WO2021218671A1 (en) Target tracking method and device, and storage medium and computer program
CN109858437B (en) Automatic luggage volume classification method based on generation query network
CN110516707B (en) Image labeling method and device and storage medium thereof
US9202138B2 (en) Adjusting a contour by a shape model
Haque et al. Two-handed bangla sign language recognition using principal component analysis (PCA) and KNN algorithm
US20140098988A1 (en) Fitting Contours to Features
CN111401113A (en) Pedestrian re-identification method based on human body posture estimation
CN111709317B (en) Pedestrian re-identification method based on multi-scale features under saliency model
Nguyen et al. Combined YOLOv5 and HRNet for high accuracy 2D keypoint and human pose estimation
Chacua et al. People identification through facial recognition using deep learning
Yu et al. Hid 2021: Competition on human identification at a distance 2021
Jiang et al. Triangulate geometric constraint combined with visual-flow fusion network for accurate 6DoF pose estimation

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant