CN111291678B - Face image clustering method and device based on multi-feature fusion - Google Patents
Face image clustering method and device based on multi-feature fusion Download PDFInfo
- Publication number
- CN111291678B CN111291678B CN202010081619.5A CN202010081619A CN111291678B CN 111291678 B CN111291678 B CN 111291678B CN 202010081619 A CN202010081619 A CN 202010081619A CN 111291678 B CN111291678 B CN 111291678B
- Authority
- CN
- China
- Prior art keywords
- face image
- probability
- face
- cluster
- feature
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 47
- 230000004927 fusion Effects 0.000 title claims abstract description 21
- 239000013598 vector Substances 0.000 claims abstract description 78
- 238000004364 calculation method Methods 0.000 claims abstract description 21
- 238000004891 communication Methods 0.000 claims abstract description 13
- 238000003062 neural network model Methods 0.000 claims description 18
- 230000008569 process Effects 0.000 claims description 12
- 230000004913 activation Effects 0.000 claims description 4
- 230000001815 facial effect Effects 0.000 claims description 2
- 238000013527 convolutional neural network Methods 0.000 description 12
- 230000009471 action Effects 0.000 description 3
- 238000010586 diagram Methods 0.000 description 3
- 238000013528 artificial neural network Methods 0.000 description 2
- 230000007704 transition Effects 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 1
- 230000000295 complement effect Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 238000005286 illumination Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000011176 pooling Methods 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/168—Feature extraction; Face representation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/23—Clustering techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
- G06V10/75—Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries
- G06V10/751—Comparing pixel values or logical combinations thereof, or feature values having positional relevance, e.g. template matching
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/172—Classification, e.g. identification
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Health & Medical Sciences (AREA)
- Multimedia (AREA)
- Artificial Intelligence (AREA)
- General Health & Medical Sciences (AREA)
- Oral & Maxillofacial Surgery (AREA)
- Evolutionary Computation (AREA)
- Bioinformatics & Computational Biology (AREA)
- General Engineering & Computer Science (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Human Computer Interaction (AREA)
- Life Sciences & Earth Sciences (AREA)
- Computing Systems (AREA)
- Databases & Information Systems (AREA)
- Medical Informatics (AREA)
- Software Systems (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a face image clustering method based on multi-feature fusion, which comprises the following steps: acquiring a face feature set of each face image in a face image set to be clustered, wherein the face feature set comprises feature vectors extracted by each feature extractor; calculating the similarity vector of each face image pair in the face image set to be clustered according to each face feature set; determining an initial cluster of each face image according to each similarity vector; and merging the initial clusters with the communication relation in each initial cluster to obtain a target cluster. In the clustering method, the similarity vector of each image pair in the face image set to be clustered is calculated for each face image face feature set, clustering is carried out according to the similarity vector, clustering results are combined to obtain target clustering, a plurality of feature vectors are adopted for calculation, and compared with clustering carried out by single face features, clustering accuracy is improved.
Description
Technical Field
The invention relates to the technical field of faces, in particular to a face image clustering method and device based on multi-feature fusion.
Background
In the face recognition process, the face images need to be clustered and archived according to the corresponding identity Information (ID) so that a plurality of face images from the same person have the same ID.
However, most of the existing face image clustering methods only use one face feature for clustering, and due to limited description capability of a single face feature, the problems of more people, multiple grades and multiple people and one grade in a clustering result occur, and the clustering precision is low.
Disclosure of Invention
In view of the above, the invention provides a face image clustering method and device based on multi-feature fusion, which are used for solving the problems that the existing face image clustering method mainly utilizes only one face feature for clustering, and the clustering accuracy is low because the description capability of a single face feature is limited, so that more people and multiple people exist in a clustering result. The specific scheme is as follows:
a face image clustering method based on multi-feature fusion is characterized by comprising the following steps:
acquiring a face feature set of each face image in a face image set to be clustered, wherein the face feature set comprises feature vectors extracted by each feature extractor;
calculating the similarity vector of each face image pair in the face image set to be clustered according to each face feature set;
determining an initial cluster of each face image according to each similarity vector;
and merging the initial clusters with the communication relation in each initial cluster to obtain a target cluster.
The above method, optionally, determines an initial cluster of each face image according to each similarity vector, including:
determining the distance of each image pair in the face image set to be clustered according to each similarity vector;
and classifying the face image pairs with the distance smaller than or equal to a preset distance threshold into the same cluster to obtain each initial cluster.
According to the above method, optionally, determining the distance between each image pair in the face image set to be clustered according to each similarity vector includes:
transmitting each similarity vector to a preset neural network model for calculation to obtain a first probability and a second probability;
calculating the target similarity of each face image pair according to each first probability and each second probability;
and determining the distance of each image pair according to the similarity of each target.
In the above method, optionally, merging the initial clusters with the communication relationship in each initial cluster to obtain the target cluster, including:
combining the initial clusters with intersections in each initial cluster to obtain each combined cluster;
and continuing to merge the merged vectors with the intersections in each merged cluster until each merged cluster does not have the intersections, thereby obtaining the target cluster.
The method, optionally, the feature extractors include: a face feature extractor and a body feature extractor.
A face image clustering device based on multi-feature fusion comprises:
the acquisition module is used for acquiring a face feature set of each face image in the face image set to be clustered, wherein the face feature set comprises feature vectors extracted by each feature extractor;
the computing module is used for computing the similarity vector of each face image pair in the face image set to be clustered according to each face feature set;
the determining module is used for determining the initial cluster of each face image according to each similarity vector;
and the merging module is used for merging the initial clusters with the communication relation in the initial clusters to obtain target clusters.
The above apparatus, optionally, the determining module includes:
the determining unit is used for determining the distance between each image pair in the face image set to be clustered according to each similarity vector;
and the dividing unit is used for classifying the face image pairs with the distance smaller than or equal to a preset distance threshold value into the same cluster to obtain each initial cluster.
The above apparatus, optionally, the determining unit includes:
the first calculation subunit is used for transmitting each similarity vector to a preset neural network model for calculation to obtain a first probability and a second probability;
the second calculating subunit is used for calculating the target similarity of each face image pair according to each first probability and each second probability;
and the determining subunit is used for determining the distance of each image pair according to each target similarity.
The above apparatus, optionally, the combining module includes:
the first merging unit is used for merging the initial clusters with intersections in each initial cluster to obtain each merged cluster;
and the second merging unit is used for continuously merging the merged vectors with the intersections in each merged cluster until each merged cluster does not have the intersections, so as to obtain a target cluster.
The above apparatus, optionally, the feature extractors in the clustering apparatus include: a face feature extractor and a body feature extractor.
Compared with the prior art, the invention has the following advantages:
the invention discloses a face image clustering method based on multi-feature fusion, which comprises the following steps: acquiring a face feature set of each face image in a face image set to be clustered, wherein the face feature set comprises feature vectors extracted by each feature extractor; calculating the similarity vector of each face image pair in the face image set to be clustered according to each face feature set; determining an initial cluster of each face image according to each similarity vector; and merging the initial clusters with the communication relation in each initial cluster to obtain a target cluster. In the clustering method, the similarity vector of each image pair in the face image set to be clustered is calculated for each face image face feature set, clustering is carried out according to the similarity vector, clustering results are combined to obtain target clustering, a plurality of feature vectors are adopted for calculation, and compared with clustering carried out by single face features, clustering accuracy is improved.
Drawings
In order to more clearly illustrate the embodiments of the invention or the technical solutions in the prior art, the drawings that are required in the embodiments or the description of the prior art will be briefly described, it being obvious that the drawings in the following description are only some embodiments of the invention, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
Fig. 1 is a flowchart of a face image clustering method based on multi-feature fusion, which is disclosed in an embodiment of the present application;
FIG. 2 is a flowchart of a face image clustering method based on multi-feature fusion according to an embodiment of the present disclosure;
fig. 3 is a schematic structural diagram of a base neural network model according to an embodiment of the present disclosure;
fig. 4 is a structural block diagram of a face image clustering device based on multi-feature fusion disclosed in an embodiment of the present application.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
The previous description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the invention. Thus, the present invention is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
The invention discloses a face image clustering method and a device based on multi-feature fusion, which are applied to the face image clustering in the face recognition process, wherein in the face image clustering analysis process, only one face feature is utilized for clustering in the prior art, and as the description capability of a single face feature is limited, the problem that more people and more people occur in the clustering result and the problem that the clustering result is in one step occurs, and the clustering precision is low, the invention provides the face image clustering method based on multi-feature fusion, which is used for solving the problems, and the execution flow of the clustering method is shown in the figure 1 and comprises the following steps:
s101, acquiring a face feature set of each face image in a face image set to be clustered, wherein the face feature set comprises feature vectors extracted by each feature extractor;
in the embodiment of the invention, the face images to be clustered collectively comprise a plurality of face images, and the face images are clustered according to different people when the face images are the same person. The feature extractors can be Convolutional Neural Network (CNN) models, the CNN models are trained face recognition or pedestrian re-recognition CNN models, the input of the models is an image, and through a series of operations such as convolution, activation, pooling and the like of the CNN models, the output of the CNN models is a feature vector with fixed dimension, and the feature vector can be used for describing the input image. The feature extraction by using the CNN model is to take a face image as input, and finally obtain the feature vector describing the image.
The generation process of the face feature set is as follows:
using a plurality of different Convolutional Neural Network (CNN) models G i Extracting facial image characteristics:
f ij ＝G i (I j ),i＝1,2,3,...,m,j＝1,2,3,...n (1)
wherein G is i Representing the ith CNN model, and m kinds of CNN models are used; i j Representing the j-th image, wherein n total images in the face image set to be clustered are represented; and f ij Representing the feature vectors extracted on the j Zhang Tuzai ith CNN model. Finally for each face image I j Corresponding face feature sets which are all reached:
F j ＝{f 1j ,f 2j ,f 3j ,...,f mj } (2)
further, as the face image of the same person may have illumination changes, occlusion and posture changes, the face features of the same person may be greatly different due to such changes. However, in some cases, the dress of the person is not changed too much, so that although the images cannot be clustered through the face features, the images can be clustered through the body features of the person, and therefore, the feature extractor in the embodiment of the invention can be all the face feature extractor, and the complementary relation existing between various features can be mined in a mode of mixing the face feature extractor and the body feature extractor, so that the purpose of improving the face clustering precision is achieved.
S102, calculating a similarity vector of each face image pair in the face image set to be clustered according to each face feature set;
in the embodiment of the invention, for each image I j And obtaining a face characteristic image set, dividing each face image in the face image set to be clustered into each face image pair according to a preset classification rule in advance, wherein each image in the face image set to be clustered needs to form an image pair with all other images except the image pair and calculate a similarity vector, and if the similarity vector of the image a and the image b is already calculated, the similarity vector of the image b and the image a can be directly used without calculation, and the preset classification rule is not limited in the embodiment of the invention.
And calculating the similarity vector of each face image pair in the face image set to be clustered according to each face feature set, wherein the calculation process of each face image pair is the same. Let us assume that we want to calculate the image I in the face image pair a And image I b For image I a F of face feature set of (2) a ＝{f 1a ,f 2a ,f 3a ,...,f ma -a }; for image I b F of face feature set of (2) b ＝{f 1b ,f 2b ,f 3b ,...,f mb Calculation mapImage I a And image I b The similarity on each feature is to calculate the similarity on each feature vector:
the similarity calculation method is shown in formula (3),
the specific calculation process is as follows:
...
thus, for each image pair, a set of similarity vectors is obtained
S103, determining an initial cluster of each face image according to each similarity vector;
in the embodiment of the invention, firstly, the distance between each image pair in the face image set to be clustered is determined according to each similarity vector, the face image pairs with the distance smaller than or equal to a preset distance threshold are classified into the same cluster, and each initial cluster is obtained, wherein the preset distance threshold can be set according to experience or specific conditions, and the specific value of the preset distance threshold in the embodiment of the invention is not takenLimiting. In image I a And image I b For example, assume image I a And image I b Distance d of (2) ab The preset distance threshold is t, when d ab When t is less than or equal to t, judging the image I a And image I b For the same cluster, when d ab At > t, image I is determined a And image I b Not the same cluster, so that each image forms an initial cluster c j ＝{I j ,I a ,I b ,I c ,...}，j＝1,2,3,...n
S104, merging the initial clusters with the communication relation in the initial clusters to obtain a target cluster.
In the embodiment of the invention, the initial cluster c is aimed at i If c a ＝{I a ,I b ,I c ,I d .., and c } e ＝{I e ,I a ,I b ,I f ,...},c a ∩c e Not equal to phi, consider c a Class c e Class communication, combining the two classes into a combined cluster c= { I a ,I b ,I c ,I d ,I e ,I f ...}. Subclass c to be merged according to the connected subgraph rule i Sequentially merging to obtain a plurality of merged clusters, continuously merging merged vectors with intersections in all merged clusters until no intersection exists in all merged clusters to obtain target clusters, and finally forming a target cluster C with k target clusters h H=1, 2,3,..k, k, clustering is completed.
The invention discloses a face image clustering method based on multi-feature fusion, which comprises the following steps: acquiring a face feature set of each face image in a face image set to be clustered, wherein the face feature set comprises feature vectors extracted by each feature extractor; calculating the similarity vector of each face image pair in the face image set to be clustered according to each face feature set; determining an initial cluster of each face image according to each similarity vector; and merging the initial clusters with the communication relation in each initial cluster to obtain a target cluster. In the clustering method, the similarity vector of each image pair in the face image set to be clustered is calculated for each face image face feature set, clustering is carried out according to the similarity vector, clustering results are combined to obtain target clustering, a plurality of feature vectors are adopted for calculation, and compared with clustering carried out by single face features, clustering accuracy is improved.
In the embodiment of the present invention, determining the distance between each image pair in the face image set to be clustered according to each similarity vector includes:
s201, transmitting each similarity vector to a preset neural network model for calculation to obtain a first probability and a second probability;
in the embodiment of the invention, a preset neural network model is pre-built, wherein the input of the preset neural network model is each vector in similarity vectors, and the output is a first probability P 1 And a second probability P 2 Wherein the first probability P 1 Representing the probability that two images are not of the same class, the first probability P 1 The larger indicates that the two images are less likely to be of the same class; second probability P 2 Representing the probability that two images are of the same class, the second probability P 2 The larger the two images are, the more likely they are to be of the same category; p (P) 1 ,P 2 ＝H(S ab ) Representing a preset neural network model H to S ab For input, two similarity values P of output are obtained 1 And P 2 。
The preset neural network model is shown in fig. 3, and the implementation process is as follows:
assuming that the input of the preset neural network model is an image pair I a And image I b Vector formed by similarity on different featuresThe expected network output is a first probability P 1 And a second probability P 2 . The neural network parameters learned through training are layer 1 network weights W (1) Bias term B (1) Layer two network weights W (2) And bias term B (2) ，
Wherein,
the activation function relu (x) =max (0, x). Let z= [ Z ] 1 ,z 2 ,z 3 ...z m ] T ，A＝[a 1 ,a 2 ,a 3 ,...,a m ] T Then:
Z＝W (1) S ab +B (1) (4)
A＝[a 1 ,a 2 ,a 3 ,...,a m ] T ＝relu(Z)＝[relu(z 1 ),relu(z 2 ),relu(z 3 ),...,relu(z m )] T (5)
[P 1 ,P 2 ] T ＝W (2) A+B (2) (6)
thus, the output first probability p of the preset neural network model can be calculated 1 And a second probability p 2 。
S202, calculating the target similarity of each face image pair according to each first probability and each second probability;
in the embodiment of the present invention, the first probability P is output due to the preset neural network model 1 And a second probability P 2 Is not strictly in [0,1 ]]In the method, for each first probability and the corresponding second probability, the similarity of the corresponding face image pair is calculated, and a sigmoid function is adopted to calculate the first probability P 1 And a second probability P 2 All transition into the (0, 1) interval and remain monotonically increasing, the transition proceeds as follows:
purpose of equation (7)Is to make the first probability P 1 And a second probability P 2 The unified value is convenient for determining a preset distance threshold value when clustering.
S203, determining the distance of each image pair according to the similarity of each target.
In the embodiment of the invention, the distance between two face images is calculated by the following formula:
in the embodiment of the invention, the clustering method utilizes the neural network to dig out the association of various face features on the similarity level, so that the complementarity between different features can be effectively utilized, and the defect of insufficient face description capability of a single feature can be overcome. Meanwhile, the clustering precision is further improved by fusion of various features.
Based on the above-mentioned face image clustering method based on multi-feature fusion, the embodiment of the invention also provides a face image clustering device based on multi-feature fusion, and the structural block diagram of the clustering device is shown in fig. 4, and the clustering device comprises:
an acquisition module 301, a calculation module 302, a determination module 303 and a combination module 304.
Wherein,
the acquiring module 301 is configured to acquire a face feature set of each face image in the set of face images to be clustered, where the face feature set includes feature vectors extracted by each feature extractor;
the computing module 302 is configured to compute a similarity vector of each face image pair in the face image set to be clustered according to each face feature set;
the determining module 303 is configured to determine an initial cluster of each face image according to each similarity vector;
the merging module 304 is configured to merge the initial clusters having the communication relationship in each initial cluster to obtain a target cluster.
The invention discloses a face image clustering device based on multi-feature fusion, which comprises: acquiring a face feature set of each face image in a face image set to be clustered, wherein the face feature set comprises feature vectors extracted by each feature extractor; calculating the similarity vector of each face image pair in the face image set to be clustered according to each face feature set; determining an initial cluster of each face image according to each similarity vector; and merging the initial clusters with the communication relation in each initial cluster to obtain a target cluster. In the clustering device, the similarity vector of each image pair in the face image set to be clustered is calculated for each face image face feature set, clustering is carried out according to the similarity vector, clustering results are combined to obtain target clustering, a plurality of feature vectors are adopted for calculation, and compared with clustering carried out by single face features, clustering accuracy is improved.
In the embodiment of the present invention, the determining module 303 includes:
a determining unit 305 and a dividing unit 306.
Wherein,
the determining unit 305 is configured to determine, according to the respective similarity vectors, a distance between each image pair in the face image set to be clustered;
the dividing unit 306 is configured to divide the face image pairs with the distance less than or equal to a preset distance threshold into the same clusters, and obtain each initial cluster.
In the embodiment of the present invention, the determining unit 305 includes:
a first calculation subunit 307, a second calculation subunit 308, and a determination subunit 309.
Wherein,
the first calculating subunit 307 is configured to transmit each similarity vector to a preset neural network model for calculation, so as to obtain a first probability and a second probability;
the second calculating subunit 308 is configured to calculate the target similarity of each face image pair according to each first probability and each second probability;
the determining subunit 309 is configured to determine a distance between the respective image pairs according to the respective object similarities.
In the embodiment of the present invention, the merging module 304 includes:
a first merging unit 310 and a second merging unit 311.
Wherein,
the first merging unit 310 is configured to merge initial clusters with intersections in each initial cluster to obtain each merged cluster;
the second merging unit 311 is configured to continue merging the merged vectors with the intersections in the merged clusters until no intersection exists in the merged clusters, so as to obtain a target cluster.
In an embodiment of the present invention, each feature extractor in the clustering device includes: a face feature extractor and a body feature extractor.
It should be noted that, in the present specification, each embodiment is described in a progressive manner, and each embodiment is mainly described as different from other embodiments, and identical and similar parts between the embodiments are all enough to be referred to each other. For the apparatus class embodiments, the description is relatively simple as it is substantially similar to the method embodiments, and reference is made to the description of the method embodiments for relevant points.
Finally, it is further noted that relational terms such as first and second, and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.
For convenience of description, the above devices are described as being functionally divided into various units, respectively. Of course, the functions of each element may be implemented in the same piece or pieces of software and/or hardware when implementing the present invention.
From the above description of embodiments, it will be apparent to those skilled in the art that the present invention may be implemented in software plus a necessary general hardware platform. Based on such understanding, the technical solution of the present invention may be embodied essentially or in a part contributing to the prior art in the form of a software product, which may be stored in a storage medium, such as a ROM/RAM, a magnetic disk, an optical disk, etc., including several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the method described in the embodiments or some parts of the embodiments of the present invention.
The above describes in detail a face image clustering method and device based on multi-feature fusion, and specific examples are applied to illustrate the principle and implementation of the invention, and the above description of the examples is only used to help understand the method and core idea of the invention; meanwhile, as those skilled in the art will have variations in the specific embodiments and application scope in accordance with the ideas of the present invention, the present description should not be construed as limiting the present invention in view of the above.
Claims (4)
1. A face image clustering method based on multi-feature fusion is characterized by comprising the following steps:
acquiring a face feature set of each face image in a face image set to be clustered, wherein the face feature set comprises feature vectors extracted by each feature extractor, and each feature extractor comprises: a face feature extractor and a body feature extractor;
calculating the similarity vector of each face image pair in the face image set to be clustered according to each face feature set;
determining an initial cluster of each face image according to each similarity vector;
wherein, the determining the initial cluster of each face image according to each similarity vector comprises: transmitting the similarity vector of each face image pair to a preset neural network model for calculation to obtain a first probability and a second probability; calculating the target similarity of each face image pair according to each first probability and each second probability; determining the distance of each face image pair according to the similarity of each target; classifying the face image pairs with the distance smaller than or equal to a preset distance threshold into the same cluster to obtain an initial cluster of each face image;
the calculation process of the first probability and the second probability comprises the following steps: obtaining a first intermediate parameter according to a first layer network weight, a first layer bias item and the similarity vector in parameters of the neural network model; substituting the first intermediate parameter into an activation function to obtain a second intermediate parameter; obtaining the first probability and the second probability according to a second-layer network weight, a second-layer bias term and the second intermediate parameter in parameters of the neural network model;
the calculating process for calculating the target similarity of each face image pair comprises the following steps: substituting each first probability and each second probability into a formula to obtain the target similarity of each face image pair, wherein the formula is as follows:
wherein,for the target similarity, p 1 For the first probability, p 2 Is the second probability;
and merging the initial clusters with the communication relation in each initial cluster to obtain a target cluster.
2. The method of claim 1, wherein merging the initial clusters having a communication relationship among the initial clusters to obtain the target cluster comprises:
combining the initial clusters with intersections in each initial cluster to obtain each combined cluster;
and continuing to merge the merged vectors with the intersections in each merged cluster until each merged cluster does not have the intersections, thereby obtaining the target cluster.
3. The utility model provides a facial image clustering device based on multi-feature fusion which characterized in that includes:
the device comprises an acquisition module, a clustering module and a clustering module, wherein the acquisition module is used for acquiring a face feature set of each face image in a face image set to be clustered, the face feature set comprises feature vectors extracted by each feature extractor, and each feature extractor comprises: a face feature extractor and a body feature extractor;
the computing module is used for computing the similarity vector of each face image pair in the face image set to be clustered according to each face feature set;
the determining module is used for determining the initial cluster of each face image according to each similarity vector;
wherein the determining module comprises: the determining unit is used for determining the distance between each image pair in the face image set to be clustered according to each similarity vector; the dividing unit is used for classifying the face image pairs with the distance smaller than or equal to a preset distance threshold value into the same cluster to obtain each initial cluster; wherein the determining unit includes: the first calculation subunit is used for transmitting each similarity vector to a preset neural network model for calculation to obtain a first probability and a second probability; a second calculating subunit, configured to calculate, according to each first probability and each second probability, a target similarity of each face image pair; a determining subunit, configured to determine a distance between each image pair according to each target similarity;
wherein the first computing subunit is specifically configured to: obtaining a first intermediate parameter according to a first layer network weight, a first layer bias item and the similarity vector in parameters of the neural network model; substituting the first intermediate parameter into an activation function to obtain a second intermediate parameter; obtaining the first probability and the second probability according to a second-layer network weight, a second-layer bias term and the second intermediate parameter in parameters of the neural network model; the second computing subunit is specifically configured to: substituting each first probability and each second probability into a formula to obtain the target similarity of each face image pair, wherein the formula is as follows:
wherein,for the target similarity, p 1 For the first probability, p 2 Is the second probability;
and the merging module is used for merging the initial clusters with the communication relation in the initial clusters to obtain target clusters.
4. The apparatus of claim 3, wherein the combining module comprises:
the first merging unit is used for merging the initial clusters with intersections in each initial cluster to obtain each merged cluster;
and the second merging unit is used for continuously merging the merged vectors with the intersections in each merged cluster until each merged cluster does not have the intersections, so as to obtain a target cluster.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010081619.5A CN111291678B (en) | 2020-02-06 | 2020-02-06 | Face image clustering method and device based on multi-feature fusion |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010081619.5A CN111291678B (en) | 2020-02-06 | 2020-02-06 | Face image clustering method and device based on multi-feature fusion |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111291678A CN111291678A (en) | 2020-06-16 |
| CN111291678B true CN111291678B (en) | 2024-01-12 |
Family
ID=71022342
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010081619.5A Active CN111291678B (en) | 2020-02-06 | 2020-02-06 | Face image clustering method and device based on multi-feature fusion |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111291678B (en) |
Families Citing this family (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111667403B (en) * | 2020-07-02 | 2023-04-18 | 北京爱笔科技有限公司 | Method and device for generating human face image with shielding |
| CN111985336A (en) * | 2020-07-22 | 2020-11-24 | 深圳供电局有限公司 | Face image clustering method and device, computer equipment and storage medium |
| CN112070144A (en) * | 2020-09-03 | 2020-12-11 | Oppo广东移动通信有限公司 | Image clustering method and device, electronic equipment and storage medium |
| CN112307938B (en) * | 2020-10-28 | 2022-11-11 | 深圳市商汤科技有限公司 | Data clustering method and device, electronic equipment and storage medium |
| CN112949710B (en) * | 2021-02-26 | 2023-06-13 | 北京百度网讯科技有限公司 | Image clustering method and device |
| CN113190701A (en) * | 2021-05-07 | 2021-07-30 | 北京百度网讯科技有限公司 | Image retrieval method, device, equipment, storage medium and computer program product |
| CN113344095B (en) * | 2021-06-21 | 2024-03-29 | 北京惠朗时代科技有限公司 | K-means worker fatigue simple discrimination method based on multi-feature operator |
| CN113344124B (en) * | 2021-06-29 | 2023-11-28 | 北京市商汤科技开发有限公司 | Track analysis method and device, storage medium and system |
| CN113762376A (en) * | 2021-08-31 | 2021-12-07 | 阿里巴巴新加坡控股有限公司 | Image clustering method and device, electronic equipment and storage medium |
| CN114333039B (en) * | 2022-03-03 | 2022-07-08 | 济南博观智能科技有限公司 | Method, device and medium for clustering human images |
| CN115909434B (en) * | 2022-09-07 | 2023-07-04 | 以萨技术股份有限公司 | Data processing system for acquiring facial image characteristics |
| CN115439676A (en) * | 2022-11-04 | 2022-12-06 | 浙江莲荷科技有限公司 | Image clustering method and device and electronic equipment |
| CN115953650B (en) * | 2023-03-01 | 2023-06-27 | 杭州海康威视数字技术股份有限公司 | Training method and device for feature fusion model |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107909104A (en) * | 2017-11-13 | 2018-04-13 | 腾讯数码（天津）有限公司 | The face cluster method, apparatus and storage medium of a kind of picture |
| CN108229419A (en) * | 2018-01-22 | 2018-06-29 | 百度在线网络技术（北京）有限公司 | For clustering the method and apparatus of image |
| CN110516586A (en) * | 2019-08-23 | 2019-11-29 | 深圳力维智联技术有限公司 | A kind of facial image clustering method, system, product and medium |
| CN110730433A (en) * | 2019-10-16 | 2020-01-24 | 北京爱笔科技有限公司 | Indoor positioning method, device and system based on iBeacon |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP5178611B2 (en) * | 2009-04-15 | 2013-04-10 | 株式会社東芝 | Image processing apparatus, image processing method, and program |
-
2020
- 2020-02-06 CN CN202010081619.5A patent/CN111291678B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107909104A (en) * | 2017-11-13 | 2018-04-13 | 腾讯数码（天津）有限公司 | The face cluster method, apparatus and storage medium of a kind of picture |
| CN108229419A (en) * | 2018-01-22 | 2018-06-29 | 百度在线网络技术（北京）有限公司 | For clustering the method and apparatus of image |
| CN110516586A (en) * | 2019-08-23 | 2019-11-29 | 深圳力维智联技术有限公司 | A kind of facial image clustering method, system, product and medium |
| CN110730433A (en) * | 2019-10-16 | 2020-01-24 | 北京爱笔科技有限公司 | Indoor positioning method, device and system based on iBeacon |
Non-Patent Citations (1)
| Title |
|---|
| 杨之光 ; 艾海舟 ; .基于聚类的人脸图像检索及相关反馈.自动化学报.2008,(第09期),全文. * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111291678A (en) | 2020-06-16 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111291678B (en) | Face image clustering method and device based on multi-feature fusion | |
| CN106228185B (en) | A kind of general image classifying and identifying system neural network based and method | |
| US7362892B2 (en) | Self-optimizing classifier | |
| CN104850633B (en) | A kind of three-dimensional model searching system and method based on the segmentation of cartographical sketching component | |
| JP7329430B2 (en) | Rapid Video Queries Using Ensemble of Deep Neural Networks | |
| CN108288051B (en) | Pedestrian re-recognition model training method and device, electronic equipment and storage medium | |
| WO2019200782A1 (en) | Sample data classification method, model training method, electronic device and storage medium | |
| CN107885778B (en) | Personalized recommendation method based on dynamic near point spectral clustering | |
| US9323886B2 (en) | Performance predicting apparatus, performance predicting method, and program | |
| CN110163258A (en) | A kind of zero sample learning method and system reassigning mechanism based on semantic attribute attention | |
| CN107169117B (en) | Hand-drawn human motion retrieval method based on automatic encoder and DTW | |
| CN109165309B (en) | Negative example training sample acquisition method and device and model training method and device | |
| WO2020114108A1 (en) | Clustering result interpretation method and device | |
| CN109635140B (en) | Image retrieval method based on deep learning and density peak clustering | |
| CN103824051A (en) | Local region matching-based face search method | |
| CN110414550B (en) | Training method, device and system of face recognition model and computer readable medium | |
| CN110751027B (en) | Pedestrian re-identification method based on deep multi-instance learning | |
| WO2022062419A1 (en) | Target re-identification method and system based on non-supervised pyramid similarity learning | |
| CN113705596A (en) | Image recognition method and device, computer equipment and storage medium | |
| CN101216886B (en) | A shot clustering method based on spectral segmentation theory | |
| CN110163130B (en) | Feature pre-alignment random forest classification system and method for gesture recognition | |
| CN108520205B (en) | motion-KNN-based human body motion recognition method | |
| CN116883746A (en) | Graph node classification method based on partition pooling hypergraph neural network | |
| Yu et al. | Research on face recognition method based on deep learning | |
| CN111160077A (en) | Large-scale dynamic face clustering method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |