CN113536907A - Social relationship identification method and system based on deep supervised feature selection - Google Patents

Social relationship identification method and system based on deep supervised feature selection Download PDF

Info

Publication number
CN113536907A
CN113536907A CN202110628155.XA CN202110628155A CN113536907A CN 113536907 A CN113536907 A CN 113536907A CN 202110628155 A CN202110628155 A CN 202110628155A CN 113536907 A CN113536907 A CN 113536907A
Authority
CN
China
Prior art keywords
feature selection
feature
weight matrix
features
social relationship
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202110628155.XA
Other languages
Chinese (zh)
Inventor
舒祥波
杨佳文
王梦荫
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Nanjing University of Science and Technology
Original Assignee
Nanjing University of Science and Technology
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Nanjing University of Science and Technology filed Critical Nanjing University of Science and Technology
Priority to CN202110628155.XA priority Critical patent/CN113536907A/en
Publication of CN113536907A publication Critical patent/CN113536907A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/211—Selection of the most significant subset of features
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/213—Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods
    • G06F18/2136—Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods based on sparsity criteria, e.g. with an overcomplete basis
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G06F18/2411—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on the proximity to a decision surface, e.g. support vector machines
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G06F18/2415—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on parametric or probabilistic models, e.g. based on likelihood ratio or false acceptance rate versus a false rejection rate
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • General Physics & Mathematics (AREA)
  • Evolutionary Computation (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Evolutionary Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • General Health & Medical Sciences (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Probability & Statistics with Applications (AREA)
  • Image Analysis (AREA)

Abstract

The invention discloses a social relationship identification method and a system based on deep supervised feature selection, wherein the identification method comprises the following steps: generating, by a plurality of feature extractors, a representation of a relationship between two persons; the deep learning algorithm is compared with2,1Combining the normal forms, and selecting an optimal feature subset from the multi-source features by learning a sparse weighting matrix; calculating a probability distribution of the input pairs over the social relationship categories using a Softmax classifier with the selected features; and (5) sparse optimization of the weight matrix. According to the invention, the accuracy of social relationship identification is improved by more effectively extracting the multi-source attribute characteristics.

Description

Social relationship identification method and system based on deep supervised feature selection
Technical Field
The invention relates to a social relationship identification technology in the field of computer vision, in particular to a social relationship identification method and a social relationship identification system based on deep supervised feature selection.
Background
In recent years, researchers have focused on mining social information from images or videos, such as parentage verification, social roles, social relationship attributes and social relationships, ramatahan et al first tried to describe the social roles people play in activities in 2013. They present roles and interactions through a conditional random field while deriving model weights and role assignments in the video. Zhang et al 2015 proposed a depth model that classifies social relationship features (e.g., warmth, friendliness, and dominance) between two or more people by capturing multiple facial attributes. In addition, they also construct a bridging layer to take advantage of the inherent correspondence between heterogeneous attribute sources. Meanwhile, many researchers are interested in analyzing relationships of relatives as subsets of social relationships. Lu et al propose a new neighborhood exclusion metric learning (NRML) method for the membership verification task. Robinson et al, 2017, collected the largest affinity dataset (i.e., the FIW dataset) and designed several baseline frameworks for affinity verification tasks and family identification tasks.
For social relationship recognition tasks, existing methods can be mostly divided into two categories: based on the environmental object and based on a plurality of facial or body attributes. In the existing social relationship awareness literature, environmental objects are considered as basic clues. Li et al, 2017, proposed a two-channel model for identifying social relationships in still images. The first channel makes a rough prediction of social relationships based on the appearance and geometric information of the paired individuals. The second channel integrates an attention mechanism into the context objects generated by the regional candidate network to refine the results of the coarse-grained prediction. Wang et al developed an end-to-end trainable graph inference model in 2018 that propagated node information through graphs using Gated Graph Neural Networks (GGNNs), and introduced a new graph attention mechanism to infer key environmental objects around people in images.
Also, facial or body attributes may provide effective information for understanding social relationships in images. Sun et al contributed a Double-Stream CaffeNet model to the classification of social domains or social relationships in 2017. From the perspective of social psychology, a plurality of attribute features of the face and the body are collected to improve the accuracy of classification. These features are then concatenated into a vector as input to the classifier. However, this cascading feature is high dimensional, which may contain noise and redundancy.
Disclosure of Invention
The invention aims to provide a social relationship recognition method and system based on deep supervised feature selection, which mainly focuses on social relationship tasks based on human faces or body attributes, selects an optimal subset from multiple source features to remove noise and improves the accuracy of social relationship recognition.
The technical solution for realizing the purpose of the invention is as follows: a social relationship identification method based on deep supervised feature selection comprises the following steps:
generating, by a plurality of feature extractors, a representation of a relationship between two persons;
the deep learning algorithm is compared with2,1Combining the normal forms, and selecting an optimal feature subset from the multi-source features by learning a sparse weighting matrix;
calculating a probability distribution of the input pairs over the social relationship categories using a Softmax classifier with the selected features;
weight matrix sparse optimization, use l2，1And the weight matrix is sparse, redundant features are removed, and the calculation efficiency and the identification precision are improved.
Further, the deep learning algorithm is combined with l2,1Combining the normal forms, selecting an optimal feature subset from the multi-source features by learning a sparse weighting matrix, and specifically comprising the following steps:
for a group-based feature selection strategy, the selected features are represented as
Figure BDA0003102679780000021
Wherein a isiA feature representing the ith attribute,
Figure BDA0003102679780000022
is a weight matrix corresponding to the attribute, k is the number of attributes, | aiI denotes aiCharacteristic dimension of dsRepresenting an attribute dimension;
for dimension-based feature selection, we concatenate the feature vectors of all attributes, i.e.
Figure BDA0003102679780000023
The selected feature may be rewritten as
Figure BDA0003102679780000024
In the formula (I), the compound is shown in the specification,
Figure BDA0003102679780000025
and
Figure BDA0003102679780000026
further, the calculating the probability distribution of the input pairs on the social relationship categories by using the selected features and the Softmax classifier specifically comprises the following steps:
calculating the probability distribution of the input pairs on the social relation classes by using a Softmax classifier by using the optimal feature subset output by the feature selection module; for the selected optimal feature subset fsThe probability calculation formula is:
Figure BDA0003102679780000031
wherein p isjIs the jth element of the vector p, representing that the input pair belongs to the relationship cjM represents M categories.
Further, the weight matrix sparse optimization specifically includes the following steps:
in each iteration, we extract B sets of human image pairs, denoted as T, and minimize the objective loss function L, the formula is:
Figure BDA0003102679780000032
wherein g istIs the true relationship (label) for the t-set,
Figure BDA0003102679780000033
represents t groupsIs predicted to be gtW represents a weight matrix, which may be the weight W in a group-based feature selection strategyiOr weights in dimension-based feature selection strategies
Figure BDA0003102679780000034
The first term in the above equation is the cross-entropy loss function, whose purpose is to train the Softmax classifier, and the second term reg (-) represents the regularization of the weight matrix. Learning the optimal weight by adopting a random gradient descent algorithm; to obtain a sparse weight matrix W, the regularization term of the loss function l is denoted by l2,1Norm expression, the parameter α is used to balance its influence in the loss function; l2，1The norm is a norm function widely used in feature selection algorithms, and is mainly used to force our weight matrix to be as sparse as possible.
The group-based feature selection strategy is to select a subset of features at the attribute level, and its regularization term can be written as:
Figure BDA0003102679780000035
wherein WiI F is WiThe Frobenius norm of (a); the dimension-based feature selection strategy is to select a feature subset on a dimension level, and the strategy eliminates most redundancy as much as possible; according to the above formula, the weight matrix is matched
Figure BDA0003102679780000036
Applications l2,1Norm, whose regularization term can be written as:
Figure BDA0003102679780000037
in the formula (I), the compound is shown in the specification,
Figure BDA0003102679780000038
to represent
Figure BDA0003102679780000039
The (r, c) -th element of (a),
Figure BDA00031026797800000310
is that
Figure BDA00031026797800000311
The c-th column vector of (1); to ensure that the proposed strategy performs feature selection at the dimensional level, we force each column in the weight matrix to be as sparse as possible.
A social relationship identification system based on deep supervised feature selection, comprising:
a feature extraction module that generates a representation of a relationship between two persons through a plurality of feature extractors;
a feature selection module for performing a deep learning algorithm with2,1Combining the normal forms, and selecting an optimal feature subset from the multi-source features by learning a sparse weighting matrix;
a classification module to calculate a probability distribution of the input pairs over social relationship categories using a Softmax classifier using the selected features;
weight matrix sparse optimization Module, use l2,1And (5) thinning the learned weight matrix by using the paradigm, and removing redundant features.
Compared with the prior art, the invention has the following remarkable advantages:
(1) the invention designs a novel deep supervision function selection framework to automatically know the social relationship in the image; (2) the present invention utilizes two feature selection strategies: selecting group characteristics and dimension characteristics, namely selecting the optimal characteristic subsets of the abstracts in the attributes and the dimensions respectively; the selected optimal feature subset has a better understanding structure, which is advantageous for investigating the contribution of different properties.
Drawings
Fig. 1 is a DSFS workflow diagram.
Detailed Description
Social relationships link everyone in the human society. Exploring social relationships in static images helps to study interpersonal behavior or features. It has been found in the literature that human facial and body attributes can be provided for social relationship identificationObjective semantic information. However, they ignore the effect of attributes on the accuracy of identification, which may contain redundancy and noise. The invention aims to improve the accuracy of social relationship identification by more effectively extracting multi-source attribute characteristics. To this end, the present invention proposes a new Deep Supervised Feature Selection (DSFS) framework to recognize social relationships in photos, which combines a deep learning algorithm with l2,1The norms are combined to learn a discriminant feature subset from the multi-source features using human face and body attributes. Experimental results on the PIPA relational data set qualitatively demonstrate the effectiveness of the proposed DSFS framework.
The invention relates to a social relationship recognition algorithm based on deep supervised feature selection, which comprises the following steps:
step 1, feature extraction: generating, by a plurality of feature extractors, a representation of a relationship between two persons; in particular, the attribute features of the face and body regions are extracted by different feature extractors as representations of pairs of people. In the feature extraction module, the invention employs 12 human face and body attributes including face/body age, face/body gender, face/body position and proportion, face appearance, face pose, face emotion, body clothing, body intimacy and physical activity. Extracting attribute features by using a corresponding classification model, and taking all the attribute features as the representation of an input human image pair;
step 2, feature selection: the module combines the deep learning algorithm with l2,1And combining the normal forms, and selecting an optimal characteristic subset from the multi-source characteristics by learning a sparse weighting matrix so as to achieve the purpose of removing noise and redundancy in the multi-source characteristics. In the present invention, we use two strategies for feature selection: group-based feature selection and dimension-based feature selection. Specifically, the goal of the group-based feature selection strategy is to select the optimal feature subset according to the contribution of attributes, and the dimension-based feature selection learns the optimal feature subset at a fine-grained level to eliminate most of the redundancy;
and step 3, classification: using the selected features, calculating a probability distribution of input pairs over social relationship categories using a Softmax classifier (Iqbal et al, 2019);
step 4, weight matrix sparse optimization, using l2,1A sparse weight matrix.
The invention is explained in more detail below with reference to the figures and examples:
examples
A social relationship identification method based on deep supervised feature selection, as shown in fig. 1, the method includes the following steps:
step 1) use a pre-trained Double-Stream cafnet (Sun et al, 2017; jia et al, 2014), pre-trained models such as CNN-CRF (Liu et al, 2015) or Multi-task RNN (Chu et al, 2015) extract attribute features of the face and body. The position and scale attributes of the face/body are composed of spatial information, including position coordinates, relative distance, and relative size ratio.
And 2) taking the feature vectors of the 12 facial and body attributes obtained in the step 1) as the input of a feature selection module, and selecting an optimal feature subset by using a group-based feature selection strategy and a dimension-based feature selection strategy.
For a group-based feature selection strategy, the selected features are represented as
Figure BDA0003102679780000051
Wherein a isiWhich represents the characteristics of one of the attributes,
Figure BDA0003102679780000052
is the weight matrix corresponding to the attribute, k is the number of attributes;
for dimension-based feature selection, we concatenate the feature vectors of all attributes, i.e.
Figure BDA0003102679780000061
The selected feature may be rewritten as
Figure BDA0003102679780000062
In the formula (I), the compound is shown in the specification,
Figure BDA0003102679780000063
and
Figure BDA0003102679780000064
step 3) using the optimal feature subset output by the feature selection module, we calculate the probability distribution of the input pairs on the social relationship classes using a Softmax classifier (Iqbal et al, 2019). For the selected optimal feature subset fsThe probability is calculated as
Figure BDA0003102679780000065
Wherein p isjIs the jth element of the vector p, representing that the input pair belongs to the relationship cjProbability of (c), M categories.
Step 4) to train our DSFS framework, in each iteration, we extract B sets of human pairs, denoted as T, and minimize the objective loss function L, formulated as,
Figure BDA0003102679780000066
wherein g istIs the true relationship to t, and W represents a weight matrix, which may be the weight W in a group-based feature selection strategyiOr weights in dimension-based feature selection strategies
Figure BDA0003102679780000067
The first term in equation (4) is the cross-entropy loss function, whose purpose is to train the Softmax classifier, and the second term reg (·) represents the regularization of the weight matrix. We adopt a random gradient descent algorithm to learn the optimal weight. To obtain a sparse weight matrix W, the regularization term of the loss function l is denoted by l2,1Norm expression, parameter α is used to balance its lossThe effect in the loss function. l2,1The norm is a norm function (Li and Tang, 2015; Li et al, 2013) that is widely used in feature selection algorithms, primarily to force our weight matrices to be as sparse as possible.
The group-based feature selection strategy is to select a subset of features at the attribute level, whose regularization term can be written as,
Figure BDA0003102679780000068
wherein WiI F is WiThe Frobenius norm of (a); a dimension-based feature selection strategy is to select a subset of features on the dimension level, which eliminates most of the redundancy as much as possible. From equation (5), we apply to the weight matrix
Figure BDA0003102679780000071
Applications l21Norm whose regularization term can be written as
Figure BDA0003102679780000072
In the formula (I), the compound is shown in the specification,
Figure BDA0003102679780000073
to represent
Figure BDA0003102679780000074
The (r, c) -th element of (a),
Figure BDA0003102679780000075
is that
Figure BDA0003102679780000076
The c-th column vector of (1). To ensure that the proposed strategy performs feature selection at the dimensional level, we force each column in the weight matrix to be as sparse as possible.
TABLE 1
Figure BDA0003102679780000077
As shown in Table 1, the comparison results of the three basic models and the model proposed by us on the PIPA-relation data set are Face + Softmax, namely, the Face feature training and Softmax classification are only used, and the result is 51.74%; body + Softmax, training only by using Body characteristics and Softmax classification, and the result is 58.25%;
face + Body + SVM Body, facial features are classified using SVM in combination. Result 57.20%; DSFS-Group, and DSFS-dimension are the models we propose, and the results are superior to the above models.
The invention also provides a social relationship recognition system based on deep supervised feature selection, comprising:
a feature extraction module that generates a representation of a relationship between two persons through a plurality of feature extractors;
a feature selection module for performing a deep learning algorithm with2,1Combining the normal forms, and selecting an optimal feature subset from the multi-source features by learning a sparse weighting matrix;
a classification module to calculate a probability distribution of the input pairs over social relationship categories using a Softmax classifier using the selected features;
weight matrix sparse optimization Module, use l2,1And (5) thinning the learned weight matrix by using the paradigm, and removing redundant features.
It should be noted that, the implementation method of each module in the device is described in detail in the social relationship identification method part based on deep supervised feature selection, and the present invention is not described repeatedly.
Furthermore, the invention also provides an electronic device, which includes a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the social relationship identification method based on deep supervised feature selection.
And a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements the above-described social relationship recognition method based on deep supervised feature selection.
The above-mentioned embodiments are only used for illustrating the technical solutions of the present invention, and not for limiting the same; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.

Claims (10)

1. A social relationship identification method based on deep supervised feature selection is characterized by comprising the following steps:
generating, by a plurality of feature extractors, a representation of a relationship between two persons;
the deep learning algorithm is compared with2，1Combining the normal forms, and selecting an optimal feature subset from the multi-source features by learning a sparse weighting matrix;
calculating a probability distribution of the input pairs over the social relationship categories using a Softmax classifier with the selected features;
weight matrix sparse optimization, use l2，1And (5) thinning the weight matrix and removing redundant features.
2. The method of claim 1, wherein the deep learning algorithm is applied to the social relationship recognition method based on deep supervised feature selection2，1Combining the normal forms, selecting an optimal feature subset from the multi-source features by learning a sparse weighting matrix, and specifically comprising the following steps:
for a group-based feature selection strategy, the selected features are represented as
Figure FDA0003102679770000011
Wherein a isiA feature representing the ith attribute,
Figure FDA0003102679770000012
is a weight matrix corresponding to the attribute, k is the number of attributes, | aiI denotes aiCharacteristic dimension of dsRepresenting an attribute dimension;
for dimension-based feature selection, the feature vectors of all attributes are concatenated, i.e.
Figure FDA0003102679770000013
The selected feature may be rewritten as
Figure FDA0003102679770000014
In the formula (I), the compound is shown in the specification,
Figure FDA0003102679770000015
and
Figure FDA0003102679770000016
3. the method for social relationship recognition based on deep supervised feature selection as recited in claim 1, wherein the probability distribution of the input pairs on the social relationship classes is calculated by using a Softmax classifier using the selected features, and the method specifically comprises the following steps:
calculating the probability distribution of the input pairs on the social relation classes by using a Softmax classifier by using the optimal feature subset output by the feature selection module; for the selected optimal feature subset fsThe probability calculation formula is:
Figure FDA0003102679770000017
wherein p isjIs the jth element of the vector p, representing that the input pair belongs to the relationship cjM represents M categories.
4. The social relationship recognition method based on deep supervised feature selection as recited in claim 1, wherein the weight matrix sparse optimization specifically includes the following steps:
in each iteration, B sets of human image pairs are extracted and represented as T, and the objective loss function L is minimized, with the formula:
Figure FDA0003102679770000021
wherein g istIs the true relationship of the t-sets,
Figure FDA0003102679770000022
prediction of the group of t is gtW represents a weight matrix, which may be the weight W in a group-based feature selection strategyiOr weights in dimension-based feature selection strategies
Figure FDA0003102679770000023
The first term in the above equation is the cross-entropy loss function, whose purpose is to train the Softmax classifier, the second term reg (-) represents the regularization of the weight matrix; learning the optimal weight by adopting a random gradient descent algorithm; to obtain a sparse weight matrix W, the regularization term of the loss function l is denoted by l2，1Norm expression, the parameter α is used to balance its influence in the loss function;
the group-based feature selection strategy is to select a subset of features at the attribute level, and its regularization term can be written as:
Figure FDA0003102679770000024
wherein WiI F is WiThe Frobenius norm of (a); the dimension-based feature selection strategy is to select a subset of features on a dimension level; according to the above formula, the weight matrix is matched
Figure FDA0003102679770000025
Application L2，1Norm, whose regularization term can be written as:
Figure FDA0003102679770000026
in the formula (I), the compound is shown in the specification,
Figure FDA0003102679770000027
to represent
Figure FDA0003102679770000028
The (r, c) -th element of (a),
Figure FDA0003102679770000029
is that
Figure FDA00031026797700000210
The c-th column vector of (1); in order to ensure that the proposed strategy performs feature selection at the dimensional level, it is mandatory that each column in the weight matrix is as sparse as possible.
5. A system for social relationship identification based on deep supervised feature selection, comprising:
a feature extraction module that generates a representation of a relationship between two persons through a plurality of feature extractors;
a feature selection module for performing a deep learning algorithm with2，1Combining the normal forms, and selecting an optimal feature subset from the multi-source features by learning a sparse weighting matrix;
a classification module to calculate a probability distribution of the input pairs over social relationship categories using a Softmax classifier using the selected features;
weight matrix sparse optimization Module, use l2，1And (5) thinning the learned weight matrix by using the paradigm, and removing redundant features.
6. The method of claim 5The social relationship recognition system based on deep supervised feature selection is characterized in that the feature selection module is used for combining a deep learning algorithm with the feature selection module2，1Combining the normal forms, and selecting an optimal feature subset from the multi-source features by learning a sparse weighting matrix, wherein the method specifically comprises the following steps:
for a group-based feature selection strategy, the selected features are represented as
Figure FDA0003102679770000031
Wherein a isiA feature representing the ith attribute,
Figure FDA0003102679770000032
is a weight matrix corresponding to the attribute, k is the number of attributes, | aiI denotes aiCharacteristic dimension of dsRepresenting an attribute dimension;
for dimension-based feature selection, the feature vectors of all attributes are concatenated, i.e.
Figure FDA0003102679770000033
The selected feature may be rewritten as
Figure FDA0003102679770000034
In the formula (I), the compound is shown in the specification,
Figure FDA0003102679770000035
and
Figure FDA0003102679770000036
7. the system of claim 5, wherein the classification module calculates the probability distribution of the input pairs over the social relationship classes using a Softmax classifier using the selected features as follows:
calculating the probability distribution of the input pairs on the social relation classes by using a Softmax classifier by using the optimal feature subset output by the feature selection module; for the selected optimal feature subset fsThe probability calculation formula is:
Figure FDA0003102679770000037
wherein p isjIs the jth element of the vector p, representing that the input pair belongs to the relationship cjM represents M categories.
8. The deep supervised feature selection based social relationship recognition system of claim 5, wherein the weight matrix sparse optimization module uses/2，1The paradigm sparsely learns the weight matrix, removes redundant features, specifically as follows:
in each iteration, B sets of human image pairs are extracted and represented as T, and the objective loss function L is minimized, with the formula:
Figure FDA0003102679770000041
wherein g istIs the true relationship of the t-sets,
Figure FDA0003102679770000042
prediction of the group of t is gtW represents a weight matrix, which may be the weight W in a group-based feature selection strategyiOr weights in dimension-based feature selection strategies
Figure FDA0003102679770000043
The first term in the above equation is the cross-entropy loss function, whose purpose is to train the Softmax classifier, the second term reg (-) represents the regularization of the weight matrix; using a stochastic gradient descent algorithmLearning an optimal weight value; to obtain a sparse weight matrix W, the regularization term of the loss function l is denoted by l2，1Norm expression, the parameter α is used to balance its influence in the loss function;
the group-based feature selection strategy is to select a subset of features at the attribute level, and its regularization term can be written as:
Figure FDA0003102679770000044
wherein WiI F is WiThe Frobenius norm of (a); the dimension-based feature selection strategy is to select a subset of features on a dimension level; according to the above formula, the weight matrix is matched
Figure FDA0003102679770000045
Applications l2，1Norm, whose regularization term can be written as:
Figure FDA0003102679770000046
in the formula (I), the compound is shown in the specification,
Figure FDA0003102679770000047
to represent
Figure FDA0003102679770000048
The (r, c) -th element of (a),
Figure FDA0003102679770000049
is that
Figure FDA00031026797700000410
The c-th column vector of (1); in order to ensure that the proposed strategy performs feature selection at the dimensional level, it is mandatory that each column in the weight matrix is as sparse as possible.
9. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the method for social relationship recognition based on deep supervised feature selection as claimed in any one of claims 1 to 4 when executing the program.
10. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out a method for social relationship recognition based on deep supervised feature selection as recited in any one of claims 1 to 4.
CN202110628155.XA 2021-06-06 2021-06-06 Social relationship identification method and system based on deep supervised feature selection Pending CN113536907A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110628155.XA CN113536907A (en) 2021-06-06 2021-06-06 Social relationship identification method and system based on deep supervised feature selection

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110628155.XA CN113536907A (en) 2021-06-06 2021-06-06 Social relationship identification method and system based on deep supervised feature selection

Publications (1)

Publication Number Publication Date
CN113536907A true CN113536907A (en) 2021-10-22

Family

ID=78124569

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110628155.XA Pending CN113536907A (en) 2021-06-06 2021-06-06 Social relationship identification method and system based on deep supervised feature selection

Country Status (1)

Country Link
CN (1) CN113536907A (en)

Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104408469A (en) * 2014-11-28 2015-03-11 武汉大学 Firework identification method and firework identification system based on deep learning of image
CN106445920A (en) * 2016-09-29 2017-02-22 北京理工大学 Sentence similarity calculation method based on sentence meaning structure characteristics
CN106555788A (en) * 2016-11-11 2017-04-05 河北工业大学 Application of the deep learning based on Fuzzy Processing in hydraulic equipment fault diagnosis
CN106845376A (en) * 2017-01-06 2017-06-13 中山大学 A kind of face identification method based on sparse coding
CN107679543A (en) * 2017-02-22 2018-02-09 天津大学 Sparse autocoder and extreme learning machine stereo image quality evaluation method
CN108664986A (en) * 2018-01-16 2018-10-16 北京工商大学 Based on lpThe multi-task learning image classification method and system of norm regularization
CN111371611A (en) * 2020-02-28 2020-07-03 广州大学 Weighted network community discovery method and device based on deep learning
CN112329539A (en) * 2020-10-10 2021-02-05 南京理工大学 Social adaptation model-based crowd behavior identification method
CN112884038A (en) * 2021-02-06 2021-06-01 河南大学 Multi-label feature selection method based on multi-distance metric learning

Patent Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104408469A (en) * 2014-11-28 2015-03-11 武汉大学 Firework identification method and firework identification system based on deep learning of image
CN106445920A (en) * 2016-09-29 2017-02-22 北京理工大学 Sentence similarity calculation method based on sentence meaning structure characteristics
CN106555788A (en) * 2016-11-11 2017-04-05 河北工业大学 Application of the deep learning based on Fuzzy Processing in hydraulic equipment fault diagnosis
CN106845376A (en) * 2017-01-06 2017-06-13 中山大学 A kind of face identification method based on sparse coding
CN107679543A (en) * 2017-02-22 2018-02-09 天津大学 Sparse autocoder and extreme learning machine stereo image quality evaluation method
CN108664986A (en) * 2018-01-16 2018-10-16 北京工商大学 Based on lpThe multi-task learning image classification method and system of norm regularization
CN111371611A (en) * 2020-02-28 2020-07-03 广州大学 Weighted network community discovery method and device based on deep learning
CN112329539A (en) * 2020-10-10 2021-02-05 南京理工大学 Social adaptation model-based crowd behavior identification method
CN112884038A (en) * 2021-02-06 2021-06-01 河南大学 Multi-label feature selection method based on multi-distance metric learning

Similar Documents

Publication Publication Date Title
CN111652066B (en) Medical behavior identification method based on multi-self-attention mechanism deep learning
CN110929164B (en) Point-of-interest recommendation method based on user dynamic preference and attention mechanism
Yu et al. Beyond bilinear: Generalized multimodal factorized high-order pooling for visual question answering
Li et al. Learning robust and discriminative subspace with low-rank constraints
Liu et al. SAANet: Siamese action-units attention network for improving dynamic facial expression recognition
Arumugam Emotion classification using facial expression
Zhi et al. Action unit analysis enhanced facial expression recognition by deep neural network evolution
CN111414461A (en) Intelligent question-answering method and system fusing knowledge base and user modeling
Praveen et al. Audio-visual fusion for emotion recognition in the valence-arousal space using joint cross-attention
Mutegeki et al. Feature-representation transfer learning for human activity recognition
Chauhan et al. Analysis of Intelligent movie recommender system from facial expression
CN113076905B (en) Emotion recognition method based on context interaction relation
CN110889505A (en) Cross-media comprehensive reasoning method and system for matching image-text sequences
Ma et al. Cascade transformer decoder based occluded pedestrian detection with dynamic deformable convolution and Gaussian projection channel attention mechanism
Jadhav et al. HDL-PI: hybrid DeepLearning technique for person identification using multimodal finger print, iris and face biometric features
CN112528077A (en) Video face retrieval method and system based on video embedding
Liu et al. Multi-type decision fusion network for visual Q&A
CN113536907A (en) Social relationship identification method and system based on deep supervised feature selection
Kambala et al. A multi-task learning based hybrid prediction algorithm for privacy preserving human activity recognition framework
Lu et al. Diversify: A General Framework for Time Series Out-of-distribution Detection and Generalization
MeshuWelde et al. Counting-based visual question answering with serial cascaded attention deep learning
CN113010772A (en) Data processing method, related equipment and computer readable storage medium
Akalya devi et al. Multimodal emotion recognition framework using a decision-level fusion and feature-level fusion approach
Prasad et al. Hybrid heuristic mechanism for occlusion aware facial expression recognition scheme using patch based adaptive CNN with attention mechanism
Mase et al. Towards privacy-preserving affect recognition: A two-level deep learning architecture

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination