CN106096535B - Face verification method based on bilinear joint CNN - Google Patents
Face verification method based on bilinear joint CNN Download PDFInfo
- Publication number
- CN106096535B CN106096535B CN201610399704.XA CN201610399704A CN106096535B CN 106096535 B CN106096535 B CN 106096535B CN 201610399704 A CN201610399704 A CN 201610399704A CN 106096535 B CN106096535 B CN 106096535B
- Authority
- CN
- China
- Prior art keywords
- cnn
- face
- bilinear
- layer
- network
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 46
- 238000012795 verification Methods 0.000 title claims abstract description 45
- 238000013527 convolutional neural network Methods 0.000 claims abstract description 109
- 238000012549 training Methods 0.000 claims abstract description 40
- 239000013598 vector Substances 0.000 claims abstract description 13
- 238000013528 artificial neural network Methods 0.000 claims abstract description 8
- 238000011176 pooling Methods 0.000 claims description 21
- 230000008569 process Effects 0.000 claims description 15
- 210000002569 neuron Anatomy 0.000 claims description 11
- 238000011478 gradient descent method Methods 0.000 claims description 6
- 239000011159 matrix material Substances 0.000 claims description 6
- 238000004422 calculation algorithm Methods 0.000 claims description 4
- 238000005457 optimization Methods 0.000 claims description 4
- 230000009467 reduction Effects 0.000 claims description 4
- 210000004205 output neuron Anatomy 0.000 claims description 2
- 230000000644 propagated effect Effects 0.000 claims description 2
- 238000005286 illumination Methods 0.000 abstract description 3
- 230000008859 change Effects 0.000 abstract description 2
- 230000006870 function Effects 0.000 description 6
- 230000000694 effects Effects 0.000 description 3
- 238000011160 research Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000006399 behavior Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000009795 derivation Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 238000006073 displacement reaction Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000003475 lamination Methods 0.000 description 1
- 230000007774 longterm Effects 0.000 description 1
- 238000003909 pattern recognition Methods 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 230000011218 segmentation Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/172—Classification, e.g. identification
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/168—Feature extraction; Face representation
Landscapes
- Engineering & Computer Science (AREA)
- Health & Medical Sciences (AREA)
- Oral & Maxillofacial Surgery (AREA)
- General Health & Medical Sciences (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a face verification method based on a bilinear convolutional neural network. The method comprises the following steps: 1) training a convolutional neural network (CNN for short) by using a human face image prepared in advance; 2) fine adjustment of bilinear CNN is carried out by using the face pictures in the training set; 3) inputting a human face picture to be verified, segmenting the two pictures, and extracting the joint feature output by the bilinear CNN. 4) And training the obtained vector through a self-coding network to obtain a final verification result. The invention is based on bilinear CNN method, and replaces two repeated inputs of original bilinear neural network with different face verification input images, provides a new face verification descriptor, which has robustness to illumination, shielding and posture change, and the feature extracted by bilinear CNN has smaller feature dimension than that of general CNN full link layer, reduces parameter quantity, thereby making the subsequent deep belief network training easier and improving the accuracy of face verification.
Description
Technical Field
The invention relates to the field of computer vision, in particular to a face verification method based on a bilinear joint Convolutional Neural Network (CNN).
Background
The face recognition is the most common identity authentication means in people's daily life, and is one of the most popular pattern recognition research topics at present. The face recognition is to dynamically capture the face of a person through a camera connected with a computer, and simultaneously compare and recognize the captured face with the face in the personnel stock recorded in advance. Other biological feature recognition methods all need human behavior coordination, and face recognition does not need passive coordination, so that the method can be used in certain hidden occasions and can be called as the most friendly biological feature identity authentication technology. The face verification is a sub-problem in the face recognition problem, and the problem of how to judge whether the face in the image is a designated person or not is solved, so that the face verification is a one-to-one matching process. Because the face recognition classification and verification have great practical application values, the subject has been taken as a research hotspot for years, a plurality of methods are proposed, and the face verification algorithm mostly starts from detecting certain features of the face and obtains a verification result by utilizing various probability models or classifiers after processing. The method based on the characteristics of classical SIFT, HOG, Gabor and the like and the method based on CNN are included, and the best result at present can be obtained by face recognition through the CNN. However, since a large number of parameters of the deep CNN network need to be obtained through learning, and many face data sets cannot meet the scale required by the deep CNN network learning at present, the deep CNN network implementation must be trained based on a large number of face data sets, which limits its application in the field of face recognition. In addition, the training process of the deep CNN network takes a lot of time, and optimizing the parameters is a long-term process. Most parameters of the CNN network are from a full link layer with higher dimensionality, the parameters of the network can be reduced by combining the tails of the two CNN networks and reducing the dimensionality, and a good identification effect is reserved. Our face verification studies will be directed to CNN-based methods.
The main idea of the CNN-based method is: firstly, performing convolution on CNN utilized by an input image to extract local features, then reducing dimension through matrix multiplication in a full link layer, and simultaneously adjusting parameters through a gradient descent method of reverse conduction to enable the whole network structure to output a classification result with the minimum difference with a training set result. Features of the second last and the third layer in the network can be regarded as global features of the original image, in the face verification model, the features are combined by various methods to calculate the probability that two face images belong to the same person, and the final recognition process is completed by comparing the face images with a reference set one by one. Many scholars have explored and improved on this basis. Y et al propose a method to increase the links of multiple partial convolution results in the original CNN model to the full link layer and combine these features to enhance the expressive power, thereby improving the recognition effect. Aruni et al multiply and combine two CNN models by a method of mutually reverse chain derivation, and propose a bilinear CNN model, reduce redundancy of features, and improve calculation speed. Wang et al have adopted the deep belief model to combine the CNN characteristic of the multiple pictures that the human face decomposes and get, carry on the human face and verify through the layer code of multiple RBM. The existing face verification research based on the CNN usually ignores the characteristics of the task of face verification, a single face image is directly used as a training sample to extract features, and two relation clues of the input faces are helpful for improving the recognition accuracy based on the CNN.
Disclosure of Invention
In order to overcome the defects of the prior art, the invention provides a face verification method based on bilinear combined CNN. The method comprises the steps of taking an obtained face front image as input, extracting and identifying face image features, comparing the face image features with a reference set face in pairs, and finally outputting a classification result of whether the face and the reference face belong to the same person.
In order to achieve the purpose, the invention adopts the technical scheme that:
a face verification method based on bilinear joint CNN comprises the following steps:
(1) for an input face image, respectively intercepting images of a plurality of parts of the face image by using a multi-scale rectangular frame to serve as input of a CNN (compressed natural number), and pre-training the CNN by using the face image in a training set to obtain an initial CNN model;
(2) combining two initial CNN models with the same parameters to form a new bilinear joint CNN; the initialization parameters of the convolution-pooling layer are given by the step (1), the respective full-link layers of the two CNN models are replaced by a combined three-layer full-link layer, the input of the three-layer full-link layer is obtained by multiplying the output matrix of the last convolution-pooling layer of the two initial CNNs, the structure of the softmax multi-class classifier of the last layer is replaced by a two-class classifier for judging whether the CNNs are the same face, and then the parameters of the three-layer full-link layer are all initialized to Gaussian distribution random values with zero mean variance sigma;
(3) pairwise matching the face images in the training set, respectively inputting two ends of a new bilinear joint CNN, and finely adjusting all parameters of the whole bilinear CNN network according to a classification training result; during fine adjustment, firstly fixing parameters of one side structure of the bilinear CNN model, then adjusting the parameters of the other side CNN model, and obtaining the bilinear CNN model for face verification after repeated iterative fine adjustment;
(4) and carrying out feature secondary classification on the reference face image and the detected face image intercepted in a multi-scale, multi-channel and multi-region mode by adopting a three-layer depth self-coding network, and finally outputting the identification accuracy of face verification.
Preferably, in the step (1), the upper part, the upper left part, the upper right part, the middle part, the left part, the right part, the lower left part, the lower right part and the original image of the face image are respectively captured at different scales by using the symmetry of the face image, that is, the face image is divided into 10 different face input screenshots; and pre-training the CNN by using the face image in the training set until the loss function of the CNN network is converged, wherein the parameter of the convolution-pooling layer in the trained CNN network structure is used as an initial value of the parameter of the convolution pooling layer of each of the two CNN models in the next step.
Preferably, in the step (1), for the input face image, images of a plurality of parts of the face image are respectively captured by a multi-scale rectangular frame to be used as input of the CNN, and the CNN is pre-trained by using the face image in the training set; the CNN network structure model uses a 19-layer VGG network model and consists of 5 convolution-pooling combined layers and three full-link layers, the first two layers are connecting layers of a common neural network method, the third full-link layer is a softmax classifier layer, classification tasks and derivative back propagation are completed, each convolution-pooling layer comprises a plurality of convolution layers and a pooling layer, parameters of all the layers are adjusted layer by layer from the next layer to the previous layer through a softmax classifier classification result of a face image by using a back gradient propagation algorithm, and therefore an initial CNN model is obtained.
Preferably, the full link layer is set to 3 layers, and zero mean white gaussian noise with a variance of 0.01 is used as an initial value of the full link layer parameter.
Preferably, in the step (3), the face images in the training set are paired pairwise, bilinear joint CNNs are respectively input in a grading manner according to the difference of direction and sequence according to whether the face images in the training set are the same person or not as a classification result, parameters of the network are finely adjusted by using a gradient descent method by using a mean square value of the difference between a network classification result and an actual classification result as an optimization criterion function, when gradient back propagation is performed, in a convolution-pooling layer output layer of two CNN models, namely an input layer of a full link layer, because matrix multiplication operation can be conducted on respective matrixes, when a fine adjustment parameter derivative is calculated, parameters of one CNN model are firstly fixed in each iteration process, parameters of the other CNN model are finely adjusted, then parameters of the other CNN model are inversely fixed, parameters of the first CNN model are finely adjusted, after the automatic parameter adjusting process is iterated for multiple times, the total loss function of the model is converged, and therefore the bilinear CNN model for face verification is obtained.
Preferably, in the step (4), for each input human face picture to be verified and each reference set picture, pairwise matching is performed by using 10 different cut pictures, new bilinear joint CNNs are respectively input, a full link layer of the bilinear joint CNNs is used as a feature vector, then all the feature vectors are linked, data dimensionality reduction and secondary classification are performed by using a three-layer deep self-coding network, and finally a result of human face verification, namely a secondary classification result, is obtained, and if the neuron value of 'belonging to the same person' output by the deep self-coding network is larger than the neuron value of 'not belonging to the same person'; and if the input image is used as a face classification task, the reference face corresponding to the highest neuron value of the input face to be classified and the reference face output belonging to the same person can be used as a classification result.
The invention discloses a face verification method based on a bilinear convolutional neural network. The method comprises the following steps: 1) training a convolutional neural network (CNN for short) by using a large number of face images in a data set prepared in advance; 2) fine adjustment of the bilinear CNN is carried out by using the face pictures in the training set and using the pre-training result of the CNN; 3) inputting two human face pictures to be verified, carrying out multi-scale multi-channel multi-region segmentation on the two pictures, respectively serving as two ends of the bilinear CNN, and extracting the joint features output by the bilinear CNN. 4) And connecting the obtained vectors, training the vectors through a deep self-coding network to obtain whether the final two-classification result of the same person is obtained, and taking the maximum output of 'yes' in the two-classification neurons as a face recognition classification result to obtain the recognition accuracy. The invention provides a new face verification descriptor based on a bilinear CNN method, and by replacing two repeated inputs of an original bilinear neural network with different face verification input images, the face verification descriptor can effectively acquire local features and global features of a face, retains the displacement and rotation invariance of the CNN, has good robustness on illumination, shielding and partial posture change, and reduces the calculated amount because the feature vector obtained by the bilinear neural network has smaller dimension than that of a general neural network, thereby leading the subsequent deep belief network training to be easier and obviously improving the accuracy of face verification.
Compared with the prior art, the technical scheme of the invention has the beneficial effects that:
1. the invention provides a new descriptor characteristic suitable for a face verification task to acquire image characteristics with identity recognition capability in face images of different people.
2. The central area utilized by the invention can effectively and adaptively extract the face image features robust to illumination and shielding problems, and the dimension of the feature vector is reduced by using the bilinear joint CNN model.
3. According to the invention, the bilinear CNN is combined with the deep self-coding network to perform face recognition, so that the recognition effect is improved.
Drawings
FIG. 1 is a general flow diagram of the present invention.
FIG. 2 is a structural and training flow chart of the bilinear joint convolution neural network of the present invention
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention. In addition, the technical features involved in the embodiments of the present invention described below may be combined with each other as long as they do not conflict with each other.
Fig. 1 shows an operation process of the present invention, and as shown in the figure, a face verification method based on a bilinear convolutional neural network includes the following steps:
(1) for an input face image, respectively intercepting images of a plurality of parts of the face image by using a multi-scale rectangular frame to serve as input of a CNN (compressed natural network), pre-training the CNN by using a large number of face images in a pre-prepared training set, wherein the CNN network structure model uses a 19-layer VGG (variable gradient G) network model and consists of 5 convolution-pooling combined layers and three full link layers, the first two full link layers are connecting layers of a common neural network method, the third full link layer is a softmax classifier layer and is used for completing classification tasks and derivative back propagation, each convolution-pooling layer comprises a plurality of lamination layers and a pooling layer, parameters of all the layers are adjusted layer by layer from the next layer to the previous layer by using a back gradient propagation algorithm according to classification results of the softmax classifier of the face image, and accordingly an initial CNN model is obtained;
(2) two initial CNN models with the same parameters are combined to form a new bilinear joint CNN. The initialization parameters of the convolution-pooling layer are given by the step (1), the respective full-link layers of the two CNN models are replaced by a combined three-layer full-link layer, the input of the three-layer full-link layer is obtained by multiplying the output matrix of the last convolution-pooling layer of the two initial CNN models, the structure of the softmax multi-class classifier of the last layer is replaced by a two-class classifier for judging whether the CNN models are the same face, and then the parameters of the three full-link layers are all initialized to zero-mean small-variance Gaussian distribution random values;
(3) pairwise matching is carried out on the face images in the training set, then the two ends of the new bilinear joint CNN are respectively input, and all parameters of the whole bilinear CNN network are finely adjusted according to the classification training result. During fine adjustment, firstly fixing parameters of one side structure of the bilinear CNN model, then adjusting the parameters of the other side CNN model, and obtaining the bilinear CNN model for face verification after 30 times of iterative fine adjustment;
(4) and carrying out feature secondary classification on the reference and detected face images intercepted in a multi-scale, multi-channel and multi-region mode by adopting a three-layer depth self-coding network, and finally outputting the identification accuracy of face verification. Further, the specific process in step (1) is as follows: the symmetry of the face image is utilized to respectively capture the upper part, the upper left part, the middle part, the left part, the lower left part and the original image of the face image in different scales, and each face image is divided into 10 different face input screenshots, so that the robustness of the CNN model is improved. And (3) acquiring a large number of face pictures by using a common face data set such as CASIA and the like to train CNN, wherein the trained network is used as a pre-training basis for the next step.
The specific process in the step (2) is as follows: and (2) constructing a bilinear joint CNN, wherein when the network is initialized, two convolution layers of the network are respectively initialized by adopting the weight of the CNN convolution layer trained in the step (1), a full link layer is set to be 3 layers, and random initialization is carried out by adopting zero mean Gaussian distribution of variance.
In the step (3), pairwise matching is carried out on the face images in the training set, bilinear joint CNNs are respectively input in times according to the fact whether the face images in the training set are the same person or not as a classification result, two face images are respectively input into bilinear joint CNNs in different directions and sequences, parameters of a network are finely adjusted by using a gradient descent method by using a mean square value of the difference between the network classification result and an actual classification result as an optimization criterion function, when gradient is propagated reversely, the parameters of one CNN model are firstly fixed in each iteration process, the parameters of the other CNN model are finely adjusted, then the parameters of the other CNN model are reversely fixed, the parameters of the first CNN model are finely adjusted, the concrete structure of the whole network model is shown in figure 2. After about 30 times of iterative automatic parameter adjustment process, the total loss function of the model is converged, so that a bilinear CNN model for face verification is obtained.
In the step (4), for each input human face picture to be verified and each reference set picture, pairwise matching is performed by using 10 different cut pictures, new bilinear joint CNNs are respectively input, a full link layer of the bilinear joint CNNs is used as a feature vector, then all the feature vectors are connected, a three-layer deep self-coding network is used for data dimension reduction and two-classification, and finally a human face verification result, namely a two-classification result is obtained, and if the neuron value of 'belonging to the same person' output by the deep self-coding network is larger than the neuron value of 'not belonging to the same person'. And if the input image is used as a face classification task, the reference face corresponding to the highest neuron value of the input face to be classified and the reference face output belonging to the same person can be used as a classification result. The specific process in the step (3) is as follows: pairwise matching is carried out on the face images in the training set, whether the face images in the training set are the same person is used as a classification result, the two face images are respectively input into bilinear joint CNNs in a grading mode according to different directions and different sequences, and a gradient descent method is used for fine adjustment of network parameters by using a mean square value of differences between a network classification result and an actual classification result as an optimization criterion function, so that a bilinear CNN model for face verification is obtained.
The specific process in the step (4) is as follows: and (2) pairing 10 different pictures obtained in the step (1) for each input face picture to be verified and each reference set picture, respectively inputting bilinear joint CNNs, taking a full link layer of the bilinear joint CNNs as a feature vector, then connecting all the feature vectors, performing data dimension reduction and two classification by using a three-layer deep self-coding network, and finally obtaining a face verification result, namely a two-classification result, wherein if the output neuron value of 'belonging to the same person' of the deep self-coding network is larger than the neuron value of 'not belonging to the same person'. And if the input image is used as a face classification task, the reference face corresponding to the highest neuron value of the input face to be classified and the reference face output belonging to the same person can be used as a classification result.
It should be understood that the above-described embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not intended to limit the embodiments of the present invention. Other variations and modifications will be apparent to persons skilled in the art in light of the above description. And are neither required nor exhaustive of all embodiments. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present invention should be included in the protection scope of the claims of the present invention.
Claims (5)
1. A face verification method based on bilinear joint CNN is characterized by comprising the following steps:
(1) for an input face image, respectively intercepting images of a plurality of parts of the face image by using a multi-scale rectangular frame to serve as input of a CNN (compressed natural number), and pre-training the CNN by using the face image in a training set to obtain an initial CNN model;
(2) combining two initial CNN models with the same parameters to form a new bilinear joint CNN; the initialization parameters of the convolution-pooling layer are given by the step (1), the respective full-link layers of the two CNN models are replaced by a combined three-layer full-link layer, the input of the three-layer full-link layer is obtained by multiplying the output matrix of the last convolution-pooling layer of the two initial CNNs, the structure of the softmax multi-class classifier of the last layer is replaced by a two-class classifier for judging whether the CNNs are the same face, and then the parameters of the three-layer full-link layer are all initialized to Gaussian distribution random values with zero mean variance of sigma;
(3) pairwise matching the face images in the training set, respectively inputting two ends of a new bilinear joint CNN, and finely adjusting all parameters of the whole bilinear CNN network according to a classification training result (Fine Tuning); during fine adjustment, firstly fixing parameters of one side structure of the bilinear CNN model each time, then adjusting the parameters of the other side CNN model by using a gradient descent method, and obtaining the bilinear CNN model for face verification after repeated iterative fine adjustment; (4) carrying out feature secondary classification on a reference face image and a detected face image intercepted in a multi-scale, multi-channel and multi-region mode by adopting a three-layer depth self-coding network, and finally outputting the identification accuracy rate of face verification;
in the step (3), pairwise matching is carried out on the face images in the training set, bilinear joint CNNs are respectively input in times according to the fact whether the face images in the training set are the same person or not as a classification result, two face images are respectively input into bilinear joint CNNs in different directions and sequences, parameters of a network are finely adjusted by using a gradient descent method by using a mean square value of a difference between a network classification result and an actual classification result as an optimization criterion function, when gradient is propagated reversely, in a convolution-pooling layer output layer of two CNN models, namely an input layer of a full-link layer, due to the fact that matrix multiplication can conduct respective matrixes, when a fine adjustment parameter derivative is calculated, parameters of one CNN model are firstly fixed in each iteration process, parameter fine adjustment is carried out on the other CNN model, then parameters of the other CNN model are reversely fixed, and parameters of the first CNN model are finely adjusted, after the automatic parameter adjusting process is iterated for multiple times, the total loss function of the model is converged, and therefore the bilinear CNN model for face verification is obtained.
2. The bilinear joint CNN-based face verification method of claim 1, wherein in step (1), the upper part, the upper left part, the upper right part, the middle part, the left part, the right part, the lower left part, the lower right part and the original image of the face image are respectively captured at different scales by using the symmetry of the face image, and each face image is divided into 10 different face input screenshots; and pre-training the CNN by using the face image in the training set until the loss function of the CNN network is converged, wherein the parameter of the convolution pooling layer in the trained CNN network structure is used as an initial value of the parameter of the convolution pooling layer of each of the two CNN models in the next step.
3. The face verification method based on bilinear joint CNN of claim 1, characterized in that, in step (1), for the input face image, the images of multiple parts of the face image are respectively intercepted by multi-scale rectangular frames as the input of CNN, and the CNN is pre-trained by using the face image in the training set; the CNN network structure model uses a 19-layer VGG network model and consists of 5 convolution-pooling combined layers and three full-link layers, the first two layers are connecting layers of a common neural network method, the third full-link layer is a softmax classifier layer, classification tasks and derivative back propagation are completed, each convolution-pooling layer comprises a plurality of convolution layers and a pooling layer, parameters of all the layers are adjusted layer by layer from the next layer to the previous layer through a softmax classifier classification result of a face image by using a back gradient propagation algorithm, and therefore an initial CNN model is obtained.
4. The bilinear joint CNN-based face verification method of claim 1, wherein the full link layer is set to 3 layers, and zero mean Gaussian white noise with a variance of 0.01 is used as an initial value of a parameter of the full link layer.
5. The face verification method based on bilinear joint CNN as claimed in claim 2, wherein in step (4), for each input face picture to be verified and each reference set picture, pairwise matching is performed using the 10 different cut pictures, new bilinear joint CNNs are respectively input, the full link layer of the bilinear joint CNNs is used as a feature vector, then all the feature vectors are combined, a three-layer deep self-coding network is used for data dimension reduction and two-classification, and finally a face verification result, namely a two-classification result, is obtained, if the output neuron value of the same person by the deep self-coding network is larger than the neuron value of the same person; if the input image is used as a face classification task, the reference face corresponding to the highest neuron value of the input face to be classified and the reference face output belonging to the same person can be used as a classification result; the self-coding network firstly adopts a Gaussian-limited Boltzmann machine (Gaussian RBM) model to carry out parameter initialization, network parameters are firstly initialized randomly by Gaussian distribution white noise with extremely small variance, and the RBM model layer is trained; and then the parameters are used as initial values to train the self-coding network.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610399704.XA CN106096535B (en) | 2016-06-07 | 2016-06-07 | Face verification method based on bilinear joint CNN |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610399704.XA CN106096535B (en) | 2016-06-07 | 2016-06-07 | Face verification method based on bilinear joint CNN |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN106096535A CN106096535A (en) | 2016-11-09 |
| CN106096535B true CN106096535B (en) | 2020-10-23 |
Family
ID=57227479
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610399704.XA Active CN106096535B (en) | 2016-06-07 | 2016-06-07 | Face verification method based on bilinear joint CNN |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106096535B (en) |
Families Citing this family (29)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106845385A (en) * | 2017-01-17 | 2017-06-13 | 腾讯科技（上海）有限公司 | The method and apparatus of video frequency object tracking |
| CN106845421B (en) * | 2017-01-22 | 2020-11-24 | 苏州飞搜科技有限公司 | Face feature recognition method and system based on multi-region feature and metric learning |
| CN106874868B (en) * | 2017-02-14 | 2020-09-18 | 北京飞搜科技有限公司 | Face detection method and system based on three-level convolutional neural network |
| CN107016370B (en) * | 2017-04-10 | 2019-10-11 | 电子科技大学 | A kind of partial occlusion face identification method based on data enhancing |
| WO2019012330A2 (en) * | 2017-07-12 | 2019-01-17 | Banuba Limited | Computer-implemented methods and computer systems configured for generating photorealistic-imitating synthetic representations of subjects |
| CN107316066B (en) * | 2017-07-28 | 2021-01-01 | 北京工商大学 | Image classification method and system based on multi-channel convolutional neural network |
| CN108090468B (en) * | 2018-01-05 | 2019-05-03 | 百度在线网络技术（北京）有限公司 | Method and apparatus for detecting face |
| WO2019153175A1 (en) * | 2018-02-08 | 2019-08-15 | 国民技术股份有限公司 | Machine learning-based occluded face recognition system and method, and storage medium |
| CN108427963B (en) * | 2018-03-02 | 2020-10-30 | 浙江工业大学 | Deep learning-based classification and identification method for melanoma skin diseases |
| CN108446617B (en) * | 2018-03-09 | 2022-04-22 | 华南理工大学 | Side face interference resistant rapid human face detection method |
| CN108510466A (en) * | 2018-03-27 | 2018-09-07 | 百度在线网络技术（北京）有限公司 | Method and apparatus for verifying face |
| CN108596090B (en) * | 2018-04-24 | 2019-08-27 | 北京达佳互联信息技术有限公司 | Facial image critical point detection method, apparatus, computer equipment and storage medium |
| CN108921106B (en) * | 2018-07-06 | 2021-07-06 | 重庆大学 | Capsule-based face recognition method |
| CN109190695B (en) * | 2018-08-28 | 2021-08-03 | 中国海洋大学 | Fish image classification method based on deep convolutional neural network |
| CN110874602A (en) * | 2018-08-30 | 2020-03-10 | 北京嘀嘀无限科技发展有限公司 | Image identification method and device |
| CN109086753B (en) * | 2018-10-08 | 2022-05-10 | 新疆大学 | Traffic sign identification method and device based on two-channel convolutional neural network |
| CN110135231B (en) * | 2018-12-25 | 2021-05-28 | 杭州慧牧科技有限公司 | Animal face recognition method and device, computer equipment and storage medium |
| TWI706378B (en) * | 2018-12-29 | 2020-10-01 | 鴻海精密工業股份有限公司 | Cloud device, terminal device, and image classification method |
| CN110472668B (en) * | 2019-07-22 | 2021-02-19 | 华北电力大学（保定） | Image classification method |
| CN110378319A (en) * | 2019-07-31 | 2019-10-25 | 广州视源电子科技股份有限公司 | Signal detection method and device, computer equipment and storage medium |
| CN110458246A (en) * | 2019-08-27 | 2019-11-15 | 上海理工大学 | A kind of Biliary Calculi classification method based on deep learning |
| CN110705352A (en) * | 2019-08-29 | 2020-01-17 | 杭州晟元数据安全技术股份有限公司 | Fingerprint image detection method based on deep learning |
| CN112766292A (en) * | 2019-11-04 | 2021-05-07 | 中移(上海)信息通信科技有限公司 | Identity authentication method, device, equipment and storage medium |
| CN110674800B (en) * | 2019-12-04 | 2020-05-19 | 图谱未来(南京)人工智能研究院有限公司 | Face living body detection method and device, electronic equipment and storage medium |
| CN111091624B (en) * | 2019-12-19 | 2021-09-28 | 南京大学 | Method for generating high-precision drivable human face three-dimensional model from single picture |
| CN111079851B (en) * | 2019-12-27 | 2020-09-18 | 常熟理工学院 | Vehicle type identification method based on reinforcement learning and bilinear convolution network |
| CN110991433B (en) * | 2020-03-04 | 2020-06-23 | 支付宝(杭州)信息技术有限公司 | Face recognition method, device, equipment and storage medium |
| CN111783670A (en) * | 2020-07-02 | 2020-10-16 | 哈尔滨理工大学 | Face recognition method based on neural network and face composition |
| CN114998964B (en) * | 2022-06-02 | 2023-04-18 | 天津道简智创信息科技有限公司 | Novel license quality detection method |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104866829A (en) * | 2015-05-25 | 2015-08-26 | 苏州大学 | Cross-age face verify method based on characteristic learning |
| CN105138993A (en) * | 2015-08-31 | 2015-12-09 | 小米科技有限责任公司 | Method and device for building face recognition model |
| WO2016074247A1 (en) * | 2014-11-15 | 2016-05-19 | Beijing Kuangshi Technology Co., Ltd. | Face detection using machine learning |
| CN105631296A (en) * | 2015-12-30 | 2016-06-01 | 北京工业大学 | Design method of safety face verification system based on CNN (convolutional neural network) feature extractor |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105960657B (en) * | 2014-06-17 | 2019-08-30 | 北京旷视科技有限公司 | Use the facial super-resolution of convolutional neural networks |
| CN105184312B (en) * | 2015-08-24 | 2018-09-25 | 中国科学院自动化研究所 | A kind of character detecting method and device based on deep learning |
| CN105184271A (en) * | 2015-09-18 | 2015-12-23 | 苏州派瑞雷尔智能科技有限公司 | Automatic vehicle detection method based on deep learning |
-
2016
- 2016-06-07 CN CN201610399704.XA patent/CN106096535B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2016074247A1 (en) * | 2014-11-15 | 2016-05-19 | Beijing Kuangshi Technology Co., Ltd. | Face detection using machine learning |
| CN104866829A (en) * | 2015-05-25 | 2015-08-26 | 苏州大学 | Cross-age face verify method based on characteristic learning |
| CN105138993A (en) * | 2015-08-31 | 2015-12-09 | 小米科技有限责任公司 | Method and device for building face recognition model |
| CN105631296A (en) * | 2015-12-30 | 2016-06-01 | 北京工业大学 | Design method of safety face verification system based on CNN (convolutional neural network) feature extractor |
Also Published As
| Publication number | Publication date |
|---|---|
| CN106096535A (en) | 2016-11-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106096535B (en) | Face verification method based on bilinear joint CNN | |
| CN107194341B (en) | Face recognition method and system based on fusion of Maxout multi-convolution neural network | |
| CN108520535B (en) | Object classification method based on depth recovery information | |
| CN108304788B (en) | Face recognition method based on deep neural network | |
| CN108182441B (en) | Parallel multichannel convolutional neural network, construction method and image feature extraction method | |
| CN108537743B (en) | Face image enhancement method based on generation countermeasure network | |
| CN108596039B (en) | Bimodal emotion recognition method and system based on 3D convolutional neural network | |
| CN111079639B (en) | Method, device, equipment and storage medium for constructing garbage image classification model | |
| CN106599797B (en) | A kind of infrared face recognition method based on local parallel neural network | |
| CN109063724B (en) | Enhanced generation type countermeasure network and target sample identification method | |
| WO2018052587A1 (en) | Method and system for cell image segmentation using multi-stage convolutional neural networks | |
| CN106096538A (en) | Face identification method based on sequencing neural network model and device | |
| CN110378208B (en) | Behavior identification method based on deep residual error network | |
| CN110728179A (en) | Pig face identification method adopting multi-path convolutional neural network | |
| CN109360170B (en) | Human face repairing method based on advanced features | |
| CN110222718B (en) | Image processing method and device | |
| CN112052772A (en) | Face shielding detection algorithm | |
| CN106650617A (en) | Pedestrian abnormity identification method based on probabilistic latent semantic analysis | |
| CN106611156B (en) | Pedestrian identification method and system based on self-adaptive depth space characteristics | |
| CN115966010A (en) | Expression recognition method based on attention and multi-scale feature fusion | |
| CN107563319A (en) | Face similarity measurement computational methods between a kind of parent-offspring based on image | |
| Kalam et al. | Rotation independent digit recognition in sign language | |
| CN104036242A (en) | Object recognition method based on convolutional restricted Boltzmann machine combining Centering Trick | |
| Liu | Human face expression recognition based on deep learning-deep convolutional neural network | |
| Shen et al. | Facial expression recognition based on bidirectional gated recurrent units within deep residual network |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20170315 Address after: 528300 East Road, Shunde District, Guangdong, Foshan, No. 9 Applicant after: Internation combination research institute of Carnegie Mellon University of Shunde Zhongshan University Applicant after: Sun Yat-sen University Address before: 528300 Guangdong province Foshan city Shunde District Daliang South Road No. 9 Research Institute Applicant before: Internation combination research institute of Carnegie Mellon University of Shunde Zhongshan University |
|
| TA01 | Transfer of patent application right | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |