CN110781856A - Heterogeneous face recognition model training method, face recognition method and related device - Google Patents
Heterogeneous face recognition model training method, face recognition method and related device Download PDFInfo
- Publication number
- CN110781856A CN110781856A CN201911067746.3A CN201911067746A CN110781856A CN 110781856 A CN110781856 A CN 110781856A CN 201911067746 A CN201911067746 A CN 201911067746A CN 110781856 A CN110781856 A CN 110781856A
- Authority
- CN
- China
- Prior art keywords
- model
- face recognition
- domain
- training
- feature extraction
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 70
- 238000012549 training Methods 0.000 title claims abstract description 69
- 238000000605 extraction Methods 0.000 claims description 68
- 238000001514 detection method Methods 0.000 claims description 49
- 238000013145 classification model Methods 0.000 claims description 29
- 238000013527 convolutional neural network Methods 0.000 claims description 7
- 238000013528 artificial neural network Methods 0.000 claims description 4
- 238000000691 measurement method Methods 0.000 claims description 3
- 239000000523 sample Substances 0.000 description 56
- 238000002372 labelling Methods 0.000 description 9
- 238000010586 diagram Methods 0.000 description 6
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 238000012545 processing Methods 0.000 description 3
- 238000004891 communication Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 239000013598 vector Substances 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000004590 computer program Methods 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 230000006386 memory function Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000003909 pattern recognition Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/168—Feature extraction; Face representation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Oral & Maxillofacial Surgery (AREA)
- Health & Medical Sciences (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Physics & Mathematics (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Evolutionary Computation (AREA)
- General Engineering & Computer Science (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Computational Biology (AREA)
- Artificial Intelligence (AREA)
- Life Sciences & Earth Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Human Computer Interaction (AREA)
- Multimedia (AREA)
- Image Analysis (AREA)
Abstract
The invention provides a heterogeneous face recognition model training method, a face recognition method and a related device, wherein the heterogeneous face recognition model comprises a feature comparison model; the feature comparison model comprises a plurality of comparison branch models formed by pairwise different domains, and the training method comprises the following steps: sequentially acquiring a plurality of groups of sample images of two different domains; wherein the sample image is labeled with a domain type of the sample image; and training the corresponding comparison branch model sequentially through the characteristics of the sample images of each group of the two different domains. Therefore, a plurality of comparison branch models are obtained, and each comparison branch model can realize heterogeneous face recognition of two domains, so that heterogeneous face recognition of more than 2 domains is realized.
Description
Technical Field
The invention relates to the technical field of computer vision and pattern recognition, in particular to a heterogeneous face recognition model training method, a face recognition method and a related device.
Background
The face recognition is a technology for carrying out identity recognition according to a face image, firstly, feature vectors are extracted from the face image, then, the similarity between the vectors is calculated through a certain similarity measurement function, the feature extraction is carried out on the basis of a convolutional neural network in the current mainstream scheme, and the similarity calculation is carried out by using a cosine function. The traditional face recognition is generally applied to 2D visible light images, and along with the abundance of sensors, a face recognition technology, namely heterogeneous face recognition, needs to be implemented on non-visible light face images (near infrared, sketch, cartoon and 3D faces).
It is proposed in patent document (CN105608450A) to improve the performance of near infrared-visible face recognition by using a large number of visible light images. Firstly, a basic network is pre-trained on the basis of a large number of visible light samples, then a cross-domain triple is constructed for network fine tuning, uniform deep layer feature expression across a near infrared domain and a visible light domain is obtained, and the problem that a convolutional network is easy to over-fit when being trained on a small-scale heterogeneous database can be solved.
However, the feature extraction is performed by using one network, so that the difference between different image domains cannot be established, and the learned feature identification capability is limited. In addition, the scheme mainly aims at the heterogeneous face recognition of 2 domains, and is difficult to popularize in heterogeneous face recognition scenes of more domains, because the expression capacity of a single model is limited, the overall accuracy is reduced along with the increase of image domain types, and along with the addition of a new domain, the whole feature extraction model needs to be retrained, and the expansibility is poor.
Disclosure of Invention
The invention mainly solves the technical problem of realizing the heterogeneous face recognition of more than 2 domains.
In order to solve the technical problems, the invention adopts a technical scheme that: the training method of the heterogeneous face recognition model is characterized in that the heterogeneous face recognition model comprises a feature comparison model; the feature comparison model comprises a plurality of comparison branch models formed by pairwise different domains, and the training method comprises the following steps: sequentially acquiring a plurality of groups of sample images of two different domains; wherein the sample image is labeled with a domain type of the sample image; and training the corresponding comparison branch model sequentially through the characteristics of the sample images of each group of the two different domains.
Wherein, the step of training the corresponding comparison branch model sequentially through the characteristics of each group of the two different domain sample images comprises the following steps: and sequentially training the corresponding comparison branch models of each group of the two sample images in different domains by using a cosine similarity measurement method.
The heterogeneous face recognition model further comprises a domain classification model and a face feature extraction model, wherein the domain classification model, the face feature extraction model and the feature comparison model are sequentially cascaded; the feature extraction model comprises a plurality of feature extraction branch models corresponding to the domain types one to one; the training method further comprises the following steps: training the domain classification model through a convolution neural network method by using a sample image with a labeled domain type; and training the feature extraction branch models of the feature extraction model respectively through the sample images marked with the identity categories.
The method comprises the following steps of respectively training the feature extraction branch models of the feature extraction model by labeling sample images of identity categories, wherein the domain types comprise visible light types, and the step of respectively training the feature extraction branch models of the feature extraction model by labeling the sample images of the identity categories comprises the following steps: establishing a first feature extraction branch model corresponding to a visible light domain, and training the first feature extraction branch model through a sample image of a visible light domain type; and establishing feature extraction branch models corresponding to other different domains based on the trained first feature extraction branch model, and sequentially training.
Wherein the step of training the domain classification model through the sample image labeled with the domain type by a convolutional neural network method further comprises: acquiring a sample image of a new annotation domain type; and training the domain classification model through the sample image of the labeled domain type and the new sample image of the labeled domain type by a convolutional neural network method.
In order to solve the technical problem, the invention adopts another technical scheme that: the method is characterized in that the face recognition model is obtained by the training method; the face recognition method comprises the following steps: acquiring characteristics of a plurality of groups of detection images in different domains; inputting the characteristics of the detection images of the plurality of groups of different domains into the corresponding comparison branch model, so as to perform characteristic comparison on the characteristics of the detection images of the different domains in the comparison branch model and obtain a comparison result.
Wherein the method further comprises: the acquiring features of the plurality of groups of detection images of different domains further comprises: acquiring a detection image; inputting the detection image into a domain classification model, so as to classify the detection image according to different domains in the domain classification model, and obtain a plurality of groups of domain category labels of the detection image of different domains; and respectively inputting the multiple groups of detection images in different domains into corresponding human face feature extraction models to obtain the features of the multiple groups of detection images in different domains.
In order to solve the technical problem, the invention adopts another technical scheme that: provided is a heterogeneous face recognition device including: the domain classification model is used for carrying out domain classification on the detection images so as to obtain a plurality of groups of domain category labels of the detection images of different domains; the human face feature extraction model comprises a plurality of human face feature extraction branch models, and each human face feature extraction branch model is used for extracting features of the detection images of the corresponding domain to obtain the features of a plurality of groups of detection images of different domains; the characteristic comparison model comprises a plurality of characteristic comparison branch models, and each characteristic comparison branch model is used for comparing the characteristics of the detection graphs of two different domains and obtaining a comparison result.
In order to solve the technical problem, the invention adopts another technical scheme that: providing a smart device, the smart device comprising: the heterogeneous face recognition model training device comprises a control circuit, a processor and a memory which are mutually coupled, wherein the memory is used for storing a program instruction for realizing any one of the above training methods of the heterogeneous face recognition model; the memory is used for storing program instructions for realizing the face recognition method based on the heterogeneous face recognition model in any one of the above items; the processor is configured to execute the program instructions stored by the memory.
In order to solve the technical problem, the invention adopts another technical scheme that: providing a storage medium storing a program file executable to implement the training method of the heterogeneous face recognition model according to any one of the above items; and the program file can be executed to realize the face recognition method based on the heterogeneous face recognition model.
The invention has the beneficial effects that: different from the prior art, in the training method of the heterogeneous face recognition model provided by the invention, the heterogeneous face recognition model comprises a feature comparison model, the feature comparison model comprises a comparison branch model formed by a plurality of pairwise different domains, and the training method comprises the steps of sequentially obtaining a plurality of groups of sample images of two different domains; wherein the sample image is labeled with a domain type of the sample image; and training the corresponding comparison branch model sequentially through the characteristics of the sample images of each group of the two different domains. Therefore, a plurality of comparison branch models can be obtained, and each comparison branch model can realize heterogeneous face recognition of two domains, so that heterogeneous face recognition of more than 2 domains is realized.
Drawings
FIG. 1 is a schematic flow chart of a training method of a heterogeneous face recognition model according to a first embodiment of the present invention;
FIG. 2 is a schematic flow chart of a training method of a heterogeneous face recognition model according to a second embodiment of the present invention;
FIG. 3 is a sub-flowchart of step S14;
FIG. 4 is a flowchart illustrating a first embodiment of a face recognition method according to the present invention;
FIG. 5 is a flowchart illustrating a second embodiment of the face recognition method of the present invention;
6 a-6 b are schematic structural diagrams of the heterogeneous face recognition model of the present invention;
FIG. 7 is a schematic diagram of the structure of the smart device of the present invention;
FIG. 8 is a schematic diagram of the structure of the storage medium of the present invention.
Detailed Description
The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
The present invention will be described in detail below with reference to the accompanying drawings and examples.
Fig. 1 is a schematic flow chart of a training method of a heterogeneous face recognition model according to a first embodiment of the present invention.
Specifically, the training method of the feature comparison model comprises the following steps:
step S11: sequentially acquiring characteristics of a plurality of groups of sample images of two different domains; wherein the sample image is labeled with a domain type of the sample image.
Obtaining a plurality of sample images, carrying out domain classification on the plurality of sample images, extracting the characteristics of different domain images according to different domains, and dividing the two sample images of different domains into a group, wherein the domain type of the image is marked on the sample image.
The domain of the image is divided into a visible light image, an infrared light image, a sketch image, and the like.
Step S12: and training the corresponding comparison branch model sequentially through the characteristics of the sample images of each group of the two different domains.
And training the corresponding comparison branch models by using the characteristics of the two groups of sample images in different domains, wherein the obtained comparison branch models can support the face recognition of two domains. Specifically, the cosine similarity measurement method can be used to train the corresponding comparison branch model.
Specifically, if the sample image includes a visible light image, an infrared light image, and a sketch image, the visible light image and the infrared light image are used to train the comparison branch model, so as to obtain a comparison branch model, which supports the feature comparison of the visible light image and the infrared light image; then, the infrared light image and the sketch image are used for training the comparison branch model, so that another comparison branch model is obtained, and the comparison branch model supports the characteristic comparison of the infrared light image and the sketch image; and training the comparison branch model by using the visible light image and the sketch image to obtain another comparison branch model which supports the characteristic comparison of the visible light image and the sketch image.
In one embodiment, if there are 3 different domain sample images, then 3 alignment branch models need to be trained; if there are 4 sample images in different domains, 6 comparison branch models need to be trained, so that the feature comparison of the sample images of two different domains can be realized. In one embodiment, if there are a total of k different fields of the sample image, then 0.5 xkX (k-1) alignment branch models are required.
In this embodiment, the corresponding comparison branch models are trained through the features of a plurality of groups of two different domain sample images, so as to obtain a plurality of comparison branch models, and heterogeneous face recognition of more than 2 domains can be realized.
Fig. 2 is a schematic flow chart of a training method of a heterogeneous face recognition model according to a second embodiment of the present invention. The method comprises the following steps:
step S13: and training the domain classification model through a convolution neural network method by marking the sample image of the domain type.
Obtaining a sample image, carrying out domain type labeling on the sample image to obtain a sample image labeled with a domain type, and training a domain classification model by combining the sample image labeled with the domain type with a convolutional neural network method.
In one embodiment, if the sample image includes a visible light image, an infrared light image, and a sketch image, the domain classification model is trained according to the visible light image, the infrared light image, and the sketch image in combination with a convolutional neural network method.
Step S14: and training the feature extraction branch models of the feature extraction model respectively through the sample images marked with the identity categories.
And acquiring a sample image, and carrying out identity type labeling on the sample image, wherein the identity type labeling is name labeling, gender labeling, height labeling and the like. And training a feature extraction model according to the features of the labeled image.
In an embodiment, since the sample material of the visible light is rich, the feature extraction model can be trained based on the visible light when training other domain models except the visible light, and then fine tuning is performed. Specifically, please refer to fig. 3, which includes:
step S141: establishing a first feature extraction branch model corresponding to a visible light domain, and training the first feature extraction branch model through a sample image of the visible light domain type.
Firstly, a first feature extraction branch model corresponding to a visible light domain is established, and then the first feature extraction branch model is trained through a sample image of the visible light domain type.
Step S142: and establishing feature extraction branch models corresponding to other different domains based on the trained first feature extraction branch model, and sequentially training.
And establishing feature extraction models corresponding to other domains on the basis of the trained first feature extraction model, and sequentially training according to the established feature extraction models.
In an embodiment, when a new sample image of the labeled domain type is obtained, only the domain classification model needs to be retrained, and the face feature extraction model and the feature comparison model adopt incremental training. Specifically, if the type of the obtained labeling domain of the new sample image is an ultraviolet image, the domain classification model is retrained according to the visible light image, the infrared light image, the sketch image and the ultraviolet image in combination with a convolutional neural network method. And then, according to the identity type of the sample image of the new mark domain type, training the corresponding feature extraction branch model, wherein in the process, the feature extraction branch model obtained by the previous training does not need to be trained again. The feature comparison model is also trained, that is, the feature of the sample image of the new annotation domain type is combined with any sample image of the previous annotation domain type to train a feature comparison branch model compatible with the sample image of the new annotation domain type and the sample image of the previous annotation domain type, and the feature comparison branch model established before does not need to be trained again.
In an embodiment, in order to improve the accuracy of the comparison result of the feature comparison model, when the feature comparison model is trained, the feature comparison model may also be trained by combining the features of the sample images of two different domains with the domain types output by the domain classification model.
Fig. 4 is a schematic flow chart of a face recognition method according to a first embodiment of the present invention.
The method comprises the following steps:
step S41: and acquiring characteristics of a plurality of groups of detection images of different domains.
And obtaining a plurality of groups of characteristics of the detection images of different domains, such as the characteristics of a visible light image, the characteristics of an infrared light image and the characteristics of a sketch image, through characteristic extraction.
Step S42: inputting the characteristics of the detection images of the plurality of groups of different domains into the corresponding comparison branch model, so as to perform characteristic comparison on the characteristics of the detection images of the different domains in the comparison branch model and obtain a comparison result.
Inputting the detected characteristics of the detection images of the two different domains into the corresponding comparison branch models, comparing the characteristics of the detection images of the two different domains in the corresponding comparison branch models, and obtaining comparison results.
Fig. 5 is a schematic flow chart of a face recognition method according to a second embodiment of the present invention.
Before step S41, the method further includes:
step S43: and acquiring a detection image.
A plurality of detection images are input, and the detection images comprise images of various different domains.
Step S44: and inputting the detection image into a domain classification model, so as to classify the detection image according to different domains in the domain classification model, and obtain a plurality of groups of domain category labels of the detection image of different domains.
And inputting the detection images into the domain classification model so as to classify the detection images according to different domain types in the domain classification model, thereby obtaining a plurality of groups of detection images with different domain types.
Step S45: and respectively inputting the multiple groups of detection images in different domains into corresponding human face feature extraction branch models to obtain the features of the multiple groups of detection images in different domains.
And respectively inputting the classified multiple groups of detection images of different domains into corresponding human face feature extraction branch models so as to extract the features of the detection images from the different human face feature extraction branch models. Specifically, if the domain type of the detected image is the visible light domain type, the detected image is input into a human face feature extraction branch model corresponding to the visible light domain type to extract features.
Fig. 6a is a schematic structural diagram of a heterogeneous face recognition model according to the present invention. The method comprises the following steps: a feature comparison model 63, a domain classification model 61 and a face feature extraction model 62. The domain classification model 61, the face feature extraction model 62 and the feature comparison model 63 are sequentially cascaded. The face feature extraction model 62 includes a plurality of domain type one-to-one corresponding feature extraction branch models, each for extracting features of sample graphics of a corresponding domain. The feature alignment model 63 includes a plurality of alignment branch models formed by two different domains. Each alignment branch model is used for aligning the characteristics of the sample images of two different domains.
Specifically, referring to fig. 6b, in an embodiment, the face feature extraction model 62 includes a face feature extraction branch model 621, a face feature extraction branch model 622, and a face feature extraction branch model 623, wherein the face feature extraction branch models 621, 622, and 623 respectively correspond to different domain types, and if the domain type of the detected image output by the domain classification model 61 is visible light, infrared ray, and sketch, the domain types respectively corresponding to the face feature extraction branch models 621, 622, and 623 are visible light, infrared ray, and sketch. The feature comparison model 63 includes a feature comparison branch model 631, a feature comparison branch model 632, and a feature comparison branch model 633, wherein the feature comparison branch models 631, 632, 633 are respectively compatible with two different domains for feature comparison. Specifically, the feature comparison branch model 631 supports comparison between the identity categories of the visible light domain and the infrared domain, the feature comparison branch model 632 supports comparison between the identity categories of the infrared domain and the sketch domain, and the feature comparison branch model 633 supports comparison between the identity categories of the visible light domain and the sketch domain.
In this embodiment, up to 3 domain types are listed, and in other embodiments, the domain types may also be 4 or 5. It can be understood that, in order to realize feature comparison of two different domains in the same feature comparison branch model, when there are k domains, there are 0.5 × k × (k-1) feature comparison branch models.
Fig. 7 is a schematic structural diagram of an intelligent device according to the present invention. The smart device comprises a memory 52 and a processor 51 and control circuitry 53 connected to each other.
The memory 52 is used for storing program instructions for implementing the training method of the heterogeneous face recognition model and the face recognition method based on the heterogeneous face recognition model in any one of the above.
The processor 51 is operative to execute program instructions stored in the memory 52.
The processor 51 may also be referred to as a CPU (Central Processing Unit). The processor 51 may be an integrated circuit chip having signal processing capabilities. The processor 51 may also be a general purpose processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), an off-the-shelf programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
The storage 52 may be a memory bank, a TF card, etc., and may store all information in the heterogeneous face recognition device, including the input original data, the computer program, the intermediate operation result, and the final operation result. It stores and retrieves information based on the location specified by the controller. With the memory, the heterogeneous face recognition device has a memory function, and can work normally. The memory of the heterogeneous face recognition apparatus may be classified into a main memory (internal memory) and an auxiliary memory (external memory) according to the purpose, and also into an external memory and an internal memory. The external memory is usually a magnetic medium, an optical disk, or the like, and can store information for a long period of time. The memory refers to a storage component on the main board, which is used for storing data and programs currently being executed, but is only used for temporarily storing the programs and the data, and the data is lost when the power is turned off or the power is cut off.
In the several embodiments provided in the present application, it should be understood that the disclosed method and apparatus may be implemented in other ways. For example, the above-described apparatus embodiments are merely illustrative, and for example, a division of a module or a unit is merely a logical division, and an actual implementation may have another division, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.
Units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the embodiment.
In addition, functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.
The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present application may be substantially implemented or contributed to by the prior art, or all or part of the technical solution may be embodied in a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a system server, a network device, or the like) or a processor (processor) to execute all or part of the steps of the method of the embodiments of the present application.
Please refer to fig. 8, which is a schematic structural diagram of a storage medium according to the present invention. The storage medium of the present application stores a program file 81 capable of implementing all the above-mentioned training methods for heterogeneous face recognition models and face recognition methods based on heterogeneous face recognition models, wherein the program file 81 may be stored in the storage medium in the form of a software product, and includes several instructions for enabling a computer device (which may be a personal computer, a server, or a network device, etc.) or a processor (processor) to execute all or part of the steps of the methods according to the embodiments of the present application. The aforementioned storage device includes: various media capable of storing program codes, such as a usb disk, a mobile hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, or terminal devices, such as a computer, a server, a mobile phone, and a tablet.
The above description is only an embodiment of the present invention, and not intended to limit the scope of the present invention, and all modifications of equivalent structures and equivalent processes, which are made by using the contents of the present specification and the accompanying drawings, or directly or indirectly applied to other related technical fields, are included in the scope of the present invention.
Claims (10)
1. A training method of a heterogeneous face recognition model is characterized in that the heterogeneous face recognition model comprises a feature comparison model; the feature comparison model comprises a plurality of comparison branch models formed by pairwise different domains, and the training method comprises the following steps:
sequentially acquiring a plurality of groups of sample images of two different domains; wherein the sample image is labeled with a domain type of the sample image;
and training the corresponding comparison branch model sequentially through the characteristics of the sample images of each group of the two different domains.
2. The training method according to claim 1, wherein the step of sequentially training the corresponding alignment branch model through the features of each group of the sample images of the two different domains comprises:
and sequentially training the corresponding comparison branch models of each group of the two sample images in different domains by using a cosine similarity measurement method.
3. The training method according to claim 1 or 2, wherein the heterogeneous face recognition model further comprises a domain classification model and a face feature extraction model, wherein the domain classification model, the face feature extraction model and the feature comparison model are cascaded in sequence; the feature extraction model comprises a plurality of feature extraction branch models corresponding to the domain types one to one;
the training method further comprises the following steps:
training the domain classification model through a convolution neural network method by using a sample image with a labeled domain type;
and training the feature extraction branch models of the feature extraction model respectively through the sample images marked with the identity categories.
4. The training method according to claim 3, wherein the domain type comprises a visible light type, and the step of sequentially training the feature extraction branch models of the feature extraction model by the sample images labeled with the identity categories comprises:
establishing a first feature extraction branch model corresponding to a visible light domain, and training the first feature extraction branch model through a sample image of a visible light domain type;
and establishing feature extraction branch models corresponding to other different domains based on the trained first feature extraction branch model, and sequentially training.
5. The training method of claim 3, wherein the step of training the domain classification model by the convolution neural network method through the sample image labeled with the domain type further comprises:
acquiring a sample image of a new annotation domain type;
and training the domain classification model through the sample image of the labeled domain type and the new sample image of the labeled domain type by a convolutional neural network method.
6. A face recognition method based on a heterogeneous face recognition model, characterized in that the face recognition model is obtained by the training method of the above claims 1-5; the face recognition method comprises the following steps:
acquiring characteristics of a plurality of groups of detection images in different domains;
inputting the characteristics of the detection images of the plurality of groups of different domains into the corresponding comparison branch model, so as to perform characteristic comparison on the characteristics of the detection images of the different domains in the comparison branch model and obtain a comparison result.
7. The method of claim 6, further comprising:
the acquiring features of the plurality of groups of detection images of different domains further comprises:
acquiring a detection image;
inputting the detection image into a domain classification model, so as to classify the detection image according to different domains in the domain classification model, and obtain a plurality of groups of domain category labels of the detection image of different domains;
and respectively inputting the multiple groups of detection images in different domains into corresponding human face feature extraction models to obtain the features of the multiple groups of detection images in different domains.
8. A heterogeneous face recognition device, comprising:
the domain classification model is used for carrying out domain classification on the detection images so as to obtain a plurality of groups of domain category labels of the detection images of different domains;
the human face feature extraction model comprises a plurality of human face feature extraction branch models, and each human face feature extraction branch model is used for extracting features of the detection images of the corresponding domain to obtain the features of a plurality of groups of detection images of different domains;
the characteristic comparison model comprises a plurality of characteristic comparison branch models, and each characteristic comparison branch model is used for comparing the characteristics of the detection graphs of two different domains and obtaining a comparison result.
9. A smart device, the smart device comprising: a control circuit, a processor and a memory coupled to each other, wherein,
the memory is used for storing program instructions for implementing a training method of the heterogeneous face recognition model according to any one of claims 1 to 5; and
the memory is used for storing program instructions for implementing the heterogeneous face recognition model-based face recognition method according to any one of claims 6 to 7;
the processor is configured to execute the program instructions stored by the memory.
10. A storage medium storing a program file executable to implement the training method of the heterogeneous face recognition model according to any one of claims 1 to 5; and
the program file can be executed to implement the heterogeneous face recognition model-based face recognition method according to any one of claims 6 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911067746.3A CN110781856B (en) | 2019-11-04 | 2019-11-04 | Heterogeneous face recognition model training method, face recognition method and related device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911067746.3A CN110781856B (en) | 2019-11-04 | 2019-11-04 | Heterogeneous face recognition model training method, face recognition method and related device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110781856A true CN110781856A (en) | 2020-02-11 |
| CN110781856B CN110781856B (en) | 2023-12-19 |
Family
ID=69388992
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911067746.3A Active CN110781856B (en) | 2019-11-04 | 2019-11-04 | Heterogeneous face recognition model training method, face recognition method and related device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110781856B (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111582066A (en) * | 2020-04-21 | 2020-08-25 | 浙江大华技术股份有限公司 | Heterogeneous face recognition model training method, face recognition method and related device |
| CN112052792A (en) * | 2020-09-04 | 2020-12-08 | 恒睿(重庆)人工智能技术研究院有限公司 | Cross-model face recognition method, device, equipment and medium |
| CN113486804A (en) * | 2021-07-07 | 2021-10-08 | 科大讯飞股份有限公司 | Object identification method, device, equipment and storage medium |
| CN113642481A (en) * | 2021-08-17 | 2021-11-12 | 百度在线网络技术（北京）有限公司 | Recognition method, training method, device, electronic equipment and storage medium |
Citations (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20100074479A1 (en) * | 2008-09-19 | 2010-03-25 | Altek Corpoartion | Hierarchical face recognition training method and hierarchical face recognition method thereof |
| CN104484650A (en) * | 2014-12-09 | 2015-04-01 | 北京信息科技大学 | Method and device for identifying sketch face |
| CN104598878A (en) * | 2015-01-07 | 2015-05-06 | 深圳市唯特视科技有限公司 | Multi-modal face recognition device and method based on multi-layer fusion of gray level and depth information |
| US20160035093A1 (en) * | 2014-07-31 | 2016-02-04 | California Institute Of Technology | Multi modality brain mapping system (mbms) using artificial intelligence and pattern recognition |
| CN105574525A (en) * | 2015-12-18 | 2016-05-11 | 天津中科智能识别产业技术研究院有限公司 | Method and device for obtaining complex scene multi-mode biology characteristic image |
| CN105608450A (en) * | 2016-03-01 | 2016-05-25 | 天津中科智能识别产业技术研究院有限公司 | Heterogeneous face identification method based on deep convolutional neural network |
| CN105701509A (en) * | 2016-01-13 | 2016-06-22 | 清华大学 | Image classification method based on cross-type migration active learning |
| CN108427939A (en) * | 2018-03-30 | 2018-08-21 | 百度在线网络技术（北京）有限公司 | model generating method and device |
| CN109192302A (en) * | 2018-08-24 | 2019-01-11 | 杭州体光医学科技有限公司 | A kind of face's multi-modality images acquisition processing device and method |
| CN109815801A (en) * | 2018-12-18 | 2019-05-28 | 北京英索科技发展有限公司 | Face identification method and device based on deep learning |
| CN109840475A (en) * | 2018-12-28 | 2019-06-04 | 深圳奥比中光科技有限公司 | Face identification method and electronic equipment |
| CN109934198A (en) * | 2019-03-22 | 2019-06-25 | 北京市商汤科技开发有限公司 | Face identification method and device |
| WO2019128646A1 (en) * | 2017-12-28 | 2019-07-04 | 深圳励飞科技有限公司 | Face detection method, method and device for training parameters of convolutional neural network, and medium |
| CN110046551A (en) * | 2019-03-18 | 2019-07-23 | 中国科学院深圳先进技术研究院 | A kind of generation method and equipment of human face recognition model |
-
2019
- 2019-11-04 CN CN201911067746.3A patent/CN110781856B/en active Active
Patent Citations (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20100074479A1 (en) * | 2008-09-19 | 2010-03-25 | Altek Corpoartion | Hierarchical face recognition training method and hierarchical face recognition method thereof |
| US20160035093A1 (en) * | 2014-07-31 | 2016-02-04 | California Institute Of Technology | Multi modality brain mapping system (mbms) using artificial intelligence and pattern recognition |
| CN104484650A (en) * | 2014-12-09 | 2015-04-01 | 北京信息科技大学 | Method and device for identifying sketch face |
| CN104598878A (en) * | 2015-01-07 | 2015-05-06 | 深圳市唯特视科技有限公司 | Multi-modal face recognition device and method based on multi-layer fusion of gray level and depth information |
| CN105574525A (en) * | 2015-12-18 | 2016-05-11 | 天津中科智能识别产业技术研究院有限公司 | Method and device for obtaining complex scene multi-mode biology characteristic image |
| CN105701509A (en) * | 2016-01-13 | 2016-06-22 | 清华大学 | Image classification method based on cross-type migration active learning |
| CN105608450A (en) * | 2016-03-01 | 2016-05-25 | 天津中科智能识别产业技术研究院有限公司 | Heterogeneous face identification method based on deep convolutional neural network |
| WO2019128646A1 (en) * | 2017-12-28 | 2019-07-04 | 深圳励飞科技有限公司 | Face detection method, method and device for training parameters of convolutional neural network, and medium |
| CN108427939A (en) * | 2018-03-30 | 2018-08-21 | 百度在线网络技术（北京）有限公司 | model generating method and device |
| CN109192302A (en) * | 2018-08-24 | 2019-01-11 | 杭州体光医学科技有限公司 | A kind of face's multi-modality images acquisition processing device and method |
| CN109815801A (en) * | 2018-12-18 | 2019-05-28 | 北京英索科技发展有限公司 | Face identification method and device based on deep learning |
| CN109840475A (en) * | 2018-12-28 | 2019-06-04 | 深圳奥比中光科技有限公司 | Face identification method and electronic equipment |
| CN110046551A (en) * | 2019-03-18 | 2019-07-23 | 中国科学院深圳先进技术研究院 | A kind of generation method and equipment of human face recognition model |
| CN109934198A (en) * | 2019-03-22 | 2019-06-25 | 北京市商汤科技开发有限公司 | Face identification method and device |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111582066A (en) * | 2020-04-21 | 2020-08-25 | 浙江大华技术股份有限公司 | Heterogeneous face recognition model training method, face recognition method and related device |
| CN111582066B (en) * | 2020-04-21 | 2023-10-03 | 浙江大华技术股份有限公司 | Heterogeneous face recognition model training method, face recognition method and related device |
| CN112052792A (en) * | 2020-09-04 | 2020-12-08 | 恒睿(重庆)人工智能技术研究院有限公司 | Cross-model face recognition method, device, equipment and medium |
| CN113486804A (en) * | 2021-07-07 | 2021-10-08 | 科大讯飞股份有限公司 | Object identification method, device, equipment and storage medium |
| CN113486804B (en) * | 2021-07-07 | 2024-02-20 | 科大讯飞股份有限公司 | Object identification method, device, equipment and storage medium |
| CN113642481A (en) * | 2021-08-17 | 2021-11-12 | 百度在线网络技术（北京）有限公司 | Recognition method, training method, device, electronic equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110781856B (en) | 2023-12-19 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Shivakumara et al. | CNN‐RNN based method for license plate recognition | |
| CN110781856A (en) | Heterogeneous face recognition model training method, face recognition method and related device | |
| EP3869385B1 (en) | Method for extracting structural data from image, apparatus and device | |
| CN107016387B (en) | Method and device for identifying label | |
| CN111242124B (en) | Certificate classification method, device and equipment | |
| CN111639648B (en) | Certificate identification method, device, computing equipment and storage medium | |
| EP4085369A1 (en) | Forgery detection of face image | |
| Zayene et al. | Multi‐dimensional long short‐term memory networks for artificial Arabic text recognition in news video | |
| CN112699775A (en) | Certificate identification method, device and equipment based on deep learning and storage medium | |
| CN112380978B (en) | Multi-face detection method, system and storage medium based on key point positioning | |
| CA3052846A1 (en) | Character recognition method, device, electronic device and storage medium | |
| CN113536856A (en) | Image recognition method and system, and data processing method | |
| CN113627395A (en) | Text recognition method, text recognition device, text recognition medium and electronic equipment | |
| CN112990172B (en) | Text recognition method, character recognition method and device | |
| Xu et al. | Multi‐pyramid image spatial structure based on coarse‐to‐fine pyramid and scale space | |
| CN112749576B (en) | Image recognition method and device, computing equipment and computer storage medium | |
| CN112396060B (en) | Identification card recognition method based on identification card segmentation model and related equipment thereof | |
| CN112396048B (en) | Picture information extraction method and device, computer equipment and storage medium | |
| CN111339910B (en) | Text processing and text classification model training method and device | |
| CN111539424A (en) | Image processing method, system, device and medium based on OCR | |
| CN113988223B (en) | Certificate image recognition method, device, computer equipment and storage medium | |
| CN109190467A (en) | A kind of more object detecting methods, system, terminal and storage medium returned based on key point | |
| CN113177543B (en) | Certificate identification method, device, equipment and storage medium | |
| CN115880702A (en) | Data processing method, device, equipment, program product and storage medium | |
| CN112395450B (en) | Picture character detection method and device, computer equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |