CN111553235B - Network training method for protecting privacy, identity recognition method and device - Google Patents

Network training method for protecting privacy, identity recognition method and device Download PDF

Info

Publication number
CN111553235B
CN111553235B CN202010323578.6A CN202010323578A CN111553235B CN 111553235 B CN111553235 B CN 111553235B CN 202010323578 A CN202010323578 A CN 202010323578A CN 111553235 B CN111553235 B CN 111553235B
Authority
CN
China
Prior art keywords
network
face
image
loss
face image
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202010323578.6A
Other languages
Chinese (zh)
Other versions
CN111553235A (en
Inventor
郭明宇
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Alipay Hangzhou Information Technology Co Ltd
Original Assignee
Alipay Hangzhou Information Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Alipay Hangzhou Information Technology Co Ltd filed Critical Alipay Hangzhou Information Technology Co Ltd
Priority to CN202010323578.6A priority Critical patent/CN111553235B/en
Publication of CN111553235A publication Critical patent/CN111553235A/en
Application granted granted Critical
Publication of CN111553235B publication Critical patent/CN111553235B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/168—Feature extraction; Face representation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/172—Classification, e.g. identification

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • General Physics & Mathematics (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • Computing Systems (AREA)
  • Software Systems (AREA)
  • Evolutionary Computation (AREA)
  • Computational Linguistics (AREA)
  • Molecular Biology (AREA)
  • Biophysics (AREA)
  • General Engineering & Computer Science (AREA)
  • Biomedical Technology (AREA)
  • Mathematical Physics (AREA)
  • Data Mining & Analysis (AREA)
  • Artificial Intelligence (AREA)
  • Multimedia (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Human Computer Interaction (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Image Analysis (AREA)
  • Collating Specific Patterns (AREA)

Abstract

The embodiment of the specification provides an identity recognition method for protecting privacy, a training method and a training device for generating an antagonistic network, and a real face image of a target object is obtained; the real face image is input into the trained generation countermeasure network, the fictitious face image with the appearance different from that of the real face image is generated through the trained generation countermeasure network, and the fictitious face image is sent to the server side, so that the server side carries out identity recognition on a target object based on identity characteristics extracted from the fictitious face image, the face image containing the real appearance of the user is prevented from being transmitted to the server side for identity recognition, and the purpose of protecting the privacy of the user is achieved.

Description

Network training method for protecting privacy, identity recognition method and device
Technical Field
The embodiment of the specification relates to the technical field of computers, in particular to a network training method, an identity recognition method and an identity recognition device for protecting privacy.
Background
After the fingerprint identification technology, the identity identification technology gradually becomes a new verification mode, and the technologies of face unlocking, face brushing payment, face brushing in and out and the like are filled in the lives of the contemporary people carelessly. The "non-contact" type operation is convenient and natural. However, the security of the identification technology needs to be ensured by other technologies, which may cause mass leakage of mass data and cause immeasurable damage if careless. Such as age estimation, gender identification, blood relationship discrimination, etc., based on the face image. Generally, our photos are used for training an identity recognition system without permission, and many users are infringed of the right of awareness and privacy without knowing the photos.
Disclosure of Invention
The embodiment of the specification provides a network training method for protecting privacy, an identity recognition method and an identity recognition device.
In a first aspect, an embodiment of the present specification provides a training method for generating a countermeasure network for protecting privacy, where the method includes: iteratively training the generating network to meet a first convergence condition, and iteratively training the discriminating network to meet a second convergence condition; wherein one iterative training of the generated network comprises: acquiring a current batch of real face image sets, wherein the current batch of real face image sets comprise more than one real face image sample; for each real face image sample in the current batch of real face image sets, inputting the real face image sample and random noise into the generation network to generate a current batch of generated image sets corresponding to the current batch of real face image samples; inputting the current batch of generated image sets into the discrimination network, calculating a first target loss according to a forward prediction result of each generated image sample in the current batch of generated image sets by the discrimination network, and optimizing network parameters of the generation network based on the first target loss, wherein the first target loss comprises a first identification loss.
In a second aspect, an embodiment of the present specification provides an identity identification method for protecting privacy, including: acquiring a real face image of a target object; inputting the real face image into a trained generation countermeasure network, and generating a fictitious face image with a different appearance from the real face image through the trained generation countermeasure network, wherein the fictitious face image and the real face image contain identity features larger than a preset similarity threshold; and sending the fictitious human face image to a server side, so that the server side extracts the identity characteristics of the fictitious human face image and identifies the target object based on the identity characteristics.
In a third aspect, an embodiment of the present specification provides an identity recognition system, including: the system comprises a client and a server, wherein the client stores a trained generation countermeasure network, and the server stores an identity characteristic extraction model; the client is used for acquiring a real face image of a target object and generating a fictitious face image with a different appearance from the real face image through the trained generation countermeasure network, wherein the fictitious face image and the real face image contain identity characteristics larger than a preset similarity threshold; the client side sends the fictitious human face image to a server side; the server is used for receiving the fictitious human face image and extracting the identity characteristics of the fictitious human face image through the identity characteristic extraction model, and the server is also used for carrying out identity recognition on the target object based on the identity characteristics.
In a fourth aspect, an embodiment of the present specification provides a training apparatus for generating a countermeasure network for protecting privacy, where the generating countermeasure network includes a generating network and a discriminating network, the apparatus includes: a generating network training module for iteratively training the generating network to meet a first convergence condition; a discriminant network training module, configured to iteratively train the discriminant network until a second convergence condition is satisfied, where the generated network training module includes: the real face image acquisition unit is used for acquiring a current batch of real face image sets, and the current batch of real face image sets comprise more than one real face image samples; an image generating unit, configured to input, to the generation network, the real face image samples and random noise for each real face image sample in the current batch of real face image sets, and generate a current batch of generated image sets corresponding to the current batch of real face image samples; a first parameter optimization unit, configured to input the current batch of generated image sets to the decision network, calculate a first target loss according to a forward prediction result of each generated image sample in the current batch of generated image sets by the decision network, and optimize network parameters of the generation network based on the first target loss, where the first target loss includes a first identification loss.
In a fifth aspect, an embodiment of the present specification provides an identity recognition apparatus for protecting privacy, including: a face image acquisition unit for acquiring a real face image of the target object; the face fictitious unit is used for inputting the real face image into a trained generation confrontation network and generating a fictitious face image with the appearance different from that of the real face image through the trained generation confrontation network, wherein the fictitious face image and the real face image contain identity characteristics larger than a preset similarity threshold; and the fictitious face sending unit is used for sending the fictitious face image to a server side, so that the server side extracts the identity characteristics of the fictitious face image and identifies the target object based on the identity characteristics.
In a sixth aspect, an embodiment of the present specification provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor, where the processor implements the method according to any one of the first aspect to the second aspect when executing the program.
In a seventh aspect, the present specification provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the method in any one of the first to second aspects is implemented.
The technical scheme provided by the embodiment of the specification at least realizes the following technical effects:
the fictitious face image with the appearance different from that of the real face image of the target object is generated by the generation countermeasure network and is high in similarity with the real face image, so that the server extracts the identity feature from the fictitious face image to identify the identity of the target object, and the real face image is not required to be faxed to the server by the client, so that the appearance privacy of the target object cannot be transmitted to the server, and the privacy information of the target object is protected, and the purpose of protecting the user privacy can be achieved.
And, because the extraction of the identity features is completed by the model stored in the server, the model for extracting the features does not need to be stored in the client. Therefore, the identity feature extraction model is stored in the server side, so that the model is convenient to optimize, and the data storage capacity of the client side is reduced.
Drawings
Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The drawings are only for purposes of illustrating the preferred embodiments and are not to be construed as limiting the specification. Also, like reference numerals are used to refer to like parts throughout the drawings. In the drawings:
FIG. 1 is a schematic diagram of an embodiment of a privacy preserving identity recognition system of the present description;
FIG. 2 is a schematic diagram of training for generating a network in an embodiment of the present disclosure;
FIG. 3 is a schematic diagram illustrating training of a discriminant network in an embodiment of the present disclosure;
FIG. 4 is a flow chart illustrating an embodiment of a privacy preserving identity recognition method;
FIG. 5 is a schematic structural diagram of a privacy-preserving generation countermeasure network training device in an embodiment of the present specification;
FIG. 6 is a schematic structural diagram of an identity recognition apparatus for protecting privacy in an embodiment of the present disclosure;
fig. 7 is a schematic structural diagram of an electronic device in an embodiment of the present specification.
Detailed Description
In order to better understand the technical solutions, the technical solutions of the embodiments of the present specification are described in detail below with reference to the drawings and specific embodiments, and it should be understood that the specific features of the embodiments and embodiments of the present specification are detailed descriptions of the technical solutions of the embodiments of the present specification, and are not limitations of the technical solutions of the present specification, and the technical features of the embodiments and embodiments of the present specification may be combined with each other without conflict.
In a first aspect, an embodiment of the present specification provides an identity recognition system for protecting privacy, fig. 1 is a system architecture diagram of the identity recognition system for protecting privacy in the embodiment of the present specification, and referring to fig. 1, the identity recognition system includes: a client 10 and a server 20. The client 10 stores the trained generation countermeasure network, and the server 20 stores the identity feature extraction model.
The client 10 is configured to acquire a real face image of a target object, and generate a fictitious face image with a different appearance from the real face image through the trained generation countermeasure network, where the fictitious face image and the real face image include an identity feature greater than a preset similarity threshold; the client 10 sends the fictitious facial image to a server 20; the server 20 is configured to receive the fictitious facial image and extract the identity feature of the fictitious facial image through the identity feature extraction model, and the server 20 is further configured to perform identity recognition on the target object based on the identity feature.
Specifically, the client 10 is a terminal device. The terminal device may be a mobile phone terminal, an access control device, an intelligent wearable device, a notebook computer, and the like, and the client 10 may be a payment application or a system application on the terminal device, such as an unlocking application, an address book application, a gallery application, a third party payment application, and the like, that need to be used after performing identity authentication. The access control device may be an intelligent door lock, an intelligent access control, etc., and the client 10 may be an application program on the access control device.
Specifically, the server 20 is a network-side device, so that the real face image of the target object does not need to be transmitted to the server 20, and in the whole recognition process, the server 20 does not obtain the face image containing the real face of the target object, but only obtains a face image with a fictional face, so that the face privacy of the target object is protected, and the purpose of protecting the user privacy is achieved.
In a first aspect, embodiments of the present specification provide a training method for generating a countermeasure network, which protects privacy. The GANS (generic adaptive Nets, generating countermeasure network) is a deep learning model, and specifically includes two network modules: a generation network (G) and a discriminant network (D). In the embodiment of the present specification, DCGAN (Deep Convolutional adaptive network) may be used.
In this embodiment of the present specification, an implementation process of training a generation countermeasure network for protecting privacy includes: the iterative training of the generation network and the iterative training of the discrimination network can be performed simultaneously or alternately. Hereinafter, the iterative training process of generating the network and the iterative training process of discriminating the network are described separately for easy understanding.
Referring to fig. 2, the training process for generating the network includes the following steps S201 to S206:
step S201, initializing the network parameters of the generated network.
And performing iterative training on the generated network after the network parameters are initialized so as to optimize the network parameters for generating the countermeasure network until the generated network meets the first convergence condition, thereby obtaining the trained generated network. The iterative training of the generated network comprises the following steps S202-S206, and the steps S202-S206 are repeatedly executed until the generated network meets the first convergence condition:
and step S202, acquiring a current batch of real human face image sets.
The method comprises the steps of obtaining a current batch of real face image sets from a pre-accurate training set, wherein the training set comprises a plurality of real face image samples. And acquiring a current batch of real face image sets from the training set, wherein the current batch of real face image sets comprise more than one real face image sample. And each real face image sample corresponds to label information. Specifically, the label information of each real face image sample includes N types of label values, where N is the number of two classification subnetworks included in the discrimination network.
Step S203, aiming at each real face image sample in the current batch of real face image sets, inputting the real face image samples and random noise into a generation network, and generating the current batch of generated image sets corresponding to the current batch of real face image samples.
The current batch of generated image set comprises more than one generated image sample, and the number of the samples is the same as that of the samples of the current batch of real human face image set. The label information of each generated image sample contains N types of label values, wherein N is the number of the two classification sub-networks contained in the discrimination network.
Next, a detailed description will be given of an implementation procedure for generating an image sample by generating a network output in step S203:
the ith real face image sample in the current batch of real face image set is recorded ask i Generating and real face image sample k i The corresponding random noise, which can be noted as Z. Real face image sample k i The generated random noise Z is input to the generation network, and a generated image sample with the random noise Z added thereto is output, which may be written as Xi = G (Z, ki).
Random noise Z and real face image sample k i The random noise Z is input into a generating network, and the generating countermeasure network performs multiple times of transposition convolution operation on the random noise Z so as to perform multiple times of upsampling on the random noise Z to obtain a noise image with a target size, wherein the noise image can be a gray image or a color image.
Noise image of target size and real face image sample k i Synthesizing to obtain a sample k of the real face image i Corresponding generated image sample X i . Or generating a confrontation network to the real face image sample k i And performing convolution operation for multiple times to obtain identity characteristics from downsampling of the real face image sample. And generating a countermeasure network, and executing transposition convolution operation according to the identity characteristics so as to perform up-sampling on the identity characteristics to obtain an identity characteristic image. And carrying out image synthesis on the noise image with the target size and the identity characteristic image to obtain a generated image sample Xi.
Specifically, the noise image of the target size may be synthesized with the real face image or the identity image by one of image synthesis methods, such as image superposition, image subtraction, image multiplication, image bitwise summation, image division, and the like.
For example, if the real face image sample is a 100 × 140 pixel picture, the target size noise image may be a 100 × 140 pixel gray scale image, or a smaller size gray scale image.
And step S204, inputting the current batch generated image set into a discrimination network, and calculating a first target loss according to a forward prediction result of each generated image sample in the current batch generated image set by the discrimination network.
In one embodiment, the discriminant network comprises three two-class subnetworks as follows: the system comprises a face discrimination sub-network, a face classification sub-network and an identity recognition sub-network, wherein based on the face discrimination sub-network, the forward prediction result of each generated image sample by the discrimination network comprises a face discrimination result obtained by the face discrimination sub-network, a face classification result obtained by the face classification sub-network and an identity recognition result obtained by the identity recognition sub-network; and calculating to obtain a first target loss according to the face discrimination result, the face classification result and the identity recognition result.
The label information corresponding to each real image sample and the label information of each generated image sample comprise: calculating face discrimination label values required by face discrimination loss, wherein the label value of a face is 1, and the label value of a non-face is 0; calculating a face classification label value required by face classification loss: the label value of the real face is 1, and the label value of the generated face is 0; calculating an identification tag value required for identification loss: the identification tag value is specifically a user ID. Of course, in the case that the first target loss includes other losses, the label information of each real face image sample, and the label information of each generated image sample also correspondingly include more types of label values. In terms of face loss, for example: calculating a gender discrimination label value required by gender discrimination loss, wherein the gender discrimination label value required by skin color discrimination loss is calculated by corresponding to different label values, and the calculation can be as follows: the label value for yellow skin is 1 and for non-yellow skin is 0.
It can be seen from the above description that the iterative training process for generating the countermeasure network is subject to multiple types of losses loss as constraint conditions, and the training process for generating the countermeasure network is not only subject to the constraints of identity recognition losses, but also subject to the constraints of other types of losses, so that the quality and the identifiability of the fictitious face image generated according to the real face image are enabled.
Next, for the case that the discrimination network includes a face discrimination sub-network, a face classification sub-network and an identity recognition sub-network, the implementation process of step S204 specifically includes the following steps S2041 to S2043:
s2041: and determining the face discrimination result of each generated image sample through a face discrimination subnetwork, and calculating to obtain a first face discrimination loss according to the face discrimination result of each generated image sample and the face discrimination label value of each generated image sample.
Specifically, the face discrimination subnetwork is a two-class network for performing real face image/generated face image discrimination, and can be denoted as D1. Based on the above, the face discrimination result of each generated image sample is determined, and the specific implementation process is as follows: and for each generated image sample, after the face discrimination sub-network extracts the face discrimination characteristics of the generated image sample, outputting the face discrimination probability value of the generated image sample according to the face discrimination characteristics, wherein the face discrimination probability value is the probability that the generated image sample is a real face image.
If the face discrimination sub-network is a binary network realized by a first Convolutional Neural Network (CNN), for each generated image sample, extracting a face discrimination feature through convolution operation of a hidden layer of the first Convolutional Neural network, outputting a face discrimination probability value of the generated image sample according to the face discrimination feature by using a logic function or a normalized exponential function (softmax function) through an output layer of the first Convolutional Neural network, wherein the face discrimination probability value is a probability value D1 (X) of an interval of 0-1 i )。
Then, the specific implementation process of calculating to obtain the first face discrimination loss is as follows: calculating a first face discrimination loss according to the face discrimination probability value of each generated image sample and the face discrimination label value of each generated image sample in the current batch of generated images, for example, calculating an L1 distance (manhattan distance) as the first face discrimination loss, which is denoted as loss1:
Figure GDA0003823904870000091
m is the number of samples of the current batch of generated image sets, y 1 (xi) To generate face discrimination probability values, f, of image samples Xi 1 (xi) The label values are discriminated for generating the faces of the image samples Xi.
S2042, determining a face classification result of each generated image sample through a face classification sub-network, and calculating to obtain a first face classification loss according to the face classification result of each generated image sample and the face classification label value of each generated image sample.
Specifically, the face classification sub-network is a two-classification network for performing face image/non-face image classification, and may be denoted as D2. Based on this, the specific implementation process for determining the face classification result of each generated image sample is as follows: and for each generated image sample, after the face classification characteristics of the generated image sample are extracted by the face classification sub-network, the face classification probability value of the generated image sample is output according to the face classification characteristics, and the face classification probability value represents the probability that the generated image sample is a face image.
If the face classification sub-network is a binary classification network realized by a second Convolutional Neural Network (CNN), for each generated image sample, performing convolution operation by a hidden layer of the second Convolutional Neural network to extract face classification features of the generated image sample, and outputting a face classification probability value of the generated image sample according to the face classification features by using a logic function or a normalized exponential function (softmax function) by an output layer of the second Convolutional Neural network, wherein the face classification probability value is a probability value D1 (X1) of an interval of 0-1 i )。
Then, the specific implementation process of calculating the loss of the first face classification is as follows: calculating a first face classification loss according to the face classification probability value of each generated image sample and the face classification label value of each generated image sample in the current batch of generated images, for example, calculating an L1 distance (manhattan distance) as the first face classification loss, which is denoted as loss2:
Figure GDA0003823904870000101
m is the number of samples of the current batch of generated image sets, y 2 (xi) To generate the face classification probability values for the image samples Xi, f2 (Xi) is the face classification label value for the generated image samples Xi.
S2043, determining the identification result of each generated image sample through an identification sub-network, and calculating to obtain a first identification loss according to the identification result of each generated image sample and the identification tag value of each generated image sample.
In the embodiment of the present specification, the identification sub-network is a two-class network for performing identification classification, and may be denoted as D3. Based on this, the specific implementation process for determining the identification result of each generated image sample is as follows: for each generated image sample, performing by means of an identity recognition subnetwork: and extracting a feature map of the generated image sample based on convolution operation, and outputting the identification probability value of the generated image sample by using the extracted feature map. If the identification sub-network is a binary network implemented by a third convolutional neural network. The feature map of the generated image sample is extracted through a hidden layer of the third convolutional neural network, and then the output layer of the third convolutional neural network outputs the identification probability value of the generated image sample according to the feature map, wherein the identification probability value can be a probability value D3 (Xi) of an interval of 0-1.
And calculating a first identification loss according to the identification probability value of each generated image sample and the identification label value of each generated image sample in the current batch of generated images, for example, calculating a cos distance (cosine similarity) as a first identification loss3.
It should be noted that steps S2041 to S2043 are independently executed steps, and are not executed in sequence in the implementation process. In the specific implementation process, the first face discrimination loss and the first face classification loss are not limited to be calculated by the L1 distance, the first identification loss is not limited to be calculated by the cos distance, and other objective functions can be constructed instead.
After obtaining the loss values in S2041 to S2043, the following step S2044 is executed: and calculating to obtain a first target loss according to the first face discrimination loss, the first face classification loss and the first identity identification. For example, the first target loss may be a sum or weighted sum of the first face discrimination loss, the first face classification loss, and the first identity recognition loss.
And S205, optimizing the network parameters of the generated network based on the first target loss.
And calculating a gradient value according to the first target loss by a gradient descent optimization method, and updating the network parameters of the generated network according to the gradient value. And performing iterative optimization on the network parameters of the generated network through the first target loss so as to continuously reduce the first target loss. After multiple iterative optimization, when the reduction value of the first target loss is smaller than a set first threshold value, representing that the generated network meets a preset convergence condition, and stopping optimizing network parameters of the generated network to obtain a trained generated network.
S206, judging whether the generated network after the current iteration optimization meets a first convergence condition; if yes, finishing iterative optimization of the generated network to obtain a trained generated network; otherwise, returning to step S202, taking the next batch of acquired real face image sets as the current batch of real face image sets, and entering the next iteration of generating the network.
Referring to fig. 3, the training process of the discriminant network includes the following steps S301 to S305:
s301, initializing and judging network parameters of the network.
After initializing the network parameters of the discriminant network, repeating the steps S302-S305 to iteratively optimize the network parameters of the discriminant network until the discriminant network satisfies the second convergence condition, so as to obtain a trained discriminant network:
and S302, inputting the current batch of real human face image sets and the current batch of generated image sets into a discrimination network.
And S303, calculating a second target loss together with the forward prediction result of each generated image sample in the current batch of generated image sets and the forward prediction result of each real face image sample in the current batch of real face images through a discrimination network, wherein the second target loss comprises a second identity recognition loss.
And S304, optimizing the network parameters of the discriminant network based on the second target loss.
S305, judging whether the judgment network after the current iteration optimization meets a second convergence condition; if yes, finishing the iterative optimization of the discrimination network to obtain a trained discrimination network; otherwise, returning to step S302, taking the obtained next batch of real face image sets as the current batch of real face image sets, correspondingly generating next batch of generated image sets as the current batch of generated image sets, and entering the next iteration of the discrimination network.
For example, a second face discrimination loss, a second face classification loss, and a second identity recognition loss are calculated according to forward prediction results of the discrimination network on a current batch of real face image sets and a current batch of generated image sets, and a sum or a weighted sum of the second face discrimination loss, the second face classification loss, and the second identity recognition loss is used as a second target loss.
In the specific implementation process, the iterative optimization process for discriminating the network is similar to the iterative optimization process for generating the network, and only the samples used in each iteration are different: and (4) carrying out iterative optimization on the discrimination network, wherein each batch of samples are obtained. For the sake of brevity of the description, the training process of the discriminant network is not described herein again.
With respect to the above steps S201 to S206, and steps 301 to 305, the network parameters of the generated network G may be optimized once after the network parameters of the discrimination network D are optimized once, or may be optimized once after the network parameters of the discrimination network are optimized several times.
In an alternative embodiment, in order to generate a face image of an imaginary face generated by the countermeasure network that is more like a real face, the discrimination network further includes more than one face classification sub-network. For example, one or more of a gender classification sub-network, an age classification sub-network, and a skin color classification sub-network. Based on the above, the method includes inputting the generated image sample to a discrimination network for generating a countermeasure network, and calculating a first target loss based on a forward prediction result of the discrimination network, and specifically includes:
s1, each of more than one sub-networks for personal classification generates an image sample in an image set aiming at the current batch, and determines the personal classification result of each generated image sample through a face personal discrimination sub-network.
And S2, calculating to obtain the target class personal loss according to the personal classification result of each generated image sample and the target class human face personal label value of each generated image sample.
And S3, calculating to obtain a first target loss according to the first face discrimination loss, the first face classification loss, the first identity recognition loss and the loss of each target type.
Similarly, when calculating the second target loss, the result of the personal classification of each personal classification sub-network may also be considered, and for the sake of brevity of the description, no further description is given here.
In a specific implementation process, in order to reduce the computation amount of the client, an iterative training process for generating the countermeasure network may be completed at the server, and the trained countermeasure network is deployed to each client. Of course, the client side can also optimize and generate the countermeasure network continuously in an iterative mode in the using process, so that the countermeasure network is optimized and generated online.
Through the generation of the countermeasure network after the training in the process, the real face image of the target object is input, and the fictitious face image with the appearance different from that of the real face image can be obtained. In the iterative training process of generating the countermeasure network, the real face image is used as the input of the generation network and is constrained based on the identification loss. Therefore, although the generated fictitious face image and the input real face image are visually different, highly similar identity characteristics can be obtained.
In a second aspect, an embodiment of the present specification provides a method for identity recognition with privacy protection, which is described with reference to fig. 1 and fig. 4, where fig. 4 is an interaction flowchart of the method for identity recognition with privacy protection provided in the embodiment of the present specification, and the method for identity recognition provided in the embodiment of the present specification includes the following steps:
s400, the client side obtains a real face image of the target object.
Specifically, the client acquires a real face image of the target object through a face image containing the target object acquired by the camera. Specifically, the face image of the target object may be a single face picture, or a video stream containing the target object.
Under different application scenes, the target object is correspondingly different. For example, in a face-brushing payment scene, the target object is a payer, in a face-brushing in-out scene, the target object is a current passing object to pass through the access control device, and in a face-brushing login scene, the target object is a user to log in to a target location (for example, a certain website or an APP). In addition, the target object may also be a certain user in a face brushing scene that needs to perform identity recognition, and for the purpose of this specification, the embodiments are not exhaustive.
S402, the client inputs the real face image into the trained generation countermeasure network, and generates a fictitious face image with the appearance different from that of the real face image through the trained generation countermeasure network, wherein the fictitious face image and the real face image contain identity characteristics larger than a preset similarity threshold value.
In a specific embodiment, a convolution operation is performed on a real face image of a target object in a trained countermeasure generation network, and identity features contained in the real face image are extracted. Specifically, the identity characteristic and the appearance characteristic of the target object in the embodiments of the present specification belong to different characteristic information.
Specifically, the implementation process of generating the fictitious human face image through the trained generation confrontation network specifically includes: carrying out convolution operation on the real face image and extracting identity characteristics; and performing transposition convolution operation according to the identity characteristics and the random noise respectively to correspondingly obtain a noise image and an identity characteristic image, and performing image synthesis on the identity characteristic image and the noise image to obtain a fictional face image. The compositing means may be, but is not limited to, image superposition, image subtraction, and the like.
Specifically, the trained generation network comprises a feature image generator, a noise image generator and a face image generator which is associated with the feature image generator and the noise image generator, wherein the feature image generator performs convolution operation on a real face image, extracts identity features, and performs transposition convolution operation according to the identity features to obtain an identity feature image with a target size. The noise image generator performs transposition convolution operation on input random noise to obtain a noise image with a target size; and the human face image generator is used for carrying out image synthesis according to the noise image with the target size and the identity characteristic with the target size to obtain a fictitious human face image.
Of course, only the real face image of the target object may be input to the trained generation countermeasure network, and the generation network outputs the fictitious face image for the real face image without the participation of random parameters.
In order to improve the accuracy of identity recognition, the fictitious human face image generated by the trained generation countermeasure network and the real human face image contain the same identity characteristics. Namely, the identity features extracted from the fictitious human face image generated by the generated countermeasure network are consistent with the identity features extracted from the real human face image, so as to ensure the accuracy of identity recognition of the target object according to the fictitious human face image.
In the iterative training process of generating the countermeasure network, the first identity recognition loss is used as a constraint condition for training the generation of the countermeasure network until the trained first identity recognition loss for generating the countermeasure network is minimum. Based on this, the virtual face image generated by the trained generation countermeasure network is visually different from the real face image. However, the identity features extracted from the fictive face image can be highly similar to the identity features of the real face image, but the fictive face image is visually different from the real appearance of the target object, for example, the gender is different, the age is different, the skin color is different, and the like. Therefore, the characteristic information of the target object, such as the sex information, the age information and the skin color information, is irrelevant to the identification of the target object, and even if the characteristic information is lacked, the identification of the target object is not influenced, so that the privacy of a user is protected to a certain extent in an identification scene.
For example, in the process of generating the fictive face image from the real face image by the trained generation countermeasure network, the presented and processed gender information, age information and skin color information are changed, for example, the real face image presents the real appearance of the target object, such as male, 25-35 years old and yellow skin, and the output fictive face image presents the appearance of female, 25-35 years old and black skin through the trained generation countermeasure network.
S404, the client sends the fictitious facial image to the server, so that after the server receives the fictitious facial image sent by the client, the identity characteristics of the fictitious facial image are extracted, and the target object is subjected to identity recognition based on the identity characteristics.
Specifically, the server side extracts the identity features from the fictitious human face image through an identity feature extraction network deployed on the server side, and the server side compares the identity features extracted from the fictitious human face image with the identity features of the target object obtained from a human face database so as to identify the identity of the target object.
In a specific implementation process, comparing the identity features extracted from the fictional face image with the identity features of the target object obtained from the face database to identify the target object, specifically comprising:
the identity feature extracted from the fictitious facial image is compared with the identity feature of each pre-stored object in the facial database (specifically, the pre-stored object in the facial database may be another fictitious facial image with a highly similar identity feature to the real facial image of the target object) to identify the identity of the target object. Specifically, identity features extracted from the fictive face image are compared with identity features of each object stored in the face database, so that a face comparison score between the fictive face image and each pre-stored object is determined, if a face comparison score larger than a preset score threshold exists (for example, the face comparison score is calculated by 0 to 100, the face comparison score is 100, the target object is identified, otherwise, the target object is not identified), it is determined that the identity recognition of the target object passes, the pre-stored object corresponding to the face comparison score larger than the preset score threshold in the face database represents the target object, and if the pre-stored object of the face comparison score larger than the preset score threshold does not exist in the face database, the identity recognition of the target object does not pass.
After the target object is identified by the identity features extracted from the fictive face image, the server 20 may perform subsequent steps according to the identification result of the target object, or feed the identification result back to the client 10, and the client 10 performs subsequent steps according to the identification result. The subsequent steps performed by the client 10 and the server 20 are different in different application scenarios, and are not limited in detail here. In the following embodiments, some specific implementations for performing the following steps based on the identification result are given.
In different application scenarios, the steps of the server after performing identity recognition on the target object based on the identity features extracted from the fictive face image are different, and several specific implementations in implementation scenarios are given below:
1. under the face-brushing payment scene, based on the privacy protection identity recognition method, the privacy protection face-brushing payment method is realized, and comprises the following steps:
step A1, responding to a face brushing payment triggering instruction, and acquiring a real face image of a payer.
Specifically, the face brushing payment triggering instruction may be initiated on a terminal device of the payee, for example, initiated by a face brushing machine of a merchant or a terminal connected to the face brushing machine. Based on this, the face brushing payment triggering instruction is triggered by the operation of the payee. The face-brushing payment triggering instruction may also be initiated on a terminal device of the payer, for example, on a client with a face-brushing payment function on the terminal device of the payer, based on which, the face-brushing payment triggering instruction is triggered by an operation of the payer.
Step A2, inputting a real face image into a trained generation confrontation network, and generating a fictitious face image with the appearance different from that of the real face image through the trained generation confrontation network, wherein the fictitious face image and the real face image contain identity characteristics larger than a preset similarity threshold value, and the fictitious face image does not contain privacy information of a target object;
and step A3, sending the fictitious facial image to a payment system, so that the payment system extracts the identity characteristics of the fictitious facial image and identifies the identity of the payer based on the identity characteristics.
And A4, determining whether to pay the payee of the payer according to the identification result of the payer, and feeding back the payment result to the payer and the payee.
Specifically, the payment system comprises an identity verification server and a payment platform, wherein the identity verification server extracts the identity characteristics of the fictitious human face image, identifies the payer based on the identity characteristics extracted from the fictitious human face image, generates a payment code comprising payer information, payee information and payment amount to the payment platform if the payer passes the identity identification, and the payment platform completes the transfer from the account of the payer to the account of the payee based on the received payment code and feeds back a message of successful payment to the payer and the payee. If the identity authentication server fails to identify the payer, the identity authentication server feeds back a message that the payment is unsuccessful to the payer and the payee.
2. Under the face-brushing login scene, based on the privacy protection identity recognition method, the privacy protection face-brushing login method is realized, and the method comprises the following steps:
and B1, responding to a face brushing login trigger instruction, and acquiring a real face image of the current user. Specifically, the face-brushing login triggering instruction may be triggered by an operation of the current user.
And B2, inputting the real face image into the trained generation countermeasure network, and generating a fictitious face image with the appearance different from that of the real face image through the trained generation countermeasure network, wherein the fictitious face image and the real face image contain identity characteristics larger than a preset similarity threshold, and the fictitious face image does not contain privacy information of the target object.
And step B3, sending the fictitious face image to a login authentication system, so that the login authentication system extracts the identity characteristics of the fictitious face image and identifies the identity of the current user based on the identity characteristics.
And step B4, judging whether the current user is allowed to log in the target position or not according to the identity recognition result of the current user. And if the identity identification result represents that the identity authentication of the current user passes, allowing the current user to be logged in to log in to the target position, otherwise, refusing the current user to be logged in to log in to the target position.
Specifically, the target location may be an application program supporting face-brushing login, a privacy protection space, a website, and the like.
3. Under the scene of face brushing access, based on the privacy protection identity recognition method, the privacy protection face brushing access method is realized, and is applied to access control equipment, and the face brushing access method comprises the following steps:
step C1, collecting real face images of more than one passing object at the access control equipment;
step C2, aiming at the real face image of each passing object, inputting the real face image into a trained generation countermeasure network, and generating a fictitious face image with the appearance different from that of the real face image through the trained generation countermeasure network, wherein the fictitious face image and the real face image contain identity characteristics larger than a preset similarity threshold;
step C3, sending the fictitious facial image to the server side, enabling the server side to extract the identity characteristics of the fictitious facial image, and carrying out identity recognition on the passing object based on the identity characteristics;
and C4, receiving the identification result fed back by the server and aiming at each passing object, and judging whether to open the access control equipment according to the identification result of each passing object. Specifically, the access control device may be an intelligent door lock or an intelligent access control. And if the identity identification result represents that the identity authentication of the passing object passes, opening the access control equipment, otherwise, not opening the access control equipment.
It should be noted that, in the face brushing entry and exit method, the face brushing login method, and the face brushing payment method for protecting privacy recited in the embodiments of the present specification, specific implementation details of the identity identification step may all refer to the description of the foregoing identity identification embodiment, and are not described herein again for brevity of the specification.
Through the technical scheme provided by the embodiment of the specification, the extraction of the identity characteristics is not required to be completed on the client, and the model for extracting the characteristics is not required to be stored on the client, so that the data storage capacity of the client is reduced. The extraction of the identity characteristics is completed by the server side, so that the model for extracting the identity characteristics is stored in the server side, and the continuous optimization of the model is facilitated. On the basis, the real face image does not need to be faxed to the server side by the client side, and only the uploaded fictitious face image with the identity characteristics highly similar to the real face image is uploaded, so that the privacy of the user is protected to a certain extent.
In a third aspect, based on the same inventive concept as the aforementioned training method for generating a countermeasure network for protecting privacy, embodiments of the present specification provide a training apparatus for generating a countermeasure network for protecting privacy, where the generation countermeasure network includes a generation network and a discrimination network, and as shown in fig. 5, the apparatus includes: a generating network training module 501 and a judging network training module 502, wherein the generating network training module 501 is used for iteratively training the generating network until a first convergence condition is met; the discriminant network training module 502 is configured to iteratively train the discriminant network until a second convergence condition is satisfied, where the generate network training module 501 includes:
the real face image acquisition unit 5011 is configured to acquire a current batch of real face image sets, where the current batch of real face image sets includes more than one real face image sample;
the image generation unit 5012 is configured to input, to the generation network, the real face image samples and random noise for each real face image sample in the current batch of real face image sets, and generate a current batch of generated image sets corresponding to the current batch of real face image samples;
the first parameter optimization unit 5013 is configured to input the current batch of generated image sets to the decision network, calculate a first target loss according to a forward prediction result of each generated image sample in the current batch of generated image sets by the decision network, and optimize network parameters of the generation network based on the first target loss, where the first target loss includes a first identification loss.
In an optional implementation manner, the discriminant network training module 502 includes:
the image input unit 5021 is used for inputting the current batch of real human face image sets into the discrimination network;
the forward prediction unit 5022 is used for calculating a second target loss according to a forward prediction result of the judgment network on the current batch of generated image sets and a forward prediction result of the judgment network on the current batch of real face images, wherein the second target loss comprises a second identity recognition loss;
a second parameter optimization unit 5023, configured to optimize the network parameters of the discrimination network according to the second target loss.
In an optional implementation manner, the image generating unit specifically includes:
a noise image generation subunit, configured to input random noise to the generation network, and generate a first noise image of a target size;
and the image synthesis subunit is used for carrying out image synthesis on the first noise image with the target size and the real face image to obtain a generated image sample corresponding to the real face image.
In an optional implementation manner, the discrimination network includes a face discrimination sub-network, a face classification sub-network, and an identity recognition sub-network, and the first parameter optimization unit includes:
the first calculating subunit is used for determining a face discrimination result of each generated image sample through the face discrimination subnetwork, and calculating to obtain a first face discrimination loss according to the face discrimination result of each generated image sample and the face discrimination label value of each generated image sample;
the second calculating subunit is used for determining a face classification result of each generated image sample through the face classification sub-network, and calculating to obtain a first face classification loss according to the face classification result of each generated image sample and the face classification label value of each generated image sample;
the third calculation subunit is used for determining the identity recognition result of each generated image sample through the identity recognition sub-network, and calculating to obtain a first identity recognition loss according to the identity recognition result of each generated image sample and the identity recognition tag value of each generated image sample;
and the loss totaling unit is used for calculating a first target loss according to the first face identification loss, the first face classification loss and the first identity identification loss.
In an optional implementation manner, the decision network further includes more than one sub-network for personal classification, and the first parameter optimization unit further includes:
the fourth calculating subunit is used for determining the personal classification result of each generated image sample in the face personal discrimination subnetwork aiming at each personal classification subnetwork in the more than one personal classification subnetworks, and calculating to obtain the target personal loss according to the personal classification result of each generated image sample and the target personal label value of each generated image sample;
and the loss summing unit is used for calculating to obtain a first target loss according to the first face identification loss, the first face classification loss, the first identity identification loss and the loss of each target class appearance.
The implementation details of each functional unit in the embodiment of the method for generating an anti-network training for privacy protection have been described in detail in the corresponding embodiment of the method, and are not described herein again for brevity of the description.
In a fourth aspect, based on the same inventive concept as the foregoing privacy protection identity recognition method, an embodiment of the present specification provides a privacy protection identity recognition apparatus, which is shown with reference to fig. 6 and includes:
a face image acquisition unit 601 configured to acquire a real face image of a target object;
a face fictitious unit 602, configured to input a real face image into a trained generative confrontation network, and generate a fictitious face image with a different appearance from the real face image through the trained generative confrontation network, where the fictitious face image and the real face image contain identity features greater than a preset similarity threshold;
and the fictitious face sending unit 603 is configured to send the fictitious face image to the server, so that the server extracts the identity feature of the fictitious face image, and performs identity recognition on the target object based on the identity feature.
In an optional implementation manner, the face reconstruction unit 602 includes:
extracting identity characteristics from the real face image by generating a countermeasure network, generating the countermeasure network, and generating an identity characteristic image according to the identity characteristics by generating the countermeasure network;
generating a second noise image from the second random noise by generating a countermeasure network;
and synthesizing the identity characteristic image and the second noise image to obtain a fictitious human face image.
The implementation details of each functional unit in the embodiment of the identity recognition apparatus for protecting privacy have been described in detail in the corresponding method embodiment, and are not repeated herein for brevity of the description.
In a fifth aspect, based on the same inventive concept, an embodiment of the present specification further provides an electronic device, configured to implement the foregoing training method for generating a countermeasure network for protecting privacy, or to interact with a server (a payment system, a verification server) to implement the foregoing method for identifying an identity for protecting privacy. As shown in fig. 7, the electronic device comprises a memory 704, a processor 702 and a computer program stored on the memory 704 and operable on the processor 702, wherein the processor 702 executes the program to implement the steps of any of the above-described identification methods.
Where in fig. 7 a bus architecture (represented by bus 700) is shown, bus 700 may include any number of interconnected buses and bridges, and bus 700 links together various circuits including one or more processors, represented by processor 702, and memory, represented by memory 704. The bus 700 may also link together various other circuits such as peripherals, voltage regulators, power management circuits, and the like, which are well known in the art, and therefore, will not be described any further herein. A bus interface 706 provides an interface between the bus 700 and the receiver 701 and transmitter 703. The receiver 701 and the transmitter 703 may be the same element, i.e., a transceiver, providing a means for communicating with various other apparatus over a transmission medium. The processor 702 is responsible for managing the bus 700 and general processing, and the memory 704 may be used for storing data used by the processor 702 in performing operations.
In a fifth aspect, based on the same invention, this specification further provides a computer-readable storage medium, on which a computer program is stored, where the computer program, when executed by a processor, implements the steps of any one of the foregoing privacy-protecting identity recognition method and the privacy-protecting training method for generating an anti-network.
The description has been presented with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the description. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
While preferred embodiments of the present specification have been described, additional variations and modifications in those embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. Therefore, it is intended that the appended claims be interpreted as including preferred embodiments and all changes and modifications that fall within the scope of the specification.
It will be apparent to those skilled in the art that various changes and modifications may be made in the present specification without departing from the spirit and scope of the specification. Thus, if such modifications and variations of the present specification fall within the scope of the claims of the present specification and their equivalents, the specification is intended to include such modifications and variations.

Claims (17)

1. A training method of a privacy-preserving generative confrontation network, the generative confrontation network including a generative network and a discriminative network, the method comprising: iteratively training the generating network to meet a first convergence condition, and iteratively training the discriminating network to meet a second convergence condition; wherein one iterative training of the generated network comprises:
acquiring a current batch of real face image sets, wherein the current batch of real face image sets comprise more than one real face image sample;
inputting the real face image samples and random noise into the generation network aiming at each real face image sample in the current batch of real face image sets to generate a current batch of generated image sets corresponding to the current batch of real face image samples;
inputting the current batch of generated image sets into the discrimination network, calculating a first target loss according to a forward prediction result of each generated image sample in the current batch of generated image sets by the discrimination network, and optimizing network parameters of the generation network based on the first target loss;
the first target loss comprises a first identification loss, the discrimination network comprises an identification sub-network, the identification result of each generated image sample is determined through the identification sub-network, the first identification loss is calculated according to the identification result of each generated image sample and the identification label value of each generated image sample, and the first identification loss is used as a constraint condition for training the generation of the countermeasure network.
2. The method of claim 1, one iteration training of the discriminative network, comprising:
inputting the current batch of real human face image sets to the discrimination network;
calculating a second target loss according to the forward prediction result of the discrimination network on the current batch of generated image sets and the forward prediction result of the discrimination network on the current batch of real face images, wherein the second target loss comprises a second identification loss, and the method comprises the following steps: calculating a second face discrimination loss, a second face classification loss and a second identity recognition loss according to forward prediction results of the discrimination network on the current batch of real face image sets and the current batch of generated image sets, and taking the sum or weighted sum of the second face discrimination loss, the second face classification loss and the second identity recognition loss as a second target loss;
optimizing the network parameters of the discrimination network according to the second target loss;
and judging whether the judgment network meets a second convergence condition, if so, obtaining a trained judgment network, otherwise, returning to the step of inputting the current batch of real human face image sets into the judgment network.
3. The method of claim 1, wherein inputting the real face image samples and random noise to the generation network comprises:
inputting the random noise into the generation network to generate a first noise image with a target size;
and carrying out image synthesis on the first noise image with the target size and the real face image to obtain a generated image sample corresponding to the real face image.
4. The method of claim 1, wherein the discriminating network further comprises a face classification sub-network and an identification sub-network, the inputting the current batch of generated image samples into the discriminating network, the calculating a first target loss based on a forward prediction result of the discriminating network for each generated image sample in the current batch of generated image samples, comprises:
determining a face discrimination result of each generated image sample through the face discrimination subnetwork, and calculating to obtain a first face discrimination loss according to the face discrimination result of each generated image sample and the face discrimination label value of each generated image sample;
determining a face classification result of each generated image sample through the face classification sub-network, and calculating to obtain a first face classification loss according to the face classification result of each generated image sample and the face classification label value of each generated image sample;
and calculating to obtain the first target loss according to the first face discrimination loss, the first face classification loss and the first identity recognition loss.
5. The method of claim 4, the discriminatory network further comprising more than one facies classification sub-network, the calculating a first target loss from the discriminatory network for forward predictions of each generated image sample in the current set of batch generated images, comprising:
for each of the more than one sub-networks, determining a face classification result of each generated image sample in the face discrimination sub-network, and calculating to obtain a target face loss according to the face classification result of each generated image sample and a target face label value of each generated image sample;
and calculating to obtain the first target loss according to the first face discrimination loss, the first face classification loss, the first identity recognition loss and each target type appearance loss.
6. An identity recognition method for protecting privacy comprises the following steps:
acquiring a real face image of a target object;
inputting the real face image into a trained generative confrontation network, and generating a fictitious face image with a different appearance from the real face image through the trained generative confrontation network, wherein the fictitious face image and the real face image contain identity features larger than a preset similarity threshold, and the trained generative confrontation network is trained through the method of any one of claims 1 to 5;
and sending the fictitious human face image to a server side, so that the server side extracts the identity characteristics of the fictitious human face image and identifies the target object based on the identity characteristics.
7. The method of claim 6, wherein generating, by the trained generative confrontation network, a fictitious facial image of a different appearance than the real facial image comprises:
extracting identity characteristics from the real face image through the generation countermeasure network, generating an identity characteristic image according to the identity characteristics through the generation countermeasure network and the generation countermeasure network;
generating a second noise image from a second random noise by the generating countermeasure network;
and synthesizing the identity characteristic image and the second noise image to obtain the fictitious human face image.
8. A privacy preserving identity recognition system comprising: the system comprises a client and a server, wherein the client stores a trained generation countermeasure network, and the server stores an identity characteristic extraction model;
the client is used for acquiring a real face image of a target object and generating a fictitious face image with a different appearance from the real face image through the trained generation countermeasure network, wherein the fictitious face image and the real face image contain identity characteristics larger than a preset similarity threshold; the client sends the fictitious facial image to a server, and the trained generative confrontation network is obtained by training according to the method of any one of claims 1 to 5;
the server is used for receiving the fictitious human face image and extracting the identity characteristics of the fictitious human face image through the identity characteristic extraction model, and the server is also used for carrying out identity recognition on the target object based on the identity characteristics.
9. A privacy preserving generative confrontation network training apparatus, the generative confrontation network including a generative network and a discriminative network, the apparatus comprising: a generating network training module for iteratively training the generating network to meet a first convergence condition; a discriminant network training module, configured to iteratively train the discriminant network until a second convergence condition is satisfied, where the generated network training module includes:
the real face image acquisition unit is used for acquiring a current batch of real face image sets, and the current batch of real face image sets comprise more than one real face image samples;
an image generating unit, configured to input, to the generation network, the real face image samples and random noise for each real face image sample in the current batch of real face image sets, and generate a current batch of generated image sets corresponding to the current batch of real face image samples;
a first parameter optimization unit, configured to input the current batch of generated image sets to the decision network, calculate a first target loss according to a forward prediction result of each generated image sample in the current batch of generated image sets by the decision network, and optimize network parameters of the generation network based on the first target loss, where the first target loss includes a first identification loss, the decision network includes an identification sub-network, determine an identification result of each generated image sample by the identification sub-network, calculate the first identification loss according to the identification result of each generated image sample and an identification tag value of each generated image sample, and the first identification loss serves as a constraint condition for training a generation countermeasure network.
10. The apparatus of claim 9, the discriminative network training module comprising:
the image input unit is used for inputting the current batch of real human face image sets to the discrimination network;
a forward prediction unit, configured to calculate a second target loss according to a forward prediction result of the decision network on the current batch of generated image sets and a forward prediction result of the decision network on the current batch of real face images, where the second target loss includes a second identification loss, and the method includes: calculating a second face discrimination loss, a second face classification loss and a second identity recognition loss according to forward prediction results of the discrimination network on the current batch of real face image sets and the current batch of generated image sets, and taking the sum or weighted sum of the second face discrimination loss, the second face classification loss and the second identity recognition loss as a second target loss;
and the second parameter optimization unit is used for optimizing the network parameters of the judgment network according to the second target loss.
11. The apparatus according to claim 9, wherein the image generating unit specifically includes:
a noise image generation subunit, configured to input the random noise to the generation network, and generate a first noise image of a target size;
and the image synthesis subunit is used for carrying out image synthesis on the first noise image with the target size and the real face image to obtain a generated image sample corresponding to the real face image.
12. The apparatus of claim 9, the discriminating network comprising a face discriminating subnetwork, a face classifying subnetwork, and an identity recognition subnetwork, the first parameter optimization unit comprising:
a first calculating subunit, configured to determine a face discrimination result of each generated image sample through the face discrimination sub-network, and calculate a first face discrimination loss according to the face discrimination result of each generated image sample and the face discrimination label value of each generated image sample;
the second calculating subunit is configured to determine a face classification result of each generated image sample through the face classification sub-network, and calculate a first face classification loss according to the face classification result of each generated image sample and the face classification label value of each generated image sample;
the third calculating subunit is used for determining the identification result of each generated image sample through the identification sub-network, and calculating the first identification loss according to the identification result of each generated image sample and the identification label value of each generated image sample;
and the loss totaling unit is used for calculating the first target loss according to the first face identification loss, the first face classification loss and the first identity identification loss.
13. The apparatus of claim 12, the discrimination network further comprising one or more personal classification subnetworks, the first parameter optimization unit further comprising:
a fourth calculating subunit, configured to determine, for each of the at least one sub-networks, a face classification result of each generated image sample through the face-face discrimination sub-network, and calculate a target-class face loss according to the face classification result of each generated image sample and a target-class face label value of each generated image sample;
the loss totaling unit is configured to calculate the first target loss according to the first face identification loss, the first face classification loss, the first identity identification loss, and each target-class appearance loss.
14. An identity recognition device for privacy protection, comprising:
the face image acquisition unit is used for acquiring a real face image of the target object;
a face fictitious unit, configured to input the real face image into a trained generative confrontation network, and generate a fictitious face image with a different appearance from the real face image through the trained generative confrontation network, wherein the fictitious face image and the real face image contain identity features larger than a preset similarity threshold, and the trained generative confrontation network is trained based on the method of any one of claims 1 to 5;
and the fictitious face sending unit is used for sending the fictitious face image to a server side, so that the server side extracts the identity characteristics of the fictitious face image and identifies the target object based on the identity characteristics.
15. The apparatus of claim 14, the face reconstruction unit comprising:
extracting identity characteristics from the real face image through the generation countermeasure network, generating an identity characteristic image according to the identity characteristics through the generation countermeasure network and the generation countermeasure network;
generating a second noise image from a second random noise by the generating countermeasure network;
and synthesizing the identity characteristic image and the second noise image to obtain the fictitious human face image.
16. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the steps of the method of any one of claims 1-7 when executing the program.
17. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the steps of the method of any one of claims 1 to 7.
CN202010323578.6A 2020-04-22 2020-04-22 Network training method for protecting privacy, identity recognition method and device Active CN111553235B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010323578.6A CN111553235B (en) 2020-04-22 2020-04-22 Network training method for protecting privacy, identity recognition method and device

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010323578.6A CN111553235B (en) 2020-04-22 2020-04-22 Network training method for protecting privacy, identity recognition method and device

Publications (2)

Publication Number Publication Date
CN111553235A CN111553235A (en) 2020-08-18
CN111553235B true CN111553235B (en) 2023-04-07

Family

ID=72001179

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010323578.6A Active CN111553235B (en) 2020-04-22 2020-04-22 Network training method for protecting privacy, identity recognition method and device

Country Status (1)

Country Link
CN (1) CN111553235B (en)

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113657350B (en) * 2021-05-12 2024-06-14 支付宝(杭州)信息技术有限公司 Face image processing method and device
CN113343878A (en) * 2021-06-18 2021-09-03 北京邮电大学 High-fidelity face privacy protection method and system based on generation countermeasure network
CN114140823A (en) * 2021-07-27 2022-03-04 支付宝(杭州)信息技术有限公司 Privacy-protecting biometric feature recognition method and device
CN114120412B (en) * 2021-11-29 2022-12-09 北京百度网讯科技有限公司 Image processing method and device

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108846350A (en) * 2018-06-08 2018-11-20 江苏大学 Tolerate the face identification method of change of age
CN109635745A (en) * 2018-12-13 2019-04-16 广东工业大学 A method of Multi-angle human face image is generated based on confrontation network model is generated
CN109815928A (en) * 2019-01-31 2019-05-28 中国电子进出口有限公司 A kind of face image synthesis method and apparatus based on confrontation study
CN110309861A (en) * 2019-06-10 2019-10-08 浙江大学 A kind of multi-modal mankind's activity recognition methods based on generation confrontation network
CN110532871A (en) * 2019-07-24 2019-12-03 华为技术有限公司 The method and apparatus of image procossing
CN110675312A (en) * 2019-09-24 2020-01-10 腾讯科技（深圳）有限公司 Image data processing method, image data processing device, computer equipment and storage medium

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10474880B2 (en) * 2017-03-15 2019-11-12 Nec Corporation Face recognition using larger pose face frontalization

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108846350A (en) * 2018-06-08 2018-11-20 江苏大学 Tolerate the face identification method of change of age
CN109635745A (en) * 2018-12-13 2019-04-16 广东工业大学 A method of Multi-angle human face image is generated based on confrontation network model is generated
CN109815928A (en) * 2019-01-31 2019-05-28 中国电子进出口有限公司 A kind of face image synthesis method and apparatus based on confrontation study
CN110309861A (en) * 2019-06-10 2019-10-08 浙江大学 A kind of multi-modal mankind's activity recognition methods based on generation confrontation network
CN110532871A (en) * 2019-07-24 2019-12-03 华为技术有限公司 The method and apparatus of image procossing
CN110675312A (en) * 2019-09-24 2020-01-10 腾讯科技（深圳）有限公司 Image data processing method, image data processing device, computer equipment and storage medium

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
基于约束性循环一致生成对抗网络的人脸表情识别方法;胡敏等;《电子测量与仪器学报》;20190415(第04期);全文 *

Also Published As

Publication number Publication date
CN111553235A (en) 2020-08-18

Similar Documents

Publication Publication Date Title
CN111553235B (en) Network training method for protecting privacy, identity recognition method and device
CN113688855B (en) Data processing method, federal learning training method, related device and equipment
US11995155B2 (en) Adversarial image generation method, computer device, and computer-readable storage medium
CN108921061B (en) Expression recognition method, device and equipment
CN112784670A (en) Object detection based on pixel differences
CN111310705A (en) Image recognition method and device, computer equipment and storage medium
CN112395979A (en) Image-based health state identification method, device, equipment and storage medium
CN114241459B (en) Driver identity verification method and device, computer equipment and storage medium
CN112766366A (en) Training method for resisting generation network and image processing method and device thereof
CN110728188A (en) Image processing method, device, system and storage medium
CN110675252A (en) Risk assessment method and device, electronic equipment and storage medium
CN111353514A (en) Model training method, image recognition method, device and terminal equipment
CN115905605A (en) Data processing method, data processing equipment and computer readable storage medium
CN114036553A (en) K-anonymity-combined pedestrian identity privacy protection method
CN116152938A (en) Method, device and equipment for training identity recognition model and transferring electronic resources
CN115953849A (en) Training method of in-vivo detection model, in-vivo detection method and system
CN115880530A (en) Detection method and system for resisting attack
CN117011909A (en) Training method of face recognition model, face recognition method and device
CN114612989A (en) Method and device for generating face recognition data set, electronic equipment and storage medium
CN114612991A (en) Conversion method and device for attacking face picture, electronic equipment and storage medium
CN114004265A (en) Model training method and node equipment
Mowla et al. Selective fuzzy ensemble learner for cognitive detection of bio-identifiable modality spoofing in MCPS
CN115082873A (en) Image recognition method and device based on path fusion and storage medium
CN113849665A (en) Multimedia data identification method, device, equipment and storage medium
CN114944950A (en) Real-name authentication method and device

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
REG Reference to a national code

Ref country code: HK

Ref legal event code: DE

Ref document number: 40036349

Country of ref document: HK

GR01 Patent grant
GR01 Patent grant