CN115713680A - Semantic guidance-based face image identity synthesis method - Google Patents
Semantic guidance-based face image identity synthesis method Download PDFInfo
- Publication number
- CN115713680A CN115713680A CN202211451581.1A CN202211451581A CN115713680A CN 115713680 A CN115713680 A CN 115713680A CN 202211451581 A CN202211451581 A CN 202211451581A CN 115713680 A CN115713680 A CN 115713680A
- Authority
- CN
- China
- Prior art keywords
- layer
- inputting
- feature
- attribute
- sampling
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Landscapes
- Image Analysis (AREA)
Abstract
A face image identity synthesis method based on semantic guidance comprises the steps of extracting identity information, attribute information and background information of each image, fusing the information in a feature fusion mode, and finally obtaining a final result from the fused information in an image generation mode. The method introduces feature key points for guiding face shape changes. Meanwhile, the generated face image with the face shape changed and stable quality is obtained by adding the background information in the training process.
Description
Technical Field
The invention relates to the field of image-level depth counterfeiting, in particular to a face image identity synthesis method based on semantic guidance.
Background
In recent years, with the breakthrough development of machine learning and graphics technology, the field of deep counterfeiting has been advanced greatly, and the synthesis direction of human face identity in the sub-direction of the field has been developed rapidly, resulting in more and more counterfeit images and videos appearing on the network. Specifically, the face identity synthesis technology is to convert the identity information of a source face to a target face through a reasonable technology, and simultaneously, attribute information (information such as background, posture, illumination and the like) of the target face in an image is not damaged. At present, human face identity synthesis is widely applied to various fields of information protection, the film and television industry, virtual entertainment and the like, advanced equipment is utilized in the film and television industry to reconstruct a facial model of an actor, illumination conditions of a scene are reconstructed, and a vivid effect can be obtained. Compared with the directions of attribute editing, image restoration and the like in the field of deep counterfeiting, the face identity synthesis is more open, and meanwhile, more innovative technologies in generating models are involved.
The traditional research of the face identity synthesis direction is mainly based on an image editing mode, and the method can be divided into two categories, namely a face image analysis and fusion mode and a 3D face modeling mode. The first traditional image editing mode needs to manually analyze the face area, and carries out face fusion through rendering, deformation and other modes, and the mode is not efficient, and consumes a great deal of time and energy. The second method needs to acquire a 3D face image of the face image, and combines a deep learning technique to generate an image, which may cause a problem of lack of illumination and background. In addition, these generation methods are less concerned about the structure of the face, resulting in a face shape problem in the generated face image.
Disclosure of Invention
In order to overcome the defects of the technology, the invention provides a face image identity synthesis method which is used for firstly semantically guiding feature key points of face shape change, then extracting identity information, attribute information and background information from an image, then fusing the information in a feature fusion mode, and finally generating an image from the fused information.
The technical scheme adopted by the invention for overcoming the technical problems is as follows:
a face image identity synthesis method based on semantic guidance comprises the following steps:
a) Extracting key points of the face image from all face images in the CelebA face image data set;
b) Establishing a PET key point adjusting network, inputting the key points of the face image into the PET key point adjusting network to obtain the characteristic key points lm fake For the feature key point lm fake Iteration is carried out to obtain optimized characteristic key points lm fake ;
c) Establishing a facial image feature extraction network, and concentrating CelebA facial image data into a source image Pic s And a target image Pic t Inputting the image data into a face image feature extraction network, and respectively outputting to obtain identity features F id And attribute feature F attr ;
d) Establishing a background feature extraction network and extracting a target image Pic t Inputting the background feature information into a background feature extraction network to obtain background feature information F bg ;
e) Establishing a generating network, and identifying the identity F id Attribute feature F attr Background feature information F bg And optimized feature key point lm fake Inputting the image into a generation network to obtain a face image Pic fake For picture Pic fake Iteration is carried out to obtain an optimized face image Pic fake ;
f) Repeating the steps b) to e) to obtain a real face image Pic with a changed face contour fake 。
Further, step a) comprises the following steps:
a-1) detecting key points of all face images in the CelebA face image data set by using a face key point detection algorithm H3R, and obtaining a source image Pic in the CelebA face image data set s The key points extracted in (1) are expressed as source key points lm s Target image Pic from CelebA face image dataset t The key points extracted in (1) are expressed as source key points lm t 。
Further, step b) comprises the following steps:
b-1) creation by the Source encoder E lms Target encoder E lmt Key point generator G lm And a similarity discriminator D S And true and false discriminator D TF Forming a PET key point adjusting network;
b-2) Source encoder E lms Comprises a first down-sampling convolutional layer, a second down-sampling convolutional layer, a third down-sampling convolutional layer, a fourth down-sampling convolutional layer and a fifth sampling convolutional layer, and the source key point lm is s Input to a source encoder E lms In the first downsampling convolutional layer, outputting the obtained characteristic informationFeature informationInputting the data into a second downsampled convolutional layer, and outputting the characteristic informationFeature informationInputting the data into a third downsampled convolutional layer, and outputting the characteristic informationFeature informationInputting the data into a fourth downsampled convolutional layer, and outputting the characteristic informationFeature informationInputting the data into a fifth downsampled convolutional layer, and outputting the characteristic information
b-3) target encoder E lmt Comprises a first full connection layer, a second full connection layer, a third full connection layer, a fourth full connection layer and a fifth full connection layer, and a source key point lm t Input to a target encoder E lmt In the first full connection layer, the characteristic information is outputFeature informationInputting the data into a second full connection layer, and outputting the data to obtain characteristic informationFeature informationInputting the data into a third full connection layer, and outputting the data to obtain characteristic informationFeature informationInputting the data into a fourth full connection layer, and outputting the data to obtain characteristic informationFeature informationInputting the data into the fifth connection layer, and outputting the obtained characteristic information
Cat () function will feature informationAnd characteristic informationStacking to obtain feature vectors
b-5) Key Point Generator G lm Composed of a first up-sampling convolutional layer, a second up-sampling convolutional layer, a third up-sampling convolutional layer, a fourth up-sampling convolutional layer and a fifth up-sampling convolutional layer, and the feature vectorInput to the keypoint generator G lm In the first upsampled convolutional layer, outputting to obtain the characteristic key pointFeature key pointsInputting the data into a second upsampling convolutional layer, and outputting to obtain a characteristic key pointFeature key pointsInputting the data into a third upsampling convolutional layer, and outputting to obtain characteristic key pointsFeature key pointsInputting the data into a fourth upsampling convolutional layer, and outputting to obtain characteristic key pointsFeature key pointsInputting the data into a fifth upsampling convolutional layer, and outputting to obtain a characteristic key point lm fake ;
b-6) similarity discriminator D S By Layer s Module, layer fake Module, layer c Modular construction, layer fake The module consists of a first full connection layer, a second full connection layer, a third full connection layer and a fourth full connection layer, and a characteristic key point lm fake Input to Layer fake In the first full connection layer of the module, outputting the characteristic informationFeature informationInput to Layer fake In the second full connection layer of the module, outputting the characteristic informationFeature informationInput to Layer fake In the third full connection layer of the module, outputting the obtained characteristic informationFeature informationInput to Layer fake In the fourth full connection layer of the module, outputting the characteristic informationLayer s The module consists of a first full connecting layer, a second full connecting layer, a third full connecting layer and a fourth full connecting layer, and a source key point lm s Input to Layer s The module outputs the characteristic information from the first full connection layerFeature informationInput to Layer s The module outputs the characteristic information from the second full connection layerFeature informationInput to Layer s The module outputs the characteristic information from the third full connection layerFeature informationInput to Layer s The module outputs the characteristic information from the fourth full connection layerCat () function is used to transfer feature informationAnd characteristic informationStacking to obtain feature vectorsLayer c The module comprises a first full connection layer, a second full connection layer, a third full connection layer and a fourth full connection layer, and the feature vectors are formedInput to Layer c In a first full connection layer of the module, outputting to obtain a similarity characteristic Fscore1, inputting the similarity characteristic Fscore1 into Layer c In a second full connection Layer of the module, outputting to obtain a similarity characteristic Fscore2, and inputting the similarity characteristic Fscore2 into the Layer c In a third full connection Layer of the module, outputting to obtain a similarity characteristic Fscore3, and inputting the similarity characteristic Fscore3 into a Layer c Outputting to obtain a similarity score in a fourth full connection layer of the module;
b-7) true and false discriminator D TF The characteristic key point lm is composed of a first full connecting layer, a second full connecting layer, a third full connecting layer, a fourth full connecting layer, a fifth full connecting layer and a sixth full connecting layer fake Input to a true or false discriminator D TF In the first fully-connected layer of (2), the output is characterizedWill be characterized byInput to the second fully-connected layer, output to obtain characteristicsWill be characterized byInput to the third fully-connected layer, output to obtain characteristicsWill be characterized byInput to the fourth full connection layer, and output is characterizedWill be characterized byInput to the fifth full-link layer, and output is characterizedFeature(s)Inputting the data into a sixth full connection layer, and outputting the data to obtain a numerical value of 1 channel
b-8) by the formula loss L1 =||lm fake -lm s || 2 Calculating to obtain the loss per point L1 In the formula | · | non-conducting phosphor 2 Is mean squared error, loss Cycle =||lm fake -lm t || 2 Calculating to obtain the loss of reconstruction Cycle By the formulaCalculating to obtain the loss of true and false DTF By the formulaCalculating to obtain the loss of similarity DS Using point-by-point loss by the back propagation method L1 Loss of reconstruction Cycle Loss of true and false DTF Loss of similarity DS Iterative optimization feature key point lm fake 。
In the step b-2), convolution kernels of the first downsampling convolution layer, the second downsampling convolution layer, the third downsampling convolution layer, the fourth downsampling convolution layer and the fifth downsampling convolution layer are all 1, step length is all 1, and padding is all 0; in the step b-5), the convolution kernels of the first up-sampling convolution layer, the second up-sampling convolution layer, the third up-sampling convolution layer, the fourth up-sampling convolution layer and the fifth up-sampling convolution layer are all 1, the step length is all 1, and the padding is all 0.
Further, step c) comprises the steps of:
c-1) establishment by identity encoder E id And attribute encoder E attr Forming a face image feature extraction network;
c-2) identity encoder E id Composed of an Arcface algorithm, and a source image Pic s Input to identity encoder E id In the method, a source image Pic is processed by an interplate () function s Adjusting to 112 × 112 resolution, inputting the 112 × 112 resolution image into an Arcface algorithm, and outputting to obtain an identity vectorWherein b is the training batch, c is the number of channels, h is the image height, w is the image width, and the identity vector is obtainedSequentially inputting the data into a filling layer and a regularization layer, and outputting to obtain an identity characteristic F id ;
c-3) Attribute encoder E attr The first downsampling residual block, the second downsampling residual block, the third downsampling residual block, the fourth downsampling residual block, the fifth downsampling residual block, the first bottleneck residual block and the second bottleneck residual block are sequentially composed of a first normalization layer, a second normalization layer, a first ReLU active layer, a second ReLU active layer, a first volume layer, a second volume layer, a downsampling layer and a residual connecting layer, the first bottleneck residual block and the second residual block are sequentially composed of the first normalization layer, the second normalization layer, the first ReLU active layer, the second volume layer, the downsampling layer and the residual connecting layer, the first bottleneck residual block and the second residual block are sequentially composed of the first normalization layer, the second normalization layer, the first ReLU active layer, the second ReLU active layer, the first volume layer, the second volume layer and the residual connecting layer, and the target image Pic is formed by the first downsampling residual block, the second residual block and the second residual block t Input to attribute encoder E attr In the first downsampling residual block, outputting the obtained attribute characteristicsCharacterizing attributesInputting the data into a second downsampling residual block, and outputting the data to obtain attribute characteristicsCharacterizing attributesInputting the data into a third downsampled residual block, and outputting to obtain attribute characteristicsCharacterizing attributesInputting the data into a fourth downsampling residual block, and outputting to obtain attribute characteristicsCharacterizing attributesInputting the data into a fifth downsampling residual block, and outputting to obtain attribute characteristicsCharacterizing attributesInputting into the first bottleneck residual block, outputting to obtain attribute characteristicsCharacterizing attributesInputting the data into a second bottleneck residual block, and outputting to obtain an attribute characteristic F attr 。
In the step c-3), a first normalization layer and a second normalization layer in the first downsampling residual block, the second downsampling residual block, the third downsampling residual block, the fourth downsampling residual block and the fifth downsampling residual block are respectively in a BatchNorm2d mode; in the step c-3), convolution kernels of the first convolution layer and the second convolution layer in the first downsampling residual block, the second downsampling residual block, the third downsampling residual block, the fourth downsampling residual block and the fifth downsampling residual block are all 3, and filling and step length are all 1.
Further, step d) comprises the following steps:
d-1) establishing a human face analysis module and a background information encoder E bg Forming a background feature extraction network;
d-2) the face analysis module is composed of a face analysis algorithm BiSeNet and used for analyzing a target image Pic t Inputting the image into a face analysis module, analyzing to obtain each part of the face, filling colors into each part of the analyzed face to obtain an image Pic only keeping a background area bg ;
d-3) background information encoder E bg The image Pic is formed by a first self-attention module, a second self-attention module, a third self-attention module, a fourth self-attention module and a fifth self-attention module, wherein the first self-attention module, the second self-attention module, the third self-attention module, the fourth self-attention module and the fifth self-attention module are sequentially formed by a down-sampling convolution layer, a self-attention layer and a ReLU activation layer, and the image Pic is formed bg Input to a background information encoder E bg In the first self-attention module, the output obtains the background featureCharacterizing the backgroundInputting the data into a second self-attention module, and outputting the data to obtain background featuresCharacterizing the backgroundInputting the data into a third self-attention module, and outputting the data to obtain background featuresCharacterizing the backgroundInputting the data into a fourth self-attention module, and outputting the obtained background featuresCharacterizing the backgroundInputting the data into a fourth self-attention module, and outputting the obtained background feature F bg 。
In the step d-3), the convolution kernels of the downsampling convolution layers of the first self-attention module, the second self-attention module, the third self-attention module, the fourth self-attention module and the fifth self-attention module are all 3, the step length is 0, and the padding is 0.
Further, step e) comprises the steps of:
e-1) establishing a generation network consisting of a fusion module, an up-sampling module and a discriminator module;
e-2) the fusion module is composed of a first fusion block, a second fusion block, a third fusion block, a fourth fusion block, a fifth fusion block and a sixth fusion block, wherein the first fusion block, the second fusion block, the third fusion block, the fourth fusion block, the fifth fusion block and the sixth fusion block are sequentially composed of a first convolution layer, a first adaptive instance normalization layer, a ReLU activation layer, a second convolution layer and a second adaptive instance normalization layer, and the attribute characteristic F is obtained by normalizing the first convolution layer, the second convolution layer and the second adaptive instance normalization layer attr Inputting the attribute characteristics into a fusion module, and obtaining the attribute characteristics after the attribute characteristics pass through a first convolution layer of a first fusion blockIdentity feature F id And attribute featuresInputting into the first adaptive instance normalization layer by formulaCalculating to obtain fusion characteristicsIn the formula sigma id As identity feature F id The standard deviation of the (c) is,as a property featureIs a channel averaging operation, and σ () is a standard deviation operation, the features are fusedInputting into ReLU activation layer to obtain characteristicsWill be characterized byAfter inputting into the second convolution layer, obtaining the attribute characteristicsIdentity feature F id And attribute featuresInputting into a second adaptive instance normalization layer by formulaCalculating to obtain fusion characteristicsIn the formulaAs a characteristic of an attributeThe channel average of (a);
e-3) mixingFusion featuresInputting the data into a fusion module, and obtaining attribute characteristics from a second fusion block after passing through a first convolution layer of the second fusion blockIdentity feature F id And attribute featuresInputting into the first adaptive instance normalization layer by formulaCalculating to obtain fusion characteristicsIn the formulaAs a characteristic of an attributeChannel average of (2), fusing the featuresInputting into ReLU activation layer to obtain characteristicsWill be characterized byAfter inputting into the second convolution layer, obtaining attribute characteristicsIdentity feature F id And attribute featuresInput to second AdaptationIn the example normalization layer, by formulaCalculating to obtain fusion characteristicsIn the formulaAs a characteristic of an attributeThe channel average of (a);
e-4) fusing featuresInputting the data into a fusion module, and obtaining attribute characteristics from a third fusion block and a first convolution layer of the third fusion blockIdentity feature F id And attribute featuresInputting into a first adaptive instance normalization layer by formulaCalculating to obtain fusion characteristicsIn the formulaAs a property featureChannel average of (2), fusing the featuresInputting into ReLU activation layer to obtain characteristicsWill be characterized byAfter inputting into the second convolution layer, obtaining attribute characteristicsIdentity feature F id And attribute featuresInputting into a second adaptive instance normalization layer by formulaCalculating to obtain fusion characteristicsIn the formulaAs a property featureThe channel average of (a);
e-5) fusing featuresInputting the data into a fusion module, and obtaining attribute characteristics after the data passes through a first convolution layer of a fourth fusion blockIdentity feature F id And attribute featuresInputting into the first adaptive instance normalization layer by formulaCalculating to obtain fusion characteristicsIn the formulaAs a property featureChannel average of (2), fusing the featuresInputting into ReLU activation layer to obtain characteristicsWill be characterized byAfter inputting into the second convolution layer, obtaining the attribute characteristicsIdentity feature F id And attribute featuresInputting into a second adaptive instance normalization layer by formulaCalculating to obtain fusion characteristicsIn the formulaAs a property featureThe channel average of (a);
e-6) fusing featuresInputting the attribute data into a fifth fusion block of the fusion module, and obtaining the attribute characteristics after the attribute data passes through a first convolution layer of the fifth fusion blockIdentity feature F id And attribute featuresInputting into the first adaptive instance normalization layer by formulaCalculating to obtain fusion characteristicsIn the formulaAs a characteristic of an attributeChannel average of (2), fusing the featuresInputting into ReLU activation layer to obtain characteristicsWill be characterized byAfter inputting into the second convolution layer, obtaining the attribute characteristicsIdentity F id And attribute featuresInputting into a second adaptive instance normalization layer by formulaCalculating to obtain fusion characteristicsIn the formulaAs a characteristic of an attributeThe channel average of (a);
e-7) fusing the featuresInputting the attribute data into a fusion module, and obtaining the attribute characteristics after the attribute data passes through a first convolution layer of a sixth fusion blockIdentity F id And attribute featuresInputting into the first adaptive instance normalization layer by formulaCalculating to obtain fusion characteristicsIn the formulaAs a property featureChannel average of (2), fusing the featuresInputting into ReLU activation layer to obtain characteristicsWill be characterized byAfter inputting into the second convolution layer, obtaining attribute characteristicsIdentity feature F id And attribute featuresInputting into a second adaptive instance normalization layer by formulaCalculating to obtain fusion characteristicsIn the formulaAs a property featureThe channel average of (a);
e-8) optimizing the feature key points lm fake Respectively inputting two convolution layers to respectively obtain characteristics F gamma And feature F beta By the formulaCalculating to obtain a fusion vector F fuse ;
e-9) the up-sampling module is composed of a first up-sampling layer, a second up-sampling layer, a third up-sampling layer, a fourth up-sampling layer and a fifth up-sampling layer, and the background characteristics are obtainedAnd a fusion vector F fuse Inputting the data into a first up-sampling layer of an up-sampling module, and outputting the obtained characteristicsWill be characterized byAnd background featuresInput into the second up-sampling layer together, and output to obtain characteristicsWill be characterized byAnd background featuresInput into the third up-sampling layer together, and output is characterizedWill be characterized byAnd background featuresInput into the fourth up-sampling layer together, and output to obtain characteristicsWill be characterized byAnd background feature F bg Inputting the images into a fifth upper sampling layer, and outputting to obtain a face image Pic fake ;
e-10) the discriminator module comprises a first downsampled convolutional layer, a second downsampled convolutional layer, a third downsampled convolutional layer,A fourth down-sampling convolutional layer, a fifth down-sampling convolutional layer, a sixth down-sampling convolutional layer, and a Sigmoid function layer, and the face image Pic fake Inputting the data into the first downsampling convolutional layer, and outputting the data to obtain the characteristicsWill be characterized byInput to the second downsampled convolutional layer, and output to obtain the characteristicsWill be characterized byInput to the third downsampling convolutional layer, and output to obtain the characteristicsWill be characterized byInputting the data into a fourth downsampled convolutional layer, and outputting the data to obtain characteristicsWill be characterized byInputting the data into a fifth downsampled convolutional layer, and outputting the result to obtain characteristicsWill be characterized byInput to the sixth downsampled convolutional layer, and output to obtain the characteristicsWill be characterized byAfter being input into a Sigmoid function layer, the values are outputTarget image Pic t After input to the first downsampled convolutional layer, output to obtain the characteristicsWill be characterized byInput into the second downsampled convolutional layer, and output to obtain the characteristicsWill be characterized byInput to the third downsampling convolutional layer, and output to obtain the characteristicsWill be characterized byInputting the data into a fourth downsampled convolutional layer, and outputting the data to obtain characteristicsWill be characterized byInputting the data into a fifth downsampled convolutional layer, and outputting the result to obtain characteristicsWill be characterized byAfter being input into the sixth downsampled convolution layer, the convolution product is outputTo obtain the characteristicsWill be characterized byAfter being input into a Sigmoid function layer, the values are output
e-11) by the formulaCalculating to obtain the identity loss l1, and obtaining the identity loss l1 through a formula l2= | | Pic fake -Pic t || 2 Calculating to obtain the reconstruction loss l2 by the formulaCalculating to obtain attribute loss l3, and iteratively optimizing the face image Pic by using the identity loss l1, the reconstruction loss l2 and the attribute loss l3 through a back propagation method fake 。
Further, in the step e-2), convolution kernels of the first convolution layer and the second convolution layer of the first fusion block, the second fusion block, the third fusion block, the fourth fusion block, the fifth fusion block and the sixth fusion block are all 3, step length is 1, and filling is 0; in the step e-8), the convolution kernels of the two convolution layers are both 1, the step length is both 1, and the filling is both 0; in the step e-9), convolution kernels of the first upper sampling layer, the second upper sampling layer, the third upper sampling layer and the fourth upper sampling layer are all 3, step length is 1, filling is 1, convolution kernels of the fifth upper sampling layer are 7, step length is 1, and filling is 0; e-10), the convolution kernels of the first down-sampling convolution layer, the second down-sampling convolution layer and the third down-sampling convolution layer are all 4 x 4, the step lengths are all 2, the padding is all 1, the convolution kernels of the fourth down-sampling convolution layer, the fifth down-sampling convolution layer and the sixth down-sampling convolution layer are all 4 x 4, the step lengths are all 1, and the padding is all 1.
The invention has the beneficial effects that: and extracting identity information, attribute information and background information of each image, fusing the information in a characteristic fusion mode, and finally obtaining a final result from the fused information in an image generation mode. The method introduces feature key points for guiding face shape changes. Meanwhile, the background information added in the training process enables the generated face image with the face shape changed and stable quality.
Drawings
FIG. 1 is a flow chart of a method of the present invention;
FIG. 2 is a diagram of the key point extraction and adjustment structure of the present invention;
FIG. 3 is a diagram of a key point discriminator network architecture of the present invention;
FIG. 4 is a diagram of an attribute extraction structure and a downsampling structure according to the present invention;
FIG. 5 is a block diagram illustrating an exemplary spatial adaptive normalization architecture according to the present invention;
FIG. 6 is a diagram of semantic parsing and background information extraction according to the present invention.
Detailed Description
The present invention is further described with reference to fig. 1 to 6.
A face image identity synthesis method based on semantic guidance comprises the following steps:
a) And extracting key points of the face image from all face images in the CelebA face image data set.
b) Establishing a PET key point adjusting network, inputting the key points of the face image into the PET key point adjusting network to obtain the characteristic key points lm fake For the feature key point lm fake Iteration is carried out to obtain optimized characteristic key points lm fake 。
c) Establishing a facial image feature extraction network, and concentrating CelebA facial image data into a source image Pic s And a target image Pic t Inputting the image data into a face image feature extraction network, and respectively outputting to obtain identity features F id And attribute feature F attr 。
d) Establishing a background feature extraction network and extracting a target image Pic t Inputting the background feature information into a background feature extraction network to obtain background feature information F bg 。
e) Establishing a generating network, and identifying the identity F id Attribute feature F attr Background feature information F bg And optimized characteristic key point lm fake Inputting the image into a generation network to obtain a face image Pic fake For picture Pic fake Iteration is carried out to obtain an optimized face image Pic fake 。
f) Repeating the steps b) to e) to obtain a real face image Pic with a changed face contour fake . The method comprises the steps of providing characteristic key points for semantically guiding face shape change, extracting identity information, attribute information and background information of each image, fusing the information in a characteristic fusion mode, and finally obtaining a final result from the fused information in an image generation mode. The method introduces feature key points for guiding face shape changes. Meanwhile, the generated face image with the face shape changed and stable quality is obtained by adding the background information in the training process.
Example 1:
the step a) comprises the following steps:
a-1) detecting key points of all face images in the CelebA face image data set by using a face key point detection algorithm H3R, and obtaining a source image Pic in the CelebA face image data set s The key points extracted in (1) are expressed as source key points lm s Target image Pic from CelebA face image dataset t The key points extracted in (1) are expressed as source key points lm t . CelebA facial image data set is composed of 30000 facial images with different identities, the resolution of each image is 512 x 512, and a source image Pic s And a target image Pic t Are all images in the CelebA dataset.
Example 2:
the step b) comprises the following steps:
b-1) creation by the Source encoder E lms Target encoder E lmt Key point generator G lm And a similarity discriminator D S And a true and false discriminator D TF The formed PET key points regulate the network.
b-2) Source encoder E lms Comprises a first down-sampling convolution layer, a second down-sampling convolution layer, a third down-sampling convolution layer, a fourth down-sampling convolution layer and a fifth sampling convolution layer, and the source key point lm is s Input to a source encoder E lms In the first downsampling convolutional layer, outputting the obtained characteristic informationFeature informationInputting the data into a second downsampled convolutional layer, and outputting the characteristic informationFeature informationInputting the data into a third downsampled convolutional layer, and outputting the characteristic informationFeature informationInputting the data into a fourth downsampled convolutional layer, and outputting the characteristic informationFeature informationInputting the data into a fifth downsampled convolutional layer, and outputting the characteristic information
b-3) target encoder E lmt Comprises a first full connection layer, a second full connection layer, a third full connection layer, a fourth full connection layer and a fifth full connection layer, and a source key point lm t Input to a target encoder E lmt First fully connected layer ofIn the method, the characteristic information is obtained by outputtingFeature informationInputting the data into a second full connection layer, and outputting the data to obtain characteristic informationFeature informationInputting the data into a third full connection layer, and outputting the data to obtain characteristic informationFeature informationInputting the data into a fourth full connection layer, and outputting the data to obtain characteristic informationFeature informationInputting the data into the fifth connection layer, and outputting the obtained characteristic information
Cat () function will feature informationAnd characteristic informationStacking to obtain feature vectors
b-5) Key Point Generator G lm Composed of a first up-sampling convolutional layer, a second up-sampling convolutional layer, a third up-sampling convolutional layer, a fourth up-sampling convolutional layer and a fifth up-sampling convolutional layer, and the feature vectorInput to the keypoint generator G lm In the first upsampled convolutional layer, outputting to obtain the characteristic key pointKey points of featuresInputting the data into a second upsampling convolutional layer, and outputting to obtain a characteristic key pointKey points of featuresInputting the data into a third upsampling convolutional layer, and outputting to obtain characteristic key pointsKey points of featuresInputting the data into a fourth upsampling convolutional layer, and outputting to obtain characteristic key pointsKey points of featuresInputting the data into a fifth upsampling convolutional layer, and outputting to obtain a characteristic key point lm fake Its dimension is 1 × 212.
b-6) similarity discriminator D S By Layer s Module, layer fake Module, layer c Modular construction, layer fake The module consists of a first full connecting layer, a second full connecting layer, a third full connecting layer and a fourth full connecting layer, and a characteristic key point lm is connected with the module fake Input to Layer fake In the first full connection layer of the module, outputting the characteristic informationFeature informationInput to Layer fake In the second full connection layer of the module, outputting the characteristic informationFeature informationInput to Layer fake In the third full connection layer of the module, outputting the obtained characteristic informationFeature informationInput to Layer fake In the fourth full connection layer of the module, outputting the characteristic informationLayer s The module consists of a first full connecting layer, a second full connecting layer, a third full connecting layer and a fourth full connecting layer, and a source key point lm s Input to Layer s The module outputs the characteristic information from the first full connection layerFeature informationInput to Layer s The module outputs the characteristic information from the second full connection layerFeature informationInput to Layer s The module outputs the characteristic information from the third full connection layerFeature informationInput to Layer s The module outputs the characteristic information from the fourth full connection layerCat () function is used to transfer feature informationAnd characteristic informationStacking to obtain feature vectorsLayer c The module consists of a first full-connection layer, a second full-connection layer, a third full-connection layer and a fourth full-connection layer, and the feature vectors are combinedInput to Layer c In a first full connection Layer of the module, outputting to obtain a similarity characteristic Fscore1, and inputting the similarity characteristic Fscore1 into a Layer c In a second full connection Layer of the module, outputting to obtain a similarity characteristic Fscore2, and inputting the similarity characteristic Fscore2 into the Layer c In a third full connection Layer of the module, outputting to obtain a similarity characteristic Fscore3, and inputting the similarity characteristic Fscore3 into a Layer c In the fourth full connection layer of the module, outputting to obtain a similarity score score。
b-7) true and false discriminator D TF The characteristic key point lm is composed of a first full connecting layer, a second full connecting layer, a third full connecting layer, a fourth full connecting layer, a fifth full connecting layer and a sixth full connecting layer fake Input to a true or false discriminator D TF In the first fully-connected layer of (2), the output is characterizedWill be characterized byInput to the second fully-connected layer, output to obtain characteristicsWill be characterized byInput to the third full-link layer, and output is characterizedWill be characterized byInput to the fourth full connection layer, and output is characterizedWill be characterized byInput to the fifth full-link layer, and output is characterizedFeature(s)Inputting the data into a sixth full connection layer, and outputting the data to obtain a numerical value of 1 channel
b-8) by the formula loss L1 =||lm fake -lm s || 2 Calculating to obtain the loss per point L1 In the formula | · | non-conducting phosphor 2 Is the mean squared error, loss Cycle =||lm fake -lm t || 2 Calculating to obtain the loss of reconstruction Cycle By the formulaCalculating to obtain the loss of true and false DTF By the formulaCalculating to obtain the loss of similarity DS Using point-by-point loss by the back propagation method L1 Loss of reconstruction Cycle Loss of true and false DTF Similarity loss DS Iterative optimization feature key point lm fake 。
Example 3:
in the step b-2), the convolution kernels of the first down-sampling convolution layer, the second down-sampling convolution layer, the third down-sampling convolution layer, the fourth down-sampling convolution layer and the fifth down-sampling convolution layer are all 1, the step length is all 1, and the padding is all 0; in the step b-5), the convolution kernels of the first up-sampling convolution layer, the second up-sampling convolution layer, the third up-sampling convolution layer, the fourth up-sampling convolution layer and the fifth up-sampling convolution layer are all 1, the step length is all 1, and the padding is all 0.
Example 4:
the step c) comprises the following steps:
c-1) establishment by the identity encoder E id Sum attribute encoder E attr Forming a face image feature extraction network;
c-2) identity encoder E id Composed of Arcface algorithm, and a source image Pic s Input to identity encoder E id In the method, a source image Pic is processed by an interplate () function s Adjusting to 112 × 112 resolution, inputting the 112 × 112 resolution image into an Arcface algorithm, and outputting to obtain an identity vectorWherein b is the training batch, c is the number of channels, h is the image height, w is the image width, and the identity vector is obtainedSequentially inputting the data into a filling layer and a regularization layer, and outputting to obtain an identity characteristic F id ;
c-3) Attribute encoder E attr The first downsampling residual block, the second downsampling residual block, the third downsampling residual block, the fourth downsampling residual block, the fifth downsampling residual block, the first bottleneck residual block and the second bottleneck residual block are sequentially composed of a first normalization layer, a second normalization layer, a first ReLU activation layer, a second ReLU activation layer, a first rolling layer, a second rolling layer, a downsampling layer and a residual connecting layer, the first bottleneck residual block and the second bottleneck residual block are sequentially composed of a first normalization layer, a second normalization layer, a first ReLU activation layer, a second ReLU activation layer, a first rolling layer, a second rolling layer and a residual connecting layer, and the target image Pic is formed by the target image Pic t Input to attribute encoder E attr In the first down-sampling residual block, the attribute characteristics are outputCharacterizing attributesInputting the data into a second downsampling residual block, and outputting the data to obtain attribute characteristicsCharacterizing attributesInputting the data into a third down-sampling residual block, and outputting to obtain attribute characteristicsCharacterizing attributesInputting the data into a fourth downsampling residual block, and outputting to obtain attribute characteristicsCharacterizing attributesInputting the data into a fifth downsampling residual block, and outputting to obtain attribute characteristicsCharacterizing attributesInputting into the first bottleneck residual block, outputting to obtain attribute characteristicsCharacterizing attributesInputting the data into a second bottleneck residual block, and outputting to obtain an attribute characteristic F attr 。
Example 5:
in the step c-3), a first normalization layer and a second normalization layer in the first downsampling residual block, the second downsampling residual block, the third downsampling residual block, the fourth downsampling residual block and the fifth downsampling residual block are respectively in a BatchNorm2d mode; in the step c-3), convolution kernels of the first convolution layer and the second convolution layer in the first downsampling residual block, the second downsampling residual block, the third downsampling residual block, the fourth downsampling residual block and the fifth downsampling residual block are all 3, and filling and step length are all 1.
Example 6:
the step d) comprises the following steps:
d-1) establishing a solution from a human faceAnalysis module and background information encoder E bg Forming a background feature extraction network;
d-2) the face analysis module is composed of a face analysis algorithm BiSeNet and used for analyzing a target image Pic t Inputting the image into a face analysis module, analyzing to obtain each part of the face, filling colors into each part of the face to obtain an image Pic only keeping a background region bg ;
d-3) background information encoder E bg The image Pic is composed of a first self-attention module, a second self-attention module, a third self-attention module, a fourth self-attention module and a fifth self-attention module, wherein the first self-attention module, the second self-attention module, the third self-attention module, the fourth self-attention module and the fifth self-attention module are sequentially composed of a down-sampling convolution layer, a self-attention layer and a ReLU activation layer, and the image Pic is formed by the image Pic bg Input to a background information encoder E bg In the first self-attention module, the output obtains the background featureCharacterizing the backgroundInputting the background feature into a second self-attention module, and outputtingCharacterizing the backgroundInputting the data into a third self-attention module, and outputting the data to obtain background featuresCharacterizing the backgroundInputting the data into a fourth self-attention module, and outputting the obtained background featuresCharacterizing the backgroundInputting the background feature into a fourth self-attention module, and outputting the background feature F bg 。
Example 7:
in the step d-3), the convolution kernels of the downsampling convolution layers of the first self-attention module, the second self-attention module, the third self-attention module, the fourth self-attention module and the fifth self-attention module are all 3, the step length is 0, and the padding is 0.
Example 8:
step e) comprises the following steps:
e-1) establishing a generation network consisting of a fusion module, an up-sampling module and a discriminator module;
e-2) the fusion module is composed of a first fusion block, a second fusion block, a third fusion block, a fourth fusion block, a fifth fusion block and a sixth fusion block, wherein the first fusion block, the second fusion block, the third fusion block, the fourth fusion block, the fifth fusion block and the sixth fusion block are sequentially composed of a first coiling layer, a first self-adaptive example normalizing layer, a ReLU activation layer, a second coiling layer and a second self-adaptive example normalizing layer, and the attribute characteristic F is obtained by sequentially combining the first coiling layer, the first self-adaptive example normalizing layer, the second coiling layer and the second self-adaptive example normalizing layer attr Inputting the attribute characteristics into a fusion module from a first fusion block, and obtaining the attribute characteristics after passing through a first convolution layer of the first fusion blockIdentity feature F id And attribute featuresInputting into the first adaptive instance normalization layer by formulaCalculating to obtain fusion characteristicsIn the formula σ id Is a special identitySign F id The standard deviation of (a) is determined,as a property featureIs the channel averaging operation, and σ () is the standard deviation operation, will fuse the featuresInputting into ReLU activation layer to obtain characteristicsWill be characterized byAfter inputting into the second convolution layer, obtaining the attribute characteristicsIdentity F id And attribute featuresInputting into a second adaptive instance normalization layer by formulaCalculating to obtain fusion characteristicsIn the formulaAs a property featureThe channel average of (a);
e-3) fusing featuresInputting the data into a fusion module, and obtaining attribute characteristics from a second fusion block after passing through a first convolution layer of the second fusion blockIdentity feature F id And attribute featuresInputting into the first adaptive instance normalization layer by formulaCalculating to obtain fusion characteristicsIn the formulaAs a property featureChannel average of (2), fusing the featuresInputting into ReLU activation layer to obtain characteristicsWill be characterized byAfter inputting into the second convolution layer, obtaining the attribute characteristicsIdentity feature F id And attribute featuresInputting into a second adaptive instance normalization layer by formulaCalculating to obtain fusion characteristicsIn the formulaAs a characteristic of an attributeA channel average of (d);
e-4) fusing the featuresInputting the data into a third fusion block of the fusion module, and obtaining attribute characteristics after passing through a first convolution layer of the third fusion blockIdentity F id And attribute featuresInputting into the first adaptive instance normalization layer by formulaCalculating to obtain fusion characteristicsIn the formulaAs a property featureChannel average of (2), fusing the featuresInputting into ReLU activation layer to obtain characteristicsWill be characterized byAfter inputting into the second convolution layer, obtaining the attribute characteristicsIdentity feature F id And attribute featuresInputting into a second adaptive instance normalization layer by formulaCalculating to obtain fusion characteristicsIn the formulaAs a property featureA channel average of (d);
e-5) fusing featuresInputting the data into a fusion module, and obtaining attribute characteristics after the data passes through a first convolution layer of a fourth fusion blockIdentity feature F id And attribute featuresInputting into the first adaptive instance normalization layer by formulaCalculating to obtain fusion characteristicsIn the formulaAs a property featureChannel average of (2), fusing the featuresInputting into ReLU activation layer to obtain characteristicsWill be characterized byAfter inputting into the second convolution layer, obtaining the attribute characteristicsIdentity feature F id And attribute featuresInputting into a second adaptive instance normalization layer by formulaCalculating to obtain fusion characteristicsIn the formulaAs a property featureThe channel average of (a);
e-6) fusing the featuresInputting the attribute data into a fifth fusion block of the fusion module, and obtaining the attribute characteristics after the attribute data passes through a first convolution layer of the fifth fusion blockIdentity feature F id And attribute featuresInputting into the first adaptive instance normalization layer by formulaCalculating to obtain fusion characteristicsIn the formulaAs a characteristic of an attributeChannel average of (2), fusing the featuresInputting into ReLU activation layer to obtain characteristicsWill be characterized byAfter inputting into the second convolution layer, obtaining the attribute characteristicsIdentity feature F id And attribute featuresInputting into a second adaptive instance normalization layer by formulaCalculating to obtain fusion characteristicsIn the formulaAs a property featureThe channel average of (a);
e-7) fusing the featuresInputting the attribute data into a fusion module, and obtaining the attribute characteristics after the attribute data passes through a first convolution layer of a sixth fusion blockIdentity feature F id And attribute featuresInputting into the first adaptive instance normalization layer by formulaCalculating to obtain fusion characteristicsIn the formulaAs a property featureChannel average of (2), fusing the featuresInputting into ReLU activation layer to obtain characteristicsWill be characterized byAfter inputting into the second convolution layer, obtaining attribute characteristicsIdentity feature F id And attribute featuresInputting into a second adaptive instance normalization layer by formulaCalculating to obtain fusion characteristicsIn the formulaAs a property featureA channel average of (d);
e-8) optimizing the feature key points lm fake Inputting two convolution layers respectively to obtain characteristics F gamma And feature F beta By the formulaCalculating to obtain a fusion vector F fuse ;
e-9) the up-sampling module is composed of a first up-sampling layer, a second up-sampling layer, a third up-sampling layer, a fourth up-sampling layer and a fifth up-sampling layer, and the background characteristics are obtainedAnd a fusion vector F fuse Inputting the data into a first up-sampling layer of an up-sampling module, and outputting the obtained characteristicsWill be characterized byAnd background featuresInput into the second up-sampling layer together, and output to obtain characteristicsWill be characterized byAnd background featuresInput into the third up-sampling layer together, and output is characterizedWill be characterized byAnd background featuresInput into the fourth up-sampling layer together, and output to obtain characteristicsWill be characterized byAnd background feature F bg Inputting the images into a fifth upper sampling layer, and outputting to obtain a face image Pic fake ;
e-10) the discriminator module comprises a first downsampling convolution layer, a second downsampling convolution layer, a third downsampling convolution layer, a fourth downsampling convolution layer, a fifth downsampling convolution layer, a sixth downsampling convolution layer and a Sigmoid function layer,the face image Pic fake After input to the first downsampled convolutional layer, output to obtain the characteristicsWill be characterized byInput to the second downsampled convolutional layer, and output to obtain the characteristicsWill be characterized byInput to a third downsampled convolutional layer, and output to obtain characteristicsWill be characterized byInput to the fourth downsampled convolutional layer, and output to obtain the characteristicsWill be characterized byInputting the data into a fifth downsampled convolutional layer, and outputting the result to obtain characteristicsWill be characterized byInput to the sixth downsampling convolutional layer, and output to obtain the characteristicsWill be characterized byAfter being input into a Sigmoid function layer, the values are outputTarget image Pic t After input to the first downsampled convolutional layer, output to obtain the characteristicsWill be characterized byInput to the second downsampled convolutional layer, and output to obtain the characteristicsWill be characterized byInput to a third downsampled convolutional layer, and output to obtain characteristicsWill be characterized byInput to the fourth downsampled convolutional layer, and output to obtain the characteristicsWill be characterized byInput to the fifth downsampled convolutional layer, and output to obtain the characteristicsWill be characterized byInput to the sixth downsampling convolutional layer, and output to obtain the characteristicsWill be characterized byAfter the data is input into a Sigmoid function layer, a numerical value is output
e-11) by the formulaCalculating to obtain the identity loss l1, and obtaining the identity loss l1 through a formula l2= | | Pic fake -Pic t || 2 Calculating to obtain the reconstruction loss l2 by the formulaCalculating to obtain attribute loss l3, and iteratively optimizing the face image Pic by using the identity loss l1, the reconstruction loss l2 and the attribute loss l3 through a back propagation method fake 。
Example 9:
convolution kernels of the first convolution layer and the second convolution layer of the first fusion block, the second fusion block, the third fusion block, the fourth fusion block, the fifth fusion block and the sixth fusion block in the step e-2) are all 3, step length is 1, and filling is 0; in the step e-8), the convolution kernels of the two convolution layers are both 1, the step length is both 1, and the filling is both 0; in the step e-9), convolution kernels of the first upper sampling layer, the second upper sampling layer, the third upper sampling layer and the fourth upper sampling layer are all 3, step length is 1, filling is 1, convolution kernels of the fifth upper sampling layer are 7, step length is 1, and filling is 0; e-10), wherein the convolution kernels of the first downsampling convolutional layer, the second downsampling convolutional layer and the third downsampling convolutional layer are all 4 x 4, the step lengths are all 2, the padding is all 1, the convolution kernels of the fourth downsampling convolutional layer, the fifth downsampling convolutional layer and the sixth downsampling convolutional layer are all 4 x 4, the step lengths are all 1, and the padding is all 1.
Finally, it should be noted that: although the present invention has been described in detail with reference to the foregoing embodiments, it will be apparent to those skilled in the art that changes may be made in the embodiments and/or equivalents thereof without departing from the spirit and scope of the invention. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (10)
1. A face image identity synthesis method based on semantic guidance is characterized by comprising the following steps:
a) Extracting key points of the face image from all face images in the CelebA face image data set;
b) Establishing a PET key point adjusting network, inputting the key points of the face image into the PET key point adjusting network to obtain the characteristic key points lm fake For the feature key point lm fake Iteration is carried out to obtain optimized characteristic key points lm fake ;
c) Establishing a facial image feature extraction network, and concentrating CelebA facial image data into a source image Pic s And a target image Pic t Inputting the image data into a face image feature extraction network, and respectively outputting to obtain identity features F id And attribute feature F attr ;
d) Establishing a background feature extraction network and extracting a target image Pic t Inputting the background feature information into a background feature extraction network to obtain background feature information F bg ;
e) Establishing a generating network, and identifying the identity F id Attribute feature F attr Background feature information F bg And optimized feature key point lm fake Inputting the image into a generation network to obtain a face image Pic fake For picture Pic fake Iteration is carried out to obtain an optimized face image Pic fake ;
f) Repeating the steps b) to e) to obtain a real face image Pic with a changed face contour fake 。
2. The face image identity synthesis method based on semantic guidance according to claim 1,
the step a) comprises the following steps:
a-1) detecting key points of all face images in the CelebA face image data set by using a face key point detection algorithm H3R, and obtaining a source image Pic in the CelebA face image data set s The key points extracted in (1) are expressed as source key points lm s Target image Pic from CelebA face image dataset t The key points extracted in (1) are expressed as source key points lm t 。
3. The face image identity synthesis method based on semantic guidance according to claim 2,
the step b) comprises the following steps:
b-1) construction by the Source encoder E lms Target encoder E lmt Key point generator G lm And a similarity discriminator D S And a true and false discriminator D TF Forming a PET key point adjusting network;
b-2) Source encoder E lms Comprises a first down-sampling convolutional layer, a second down-sampling convolutional layer, a third down-sampling convolutional layer, a fourth down-sampling convolutional layer and a fifth sampling convolutional layer, and the source key point lm is s Input to a source encoder E lms In the first downsampling convolutional layer, outputting the obtained characteristic informationFeature informationInputting the data into a second downsampled convolutional layer, and outputting the characteristic informationFeature informationInputting the data into a third downsampling convolutional layer, and outputting the characteristic informationFeature informationInputting the data into a fourth downsampled convolutional layer, and outputting the characteristic informationFeature informationInputting the data into a fifth downsampled convolutional layer, and outputting the characteristic information
b-3) target encoder E lmt Comprises a first full connection layer, a second full connection layer, a third full connection layer, a fourth full connection layer and a fifth full connection layer, and a source key point lm t Input to a target encoder E lmt In the first full connection layer, the characteristic information is outputFeature informationInputting the data into a second full-connection layer, and outputting the data to obtain characteristic informationFeature informationInputting the data into a third full connection layer, and outputting the data to obtain characteristic informationFeature informationInputting the data into a fourth full connection layer, and outputting the data to obtain characteristic informationFeature informationInputting the data into the fifth connection layer, and outputting the obtained characteristic information
Cat () function will feature informationAnd characteristic informationStacking to obtain feature vectors
b-5) Key Point Generator G lm Composed of a first up-sampling convolutional layer, a second up-sampling convolutional layer, a third up-sampling convolutional layer, a fourth up-sampling convolutional layer and a fifth up-sampling convolutional layer, and the feature vectorInput to the keypoint generator G lm In the first upsampled convolutional layer, outputting to obtain the characteristic key pointKey points of featuresIs inputted intoIn the second up-sampling convolution layer, outputting to obtain the characteristic key pointKey points of featuresInputting the data into a third upsampling convolutional layer, and outputting to obtain characteristic key pointsKey points of featuresInputting the data into a fourth upsampling convolutional layer, and outputting to obtain characteristic key pointsFeature key pointsInputting the data into a fifth upsampling convolutional layer, and outputting to obtain a characteristic key point lm fake ;
b-6) similarity discriminator D S By Layer s Module, layer fake Module, layer c Modular construction, layer fake The module consists of a first full connecting layer, a second full connecting layer, a third full connecting layer and a fourth full connecting layer, and a characteristic key point lm is connected with the module fake Input to Layer fake In the first full connection layer of the module, outputting the characteristic informationFeature informationInput to Layer fake In the second full connection layer of the module, outputting the characteristic informationFeature informationInput to Layer fake In the third full connection layer of the module, outputting the obtained characteristic informationFeature informationInput to Layer fake In the fourth full connection layer of the module, outputting the characteristic informationLayer s The module consists of a first full connecting layer, a second full connecting layer, a third full connecting layer and a fourth full connecting layer, and a source key point lm s Input to Layer s The module outputs the characteristic information from the first full connection layerFeature informationInput to Layer s The module outputs the characteristic information from the second full connection layerFeature informationInput to Layer s The module outputs the characteristic information from the third full connection layerFeature informationInput to Layer s The module outputs the characteristic information from the fourth full connection layerCat () function is used to transfer feature informationAnd characteristic informationStacking to obtain feature vectorsLayer c The module consists of a first full-connection layer, a second full-connection layer, a third full-connection layer and a fourth full-connection layer, and the feature vectors are combinedInput to Layer c In a first full connection Layer of the module, outputting to obtain a similarity characteristic Fscore1, and inputting the similarity characteristic Fscore1 into a Layer c In the second full connection Layer of the module, outputting to obtain a similarity characteristic Fscore2, and inputting the similarity characteristic Fscore2 into the Layer c In a third full connection Layer of the module, outputting to obtain a similarity characteristic Fscore3, and inputting the similarity characteristic Fscore3 into a Layer c Outputting to obtain a similarity score in a fourth full connection layer of the module;
b-7) true and false discriminator D TF The characteristic key point lm is composed of a first full connecting layer, a second full connecting layer, a third full connecting layer, a fourth full connecting layer, a fifth full connecting layer and a sixth full connecting layer fake Input to a true or false discriminator D TF In the first full connection layer of (2), the output is characterizedWill be characterized byInput to the second fully-connected layer, output is characterizedWill be characterized byInput to the third full-link layer, and output is characterizedWill be characterized byInput to the fourth fully-connected layer, and output to obtain characteristicsWill be characterized byInput to the fifth fully-connected layer, output to obtain characteristicsFeature(s)Inputting the data into a sixth full connection layer, and outputting the data to obtain a numerical value of 1 channel
b-8) by the formula loss L1 =||lm fake -lm s || 2 Calculating to obtain the loss per point L1 In the formula | · | non-conducting phosphor 2 Is mean squared error, loss Cycle =||lm fake -lm t || 2 Calculating to obtain the loss of reconstruction Cycle By the formulaCalculating to obtain the loss of true and false DTF By the formulaCalculating to obtain the loss of similarity DS Using point-by-point loss by the back propagation method L1 Loss of reconstruction Cycle Loss of true and false DTF Loss of similarity DS Iterative optimization feature key point lm fake 。
4. The semantic guidance-based face image identity synthesis method according to claim 1, characterized in that: in the step b-2), the convolution kernels of the first down-sampling convolution layer, the second down-sampling convolution layer, the third down-sampling convolution layer, the fourth down-sampling convolution layer and the fifth down-sampling convolution layer are all 1, the step length is all 1, and the padding is all 0; in the step b-5), the convolution kernels of the first up-sampling convolution layer, the second up-sampling convolution layer, the third up-sampling convolution layer, the fourth up-sampling convolution layer and the fifth up-sampling convolution layer are all 1, the step length is all 1, and the padding is all 0.
5. The method for synthesizing the identity of the human face image based on semantic guidance according to claim 1, wherein the step c) comprises the following steps:
c-1) establishment by identity encoder E id And attribute encoder E attr Forming a face image feature extraction network;
c-2) identity encoder E id Composed of Arcface algorithm, and a source image Pic s Input to identity encoder E id In the method, a source image Pic is processed by an interplate () function s Adjusting to 112 × 112 resolution, inputting the 112 × 112 resolution image into an Arcface algorithm, and outputting to obtain an identity vectorWherein b is the training batch, c is the number of channels, h is the image height, w is the image width, and the identity vector is obtainedSequentially inputting the data into a filling layer and a regularization layer, and outputting to obtain an identity characteristic F id ;
c-3) Attribute encoder E attr The first downsampling residual block, the second downsampling residual block, the third downsampling residual block, the fourth downsampling residual block, the fifth downsampling residual block, the first bottleneck residual block and the second bottleneck residual block are sequentially composed of a first normalization layer, a second normalization layer, a first ReLU activation layer, a second ReLU activation layer, a first rolling layer, a second rolling layer, a downsampling layer and a residual connecting layer, the first bottleneck residual block and the second bottleneck residual block are sequentially composed of a first normalization layer, a second normalization layer, a first ReLU activation layer, a second ReLU activation layer, a first rolling layer, a second rolling layer and a residual connecting layer, and the target image Pic is formed by the target image Pic t Input to attribute encoder E attr In the first downsampling residual block, outputting the obtained attribute characteristicsCharacterizing attributesInputting the data into a second downsampling residual block, and outputting the data to obtain attribute characteristicsCharacterizing attributesInputting the data into a third down-sampling residual block, and outputting to obtain attribute characteristicsCharacterizing attributesInputting the data into a fourth downsampling residual block, and outputting to obtain attribute characteristicsCharacterizing attributesInputting the data into a fifth downsampling residual block, and outputting to obtain attribute characteristicsCharacterizing attributesInputting into the first bottleneck residual block, outputting to obtain attribute characteristicsCharacterizing attributesInputting the data into a second bottleneck residual block, and outputting to obtain an attribute characteristic F attr 。
6. The semantic guidance-based human face image identity synthesis method according to claim 1, characterized in that: in the step c-3), a first normalization layer and a second normalization layer in the first downsampling residual block, the second downsampling residual block, the third downsampling residual block, the fourth downsampling residual block and the fifth downsampling residual block are respectively a BatchNorm2d; convolution kernels of the first convolution layer and the second convolution layer in the first downsampling residual block, the second downsampling residual block, the third downsampling residual block, the fourth downsampling residual block and the fifth downsampling residual block in the step c-3) are all 3, and filling and step length are all 1.
7. The method for synthesizing the identity of the human face image based on semantic guidance according to claim 1, wherein the step d) comprises the following steps:
d-1) establishing a human face analysis module and a background information encoder E bg A constructed background feature extraction network;
d-2) the face analysis module is composed of a face analysis algorithm BiSeNet and used for analyzing a target image Pic t Inputting the image into a face analysis module, analyzing to obtain each part of the face, filling colors into each part of the face to obtain an image Pic only keeping a background region bg ;
d-3) background information encoder E bg The image Pic is composed of a first self-attention module, a second self-attention module, a third self-attention module, a fourth self-attention module and a fifth self-attention module, wherein the first self-attention module, the second self-attention module, the third self-attention module, the fourth self-attention module and the fifth self-attention module are sequentially composed of a down-sampling convolution layer, a self-attention layer and a ReLU activation layer, and the image Pic is formed by the image Pic bg Input to a background information encoder E bg In the first self-attention module, the output obtains the background featureCharacterizing the backgroundInputting the data into a second self-attention module, and outputting the data to obtain background featuresCharacterizing the backgroundInputting the data into a third self-attention module, and outputting the data to obtain background featuresCharacterizing the backgroundInputting the data into a fourth self-attention module, and outputting the obtained background featuresCharacterizing the backgroundInputting the background feature into a fourth self-attention module, and outputting the background feature F bg 。
8. The semantic guidance-based human face image identity synthesis method according to claim 1, characterized in that: in the step d-3), convolution kernels of the downsampling convolution layers of the first self-attention module, the second self-attention module, the third self-attention module, the fourth self-attention module and the fifth self-attention module are all 3, step length is all 0, and padding is all 0.
9. The method for synthesizing the identity of the human face image based on semantic guidance according to claim 7, wherein the step e) comprises the following steps:
e-1) establishing a generation network consisting of a fusion module, an up-sampling module and a discriminator module;
e-2) the fusion module is composed of a first fusion block, a second fusion block, a third fusion block, a fourth fusion block, a fifth fusion block and a sixth fusion block, wherein the first fusion block, the second fusion block, the third fusion block, the fourth fusion block, the fifth fusion block and the sixth fusion block are sequentially composed of a first convolution layer, a first adaptive instance normalization layer, a ReLU activation layer, a second convolution layer and a second adaptive instance normalization layer, and the attribute characteristic F is obtained by normalizing the first convolution layer, the second convolution layer and the second adaptive instance normalization layer attr Inputting the attribute characteristics into a fusion module, and obtaining the attribute characteristics after the attribute characteristics pass through a first convolution layer of a first fusion blockIdentity feature F id And attribute featuresInputting into the first adaptive instance normalization layer by formulaCalculating to obtain fusion characteristicsIn the formula σ id As identity feature F id The standard deviation of the (c) is,as a characteristic of an attributeIs a channel averaging operation, and σ () is a standard deviation operation, the features are fusedInputting into ReLU activation layer to obtain characteristicsWill be characterized byAfter inputting into the second convolution layer, obtaining the attribute characteristicsIdentity feature F id And attribute featuresInputting into a second adaptive instance normalization layer by formulaCalculating to obtain fusion characteristicsIn the formulaAs a characteristic of an attributeThe channel average of (a);
e-3) fusing featuresInputting the data into a fusion module, and obtaining attribute characteristics after the data passes through a first convolution layer of a second fusion blockIdentity feature F id And attribute featuresInputting into the first adaptive instance normalization layer by formulaCalculating to obtain fusion characteristicsIn the formulaAs a property featureChannel average of (2), fusing the featuresInputting into ReLU activation layer to obtain characteristicsWill be characterized byAfter inputting into the second convolution layer, obtaining attribute characteristicsIdentity F id And attribute featuresInputting into a second adaptive instance normalization layer by formulaCalculating to obtain fusion characteristicsIn the formulaAs a characteristic of an attributeA channel average of (d);
e-4) fusing featuresInputting the data into a third fusion block of the fusion module, and obtaining attribute characteristics after passing through a first convolution layer of the third fusion blockIdentity feature F id And attribute featuresInputting into a first adaptive instance normalization layer by formulaCalculating to obtain fusion characteristicsIn the formulaAs a property featureChannel average of (2), fusing the featuresInputting into ReLU activation layer to obtain characteristicsWill be characterized byAfter inputting into the second convolution layer, obtaining the attribute characteristicsIdentity feature F id And attribute featuresInputting into a second adaptive instance normalization layer by formulaCalculating to obtain fusion characteristicsIn the formulaAs a property featureThe channel average of (a);
e-5) fusing featuresInputting the data into a fusion module, and obtaining attribute characteristics after the data passes through a first convolution layer of a fourth fusion blockIdentity feature F id And attribute featuresInputting into the first adaptive instance normalization layer by formulaCalculating to obtain fusion characteristicsIn the formulaAs a property featureChannel average of (2), fusing the featuresInputting into ReLU activation layer to obtain characteristicsWill be characterized byAfter inputting into the second convolution layer, obtaining the attribute characteristicsIdentity feature F id And attribute featuresInputting into a second adaptive instance normalization layer by formulaCalculating to obtain fusion characteristicsIn the formulaAs a characteristic of an attributeThe channel average of (a);
e-6) fusing featuresInputting the attribute data into a fifth fusion block of the fusion module, and obtaining the attribute characteristics after the attribute data passes through a first convolution layer of the fifth fusion blockIdentity feature F id And attribute featuresInputting into a first adaptive instance normalization layer by formulaCalculating to obtain fusion characteristicsIn the formulaAs a characteristic of an attributeChannel average of (2), fusing the featuresInputting into ReLU activation layer to obtain characteristicsWill be characterized byAfter inputting into the second convolution layer, obtaining the attribute characteristicsIdentity feature F id And attribute featuresInputting into a second adaptive instance normalization layer by formulaCalculating to obtain fusion characteristicsIn the formulaAs a property featureA channel average of (d);
e-7) fusing the featuresInputting the attribute characteristics into a fusion module from a sixth fusion block, and obtaining the attribute characteristics after passing through a first convolution layer of the sixth fusion blockIdentity feature F id And attribute featuresInputting into the first adaptive instance normalization layer by formulaCalculating to obtain fusion characteristicsIn the formulaAs a property featureChannel average of (2), fusing the featuresInputting into ReLU activation layer to obtain characteristicsWill be characterized byAfter inputting into the second convolution layer, obtaining the attribute characteristicsIdentity feature F id And attribute featuresInputting into a second adaptive instance normalization layer by formulaCalculating to obtain fusion characteristicsIn the formulaAs a property featureA channel average of (d);
e-8) optimizing the feature key points lm fake Respectively inputting two convolution layers to respectively obtain characteristics F gamma And feature F beta By the formulaCalculating to obtain a fusion vector F fuse ;
e-9) the up-sampling module comprises a first up-sampling layer, a second up-sampling layer, a third up-sampling layer, a fourth up-sampling layer and a fifth up-sampling layer, and the background characteristics are obtainedAnd a fusion vector F fuse Inputting the data into a first up-sampling layer of an up-sampling module, and outputting the obtained characteristicsWill be characterized byAnd background featuresInput into the second up-sampling layer together, and output to obtain characteristicsWill be characterized byAnd background featuresInput into the third up-sampling layer together, and output to obtain characteristicsWill be characterized byAnd background featuresInput into the fourth up-sampling layer together, and output to obtain characteristicsWill be characterized byAnd background feature F bg Inputting the images into a fifth upper sampling layer, and outputting to obtain a face image Pic fake ;
e-10) the discriminator module comprises a first down-sampling convolution layer, a second down-sampling convolution layer, a third down-sampling convolution layer, a fourth down-sampling convolution layer, a fifth down-sampling convolution layer, a sixth down-sampling convolution layer and a Sigmoid function layer, and the discriminator module is used for discriminating the face image Pic fake Is inputted intoAfter the first downsampling convolutional layer, outputting to obtain the characteristicsWill be characterized byInput to the second downsampled convolutional layer, and output to obtain the characteristicsWill be characterized byInput to the third downsampling convolutional layer, and output to obtain the characteristicsWill be characterized byInputting the data into a fourth downsampled convolutional layer, and outputting the data to obtain characteristicsWill be characterized byInput to the fifth downsampled convolutional layer, and output to obtain the characteristicsWill be characterized byInput to the sixth downsampling convolutional layer, and output to obtain the characteristicsWill be characterized byAfter being input into a Sigmoid function layer, the values are outputThe target image Pic t After input to the first downsampled convolutional layer, output to obtain the characteristicsWill be characterized byInput to the second downsampled convolutional layer, and output to obtain the characteristicsWill be characterized byInput to the third downsampling convolutional layer, and output to obtain the characteristicsWill be characterized byInputting the data into a fourth downsampled convolutional layer, and outputting the data to obtain characteristicsWill be characterized byInputting the data into a fifth downsampled convolutional layer, and outputting the result to obtain characteristicsWill be characterized byInput to the sixth downsampling convolutional layer, and output to obtain the characteristicsWill be characterized byAfter being input into a Sigmoid function layer, the values are outpute-11) by the formulaCalculating to obtain the identity loss l1, and obtaining the identity loss l1 through a formula l2= | | Pic fake -Pic t || 2 Calculating to obtain the reconstruction loss l2 by the formulaCalculating to obtain attribute loss l3, and iteratively optimizing the face image Pic by using the identity loss l1, the reconstruction loss l2 and the attribute loss l3 through a back propagation method fake 。
10. The semantic guidance-based face image identity synthesis method according to claim 1, characterized in that: convolution kernels of the first convolution layer and the second convolution layer of the first fusion block, the second fusion block, the third fusion block, the fourth fusion block, the fifth fusion block and the sixth fusion block in the step e-2) are all 3, step length is 1, and filling is 0; in the step e-8), the convolution kernels of the two convolution layers are both 1, the step length is both 1, and the filling is both 0; in the step e-9), convolution kernels of the first up-sampling layer, the second up-sampling layer, the third up-sampling layer and the fourth up-sampling layer are all 3, step length is 1, filling is 1, convolution kernels of the fifth up-sampling layer are 7, step length is 1, and filling is 0; e-10), wherein the convolution kernels of the first downsampling convolutional layer, the second downsampling convolutional layer and the third downsampling convolutional layer are all 4 x 4, the step lengths are all 2, the padding is all 1, the convolution kernels of the fourth downsampling convolutional layer, the fifth downsampling convolutional layer and the sixth downsampling convolutional layer are all 4 x 4, the step lengths are all 1, and the padding is all 1.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202211451581.1A CN115713680B (en) | 2022-11-18 | 2022-11-18 | Semantic guidance-based face image identity synthesis method |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202211451581.1A CN115713680B (en) | 2022-11-18 | 2022-11-18 | Semantic guidance-based face image identity synthesis method |
Publications (2)
Publication Number | Publication Date |
---|---|
CN115713680A true CN115713680A (en) | 2023-02-24 |
CN115713680B CN115713680B (en) | 2023-07-25 |
Family
ID=85233817
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN202211451581.1A Active CN115713680B (en) | 2022-11-18 | 2022-11-18 | Semantic guidance-based face image identity synthesis method |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN115713680B (en) |
Cited By (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN116246022A (en) * | 2023-03-09 | 2023-06-09 | 山东省人工智能研究院 | Face image identity synthesis method based on progressive denoising guidance |
CN116612211A (en) * | 2023-05-08 | 2023-08-18 | 山东省人工智能研究院 | Face image identity synthesis method based on GAN and 3D coefficient reconstruction |
Citations (13)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20190122103A1 (en) * | 2017-10-24 | 2019-04-25 | International Business Machines Corporation | Attention based sequential image processing |
CN110197167A (en) * | 2019-06-05 | 2019-09-03 | 清华大学深圳研究生院 | A kind of video actions moving method |
CN111368662A (en) * | 2020-02-25 | 2020-07-03 | 华南理工大学 | Method, device, storage medium and equipment for editing attribute of face image |
CN111932444A (en) * | 2020-07-16 | 2020-11-13 | 中国石油大学(华东) | Face attribute editing method based on generation countermeasure network and information processing terminal |
CN112734634A (en) * | 2021-03-30 | 2021-04-30 | 中国科学院自动化研究所 | Face changing method and device, electronic equipment and storage medium |
CN112766160A (en) * | 2021-01-20 | 2021-05-07 | 西安电子科技大学 | Face replacement method based on multi-stage attribute encoder and attention mechanism |
CN113112411A (en) * | 2020-01-13 | 2021-07-13 | 南京信息工程大学 | Human face image semantic restoration method based on multi-scale feature fusion |
CN113627233A (en) * | 2021-06-17 | 2021-11-09 | 中国科学院自动化研究所 | Visual semantic information-based face counterfeiting detection method and device |
CN113689328A (en) * | 2021-09-13 | 2021-11-23 | 中国海洋大学 | Image harmony system based on self-attention transformation |
WO2021258920A1 (en) * | 2020-06-24 | 2021-12-30 | 百果园技术(新加坡)有限公司 | Generative adversarial network training method, image face swapping method and apparatus, and video face swapping method and apparatus |
CN114078172A (en) * | 2020-08-19 | 2022-02-22 | 四川大学 | Text image generation method for progressively generating confrontation network based on resolution |
WO2022151535A1 (en) * | 2021-01-15 | 2022-07-21 | 苏州大学 | Deep learning-based face feature point detection method |
CN115311720A (en) * | 2022-08-11 | 2022-11-08 | 山东省人工智能研究院 | Defekake generation method based on Transformer |
-
2022
- 2022-11-18 CN CN202211451581.1A patent/CN115713680B/en active Active
Patent Citations (13)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20190122103A1 (en) * | 2017-10-24 | 2019-04-25 | International Business Machines Corporation | Attention based sequential image processing |
CN110197167A (en) * | 2019-06-05 | 2019-09-03 | 清华大学深圳研究生院 | A kind of video actions moving method |
CN113112411A (en) * | 2020-01-13 | 2021-07-13 | 南京信息工程大学 | Human face image semantic restoration method based on multi-scale feature fusion |
CN111368662A (en) * | 2020-02-25 | 2020-07-03 | 华南理工大学 | Method, device, storage medium and equipment for editing attribute of face image |
WO2021258920A1 (en) * | 2020-06-24 | 2021-12-30 | 百果园技术(新加坡)有限公司 | Generative adversarial network training method, image face swapping method and apparatus, and video face swapping method and apparatus |
CN111932444A (en) * | 2020-07-16 | 2020-11-13 | 中国石油大学(华东) | Face attribute editing method based on generation countermeasure network and information processing terminal |
CN114078172A (en) * | 2020-08-19 | 2022-02-22 | 四川大学 | Text image generation method for progressively generating confrontation network based on resolution |
WO2022151535A1 (en) * | 2021-01-15 | 2022-07-21 | 苏州大学 | Deep learning-based face feature point detection method |
CN112766160A (en) * | 2021-01-20 | 2021-05-07 | 西安电子科技大学 | Face replacement method based on multi-stage attribute encoder and attention mechanism |
CN112734634A (en) * | 2021-03-30 | 2021-04-30 | 中国科学院自动化研究所 | Face changing method and device, electronic equipment and storage medium |
CN113627233A (en) * | 2021-06-17 | 2021-11-09 | 中国科学院自动化研究所 | Visual semantic information-based face counterfeiting detection method and device |
CN113689328A (en) * | 2021-09-13 | 2021-11-23 | 中国海洋大学 | Image harmony system based on self-attention transformation |
CN115311720A (en) * | 2022-08-11 | 2022-11-08 | 山东省人工智能研究院 | Defekake generation method based on Transformer |
Non-Patent Citations (2)
Title |
---|
ZHILIANG XU 等: "StyleSwap: Style-Based Generator Empowers Robust Face Swapping", 《ARXIV》, vol. 2022, pages 1 - 21 * |
李欢: "基于归一化SAGAN的奶山羊图像生成算法研究", 《中国优秀硕士学位论文全文数据库农业科技辑》, vol. 2022, no. 1, pages 050 - 152 * |
Cited By (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN116246022A (en) * | 2023-03-09 | 2023-06-09 | 山东省人工智能研究院 | Face image identity synthesis method based on progressive denoising guidance |
CN116246022B (en) * | 2023-03-09 | 2024-01-26 | 山东省人工智能研究院 | Face image identity synthesis method based on progressive denoising guidance |
CN116612211A (en) * | 2023-05-08 | 2023-08-18 | 山东省人工智能研究院 | Face image identity synthesis method based on GAN and 3D coefficient reconstruction |
CN116612211B (en) * | 2023-05-08 | 2024-02-02 | 山东省人工智能研究院 | Face image identity synthesis method based on GAN and 3D coefficient reconstruction |
Also Published As
Publication number | Publication date |
---|---|
CN115713680B (en) | 2023-07-25 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
Wen et al. | Cycle4completion: Unpaired point cloud completion using cycle transformation with missing region coding | |
Yuan et al. | Tokens-to-token vit: Training vision transformers from scratch on imagenet | |
Liu et al. | Convtransformer: A convolutional transformer network for video frame synthesis | |
CN115713680A (en) | Semantic guidance-based face image identity synthesis method | |
WO2023072067A1 (en) | Face attribute editing model training and face attribute editing methods | |
CN113140020B (en) | Method for generating image based on text of countermeasure network generated by accompanying supervision | |
Li et al. | Learning face image super-resolution through facial semantic attribute transformation and self-attentive structure enhancement | |
CN116309913B (en) | Method for generating image based on ASG-GAN text description of generation countermeasure network | |
CN111161158B (en) | Image restoration method based on generated network structure | |
CN116612211B (en) | Face image identity synthesis method based on GAN and 3D coefficient reconstruction | |
CN116246022B (en) | Face image identity synthesis method based on progressive denoising guidance | |
CN115311720A (en) | Defekake generation method based on Transformer | |
CN113379597A (en) | Face super-resolution reconstruction method | |
Bhunia et al. | Word level font-to-font image translation using convolutional recurrent generative adversarial networks | |
CN112949707A (en) | Cross-mode face image generation method based on multi-scale semantic information supervision | |
CN115063463A (en) | Fish-eye camera scene depth estimation method based on unsupervised learning | |
CN110415261B (en) | Expression animation conversion method and system for regional training | |
CN115909160A (en) | Method and device for detecting depth video frame insertion and computer readable storage medium | |
Endo et al. | Few-shot semantic image synthesis using stylegan prior | |
CN114463214A (en) | Double-path iris completion method and system guided by regional attention mechanism | |
Luan et al. | Learning unsupervised face normalization through frontal view reconstruction | |
CN114155139A (en) | Deepfake generation method based on vector discretization representation | |
Lai et al. | Generative focused feedback residual networks for image steganalysis and hidden information reconstruction | |
CN115496134B (en) | Traffic scene video description generation method and device based on multi-mode feature fusion | |
CN108305219A (en) | A kind of image de-noising method based on uncorrelated sparse dictionary |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
GR01 | Patent grant | ||
GR01 | Patent grant |