CN113658035B

CN113658035B - Face transformation method, device, equipment, storage medium and product

Info

Publication number: CN113658035B
Application number: CN202110944348.6A
Authority: CN
Inventors: 颜剑锋
Original assignee: Beijing Baidu Netcom Science and Technology Co Ltd
Current assignee: Beijing Baidu Netcom Science and Technology Co Ltd
Priority date: 2021-08-17
Filing date: 2021-08-17
Publication date: 2023-08-08
Anticipated expiration: 2041-08-17
Also published as: CN113658035A

Abstract

The disclosure provides a face transformation method, a face transformation device, face transformation equipment, a face transformation storage medium and a face transformation product, relates to the technical field of artificial intelligence, in particular to the technical field of computer vision and deep learning, and can be applied to scenes such as face recognition. The specific implementation scheme is as follows: determining a source face image and a target face image; replacing the facial organs of the source face image with the facial organs in the target face image to obtain a facial organ image; extracting distance data between the face area of the source face image and the face area of the target face image to obtain face shape distance information; and according to the face shape distance information, adjusting the face shape in the facial organ image according to the face shape of the source face image to obtain a target face-changing image. The technical scheme of the disclosure improves the face transformation precision.

Description

Face transformation method, device, equipment, storage medium and product

Technical Field

The disclosure relates to the technical field of artificial intelligence, in particular to the technical field of computer vision and deep learning, which can be applied to scenes such as face recognition, and particularly relates to a face transformation method, a device, equipment, a storage medium and a product.

Background

In scenes such as advertisements, movies, entertainment, etc., the application of face image replacement is becoming wider and wider. For example, in a cartoon video with a duration of 10 minutes in an entertainment scene, the faces of cartoon characters in the cartoon video can be replaced by the faces of the source images in order to improve the interest. Since a video is composed of a plurality of images, when replacing a face in the video, it is necessary to replace the face with each target image in the video. The existing face-changing realization effect is not accurate enough.

Disclosure of Invention

The present disclosure provides a face transformation method, apparatus, device, storage medium, and product.

According to a first aspect of the present disclosure, there is provided a face transform method including:

determining a source face image and a target face image;

replacing the facial organs of the source face image with the facial organs in the target face image to obtain a facial organ image;

extracting distance data between the face area of the source face image and the face area of the target face image to obtain face shape distance information;

and according to the face shape distance information, adjusting the face shape in the facial organ image according to the face shape of the source face image to obtain a target face-changing image.

According to a second aspect of the present disclosure, there is provided a face transforming apparatus including:

an image determination unit configured to determine a source face image and a target face image;

an organ replacement unit configured to replace a facial organ of the source face image with a facial organ in the target face image, obtaining a facial organ image;

a distance extraction unit for extracting distance data between the face area of the source face image and the face area of the target face image to obtain face shape distance information;

and the facial form adjusting unit is used for adjusting the facial form in the facial organ image according to the facial form distance information and obtaining a target face-changing image according to the facial form of the source face image.

According to a third aspect of the present disclosure, there is provided an electronic device comprising:

at least one processor; and

a memory communicatively coupled to the at least one processor; wherein,,

the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of the first aspect.

According to a fourth aspect of the present disclosure, there is provided a non-transitory computer readable storage medium storing computer instructions for causing the computer to perform the method of the first aspect.

According to a fifth aspect of the present disclosure, there is provided a computer program product comprising: a computer program stored in a readable storage medium, from which it can be read by at least one processor of an electronic device, the at least one processor executing the computer program causing the electronic device to perform the method of the first aspect.

According to the technology disclosed by the invention, the problem that the face replacement is not accurate enough due to the fact that only the facial organ of the source image is replaced by the facial organ of the target image is solved, the face distance information is obtained by calculating the face difference between the source face image and the target face image, and then the face of the image after the facial organ replacement is subjected to face adjustment by utilizing the face distance information, so that the target face-changing image is obtained. The obtained target face-changing image is more matched with the face shape of the source face image, and the accuracy and precision of face changing are improved.

It should be understood that the description in this section is not intended to identify key or critical features of the embodiments of the disclosure, nor is it intended to be used to limit the scope of the disclosure. Other features of the present disclosure will become apparent from the following specification.

Drawings

The drawings are for a better understanding of the present solution and are not to be construed as limiting the present disclosure. Wherein:

FIG. 1 is a network architecture diagram of a face transformation method provided in accordance with the present disclosure;

fig. 2 is a flowchart of a face transformation method provided according to a first embodiment of the present disclosure;

fig. 3 is a flowchart illustrating a face transformation method according to a second embodiment of the present disclosure;

FIG. 4 is a graphical schematic diagram of face distance information provided in accordance with the present disclosure;

FIG. 5 is a graphical schematic of keypoints provided according to the disclosure;

fig. 6 is a flowchart of a face transformation method provided according to a third embodiment of the present disclosure;

FIG. 7 is a schematic illustration of a mask image provided in accordance with the present disclosure;

fig. 8 is a flowchart of a face transformation method provided according to a fourth embodiment of the present disclosure;

FIG. 9 is a schematic application diagram of a face transformation method provided in accordance with the present disclosure;

fig. 10 is a schematic structural view of a face transforming device provided according to a fifth embodiment of the present disclosure;

fig. 11 is a block diagram of an electronic device for implementing a face transformation method of an embodiment of the present disclosure.

Detailed Description

Exemplary embodiments of the present disclosure are described below in conjunction with the accompanying drawings, which include various details of the embodiments of the present disclosure to facilitate understanding, and should be considered as merely exemplary. Accordingly, one of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the present disclosure. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.

The technical scheme disclosed by the invention can be applied to any scene requiring face shape transformation, particularly the technical field of artificial intelligence, in particular the technical field of computer vision and deep learning, can be applied to scenes such as face recognition and the like, effectively constrains the face transformation through face distance information, can realize automatic face transformation in video, and improves the face transformation efficiency and accuracy.

In the prior art, facial transformation is mostly performed by facial organ transformation, for example, a common facial transformation method based on GAN (Generative Adversarial Nets) algorithm for generating an countermeasure network is a direct facial organ replacement. And the face fusion scheme is used for replacing the face in the source image with the face in the target image. In general, the GAN algorithm may extract image identification information, i.e., ID information, of a source image using an identification encoder, and may include information of the shape of eyes, the distance between the mouth and eyes, a bending procedure of the mouth, and the like. Meanwhile, the attribute extractor is utilized to extract image attribute information of the target image, such as information of the pose, outline, facial expression, hairstyle, skin color, scene illumination and the like of the face. Then, the image identification information of the source image and the image attribute information of the target image are input to a face-changing model, for example, an AAD (Adaptive Attention Denormalization Generator, adaptive non-normalized generator) generator, and the facial organ in the source image is replaced with the facial organ of the target image to obtain a face-changing image corresponding to the target image. The replacement of facial organs can be achieved for any target image in the video. However, the face-changing scheme of the existing GAN algorithm can only replace facial organs in the target image, such as facial features, and has poor face-changing effect and low accuracy.

In order to solve the technical problems, the inventor finds that in the existing face-changing scheme, the face-changing effect is not high because the influence of the face shape is ignored, the face shape of the actually obtained face-changing image is still the original face shape in the target image, and the difference is larger compared with the face shape in the source image, so that the face-changing is inaccurate and the effect is poor.

Accordingly, the inventors propose a technical solution of the present disclosure. In the embodiment of the disclosure, after the source face image and the target face image are determined, the facial organs of the source face image may be replaced with the facial organs in the target face image, so as to obtain a facial organ image, and complete the preliminary transformation of the facial organs. Then extracting distance data between the face area of the source face image and the face area of the target face image to obtain face shape distance information; and then according to the facial form distance information, the facial form in the facial organ image is adjusted according to the facial form of the source facial image, so as to obtain a target face-changing image. And performing facial form adjustment on the image after facial organ replacement by using the facial form distance information to obtain a target face-changing image. The obtained target face-changing image is more matched with the face shape of the source face, and the face-changing accuracy is improved.

The disclosure provides a face transformation method, a device, equipment, a storage medium and a product, which are applied to the technical field of computer vision and deep learning in artificial intelligence technology, and can be applied to scenes such as face recognition and the like so as to achieve the technical effect of accurately realizing face transformation.

The technical scheme of the present disclosure will be described in detail with reference to the accompanying drawings.

Fig. 1 is a network architecture diagram of one application of a face transformation method for images or videos provided in accordance with the present disclosure. As shown in fig. 1, the network architecture may include an electronic device, such as a server 1 and a user device 2 connected to the server 1 through a lan or wan, where the user device is assumed to be a personal computer 2. The electronic device may be, for example, a general server, a cloud server, or other type of server, or a computer, a notebook, an supercomputer, or other device, and the specific type of the electronic device is not limited in this disclosure. The user device 2 may be, for example, a terminal device such as a computer, a notebook, a tablet computer, a wearable device, an intelligent home appliance, a vehicle-mounted device, etc., and the specific type of the user device in the embodiment of the present disclosure is not limited too much. The user device 2 may provide an image to the server 1. The server 1 may perform face conversion on the image using the face conversion method shown in the following embodiment to obtain a target face-change image. The target face-change image is then fed back to the user device 2.

As shown in fig. 2, which is a flowchart of a face transformation method according to a first embodiment of the present disclosure, as shown in fig. 2, an execution subject of the face transformation method provided in the present application is a face transformation device, and the face transformation device is located in an electronic device, and the face transformation method provided in the embodiment may include the following steps:

201: a source face image and a target face image are determined.

The source face image may be an image corresponding to a face region in the initial source image. The target face image may be an image corresponding to a face region in the initial target image.

The initial source image may be an image providing a replacement facial organ and the initial target image may be an image of the replaced facial organ. The face in the initial target image is replaced with the face shape of the initial target image, and no replacement occurs in the background area other than the face in the initial target image.

202: and replacing the facial organs of the source face image with the facial organs in the target face image to obtain a facial organ image.

Wherein the facial organ may be at least one of eyebrow, eye, ear, mouth, nose.

Replacing the facial organ of the source face image with the facial organ in the target face image, obtaining the facial organ image may include: all facial organs displayed by the source facial image are replaced with facial organs of the target facial image. In some special scenarios, the ears may not be displayed in the source face image, and then the eyebrows, eyes, mouth, and nose in the source face image may be replaced with the facial organs of the target face image when the facial organs are replaced.

203: distance data between the face region of the source face image and the face region of the target face image are extracted, and face shape distance information is obtained.

The face type distance information may be distance data between a face region of the source face image and a face region of the target face image. The face region may be an image region contained by the face contour.

The face type distance information may be distances between a plurality of pixel point positions in the face region of the source face image and a plurality of pixel point positions in the face region of the target face image, the face type distance information including a position distance between the pixel points.

204: and according to the facial form distance information, adjusting the facial form in the facial organ image according to the facial form of the source face image to obtain a target face-changing image.

According to the face shape distance information, the face shape in the facial organ image is adjusted according to the face shape of the source face image, and the obtaining of the target face change image specifically comprises the following steps: and adjusting the facial form in the facial organ image according to the facial form distance information so as to enable the facial form of the facial organ image to be more matched with the facial form of the source face image, and obtaining a target face-changing image.

In this embodiment, after the source face image and the target face image are determined, the facial organs of the source face image may be replaced with the facial organs in the target face image, so as to obtain a facial organ image, and complete the preliminary transformation of the facial organs. Then extracting distance data between the face area of the source face image and the face area of the target face image to obtain face shape distance information; and then according to the facial form distance information, the facial form in the facial organ image is adjusted according to the facial form of the source facial image, so as to obtain a target face-changing image. And performing facial form adjustment on the image after facial organ replacement by using the facial form distance information to obtain a target face-changing image. The obtained target face-changing image is more matched with the face shape of the source face, and the face-changing accuracy is improved.

In order to extract accurate face shape distance information, referring to fig. 3, in the above embodiment, step 203: extracting distance data between a face region of a source face image and a face region of a target face image to obtain face shape distance information may include:

301: the source face image and the target face image are divided into at least one image group.

Wherein any one of the image groups includes a first region image of the source face image and a second region image of the target face image. The face region positions of the first region image and the second region image are the same.

The face region of the source face image may be formed by combining the first region images respectively corresponding to the at least one image group according to the respective region position groups.

The face region of the target face image may be formed by combining the second region images respectively corresponding to the at least one image group according to the respective region positions.

The face region may be an image region included in the face contour.

302: and for any image group, calculating the pixel point distance between the first area image and the second area image in the image group, and obtaining the pixel point distance corresponding to the image group so as to obtain the pixel point distance respectively corresponding to at least one image group.

The pixel point distance corresponding to any one image group may be a position distance between all pixel points in the first area image and all pixel points in the second area image of the image group, and may specifically be formed by a plurality of position distances.

303: and carrying out distance fusion on the distances of the pixel points of at least one image group according to the positions of the face areas, and obtaining face shape distance information.

The face type distance information may be obtained by performing distance fusion on the respective pixel distances of at least one image group according to the respective face region positions, and specifically, the respective plurality of position distances of at least one image group may be spliced according to the respective face region positions to form the plurality of position distances of the face region in a combined manner.

Alternatively, the face distance information may be a data stream (flow) corresponding to a plurality of position distances, and for convenience of understanding, reference may be specifically made to the schematic diagram of the face distance information 401 shown in fig. 4 when the face distance information is graphically displayed.

The face shape in the facial organ image is adjusted according to the face shape of the source face image according to the face shape distance information, and the target face-changing image can be obtained specifically by carrying out pixel point difference processing on all pixel points in the facial organ image by utilizing a plurality of position distances corresponding to the face shape distance information.

In this embodiment, when calculating the distance between the source face image and the target face image, the images may be partitioned to calculate the distances between different regions, and the pixel point distances of the regions are fused to obtain final face shape distance information. The distance calculation is carried out by taking the pixel points as units, so that the distance between the images can be accurately calculated, and the calculation accuracy of the distance is improved.

In order to improve the efficiency of dividing the image group, in the above embodiment, step 301: the dividing of the source face image and the target face image into at least one image group may include:

and extracting a first key point corresponding to the face area in the source face image and a second key point corresponding to the face area in the target face image.

And generating a triangular network based on the first key point and the second key point respectively, and obtaining a first triangular network corresponding to the first key point and a second triangular network corresponding to the second key point.

The source face image is divided into at least one first region image according to a first triangle network.

The target face image is divided into at least one second region image according to a second triangle network.

And dividing at least one first area image and at least one second area image into at least one image group according to the grid corresponding relation of the first triangle network and the second triangle network.

Alternatively, a keypoint model may be employed to extract a first keypoint of the source facial image and a second keypoint of the target facial image, respectively. The key point extraction mode of the source face image and the target face image is the same.

The first keypoints may comprise a plurality and the second keypoints may comprise a plurality. In practical application, each extracted key point can be numbered, so that the key point position corresponding relation of the first key point and the second key point is more definite. Assuming that the number of the extracted first key points is 150, the number of the extracted second key points is also 150, the same coding mode can be adopted, and numbers are set for each first key point and each second key point. For example, the number of the key point where the tip of the nose is located may be 57 in both the first key point and the second key point.

For ease of understanding, fig. 5 shows a schematic diagram of key points of an image provided in the present disclosure, and referring to fig. 5, a face image used in the present disclosure may be a simulated face image, in practical application, a real face image, and fig. 5 is merely for illustrating positions of key points of the face image, and should not constitute a specific limitation on the face image of the present disclosure. In practical applications, a corresponding identifier may be set for each key point, such as the number set for a part of key points in fig. 5: 1-6, fig. 5 does not set numbers for all key points, it will be understood that in practical applications, all key points may be set with corresponding numbers, and no longer set one by one.

When the first triangle network of the first key point and the second triangle network corresponding to the second key point are generated, a predetermined triangle network generation rule can be adopted for generation. For example, two key points having a connection relationship may be specified in the triangle network generation rule, and referring to fig. 5, the predetermined triangle network generation rule may specify that the number is that the key point 1 has a connection relationship with the key point 2 and the key point 6, the key point 2 has a connection relationship with the key point 1 and the key point 6, the key point 6 has a connection relationship with the key point 5 and the key point 3, the key point 3 is connected with the key point 4, and the key point 4 is connected with the key point 5. The three key points are connected with each other, and can be considered to form a triangle area. For example, keypoint 1, keypoint 6, and keypoint 2 may form a triangular region 201. Any one of the triangular networks may be constituted by at least one triangular region. Of course, the connection relationship shown in fig. 5 is only schematic, and the connection relationship of all key points is not shown in the drawing, and in practical application, all key points of any face image may be in a triangle network to form a triangle network that may cover the whole face area.

Wherein each triangular network may comprise at least one triangular region.

Dividing the source face image into at least one first region image according to the first triangle network may specifically include: and respectively extracting the region images of the source face image according to at least one first triangle region corresponding to the first triangle network, and obtaining first region images corresponding to the at least one first triangle region respectively so as to obtain at least one first region image.

Dividing the target face image into at least one second region image according to the first triangle network may specifically include: and respectively extracting the region images of the target face image according to at least one second triangular region corresponding to the second triangular network to obtain second region images corresponding to the at least one second triangular region respectively so as to obtain at least one second region image.

Alternatively, since the key point connection manner of each triangle area and the position relative to the face are preset, a corresponding mesh number may be set for each triangle area. After the division of the area image using the triangle network, the mesh number of each triangle area may be set to the mesh number of the corresponding area image. At this time, dividing the at least one first area image and the at least one second area image into at least one image group according to the mesh correspondence of the first triangle network and the second triangle network may include: determining respective grid numbers of at least one first region image according to respective grid numbers of at least one first triangle region in the first triangle network; determining respective grid numbers of at least one second region image according to respective grid numbers of at least one second triangle region in the second triangle network; the first area image and the second area image with the same grid number are divided into the same image group to obtain at least one image group. Wherein the number of at least one image group is equal to the number of triangle areas in the triangle network. The mesh number of any one region image is the mesh number of the triangular region corresponding to the region image.

In practical application, besides the above number setting mode, machine learning algorithms such as nearest neighbor distance division and the like can be adopted to divide the image group, and the dividing mode is the same as that of the prior art, so that description is omitted here for brevity.

In this embodiment, according to the manner of extracting key points and constructing a triangle network by using the key points, the face image is subjected to region division, so as to realize accurate division of the face image, so that the region image of the source face image and the region image of the target face image are divided into the same image group by using the positions of the triangular regions, so that the distance calculation is conveniently performed on the region images corresponding to the grids at the same positions, the accurate distance calculation of the region images is realized, and the obtained pixel point distance is more accurate.

In one possible design, in order to accurately calculate the pixel distance, after the image area division is performed by using the triangle network in the above embodiment to perform the grouping of the area images, when calculating the pixel distance, the pixel distance between the first area image and the second area image is calculated for any one image group, so as to obtain the pixel distance corresponding to the image group, which may be implemented by the following steps:

And determining a first sub-key point of the triangular region corresponding to the first region image in the image group and a second sub-key point of the triangular region corresponding to the second region image in the image group aiming at any one image group.

And calculating a position distance matrix of the first sub-key point and the second sub-key point.

And mapping the pixel positions of the first pixel points in the first area image into a plurality of mapping positions by using the position distance matrix.

And calculating the position difference between the mapping positions and the pixel positions of each of the plurality of second pixels of the second region image to obtain the pixel point distance corresponding to the image group.

The first sub-keypoints may be keypoints of the first region image that constitute a triangular region corresponding to the first region image. Likewise, the second sub-keypoints may be keypoints of the second region image that constitute a corresponding triangle region in the second keypoint. For example, the triangle 501 in fig. 5 corresponds to the first sub-keypoint being keypoint 1, keypoint 2 and keypoint 6, i.e. the three vertices of the triangle 501.

The location distance matrix may be a transformation matrix of the first sub-keypoint and the second sub-keypoint. Assuming that the location distance matrix is represented by W, the first sub-keypoint by D1 and the second sub-keypoint by D2, the formula can be transformed by the matrix: wd1=d2, a conversion matrix W, i.e. a position distance matrix, is calculated.

Mapping the pixel positions of each of the plurality of first pixel points in the first region image into a plurality of mapped positions using the position distance matrix may include: and determining the matrix corresponding to each pixel position of a plurality of first pixel points of the first area image as G1, calculating the matrix product of the first area image corresponding matrix G1 and the position distance matrix W, and obtaining a plurality of mapping positions formed by the mapping matrix Y. The conversion formula may be wg1=y.

Assuming that a matrix of pixel positions of each of a plurality of second pixels of the second region image is G2, the pixel point distance can be obtained by calculating the distance between G1 and G2. The pixel point distance between G1 and G2 can be calculated by a distance calculation formula. For example, the Huffman distance formula, the 1-norm distance formula, the 2-norm distance formula, etc., are not described in detail herein.

In general, the face shape of the target face image also has a certain influence on the face shape transformation, and the mask image of the target face image can be adopted to restrict the process of the face shape transformation so as to realize the accurate transformation of the face. As shown in fig. 6, there is provided a further flowchart of a face transformation method according to a third embodiment of the present disclosure, which may include the steps of:

601: a source face image and a target face image are determined.

In this embodiment, some steps are the same as those in the foregoing embodiments, and for brevity of description, detailed description and technical effects of each identical step are not repeated here.

602: face region information and facial organ region information in the target face image are determined.

603: a target mask image of a target face image is generated based on the face region information and the face organ region information.

The target mask image may be generated from the face region information and the facial organ region information.

As a possible implementation manner, the face identifier may be set to a pixel point corresponding to another region out of the five-element region in the face region in the target face image. And setting organ identifications of pixel points corresponding to the organ areas in the target face image according to the respective organ names. For example, the pixel of the eye region may be set to 1, the pixel of the nose region may be set to 2, the pixel of the eyebrow region may be set to 3, the pixel of the mouth region may be set to 4, the pixel of the ear region may be set to 5, and the pixels corresponding to the other regions except the organ region in the face region may be set to 6. The facial region information may be pixel point position information identified as 6, and the facial organ region information may be corresponding position information identified for the pixel points of each organ.

The target mask image may be a binary image corresponding to any image region smaller than the face region information and larger than the face organ region information. As shown in fig. 7, a schematic diagram of a mask image is shown, where the face area is 701.

604: and replacing the facial organs of the source face image with the facial organs in the target face image to obtain a facial organ image.

605: distance data between the face region of the source face image and the face region of the target face image are extracted, and face shape distance information is obtained.

606: based on the face distance information and the target mask image, the face of the facial organ image is processed according to the face of the source face image, and a target face-changing image is obtained.

The target face-change image may be obtained by performing image processing on the facial organ image based on the face-shape distance information and the target mask image.

In this embodiment, when the face shape is changed, the target mask image of the target face image is used to accurately adjust the face shape of the target face image according to the face shape distance information, so as to achieve accurate adjustment of the face shape.

In the actual adjustment using the target mask image and the face distance information, the adjustment may be performed by using an affine transformation method. In the above embodiment, step 606: based on the face distance information and the target mask image, the face of the facial organ image is processed according to the face of the source face image to obtain a target face-changing image, which specifically includes:

And performing face calibration calculation on the facial organ image based on the target mask image to obtain an intermediate image.

Affine transformation calculation is carried out on the intermediate image by using the face type distance information, and the target face-changing image is obtained.

Wherein performing face shape calibration calculation on the facial organ image based on the target mask image may include: and inputting the target mask image, the facial organ image and the target face image into a face type calibration formula to obtain an intermediate image. The face calibration formula can be expressed as:

G _img *mask+X _t *(1-mask)

wherein G is _img Mask image for facial organ image, mask for object, X _t Is the target face image. The result obtained by the calculation of the formula can be an intermediate image.

Performing affine transformation calculation on the intermediate image by using face shape distance information, the obtaining of the target face change image may include: and inputting the face shape distance information and the intermediate image into an affine transformation formula, and calculating to obtain the target face-changing image. The affine transformation formula can be expressed as:

result＝warp(G _img *mask+X _t *(1-mask),flow)

the flow is face type distance information, and result is a target face changing image.

In the embodiment, the affine transformation mode is adopted to adjust and calculate the face shape, so that the accurate transformation of the face shape is realized, and the transformation efficiency and the transformation accuracy are improved.

The step of replacing the facial organ in the above embodiment, replacing the facial organ of the source facial image with the facial organ in the target facial image, obtaining the facial organ image, may include:

based on a preset plurality of feature extractors, a plurality of image feature information of the source face image is extracted.

Wherein, the extraction modes of the plurality of feature extractors are different.

The feature extractor may be a mathematical model that extracts identifying features of the source face image. The identification feature may be a facial special feature that is capable of identifying the source facial image so that features that may be used to distinguish different source facial images may also be referred to as image ID (Identity document, identification) features. The plurality of feature extractors may include, for example: arcface (Additive angular margin loss for deep face recognition, depth face recognition based on modified loss function), cosface (Large Margin Cosine Loss for Deep Face Recognition, depth face recognition based on large amplitude cosine loss function), baidouce (hundred degree face recognition), sphere face recognition (Deep Hypersphere Embedding for Face Recognition), and the like. For example, 3 feature extractors therein may be taken to extract 3 image feature information.

And extracting image attribute information of the target face image based on the preset attribute extractor.

The plurality of image feature information and the image attribute information are simultaneously input into a face-changing model to replace a facial organ in the target face image with a facial organ of the source face image, and a facial organ image is obtained.

In this embodiment, a plurality of feature extractors with different extraction modes are used to extract a plurality of image feature information of the source face image, so as to realize extraction of a plurality of feature information, so that the feature information of the source face image is more comprehensive, and an organ transformation effect with higher accuracy is obtained.

The source face image and the target face image in the above embodiments may be face-posture-corrected images. Referring to fig. 8, in the face transformation method according to the present disclosure, the specific implementation steps of determining the source face image and the target face image may include:

801: and acquiring an initial source image to be replaced and an initial target image.

In one possible design, the initial source image and the initial target image may be provided by a target user initiating the face transformation. Specifically, an initial source image and an initial target image provided by a target user can be obtained in response to a face transformation request triggered by the target user.

In yet another possible design, the initial target image may be an image in the video, specifically, an image with a certain target face in the video may be determined as a target image, so as to determine all target images in the video, further, all target images in the video are respectively used as initial target images, and the face of the source initial image replaces the face in all initial target images, so as to complete the face replacement of the video.

802: the face region of the initial source image is extracted to correspond to the first face image and the face region of the initial target image is extracted to correspond to the second face image.

Alternatively, extracting the face region of the initial source image corresponding to the first face image and the face region of the initial target image corresponding to the second face image may include:

according to a face recognition algorithm, a first face image corresponding to a face rectangular area of an initial source image and a second face image corresponding to a face rectangular area of an initial target image are extracted.

The face recognition algorithm may be an existing face recognition algorithm or the like.

803: and correcting the face posture of the first face image to obtain a source face image corresponding to the initial source image.

804: and correcting the face posture of the second face image to obtain a target face image corresponding to the initial target image.

In this embodiment, after face region extraction and face pose correction are performed on the initial source image and the initial target image, a source face image and a source target image with corrected poses are obtained, a transformation basis is provided for subsequent image transformation, and the transformation effect of the corrected source face image and source target image is higher.

In practical application, if the initial source image provided by the user is an image of the face area, and the initial target image is also an image of the face area, the initial source image and the initial target image do not need to be extracted from the face area, and the initial source image can be directly determined to be the first face image and the initial source image to be the second face image.

In practical applications, if the initial source image provided by the user includes an image of other regions besides the face region, for example, an image of a body part and/or an image of an environmental region, the image of the other regions except the face region may be referred to as a background image. That is, the initial source image may be divided into a first face image and a first background image, and the initial target image may be divided into a first face image and a second background image.

On the basis of the foregoing embodiment, after the target face-changing image is acquired, the target face-changing image may be restored to the face pose of the second face image, to obtain the target pose image, and then the target pose image and the second background image may be spliced to form a final target image.

Wherein the restoring the target face-change image to the face pose of the second face image, the obtaining the target pose image may include: and performing matrix conversion on the target face-changing image according to the inverse matrix of the second transformation matrix to obtain a target attitude image.

As one embodiment, step 802 above: performing face pose correction on the first face image to obtain a source face image corresponding to the initial source image, which may include:

and correcting the face posture of the first face image based on a preset standard face image to obtain a source face image corresponding to the initial source image.

Wherein, the included angle between the two-dimensional plane of the face in the standard face image and the horizontal plane is a right angle.

Step 803 described above: performing face pose correction on the second face image to obtain a target face image corresponding to the target initial image, which may include:

and correcting the face posture of the second face image based on the standard face image to obtain a target face image corresponding to the initial target image.

The standard face image may be obtained by setting in advance. The face of the standard face image has normal pose, and specifically can be that the included angle between the two-dimensional plane where the face is positioned and the horizontal plane is a right angle. The center points of the two eyes in the standard facial image are connected to form a first line segment, and the midpoint of the first line segment is connected with the nose tip point to form a second line segment. The first line segment is perpendicular to the second line segment.

In this embodiment, the face pose correction is performed on the first face image and the second face image by using the standard face image, so that the face pose is accurately corrected, the face correction is matched with the pose of the standard face image, the pose correction is more effective, and the accuracy is higher.

In one possible design, performing facial pose correction on the first face image based on a preset standard face image, obtaining a source face image corresponding to the initial source image may include:

and extracting a third key point of the first face image, and determining a standard key point of the standard face image.

And calculating a first transformation matrix according to the third key point and the standard key point.

And performing matrix calibration calculation on the first face image by using the first transformation matrix to obtain a source face image.

Alternatively, a third keypoint of the first facial image and a fourth keypoint of the second facial image may be extracted, respectively, using a keypoint model. The key point extraction mode of the first face image and the second face image is the same. The third keypoints may comprise a plurality and the fourth keypoints may comprise a plurality.

Wherein, according to the third key point and the standard key point, calculating the first transformation matrix may include: and performing matrix calculation through the first transformation matrix according to the third key points to obtain a matrix conversion principle of the standard key points, and determining the first transformation matrix. The first transformation matrix may be a product of a position matrix of the standard keypoint and an inverse of a position matrix of the third keypoint.

In this embodiment, the first face image is subjected to pose correction through the first transformation matrix, so that accurate pose correction of the image is realized, and the obtained source face image is more accurate.

In yet another possible design, performing facial pose correction on the second face image based on the standard face image to obtain a target face image corresponding to the initial target image may include:

and extracting a fourth key point of the second face image, and determining a standard key point of the standard face image.

And calculating a second transformation matrix according to the fourth key point and the standard key point.

And performing matrix calibration calculation on the second face image by using the second transformation matrix to obtain a target face image.

Wherein calculating the second transformation matrix according to the fourth keypoint and the standard keypoint may comprise: and performing matrix calculation through the second transformation matrix according to the fourth key points to obtain a matrix conversion principle of the standard key points, and determining the second transformation matrix. The second transformation matrix may be a product of a position matrix of the standard keypoint and an inverse of a position matrix of the fourth keypoint.

In this embodiment, the second transformation matrix is used to correct the pose of the second face image, so as to correct the accurate pose of the image, and the obtained target face image is more accurate.

For ease of understanding, as shown in fig. 9, a schematic diagram of an application of face transformation implemented according to the face transformation method provided in the present disclosure is shown. The user may input the source face image 901 and the target face image 902 into an electronic device configured with a face transformation apparatus, such as a cloud server M2, through a user device, such as a tablet M1. The face transformation device can implement the face transformation method of the present disclosure.

The electronic device M2 may perform the face transformation method of the embodiment of the present disclosure, that is: and replacing the facial organs of the source face image with the facial organs in the target face image to obtain a facial organ image. Distance data between the face region of the source face image and the face region of the target face image are extracted, and face shape distance information is obtained. And according to the facial form distance information, adjusting the facial form in the facial organ image according to the facial form of the source facial image to obtain a target face-changing image. The obtained target face-change image may be as shown at 903. The matching degree of the face shape of the target face image 903 and the face shape of the source face image 901 is greatly improved, and the face changing precision is higher.

As shown in fig. 10, a schematic structural diagram of an embodiment of a face transformation device 1000 according to a fourth embodiment of the present disclosure may include the following units:

the image determination unit 1001: for determining a source face image and a target face image.

Organ replacement unit 1002: for replacing the facial organs of the source face image with the facial organs in the target face image, obtaining a facial organ image.

Distance extraction unit 1003: and the face type distance information is obtained by extracting the distance data between the face area of the source face image and the face area of the target face image.

Face adjusting section 1004: the facial mask image processing method is used for adjusting the facial mask in the facial organ image according to the facial mask of the source face image according to the facial mask distance information to obtain a target face-changing image.

As one embodiment, wherein the distance extraction unit includes:

a first dividing module for dividing the source face image and the target face image into at least one image group; wherein any one of the image groups includes a first region image of the source face image and a second region image of the target face image; the face area positions of the first area image and the second area image are the same;

the distance calculation module is used for calculating the pixel point distances of the first area image and the second area image in the image group aiming at any one image group, and obtaining the pixel point distances corresponding to the image groups so as to obtain the pixel point distances respectively corresponding to at least one image group;

and the distance fusion module is used for carrying out distance fusion on the pixel point distances of at least one image group according to the face areas to obtain face shape distance information.

In one possible design, the first dividing module includes:

the first determining submodule is used for extracting a first key point corresponding to a face area in the source face image and a second key point corresponding to the face area in the target face image;

the network generation sub-module is used for generating a triangular network based on the first key point and the second key point respectively to obtain a first triangular network corresponding to the first key point and a second triangular network corresponding to the second key point;

The first dividing sub-module is used for dividing the source face image into at least one first area image according to a first triangle network;

the second dividing sub-module is used for dividing the target face image into at least one second area image according to a second triangular network;

the region matching sub-module is used for dividing at least one first region image and at least one second region image into at least one image group according to the grid corresponding relation of the first triangle network and the second triangle network.

In some embodiments, the distance calculation module comprises:

the second determining submodule is used for determining a first sub-key point of a triangular region corresponding to the first region image in the image group and a second sub-key point of a triangular region corresponding to the second region image in the image group aiming at any image group;

the distance calculation sub-module is used for calculating a position distance matrix of the first sub-key point and the second sub-key point;

the position mapping sub-module is used for mapping the pixel positions of the first pixel points into a plurality of mapping positions by utilizing the position distance matrix;

and the distance calculation sub-module is used for calculating the position difference value between the plurality of mapping positions and the pixel positions of the plurality of second pixel points to obtain the pixel point distance corresponding to the image group.

As a possible implementation manner, the method further includes:

a region determination unit configured to determine face region information and facial organ region information in a target face image;

a mask determining unit for generating a target mask image of the target face image based on the face area information and the facial organ area information;

a face adjustment unit comprising:

the first adjusting module is used for processing the facial form in the facial organ image according to the facial form of the source facial image based on the facial form distance information and the target mask image to obtain a target face change image.

In some embodiments, wherein the first adjustment module comprises:

the calibration calculation sub-module is used for performing facial form calibration calculation on the facial organ image based on the target mask image to obtain an intermediate image;

and the transformation calculation sub-module is used for carrying out affine transformation calculation on the intermediate image by using the face shape distance information to obtain a target face-changing image.

In one possible design, wherein the facial organ replacement unit comprises:

a second extraction module for extracting a plurality of image feature information of the source face image based on a preset plurality of feature extractors; wherein, the extraction modes of the plurality of feature extractors are different;

The third extraction module is used for extracting image attribute information of the target face image based on a preset attribute extractor;

and the identification input module is used for inputting the plurality of image characteristic information and the image attribute information into the face changing model at the same time so as to replace the facial organ in the target face image with the facial organ of the source face image and obtain a facial organ image.

As a possible implementation manner, the image determining unit includes:

the image acquisition module is used for acquiring an initial source image and an initial target image to be replaced;

the area extraction module is used for extracting that the face area of the initial source image corresponds to the first face image and the face area of the initial target image corresponds to the second face image;

the first correction module is used for correcting the face posture of the first face image to obtain a source face image corresponding to the initial source image;

and the second correction module is used for correcting the facial pose of the second facial image to obtain a target facial image corresponding to the initial target image.

In certain embodiments, wherein the first correction module comprises:

the first correction submodule is used for correcting the face posture of the first face image based on a preset standard face image to obtain a source face image corresponding to the initial source image; wherein, the included angle between the two-dimensional plane of the face in the standard face image and the horizontal plane is a right angle;

A second correction module comprising:

and the second correction sub-module is used for carrying out face posture correction on the second face image based on the standard face image to obtain a target face image of the initial target image.

In one possible design, wherein the first correction sub-module comprises:

a first extraction unit for extracting a third key point of the first face image and determining a standard key point of the standard face image;

the first calculation unit is used for calculating a first transformation matrix according to the third key point and the standard key point;

and the first obtaining unit is used for performing matrix calibration calculation on the initial source image by using the first transformation matrix to obtain a source face image.

In yet another possible design, wherein the second correction sub-module comprises:

a second extraction unit for extracting a fourth key point of the second face image and determining a standard key point of the standard face image;

the second calculation unit is used for calculating a second transformation matrix according to the fourth key point and the standard key point;

and the second obtaining unit is used for performing matrix calibration calculation on the second face image by using a second transformation matrix to obtain a target face image.

Furthermore, the present disclosure also provides a non-transitory computer-readable storage medium storing computer instructions for causing a computer to perform a method according to any one of the embodiments provided in the present disclosure.

And a computer program product comprising a computer program which, when executed by a processor, implements a method according to any of the embodiments provided by the present disclosure.

In the technical scheme of the disclosure, the acquisition, storage, application and the like of the related user personal information all conform to the regulations of related laws and regulations, and the public sequence is not violated.

According to embodiments of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.

According to an embodiment of the present disclosure, the present disclosure also provides a computer program product comprising: a computer program stored in a readable storage medium, from which at least one processor of an electronic device can read, the at least one processor executing the computer program causing the electronic device to perform the solution provided by any one of the embodiments described above.

Fig. 11 illustrates a schematic block diagram of an example electronic device 1100 that can be used to implement embodiments of the present disclosure. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular telephones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be exemplary only, and are not meant to limit implementations of the disclosure described and/or claimed herein.

As shown in fig. 11, the apparatus 1100 includes a computing unit 1101 that can perform various appropriate actions and processes according to a computer program stored in a Read Only Memory (ROM) 1102 or a computer program loaded from a storage unit 1108 into a Random Access Memory (RAM) 1103. In the RAM 1103, various programs and data required for the operation of the device 1100 can also be stored. The computing unit 1101, ROM1102, and RAM 1103 are connected to each other by a bus 1104. An input/output (I/O) interface 1105 is also connected to bus 1104.

Various components in device 1100 are connected to I/O interface 1105, including: an input unit 1106 such as a keyboard, a mouse, etc.; an output unit 1107 such as various types of displays, speakers, and the like; a storage unit 1108, such as a magnetic disk, optical disk, etc.; and a communication unit 1109 such as a network card, modem, wireless communication transceiver, or the like. The communication unit 1109 allows the device 1100 to exchange information/data with other devices through a computer network such as the internet and/or various telecommunication networks.

The computing unit 1101 may be a variety of general purpose and/or special purpose processing components having processing and computing capabilities. Some examples of computing unit 1101 include, but are not limited to, a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), various specialized Artificial Intelligence (AI) computing chips, various computing units running machine learning model algorithms, a Digital Signal Processor (DSP), and any suitable processor, controller, microcontroller, etc. The calculation unit 1101 performs the respective methods and processes described above, for example, the face transform method. For example, in some embodiments, the face transformation method may be implemented as a computer software program tangibly embodied on a machine-readable medium, such as storage unit 1108. In some embodiments, some or all of the computer programs may be loaded and/or installed onto device 1100 via ROM1102 and/or communication unit 1109. When the computer program is loaded into the RAM 1103 and executed by the computing unit 1101, one or more steps of the face transform method described above may be performed. Alternatively, in other embodiments, the computing unit 1101 may be configured to perform the face transformation method by any other suitable means (e.g. by means of firmware).

Various implementations of the systems and techniques described here above may be implemented in digital electronic circuitry, integrated circuit systems, field Programmable Gate Arrays (FPGAs), application Specific Integrated Circuits (ASICs), application Specific Standard Products (ASSPs), systems On Chip (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs, the one or more computer programs may be executed and/or interpreted on a programmable system including at least one programmable processor, which may be a special purpose or general-purpose programmable processor, that may receive data and instructions from, and transmit data and instructions to, a storage system, at least one input device, and at least one output device.

Program code for carrying out methods of the present disclosure may be written in any combination of one or more programming languages. These program code may be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus such that the program code, when executed by the processor or controller, causes the functions/operations specified in the flowchart and/or block diagram to be implemented. The program code may execute entirely on the machine, partly on the machine, as a stand-alone software package, partly on the machine and partly on a remote machine or entirely on the remote machine or server.

In the context of this disclosure, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. The machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and pointing device (e.g., a mouse or trackball) by which a user can provide input to the computer. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic input, speech input, or tactile input.

The systems and techniques described here can be implemented in a computing system that includes a background component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such background, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), wide Area Networks (WANs), and the internet.

The computer system may include a client and a server. The client and server are typically remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server can be a cloud server, also called a cloud computing server or a cloud host, and is a host product in a cloud computing service system, so that the defects of high management difficulty and weak service expansibility in the traditional physical hosts and VPS service ("Virtual Private Server" or simply "VPS") are overcome. The server may also be a server of a distributed system or a server that incorporates a blockchain.

It should be appreciated that various forms of the flows shown above may be used to reorder, add, or delete steps. For example, the steps recited in the present disclosure may be performed in parallel or sequentially or in a different order, provided that the desired results of the technical solutions of the present disclosure are achieved, and are not limited herein.

The above detailed description should not be taken as limiting the scope of the present disclosure. It will be apparent to those skilled in the art that various modifications, combinations, sub-combinations and alternatives are possible, depending on design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present disclosure are intended to be included within the scope of the present disclosure.

Claims

1. A face transformation method, comprising:

determining a source face image and a target face image;

replacing the facial organs in the target face image with the facial organs in the source face image to obtain a facial organ image;

dividing the source face image and the target face image into at least one image group; wherein any one of the image groups includes a first region image of the source face image and a second region image of the target face image; the face area positions of the first area image and the second area image are the same;

For any image group, determining a first sub-key point of a triangular region corresponding to a first region image in the image group and a second sub-key point of a triangular region corresponding to a second region image in the image group;

calculating a position distance matrix of the first sub-key points and the second sub-key points;

mapping the pixel positions of each of a plurality of first pixel points in the first region image into a plurality of mapping positions by using the position distance matrix;

calculating position differences between the mapping positions and the pixel positions of each of the plurality of second pixels of the second region image to obtain pixel point distances corresponding to the image groups, so as to obtain pixel point distances corresponding to the at least one image group respectively;

performing distance fusion on the pixel point distances of each image group according to the positions of the face areas to obtain face shape distance information;

2. The method of claim 1, wherein the dividing the source face image and the target face image into at least one image group comprises:

Extracting a first key point corresponding to a face area in the source face image and a second key point corresponding to the face area in the target face image;

generating a triangular network based on the first key point and the second key point respectively to obtain a first triangular network corresponding to the first key point and a second triangular network corresponding to the second key point;

dividing the source face image into at least one first area image according to the first triangle network;

dividing the target face image into at least one second area image according to the second triangle network;

and dividing the at least one first area image and the at least one second area image into the at least one image group according to the grid correspondence of the first triangle network and the second triangle network.

3. The method of claim 1 or 2, further comprising:

determining facial region information and facial organ region information in the target facial image;

generating a target mask image of the target face image according to the face area information and the facial organ area information;

the step of adjusting the face shape in the facial organ image according to the face shape of the source face image according to the face shape distance information to obtain a target face change image comprises the following steps:

And processing the face shape of the facial organ image according to the face shape of the source face image based on the face shape distance information and the target mask image to obtain the target face change image.

4. The method of claim 3, wherein the processing the face shape in the facial organ image according to the face shape of the source face image based on the face shape distance information and the target mask image to obtain the target face change image comprises:

performing face shape calibration calculation on the facial organ image based on the target mask image to obtain an intermediate image;

and carrying out affine transformation calculation on the intermediate image by using the face shape distance information to obtain the target face change image.

5. The method of any of claims 1-2, 4, wherein the replacing facial organs in the target facial image with facial organs in the source facial image to obtain a facial organ image comprises:

extracting a plurality of image feature information of the source face image based on a preset plurality of feature extractors; wherein the extraction modes of the plurality of feature extractors are different;

extracting image attribute information of the target face image based on a preset attribute extractor;

And simultaneously inputting the plurality of image characteristic information and the image attribute information into a face-changing model to replace a facial organ in the target face image with a facial organ of the source face image, so as to obtain the facial organ image.

6. The method of any one of claims 1-2, 4, wherein determining the source face image and the target face image comprises:

acquiring an initial source image to be replaced and an initial target image;

extracting a first face image corresponding to the face area of the initial source image and a second face image corresponding to the face area of the initial target image;

correcting the face posture of the first face image to obtain a source face image corresponding to the initial source image;

and correcting the face posture of the second face image to obtain a target face image corresponding to the initial target image.

7. The method of claim 6, wherein the performing facial pose correction on the first face image to obtain a source face image corresponding to the initial source image comprises:

based on a preset standard face image, carrying out face posture correction on the first face image to obtain a source face image corresponding to the initial source image; wherein, the included angle between the two-dimensional plane of the face in the standard face image and the horizontal plane is a right angle;

The correcting the face posture of the second face image to obtain a target face image corresponding to the initial target image includes:

and correcting the face posture of the second face image based on the standard face image to obtain a target face image of the initial target image.

8. The method of claim 7, wherein the performing facial pose correction on the first face image based on a preset standard face image to obtain the initial source image corresponding to a source face image comprises:

extracting a third key point of the first face image, and determining a standard key point of the standard face image;

calculating a first transformation matrix according to the third key point and the standard key point;

and performing matrix calibration calculation on the first face image by using the first transformation matrix to obtain the source face image.

9. The method of claim 7, wherein the pose correcting the second face image based on the standard face image to obtain a target face image of the initial target image comprises:

extracting a fourth key point of the second face image, and determining a standard key point of the standard face image;

Calculating a second transformation matrix according to the fourth key point and the standard key point;

and performing matrix calibration calculation on the second face image by using the second transformation matrix to obtain the target face image.

10. A face transform device, comprising:

an organ replacement unit configured to replace a facial organ of the target face image with a facial organ in the source face image, obtaining a facial organ image;

a distance extraction unit comprising: the device comprises a first dividing module, a distance calculating module and a distance fusing module;

the first dividing module is used for dividing the source face image and the target face image into at least one image group; wherein any one of the image groups includes a first region image of the source face image and a second region image of the target face image; the face area positions of the first area image and the second area image are the same;

the distance calculation module is used for determining a first sub-key point of a triangular region corresponding to a first region image in the image group and a second sub-key point of a triangular region corresponding to a second region image in the image group aiming at any image group;

the distance fusion module is used for carrying out distance fusion on the respective pixel point distances of the at least one image group according to the respective face area positions to obtain face shape distance information;

11. The apparatus of claim 10, wherein the first partitioning module comprises:

the first determining submodule is used for extracting a first key point corresponding to a face area in the source face image and a second key point corresponding to a face area in the target face image;

The network generation sub-module is used for respectively generating a triangular network based on the first key point and the second key point to obtain a first triangular network corresponding to the first key point and a second triangular network corresponding to the second key point;

the first dividing sub-module is used for dividing the source face image into at least one first area image according to the first triangle network;

the second dividing sub-module is used for dividing the target face image into at least one second area image according to the second triangular network;

and the region matching sub-module is used for dividing the at least one first region image and the at least one second region image into the at least one image group according to the grid corresponding relation of the first triangle network and the second triangle network.

12. The apparatus of claim 10 or 11, further comprising:

a region determining unit configured to determine face region information and facial organ region information in the target face image;

a mask determining unit configured to generate a target mask image of the target face image based on the face area information and the facial organ area information;

The face shape adjusting unit includes:

and the first adjusting module is used for processing the facial form in the facial organ image according to the facial form of the source facial image based on the facial form distance information and the target mask image to obtain the target face-changing image.

13. The apparatus of claim 12, wherein the first adjustment module comprises:

a calibration calculation sub-module, configured to perform facial form calibration calculation on the facial organ image based on the target mask image, to obtain an intermediate image;

and the transformation calculation sub-module is used for carrying out affine transformation calculation on the intermediate image by utilizing the face shape distance information to obtain the target face-changing image.

14. The apparatus of any one of claims 10-11, 13, wherein the organ replacement unit comprises:

a second extraction module for extracting a plurality of image feature information of the source face image based on a preset plurality of feature extractors; wherein the extraction modes of the plurality of feature extractors are different;

the third extraction module is used for extracting the image attribute information of the target face image based on a preset attribute extractor;

and the identification input module is used for inputting the plurality of image characteristic information and the image attribute information into a face changing model at the same time so as to replace a facial organ in the target face image with a facial organ of the source face image and obtain the facial organ image.

15. The apparatus according to any one of claims 10-11, 13, wherein the image determination unit comprises:

the region extraction module is used for extracting a first face image corresponding to the face region of the initial source image and a second face image corresponding to the face region of the initial target image;

the first correction module is used for correcting the facial pose of the first facial image to obtain a source facial image corresponding to the initial source image;

16. The apparatus of claim 15, wherein the first correction module comprises:

the first correction sub-module is used for correcting the face posture of the first face image based on a preset standard face image to obtain a source face image corresponding to the initial source image; wherein, the included angle between the two-dimensional plane of the face in the standard face image and the horizontal plane is a right angle;

the second correction module includes:

and the second correction sub-module is used for correcting the face posture of the second face image based on the standard face image to obtain a target face image of the initial target image.

17. The apparatus of claim 16, wherein the first correction sub-module comprises:

a first extraction unit configured to extract a third key point of the first face image, and determine a standard key point of the standard face image;

and the first obtaining unit is used for performing matrix calibration calculation on the initial source image by using the first transformation matrix to obtain the source face image.

18. The apparatus of claim 16, wherein the second correction sub-module comprises:

a second extraction unit configured to extract a fourth key point of the second face image, and determine a standard key point of the standard face image;

and the second obtaining unit is used for performing matrix calibration calculation on the second face image by using the second transformation matrix to obtain the target face image.

19. An electronic device, comprising:

at least one processor; and

A memory communicatively coupled to the at least one processor; wherein,,

the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-9.

20. A non-transitory computer readable storage medium storing computer instructions for causing the computer to perform the method of any one of claims 1-9.