CN115861533A - Digital face generation method and device, electronic equipment and storage medium - Google Patents
Digital face generation method and device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN115861533A CN115861533A CN202211526834.7A CN202211526834A CN115861533A CN 115861533 A CN115861533 A CN 115861533A CN 202211526834 A CN202211526834 A CN 202211526834A CN 115861533 A CN115861533 A CN 115861533A
- Authority
- CN
- China
- Prior art keywords
- face
- facial
- pinching
- distribution
- human face
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Landscapes
- Processing Or Creating Images (AREA)
Abstract
The application provides a digital face generation method, a digital face generation device, electronic equipment and a storage medium, wherein the method comprises the following steps: extracting face features in the face image, wherein the face features indicate distribution features of facial features; determining model characteristics of the human face through a preset human face three-dimensional model, wherein the human face three-dimensional model is constructed based on a plurality of facial feature distribution pinching coefficients, the facial feature distribution pinching coefficients are used for adjusting relative distances between the facial features of the human face in the human face three-dimensional model, and the model characteristics indicate the facial feature distribution characteristics of the human face in the human face three-dimensional model; under the condition that the characteristic error between the human face characteristic and the model characteristic is larger than a preset error threshold value, iteratively optimizing the facial pinching coefficient of the facial features distribution until the iteration times or the characteristic error meets a preset stop condition; and generating the digital face of the face image through the optimized three-dimensional model of the face. The method and the device improve the accuracy of the digital face.
Description
Technical Field
The present application relates to the field of AI face pinching technologies, and in particular, to a digital face generation method and apparatus, an electronic device, and a storage medium.
Background
With the development of scientific technology and the need of modern industrial technology, 3D virtual digital people become a common research hotspot in academic circles and industrial circles, and the technology has wide application in virtual characters, other interactive games and entertainment and the like. In these applications, AI face pinching is an important technical direction, i.e. inputting a face picture, and reconstructing the digital human image of the person by an algorithm, where the reconstructed image has a higher similarity to the input picture.
In the prior art, due to the limitation of reconstruction technology, personalization of digital people is often realized in a material matching mode, however, five sense organs of people have diversity, even if the five sense organs of the same type are distributed on different faces, the feeling of an observer is different, the personalized five sense organs distribution characteristic information of a human face is difficult to accurately represent by using limited materials, and the accuracy of the digital human face is low.
Aiming at the problem of low accuracy of the current digital face, no good solution is available at present.
Disclosure of Invention
In order to solve the technical problems described above or at least partially solve the technical problems, the present application provides a digital face generation method, apparatus, electronic device and storage medium.
In a first aspect, the present application provides a digital face generation method, where the method includes:
extracting face features in the face image, wherein the face features indicate facial feature distribution features;
determining model characteristics of the human face through a preset human face three-dimensional model, wherein the human face three-dimensional model is constructed based on a plurality of facial feature distribution pinching coefficients, the facial feature distribution pinching coefficients are used for adjusting relative distances between the facial features of the human face in the human face three-dimensional model, and the model characteristics indicate the facial feature distribution characteristics of the human face in the human face three-dimensional model;
under the condition that the characteristic error between the human face characteristic and the model characteristic is larger than a preset error threshold value, iteratively optimizing the facial pinching coefficient of the facial features distribution until the iteration times or the characteristic error meets a preset stop condition;
and generating the digital face of the face image through the optimized three-dimensional model of the face.
Optionally, the determining the model feature of the face through a preset three-dimensional model of the face includes:
calculating a plurality of key point coordinates of the human face through a preset human face three-dimensional model;
determining a plurality of characteristic values based on the position relation between the key point coordinates of each organ, wherein the characteristic values are used for indicating the relative distance between the five sense organs of the human face in the human face three-dimensional model;
and taking the plurality of characteristic values as the model characteristics.
Optionally, before the calculating the coordinates of the plurality of key points of the face through the preset three-dimensional model of the face, the method further includes:
setting a single facial pinching coefficient of facial features distribution to be maximum each time, and obtaining a facial pinching coefficient distribution deformation target corresponding to the facial pinching coefficient of facial features distribution, wherein the facial pinching coefficient distribution deformation target of facial features is a model for outputting a deformation digital face, the deformation digital face is obtained relative to a preset basic digital face, and the model for outputting the basic digital face is obtained according to a default facial pinching coefficient of facial features distribution;
initializing the facial pinching coefficient of the distribution of the five sense organs;
and obtaining a human face three-dimensional model based on the model of the output basic digital human face, the initialized facial pinching coefficient of the distribution of the five sense organs and the distribution deformation target of the five sense organs.
Optionally, the obtaining a three-dimensional face model based on the model of the output-based digital face, the initialized facial pinching coefficient of the distribution of the five sense organs and the deformation target of the distribution of the five sense organs includes:
calculating a deformation difference value between each five sense organs distribution deformation target and the model of the output basic digital human face;
weighting and summing each initialized facial pinching coefficient distributed in the five sense organs and the corresponding deformation difference value to obtain a weighting result;
and calculating to obtain the human face three-dimensional model according to the model of the output basic digital human face and the sum of the weighting results.
Optionally, the calculating the coordinates of the plurality of key points of the face through a preset three-dimensional model of the face includes:
acquiring a face material map, wherein pixel points on the face material map and grid points on the face three-dimensional model have a preset corresponding relationship;
determining a plurality of key points of each organ in the face three-dimensional model according to the key points of each organ in the face material map and the corresponding relation;
and coding each key point by adopting a gravity center coordinate method to obtain the key point coordinate of each key point in each organ.
Optionally, each time before setting the single facial coefficient of facial pinching of the five sense organs to maximum, the method further comprises:
keeping the facial pinching coefficient of the distribution of the five sense organs as a default coefficient, and setting the facial pinching coefficient of the shape of a single five sense organs to be the maximum every time to obtain a facial pinching coefficient distribution deformation target of the five sense organs corresponding to the facial pinching coefficient of the distribution of the five sense organs;
initializing the facial pinching coefficient of the shape of the five sense organs;
obtaining an initial human face three-dimensional model based on a model of an output basic digital human face, initialized facial pinching coefficients of facial shapes of five sense organs and a distribution deformation target of the five sense organs;
and generating the shape of the five sense organs based on the initial human face three-dimensional model.
Optionally, the extracting the facial features in the facial image includes:
extracting key points of the face in the face image through a key point detection model;
selecting at least three reference points from the face key points to carry out face alignment;
and extracting the distribution characteristics of the five sense organs based on the aligned human face.
In a second aspect, an apparatus for generating a digital face is provided, the apparatus including:
the extraction module is used for extracting the face features in the face image, wherein the face features indicate the distribution features of the five sense organs of the face;
the determining module is used for determining model characteristics of the human face through a preset human face three-dimensional model, wherein the human face three-dimensional model is constructed based on a plurality of facial feature distribution pinching coefficients, the facial feature distribution pinching coefficients are used for adjusting the relative distances between the facial features of the human face in the human face three-dimensional model, and the model characteristics indicate the facial feature distribution characteristics of the human face in the human face three-dimensional model;
the optimization module is used for iteratively optimizing the facial pinching coefficient of the facial features under the condition that the feature error between the facial features and the model features is larger than a preset error threshold value until the iteration times or the feature error meet a preset stop condition;
and the generating module is used for generating the digital face of the face image through the optimized face three-dimensional model.
In a third aspect, an electronic device is provided, which includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory complete communication with each other through the communication bus;
a memory for storing a computer program;
and the processor is used for realizing any step of the digital human face generation method when executing the program stored in the memory.
In a fourth aspect, a computer-readable storage medium is provided, in which a computer program is stored, which computer program, when being executed by a processor, realizes any of the steps of the digital face generation method.
Compared with the prior art, the technical scheme provided by the embodiment of the application has the following advantages:
according to the method provided by the embodiment of the application, the face three-dimensional model of the human face is constructed based on the facial pinching coefficients distributed by the five sense organs, if the characteristic error between the model characteristic output by the three-dimensional model of the human face and the human face characteristic of the human face image is larger than the preset error threshold value, the facial pinching coefficients distributed by the five sense organs are optimized, the model characteristic is enabled to be continuously close to the human face characteristic by adjusting the relative distance between the five sense organs of the human face in the three-dimensional model of the human face, the facial pinching platform can accurately represent the personalized facial distribution characteristic information of the five sense organs of the human face, the limitation of materials is avoided, and the accuracy of the digital human face is improved.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and together with the description, serve to explain the principles of the invention.
In order to more clearly illustrate the embodiments or technical solutions in the prior art of the present invention, the drawings used in the description of the embodiments or the prior art will be briefly described below, and it is obvious for those skilled in the art to obtain other drawings without inventive labor.
Fig. 1 is a schematic diagram of a hardware environment of a digital face generation method according to an embodiment of the present application;
fig. 2 is a flowchart of a digital face generation method according to an embodiment of the present application;
fig. 3 is a schematic diagram of a 3D grid provided by an embodiment of the present application;
fig. 4 is a flowchart of a digital face generation method according to an embodiment of the present application;
fig. 5 is a schematic structural diagram of a digital face generating device according to an embodiment of the present application;
fig. 6 is a schematic structural diagram of an electronic device according to an embodiment of the present application.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are some embodiments of the present application, but not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
In the following description, suffixes such as "module", "component", or "unit" used to denote elements are used only for the convenience of description of the present application, and have no specific meaning in themselves. Thus, "module" and "component" may be used in a mixture.
In order to solve the problems mentioned in the background art, according to an aspect of the embodiments of the present application, an embodiment of a digital face generation method is provided.
Alternatively, in the embodiment of the present application, the above digital face generation method may be applied to a hardware environment formed by the terminal 101 and the server 103 as shown in fig. 1. As shown in fig. 1, a server 103 is connected to a terminal 101 through a network, which may be used to provide services for the terminal or a client installed on the terminal, and a database 105 may be provided on the server or separately from the server, and is used to provide data storage services for the server 103, and the network includes but is not limited to: wide area network, metropolitan area network, or local area network, and the terminal 101 includes but is not limited to a PC, a cell phone, a tablet computer, and the like.
The digital face generation method in the embodiment of the application can be executed by the server 103 and is used for accurately representing the facial features and generating an accurate digital face.
A detailed description will be given below of the digital face generation method provided in the embodiment of the present application with reference to a specific implementation manner, as shown in fig. 2, the specific steps are as follows:
step 201: and extracting the face features in the face image.
Wherein the face features indicate facial feature distribution characteristics.
In the embodiment of the application, the terminal sends a face image to the server, the server obtains key points (landmark) of the face image based on the face image, wherein the key points comprise eyebrows, eyes, a nose, a mouth, a face contour and the like, and each key point has respective semantic information. The method and the device can obtain the key points by using a key point detection model with 106 points, and can also obtain the key points by using other detection models.
After the server obtains the key points of the face, at least three reference points are selected for aligning the face, illustratively, a point at the center of a left pupil and a point at the center of a right pupil and a point at the tip of the nose are selected as the reference points, specifically, a transformation matrix is obtained by calculating the position mapping relation between the three reference points and the three points on the preset template, and the transformation matrix is applied to the whole face image, so that the face can be aligned. The aligned face can be regarded as a front face, and the transformation matrix is applied to the face key points to obtain the face key points of the aligned face image.
The server determines the face features according to the relative positions of the face key points, namely the facial feature distribution characteristics. Illustratively, the relative positions between the face key points include: the distance between the eyebrows and the eyes, the inner distance between the eyebrows, the inner distance between the eyes, the distance between the outside of the eyes and the temple, the nose length, the nose width, the distance between the nose and the upper lip (the length of the person in the middle), the lip width, the distance between the lower lip and the chin, and the like.
Step 202: and determining the model characteristics of the human face through a preset human face three-dimensional model.
The human face three-dimensional model is constructed based on a plurality of facial features distribution pinching coefficients, the facial features distribution pinching coefficients are used for adjusting the relative distance between the facial features of the human face in the human face three-dimensional model, and the model features indicate the facial features distribution characteristics of the facial features in the human face three-dimensional model.
In the embodiment of the application, the server constructs a three-dimensional model of the human face, namely a 3D mesh (mesh), based on a plurality of facial pinching coefficients distributed by five sense organs, and fig. 3 is a schematic diagram of the 3D mesh. The facial pinching coefficients distributed in the five sense organs are used for adjusting the relative distance between the five sense organs of the human face in the human face three-dimensional model, and specifically, the facial pinching coefficients distributed in the five sense organs adjust the relative distance between the five sense organs by adjusting the distance between any two key points. The server obtains the distribution characteristics of the facial features based on the three-dimensional model of the face, and at the initial stage of model output, the difference between the model characteristics and the facial features is possibly large, namely the distribution difference between the facial features in the current digital face and the facial image is large.
Step 203: and under the condition that the characteristic error between the human face characteristic and the model characteristic is larger than a preset error threshold value, iteratively optimizing the facial pinching coefficient of the distribution of the five sense organs until the iteration times or the characteristic error meets a preset stop condition.
In the embodiment of the application, the server compares the face features with the model features, calculates a loss function, optimizes and updates facial pinching coefficients of facial features according to gradient descent and a back propagation algorithm according to a chain rule if a feature error (loss function) is larger than a preset error threshold value, which indicates that the difference between the current digital face and facial features in a face image is larger, and stops optimization when the iteration frequency reaches a maximum frequency threshold value or the feature error is smaller than or equal to the preset error threshold value, and the face three-dimensional model is optimized and completed at this moment.
The method uses a Pythroch deep learning framework for optimization, the used optimizer is an Adam optimizer, and the learning rate (lr) is set to be 1e -4 。
Step 204: and generating a digital face of the face image through the optimized three-dimensional model of the face.
In the embodiment of the application, the server returns the facial pinching coefficients of the distribution of the five sense organs obtained by optimization to the facial pinching platform, the facial pinching platform obtains an optimized human face three-dimensional model according to the facial pinching coefficients of the distribution of the five sense organs returned, and outputs the digital human face based on the human face three-dimensional model, and at the moment, the distribution of the digital human face is closer to that of the five sense organs of the human face image.
In the application, the server constructs the human face three-dimensional model based on the facial pinching coefficients distributed by the five sense organs, if the characteristic error between the model characteristic output by the human face three-dimensional model and the human face characteristic of a human face image is larger than a preset error threshold value, the facial pinching coefficients distributed by the five sense organs are optimized, the model characteristic is continuously close to the human face characteristic by adjusting the relative distance between the five sense organs of the human face in the human face three-dimensional model, the facial pinching platform can accurately represent the personalized facial feature distribution information of the human face, the facial pinching platform is not limited by materials, and the accuracy of the digital human face is improved. The method and the device fit the distribution of facial features of the facial image to the distribution of facial features in the model, and can realize personalized 3D digital human reconstruction by matching with a personalized registration algorithm.
As an optional implementation manner, the determining, by using a preset three-dimensional model of a human face, a model feature of the human face includes: calculating a plurality of key point coordinates of the human face through a preset human face three-dimensional model; determining a plurality of characteristic values based on the position relation between the key point coordinates of each organ, wherein the characteristic values are used for indicating the relative distance between the five sense organs of the face in the three-dimensional model of the face; and taking a plurality of characteristic values as model characteristics.
In the embodiment of the application, the face material map is determined, the pixel points on the face material map and the grid points on the face three-dimensional model have a preset corresponding relationship, and after the server determines the key points of the five sense organs in the face material map, the key points of the five sense organs in the face three-dimensional model can be determined according to the corresponding relationship.
The key points are represented by U and V, the key points are located in a triangle in the mesh, and the server encodes each key point by adopting a gravity center coordinate method to obtain the key point coordinates of each key point in each organ. If the triangle has three vertices with 3D coordinates a, B, and C, the barycentric coordinates U and V define a point in 3D, and the key point coordinate positions are as follows: p = A × U + B × V + C × (1-U-V), where A, B, C are the three vertices of one face in the mesh.
As in fig. 3, 2212 indicates a key point in mesh, which is located inside the triangle, and then its three vertices around it are a, B and C.
After obtaining the coordinates of the key points, the server determines a plurality of characteristic values based on the position relationship among the coordinates of the key points of each organ, wherein the characteristic values are used for indicating the relative distance between the five sense organs of the human face in the human face three-dimensional model, such as the ratio of the distance between the eyebrows and the eyes to the length of the face, the ratio of the width of the nose to the width of the face, and the like. The server takes the plurality of characteristic values as model characteristics.
And the server calculates the difference of the same characteristic value in the face characteristic and the model characteristic, and then sums the difference values of a plurality of key points to obtain a characteristic error. The calculation formula of the characteristic error is as follows:
wherein, L represents the characteristic error,
represents the ith characteristic value of the input image, is greater than or equal to>
Representing the ith eigenvalue of the 3D grid.
As an optional implementation manner, before the multiple key point coordinates of the human face are calculated through the preset three-dimensional model of the human face, the method further includes: setting the single facial pinching coefficient of facial features distribution to be maximum each time, and obtaining a facial pinching coefficient distribution deformation target corresponding to the facial pinching coefficient of facial features distribution, wherein the facial pinching coefficient distribution deformation target of the facial features distribution is a model of an output deformation digital face, the deformation digital face is obtained relative to a preset basic digital face, and the model of the output basic digital face is obtained according to the default facial pinching coefficient of the facial features distribution; initializing facial pinching coefficients of distribution of five sense organs; and obtaining a human face three-dimensional model based on the basic digital human face, the initialized facial pinching coefficient of the distribution of the five sense organs and the distribution deformation target of the five sense organs.
In the present embodiment, the server first needs to make a deformation target, also referred to as a hybrid shape. Blended shapes (blendshapes) are techniques where a single mesh is deformed to achieve a combination between many predefined shapes and any number. E.g. the basic shape s of a single model being a neutral expression 1 The technician adjusts a smiling expression model s according to the model 2 Then s 2 Can be taken as s 1 Or a mixed shape, otherwise known as a deformed object. The neutral expression of the model can be changed into a smiling expression by giving the weight of the morphing target, the larger the weight, the larger the smiling amplitude, and generally the weight value is between 0 and 1.
The 3D model with default facial pinching coefficients for facial features distribution outputs a base digital face. The server sets the single facial pinching coefficient of the five sense organs to be the maximum, and obtains the facial pinching distribution deformation target corresponding to the facial pinching coefficient of the five sense organs, that is, the blendshape is a model obtained by setting the facial pinching coefficient of the five sense organs to be the maximum (for example, 1), and the server obtains the facial pinching distribution deformation target corresponding to each facial pinching distribution coefficient of the five sense organs according to the mode. After all facial organ distribution deformation targets are manufactured by the server, facial organ distribution pinching coefficients are initialized, such as all 0 or random values, and a human face three-dimensional model is obtained based on the model of the output basic digital human face, the initialized facial organ distribution pinching coefficients and the facial organ distribution deformation targets.
Specifically, the manner of obtaining the three-dimensional model of the human face is as follows: the server calculates a deformation difference value between each facial feature distribution deformation target and the model of the output basic digital face, weights and sums each initialized facial feature distribution pinching coefficient and the corresponding deformation difference value to obtain a weighting result, and calculates and obtains a face three-dimensional model according to the sum value of the model of the output basic digital face and the weighting result.
The formula for obtaining the human face three-dimensional model is as follows:
S＝base+λ 1 (S 1 -base)+λ 2 (S 2 -base)+…+λ n (S n -base)
wherein S is a calculated three-dimensional model of the face, λ 1 ...λ n Is facial pinching coefficient of facial features distribution, base is model of output basic digital human face, S 1 ...S n Is a five sense organs distribution deformation target corresponding to the five sense organs distribution pinching coefficient.
As an alternative embodiment, each time before setting the single facial pinching coefficient of the five sense organs to maximum, the method further comprises: keeping the facial pinching coefficient of the distribution of the five sense organs as a default coefficient, and setting the facial pinching coefficient of the shape of a single five sense organs to be the maximum every time to obtain a facial pinching coefficient distribution deformation target of the five sense organs corresponding to the facial pinching coefficient of the distribution of the five sense organs; initializing facial pinching coefficients of the shape of five sense organs; obtaining an initial human face three-dimensional model based on the model of the output basic digital human face, the initialized facial pinching coefficient of the shape of the five sense organs and the distribution deformation target of the five sense organs; and generating the shape of the five sense organs based on the initial human face three-dimensional model.
Before outputting the distribution of the five sense organs, the three-dimensional human face model needs to generate the shape of the five sense organs in advance, the method for optimizing the face pinching coefficient is also adopted for generating the shape of the five sense organs, the logic of the method is the same as that of optimizing the distribution of the five sense organs, and the method for generating the shape of the five sense organs is used for optimizing the face pinching coefficient of the shape of the five sense organs. Keeping the facial pinching coefficient of the distribution of the five sense organs as a default coefficient, setting the facial pinching coefficient of the shape of a single five sense organ to be the maximum every time, obtaining a facial distribution deformation target corresponding to the facial pinching coefficient of the distribution of the five sense organs, initializing the facial pinching coefficient of the shape of the five sense organs, and obtaining an initial facial three-dimensional model based on the model of the output basic digital facial, the initialized facial pinching coefficient of the shape of the five sense organs and the distribution deformation target of the five sense organs; and generating the shape of the five sense organs based on the initial human face three-dimensional model.
Based on the same technical concept, the embodiment of the present application further provides a digital face generation flow chart, as shown in fig. 4.
Based on the same technical concept, the embodiment of the present application further provides a digital face generating device for generating a digital face, as shown in fig. 5.
The extracting module 501 is configured to extract a face feature in the face image, where the face feature indicates a distribution feature of five sense organs of the face;
a determining module 502, configured to determine a model feature of a human face through a preset human face three-dimensional model, where the human face three-dimensional model is constructed based on a plurality of facial feature distribution pinching coefficients, the facial feature distribution pinching coefficients are used to adjust a relative distance between facial features of the human face in the human face three-dimensional model, and the model feature indicates a facial feature distribution feature of the human face in the human face three-dimensional model;
the optimization module 503 is configured to iteratively optimize the facial pinching coefficients distributed by the five sense organs under the condition that a feature error between the face feature and the model feature is greater than a preset error threshold value until the iteration number or the feature error meets a preset stop condition;
and a generating module 504, configured to generate a digital face of the face image through the optimized three-dimensional model of the face.
Optionally, the determining module 502 is configured to:
calculating a plurality of key point coordinates of the human face through a preset human face three-dimensional model;
determining a plurality of characteristic values based on the position relation between the key point coordinates of each organ, wherein the characteristic values are used for indicating the relative distance between the five sense organs of the face in the three-dimensional model of the face;
and taking a plurality of characteristic values as model characteristics.
Optionally, the apparatus is further configured to:
setting the single facial pinching coefficient of facial features distribution to be maximum each time, and obtaining a facial pinching coefficient distribution deformation target corresponding to the facial pinching coefficient of facial features distribution, wherein the facial pinching coefficient distribution deformation target of the facial features distribution is a model of an output deformation digital face, the deformation digital face is obtained relative to a preset basic digital face, and the model of the output basic digital face is obtained according to the default facial pinching coefficient of the facial features distribution;
initializing facial pinching coefficients of distribution of five sense organs;
and obtaining a human face three-dimensional model based on the model of the output basic digital human face, the initialized facial pinching coefficient of the distribution of the five sense organs and the deformation target of the distribution of the five sense organs.
Optionally, the apparatus is further configured to:
calculating a deformation difference value between each five sense organs distribution deformation target and a model of the output basic digital human face;
weighting and summing each initialized facial pinching coefficient distributed in the five sense organs and the corresponding deformation difference value to obtain a weighting result;
and calculating to obtain a human face three-dimensional model according to the sum of the model of the output basic digital human face and the weighting result.
Optionally, the determining module 502 is configured to:
acquiring a face material map, wherein pixel points on the face material map and grid points on a face three-dimensional model have a preset corresponding relationship;
determining a plurality of key points of each organ in the human face three-dimensional model according to the key points and the corresponding relation of each organ in the human face material map;
and coding each key point by adopting a gravity center coordinate method to obtain the key point coordinates of each key point in each organ.
Optionally, the apparatus is further configured to:
keeping the facial pinching coefficient of the distribution of the five sense organs as a default coefficient, and setting the facial pinching coefficient of the shape of a single five sense organs to be the maximum each time to obtain a facial pinching coefficient distribution deformation target corresponding to the facial pinching coefficient of the distribution of the five sense organs;
initializing facial pinching coefficients of the shape of five sense organs;
obtaining an initial human face three-dimensional model based on the model of the output basic digital human face, the initialized facial pinching coefficient of the shape of the five sense organs and the distribution deformation target of the five sense organs;
and generating the shape of the five sense organs based on the initial human face three-dimensional model.
Optionally, the determining module 502 is configured to:
extracting key points of the face in the face image through a key point detection model;
selecting at least three reference points from the key points of the human face to align the human face;
and extracting the distribution characteristics of the five sense organs based on the aligned human face.
Based on the same technical concept, an embodiment of the present invention further provides an electronic device, as shown in fig. 6, including a processor 601, a communication interface 602, a memory 603, and a communication bus 604, where the processor 601, the communication interface 602, and the memory 603 complete mutual communication through the communication bus 604,
a memory 603 for storing a computer program;
the processor 601 is configured to implement the above steps when executing the program stored in the memory 603.
The communication bus mentioned in the electronic device may be a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, or the like. The communication bus may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is shown, but this is not intended to represent only one bus or type of bus.
The communication interface is used for communication between the electronic equipment and other equipment.
The Memory may include a Random Access Memory (RAM) or a Non-Volatile Memory (NVM), such as at least one disk Memory. Optionally, the memory may also be at least one memory device located remotely from the processor.
The Processor may be a general-purpose Processor, including a Central Processing Unit (CPU), a Network Processor (NP), and the like; but also Digital Signal Processors (DSPs), application Specific Integrated Circuits (ASICs), field Programmable Gate Arrays (FPGAs) or other Programmable logic devices, discrete Gate or transistor logic devices, discrete hardware components.
In a further embodiment provided by the present invention, there is also provided a computer readable storage medium having stored therein a computer program which, when executed by a processor, implements the steps of any of the methods described above.
In a further embodiment provided by the present invention, there is also provided a computer program product containing instructions which, when run on a computer, cause the computer to perform any of the methods of the above embodiments.
In the above embodiments, the implementation may be wholly or partially realized by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When loaded and executed on a computer, cause the processes or functions described in accordance with the embodiments of the invention to occur, in whole or in part. The computer may be a general purpose computer, a special purpose computer, a network of computers, or other programmable device. The computer instructions may be stored in a computer readable storage medium or transmitted from one computer readable storage medium to another computer readable storage medium, for example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital Subscriber Line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device, such as a server, a data center, etc., that incorporates one or more of the available media. The usable medium may be a magnetic medium (e.g., floppy Disk, hard Disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid State Disk (SSD)), among others.
It is noted that, in this document, relational terms such as "first" and "second," and the like, may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrases "comprising a," "8230," "8230," or "comprising" does not exclude the presence of additional like elements in a process, method, article, or apparatus that comprises the element.
The foregoing are merely exemplary embodiments of the present invention, which enable those skilled in the art to understand or practice the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the invention. Thus, the present invention is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims (10)
1. A method for generating a digital face, the method comprising:
extracting face features in the face image, wherein the face features indicate distribution features of facial features;
determining model characteristics of the face through a preset three-dimensional model of the face, wherein the three-dimensional model of the face is constructed based on a plurality of facial feature distribution pinching coefficients, the facial feature distribution pinching coefficients are used for adjusting the relative distance between the facial features of the face in the three-dimensional model of the face, and the model characteristics indicate the distribution characteristics of the facial features of the face in the three-dimensional model of the face;
under the condition that the characteristic error between the human face characteristic and the model characteristic is larger than a preset error threshold value, iteratively optimizing the facial pinching coefficient of the facial features distribution until the iteration times or the characteristic error meets a preset stop condition;
and generating the digital face of the face image through the optimized three-dimensional model of the face.
2. The method of claim 1, wherein the determining the model feature of the human face through a preset human face three-dimensional model comprises:
calculating a plurality of key point coordinates of the human face through a preset human face three-dimensional model;
determining a plurality of characteristic values based on the position relation between the key point coordinates of each organ, wherein the characteristic values are used for indicating the relative distance between the five sense organs of the human face in the human face three-dimensional model;
and taking the plurality of characteristic values as the model characteristic.
3. The method according to claim 2, wherein before the calculating the coordinates of the plurality of key points of the human face through the preset three-dimensional model of the human face, the method further comprises:
setting a single facial pinching coefficient of facial features distribution to be maximum each time, and obtaining a facial pinching coefficient distribution deformation target corresponding to the facial pinching coefficient of facial features distribution, wherein the facial pinching coefficient distribution deformation target of facial features is a model for outputting a deformation digital face, the deformation digital face is obtained relative to a preset basic digital face, and the model for outputting the basic digital face is obtained according to a default facial pinching coefficient of facial features distribution;
initializing the facial pinching coefficient of the distribution of the five sense organs;
and obtaining a human face three-dimensional model based on the model of the output basic digital human face, the initialized facial pinching coefficient of the distribution of the five sense organs and the distribution deformation target of the five sense organs.
4. The method according to claim 3, wherein the obtaining a three-dimensional face model based on the model of the output base digital face, the initialized facial feature distribution pinching coefficients and the facial feature distribution deformation target comprises:
calculating a deformation difference value between each five sense organs distribution deformation target and the model of the output basic digital human face;
weighting and summing each initialized facial pinching coefficient distributed in the five sense organs and the corresponding deformation difference value to obtain a weighting result;
and calculating to obtain the human face three-dimensional model according to the model of the output basic digital human face and the sum of the weighting results.
5. The method of claim 2, wherein the calculating the plurality of key point coordinates of the human face through a preset human face three-dimensional model comprises:
acquiring a face material map, wherein pixel points on the face material map and grid points on the face three-dimensional model have a preset corresponding relationship;
determining a plurality of key points of each organ in the face three-dimensional model according to the key points of each organ in the face material map and the corresponding relation;
and coding each key point by adopting a gravity center coordinate method to obtain the key point coordinate of each key point in each organ.
6. A method according to claim 3, wherein each time before setting a single facial feature distribution pinching coefficient to maximum, the method further comprises:
keeping the facial pinching coefficient of the distribution of the five sense organs as a default coefficient, and setting the facial pinching coefficient of the shape of a single five sense organs to be the maximum every time to obtain a facial pinching coefficient distribution deformation target of the five sense organs corresponding to the facial pinching coefficient of the distribution of the five sense organs;
initializing the facial pinching coefficient of the shape of the five sense organs;
obtaining an initial human face three-dimensional model based on a model of an output basic digital human face, initialized facial pinching coefficients of facial shapes of five sense organs and a distribution deformation target of the five sense organs;
and generating the shape of the five sense organs based on the initial human face three-dimensional model.
7. The method of claim 1, wherein the extracting the facial features in the facial image comprises:
extracting key points of the face in the face image through a key point detection model;
selecting at least three reference points from the face key points to carry out face alignment;
and extracting the distribution characteristics of the five sense organs based on the aligned human face.
8. An apparatus for digital face generation, the apparatus comprising:
the extraction module is used for extracting the face features in the face image, wherein the face features indicate the distribution features of the five sense organs of the face;
the determining module is used for determining model characteristics of the human face through a preset human face three-dimensional model, wherein the human face three-dimensional model is constructed based on a plurality of facial feature distribution pinching coefficients, the facial feature distribution pinching coefficients are used for adjusting the relative distances between the facial features of the human face in the human face three-dimensional model, and the model characteristics indicate the facial feature distribution characteristics of the human face in the human face three-dimensional model;
the optimization module is used for iteratively optimizing the facial pinching coefficient of the facial features under the condition that the feature error between the facial features and the model features is larger than a preset error threshold value until the iteration times or the feature error meet a preset stop condition;
and the generating module is used for generating the digital face of the face image through the optimized face three-dimensional model.
9. An electronic device is characterized by comprising a processor, a communication interface, a memory and a communication bus, wherein the processor and the communication interface are used for realizing mutual communication by the memory through the communication bus;
a memory for storing a computer program;
a processor for implementing the method of any one of claims 1 to 7 when executing a program stored in the memory.
10. A computer-readable storage medium, in which a computer program is stored which, when being executed by a processor, carries out the method of any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211526834.7A CN115861533A (en) | 2022-11-30 | 2022-11-30 | Digital face generation method and device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211526834.7A CN115861533A (en) | 2022-11-30 | 2022-11-30 | Digital face generation method and device, electronic equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115861533A true CN115861533A (en) | 2023-03-28 |
Family
ID=85668706
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211526834.7A Pending CN115861533A (en) | 2022-11-30 | 2022-11-30 | Digital face generation method and device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115861533A (en) |
-
2022
- 2022-11-30 CN CN202211526834.7A patent/CN115861533A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109255830B (en) | Three-dimensional face reconstruction method and device | |
| CN108898186B (en) | Method and device for extracting image | |
| KR20220137848A (en) | Training method of virtual image generation model and virtual image generation method | |
| US11276218B2 (en) | Method for skinning character model, device for skinning character model, storage medium and electronic device | |
| CN111325851A (en) | Image processing method and device, electronic equipment and computer readable storage medium | |
| JP7327140B2 (en) | Image processing method and information processing apparatus | |
| CN112085041B (en) | Training method and training device of neural network and electronic equipment | |
| CN114187633B (en) | Image processing method and device, and training method and device for image generation model | |
| CN106776673A (en) | Multimedia document is summarized | |
| CN111768477A (en) | Three-dimensional facial expression base establishment method and device, storage medium and electronic equipment | |
| CN111814804B (en) | Human body three-dimensional size information prediction method and device based on GA-BP-MC neural network | |
| CN112419144B (en) | Face image processing method and device, electronic equipment and storage medium | |
| CN114925748B (en) | Model training and modal information prediction method, related device, equipment and medium | |
| WO2022110851A1 (en) | Facial information processing method and apparatus, electronic device, and storage medium | |
| KR102488517B1 (en) | A method, a device, an electronic equipment and a storage medium for changing hairstyle | |
| US20210365718A1 (en) | Object functionality predication methods, computer device, and storage medium | |
| US20230100427A1 (en) | Face image processing method, face image processing model training method, apparatus, device, storage medium, and program product | |
| EP4345770A1 (en) | Information processing method and apparatus, computer device, and storage medium | |
| CN108509904A (en) | Method and apparatus for generating information | |
| CN116310219A (en) | Three-dimensional foot shape generation method based on conditional diffusion model | |
| CN117372604B (en) | 3D face model generation method, device, equipment and readable storage medium | |
| CN117078790A (en) | Image generation method, device, computer equipment and storage medium | |
| WO2022213623A1 (en) | Image generation method and apparatus, three-dimensional facial model generation method and apparatus, electronic device and storage medium | |
| CN111460910A (en) | Face type classification method and device, terminal equipment and storage medium | |
| CN115861533A (en) | Digital face generation method and device, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |