CN112330805B

CN112330805B - Face 3D model generation method, device, equipment and readable storage medium

Info

Publication number: CN112330805B
Application number: CN202011340768.5A
Authority: CN
Inventors: 彭昊天
Original assignee: Beijing Baidu Netcom Science and Technology Co Ltd
Current assignee: Beijing Baidu Netcom Science and Technology Co Ltd
Priority date: 2020-11-25
Filing date: 2020-11-25
Publication date: 2023-08-08
Anticipated expiration: 2040-11-25
Also published as: CN112330805A

Abstract

The application discloses a face 3D model generation method, device and equipment and a readable storage medium, and relates to the technical fields of artificial intelligence such as image processing and augmented reality technology. The specific implementation scheme is as follows: the server generates a second 3D model aiming at a target object in the two-dimensional image, determines a bone driving coefficient set according to the second 3D model and a local preset skin bone model, and sends the bone driving coefficient set to the electronic equipment. The electronic device deforms a locally preset skinned bone model according to bone driving coefficients in the bone driving coefficient set, so that a first 3D model is obtained. In the process, the server only needs to send the skeleton driving coefficient set to the electronic device, and the second 3D model is not needed to be sent, so that the consumption of bandwidth between the electronic device and the server can be reduced.

Description

Face 3D model generation method, device, equipment and readable storage medium

Technical Field

The application relates to the technical field of artificial intelligence such as image processing and augmented reality technology, in particular to a face 3D model generation method, a device, equipment and a readable storage medium.

Background

With the rapid development of technology, more and more electronic devices have Photo-To-Avatar (P2A) functions. Based on the P2A function, a user imports a face photo into the electronic equipment, the electronic equipment automatically generates a virtual 3D model of the face, and the electronic equipment further generates a 3D image or animation and the like by utilizing the virtual face 3D model.

In the process of generating the face 3D model based on the P2A function, the electronic equipment sends the face photo to the server, and the server generates n mixed shape (blendhape) coefficients aiming at the face image, wherein the n blendhape coefficients respectively correspond to different blendhape models. And then, the server weights the n blendscape models according to the n blendscape coefficients, so that a face 3D model corresponding to the face photo is obtained. And finally, the server sends the face 3D model to the electronic equipment.

The face 3D model is obtained by weighting n blendscape models according to n blendscape coefficients by the server, which has huge volume and causes serious bandwidth consumption in the process of transmitting the face 3D model to the electronic equipment by the server.

Disclosure of Invention

The application provides a face 3D model generation method, device, equipment and readable storage medium, wherein a server calculates a bone driving coefficient aiming at a two-dimensional image and sends the bone driving coefficient to electronic equipment, and the electronic equipment generates a face 3D model according to the bone driving coefficient and a locally deployed skin bone model, so that bandwidth consumption is reduced.

In a first aspect, an embodiment of the present application provides a method for generating a face 3D model, including: the method comprises the steps that an electronic device receives a bone driving coefficient set of a target face in a two-dimensional image, the bone driving coefficient set comprises bone driving coefficients of each node in a skin bone model which is preset in the local electronic device, and the electronic device deforms the skin bone model by utilizing the bone driving coefficients in the bone driving coefficient set to obtain a first 3D model of the target face in the two-dimensional image.

In a second aspect, an embodiment of the present application provides a method for generating a face 3D model, including: the method comprises the steps that a server generates a second 3D model according to a target face in a two-dimensional image, the server determines a bone driving coefficient of each node in a skin bone model according to the second 3D model and a preset skin bone model to obtain a bone driving coefficient set, and the server sends the bone driving coefficient set to electronic equipment.

In a third aspect, an embodiment of the present application provides a face 3D model generating device, including:

the receiving module is used for receiving a bone driving coefficient set of a target face in a two-dimensional image, wherein the bone driving coefficient set comprises bone driving coefficients of each node in a skin bone model preset in the local electronic equipment.

And the processing module is used for deforming the skinned bone model by utilizing the bone driving coefficients in the bone driving coefficient set so as to obtain a first 3D model of the target face in the two-dimensional image.

In a fourth aspect, an embodiment of the present application provides a face 3D model generating device, including:

the first determining unit is used for determining a global rigid pose matrix of the corresponding node according to the bone driving coefficient of each node in the skin bone model to obtain the global rigid pose matrix of each node in the skin bone model.

And the second determining unit is used for determining a rigid transformation matrix of each node in the skinned skeleton model according to the global rigid pose matrix of each node in the skinned skeleton model.

And the transformation unit is used for transforming the positions of the vertexes in the skinned skeleton model according to the rigid transformation matrix of each node in the skinned skeleton model so as to obtain a first 3D model of the target face in the two-dimensional image.

In a fifth aspect, an embodiment of the present application provides an electronic device, including:

at least one processor; and

a memory communicatively coupled to the at least one processor; wherein,,

The memory stores instructions executable by the at least one processor to enable the at least one processor to perform the first aspect or any possible implementation of the first aspect.

In a sixth aspect, an embodiment of the present application provides an electronic device, including:

at least one processor; and

a memory communicatively coupled to the at least one processor; wherein,,

the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the second aspect or any possible implementation of the second aspect.

In a seventh aspect, embodiments of the present application provide a computer program product comprising instructions which, when run on an electronic device, cause the electronic device computer to perform the method of the first aspect or various possible implementations of the first aspect.

In an eighth aspect, embodiments of the present application provide a computer program product comprising instructions which, when run on an electronic device, cause the electronic device computer to perform the method of the second aspect or various possible implementations of the second aspect described above.

In a ninth aspect, an embodiment of the present application provides a method for generating a face 3D model, including: acquiring a bone driving coefficient set of a target object in a two-dimensional image, wherein the bone driving coefficient set comprises bone driving coefficients of each node in a skin bone model preset in the local electronic equipment, and deforming the skin bone model by utilizing the bone driving coefficients in the bone driving coefficient set to obtain a 3D model of the target object in the two-dimensional image.

In a tenth aspect, the present application provides a computer program product comprising: a computer program stored in a readable storage medium, from which it can be read by at least one processor of an electronic device, the at least one processor executing the computer program causing the electronic device to perform the method of the first aspect or of the various possible implementations of the first aspect or to perform the method of the ninth aspect.

In an eleventh aspect, the present application provides a computer program product comprising: a computer program stored in a readable storage medium from which at least one processor of a server can read, the at least one processor executing the computer program causing the server to perform the method of the second aspect or the various possible implementations of the second aspect.

According to the technology of the application, the server only needs to send the skeleton driving coefficient set to the electronic device, and the second 3D model is not needed to be sent, so that the consumption of bandwidth between the electronic device and the server can be reduced. Moreover, the first 3D model is obtained based on the skinned skeleton model and the skeleton driving coefficient, is a lightweight model, occupies a small space and is easy to deploy. In addition, the electronic equipment does not need to manually adjust the skeleton driving coefficient of the skin skeleton model, but can obtain the first 3D model automatically and in a second level, does not need to carry out weighted calculation, reduces the consumption of computing resources of the electronic equipment, and avoids the dead halt of the electronic equipment.

It should be understood that the description of this section is not intended to identify key or critical features of the embodiments of the application or to delineate the scope of the application. Other features of the present application will become apparent from the description that follows.

Drawings

The drawings are for better understanding of the present solution and do not constitute a limitation of the present application. Wherein:

fig. 1 is a network architecture schematic diagram of a face 3D model generating method provided in an embodiment of the present application;

fig. 2 is a flowchart of a face 3D model generating method provided in an embodiment of the present application;

FIG. 3 is a schematic diagram of nodes and vertices in a face 3D model generating method according to an embodiment of the present application;

fig. 4 is a schematic diagram of respective deployment conditions of a server and an electronic device in the face 3D model generating method provided in the embodiment of the present application;

FIG. 5 is a schematic diagram of a related node set and an unrelated node set in the face 3D model generating method provided in the embodiment of the present application;

fig. 6 is a schematic diagram of a change in the position of a vertex under a single node in the face 3D model generating method provided in the embodiment of the present application;

fig. 7 is a schematic structural diagram of a face 3D model generating device according to an embodiment of the present application;

fig. 8 is another face 3D model generating device provided in an embodiment of the present application;

fig. 9 is a schematic structural diagram of another face 3D model generating device according to an embodiment of the present application;

fig. 10 is another face 3D model generating apparatus provided in the embodiment of the present application;

fig. 11 is a block diagram of an electronic device for implementing a face 3D model generation method according to an embodiment of the present application.

Detailed Description

Exemplary embodiments of the present application are described below in conjunction with the accompanying drawings, which include various details of the embodiments of the present application to facilitate understanding, and should be considered as merely exemplary. Accordingly, one of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the present application. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.

The face 3D model is the basis for generating 3D animations or 3D images. In the prior art, a server typically generates a face 3D model for a face in a face photo. The method comprises the steps that n blendscape models are deployed locally by a server, n blendscape coefficients are generated for faces in a face photo in the process of generating the face 3D model by the server, and then the server weights the n blendscape models according to the n blendscape coefficients, so that the face 3D model corresponding to the face photo is obtained. And finally, the server sends the face 3D model to the electronic equipment.

The face 3D model is huge in size, and bandwidth and the like between a server and electronic equipment are easy to consume in the sending process. Furthermore, resources such as memory and hard disk of the electronic device are limited, and the bulky 3D model is difficult to deploy on the electronic device. In addition, compared with a server, resources such as a memory and a hard disk of the electronic device are limited, if n blendrope models are deployed on the electronic device, the electronic device self-calculates the blendrope coefficients, weights the n blendrope models according to the n blendrope coefficients in real time, and has large calculation amount, seriously consumes the resources of the electronic device, and easily causes the electronic device to generate the phenomenon of dead halt and the like.

Another approach is to deploy a skinned bone model on an electronic device, and generate a 3D model of the face for the two-dimensional image by the user by manually adjusting the bone drive coefficients. However, manual adjustment on the one hand does not satisfy the automation process; on the other hand, man-made time consuming is excessive. Moreover, as the hierarchical structure of the skeleton is not easy to understand, the phenomenon that the ideal face 3D model cannot be obtained by repeated debugging of father-son multilevel nodes exists.

The embodiment of the application provides a face 3D model generation method, device, equipment and readable storage medium, wherein a server calculates a bone driving coefficient aiming at a two-dimensional image and sends the bone driving coefficient to electronic equipment, and the electronic equipment generates a lightweight face 3D model according to the bone driving coefficient and a locally deployed skin bone model, so that bandwidth consumption is reduced. Meanwhile, the electronic equipment does not need to carry out weight calculation, so that the consumption of computing resources of the electronic equipment is reduced, and the electronic equipment is prevented from being halted.

Fig. 1 is a network architecture schematic diagram of a face 3D model generating method according to an embodiment of the present application. Referring to fig. 1, the network architecture includes an electronic device 1, a server 2 and a network 3, where the electronic device 1 and the server 2 establish a network connection through the network 3. The network 3 includes various network connection types such as wired, wireless communication links, or fiber optic cables.

A user interacts with the server 2 via the network 3 using the electronic device 1 to receive or send messages or the like. The electronic device 1 is installed with various communication client applications, such as a video playing application, a shopping application, a searching application, an instant messaging tool, a mailbox client, social platform software, and the like.

The electronic device 1 may be hardware or software. When the electronic device 1 is hardware, the electronic device 1 is, for example, a mobile phone, a tablet computer, an electronic book reader, a laptop, a desktop computer, or the like. When the electronic apparatus 1 is software, it may be installed in the above-listed hardware apparatus, and in this case, the electronic apparatus 1 is, for example, a plurality of software modules or a single software module, etc., the embodiment of the present application is not limited.

The server 2 is a server capable of providing a plurality of servers for generating a set of bone driving coefficients for a two-dimensional image and transmitting the set of bone driving coefficients to the electronic device 1 such that the electronic device 1 generates a first 3D model of a target object in the 2D image using the set of bone driving coefficients and a locally preset skinned bone model. In the process of generating the bone driving coefficient set by the server 2, a second 3D model is generated for the two-dimensional image, and then the bone driving coefficient set is determined according to the second 3D model and a preset skinned bone model.

The server 2 may be hardware or software. When the server 2 is hardware, the server 2 is a single server or a distributed server cluster composed of a plurality of servers. When the server 2 is software, it may be a plurality of software modules or a single software module, etc., and the embodiment of the present application is not limited.

It should be understood that the number of electronic devices 1, servers 2 and networks 3 in fig. 1 is merely illustrative. In an actual implementation, any number of electronic devices 1, servers 2 and networks 3 are deployed according to actual requirements.

In addition, in the embodiment of the present application, a skin skeleton model is preset on both the electronic device 1 and the server 2. The electronic device 1 deforms the skinned bone model according to the bone driving coefficients in the bone driving coefficient set to obtain a lightweight face 3D model. The bone driving coefficients are generated by the server 2 according to a face 3D model and a skinned bone model, and the face 3D model is generated by the server 2 based on a blendscape model and the like. The face 3D model is also called a patch model. For clarity, the lightweight face 3D model generated by the electronic device 1 is referred to as a first 3D model, and the face 3D model generated by the server 2 for calculating the set of bone driving coefficients is referred to as a second 3D model.

Next, a detailed description will be given of a face 3D model generating method according to an embodiment of the present application based on the network architecture shown in fig. 1. For example, please refer to fig. 2.

Fig. 2 is a flowchart of a face 3D model generating method provided in an embodiment of the present application. The present embodiment is described in terms of interaction between a server and an electronic device, and includes:

101. and the server generates a second 3D model according to the target face in the two-dimensional image.

After the server acquires the two-dimensional image, a second 3D model is generated for the target object in the two-dimensional image. For example, a server receives a two-dimensional image from an electronic device; for another example, the server reads a local two-dimensional image. The two-dimensional image typically includes at least one object, such as a person, animal, etc. The target object is any one of the at least one object.

102. And the server determines the bone driving coefficient of each node in the skinned bone model according to the second 3D model and a preset skinned bone model so as to obtain a bone driving coefficient set.

Illustratively, the bone driving coefficients in the bone driving coefficient set are used to enable the similarity of the second 3D model and the first 3D model to exceed a preset similarity, and the first 3D model is a face 3D model generated by the electronic device according to the bone driving coefficient set and the skinned bone model.

103. The server sends the set of bone drive coefficients to an electronic device.

Accordingly, the electronic device receives the set of bone drive coefficients.

The server transmits the set of bone drive coefficients to the electronic device, illustratively, over a network. In this process, the server only transmits the set of bone driving coefficients, and does not transmit the second 3D model, and the set of bone driving coefficients is much smaller than the size of the second 3D model, so that bandwidth consumption between the server and the electronic device can be reduced.

104. The electronic device deforms the skinned bone model by utilizing the bone driving coefficients in the bone driving coefficient set to obtain a first 3D model of the target face in the two-dimensional image.

After receiving the skeleton driving coefficient set, the electronic equipment deforms a locally preset skin skeleton model by utilizing the skeleton driving coefficients in the skeleton driving coefficient set, so that a first 3D model of the target object is obtained.

In the embodiment of the application, the skin skeleton model is composed of a hierarchical structure of tree stake nodes (Node), and each Node stores own local rigidity pose which comprises parameters such as translational rotation scaling (Translate, rotation, scale, TRS) and the like. Each node also has a global rigid pose. When a node is a root node, the global rigid pose of the node is the local rigid pose of the node. When a node is a child node, the global rigid pose of the node can be determined according to the local rigid pose of the parent node. The global rigid pose and the local rigid pose are, for example, a 4×4 matrix containing TRS information in three-dimensional space.

The rigid transformation matrix is also a 4 x 4 matrix, which also contains TRS information in three dimensions. And multiplying the rigid transformation matrix by one rigid pose matrix to obtain the other rigid pose matrix.

Nodes in the skinned skeletal model are enveloped by a skin, each node having a skin region containing a plurality of vertices (vertexes). The server-generated second 3D model is composed of a plurality of patches, each patch having 3 vertices, each vertex being controlled by one or more nodes. For a particular vertex, the nodes controlling that vertex are weighted differently, with a total weight of 1. When a node is rigidly transformed, i.e., the node translates, rotates or scales, the position of the vertex controlled by the node also changes, the magnitude of the change being related to the weight of the node to the vertex.

In this embodiment of the present application, the topologies of the second 3D model and the skin skeleton model are the same, that is, the numbers of vertices of the second 3D model and the skin skeleton model are the same, the positions of the corresponding vertices are the same, the connection order of the vertices is the same, and the connection order of the patches is the same.

In this embodiment, the bone driving coefficients in the bone driving coefficient set include 9 values, which are 3 translation values, 3 euler angle rotation values, and 3 scaling values of xyz. The bone drive coefficients and the rigid pose matrix of the local rigid pose may be mutually transformed.

According to the face 3D model generation method, a server generates a second 3D model aiming at a target object in a two-dimensional image, a bone driving coefficient set is determined according to the second 3D model and a local preset skin bone model, and the bone driving coefficient set is sent to electronic equipment. The electronic device deforms a locally preset skinned bone model according to bone driving coefficients in the bone driving coefficient set, so that a first 3D model is obtained. In the process, the server only needs to send the skeleton driving coefficient set to the electronic device, and the second 3D model is not needed to be sent, so that the consumption of bandwidth between the electronic device and the server can be reduced. Moreover, the first 3D model is obtained based on the skinned skeleton model and the skeleton driving coefficient, is a lightweight model, occupies a small space and is easy to deploy. In addition, the electronic equipment does not need to manually adjust the skeleton driving coefficient of the skin skeleton model, but can obtain the first 3D model automatically and in a second level, does not need to carry out weighted calculation, reduces the consumption of computing resources of the electronic equipment, and avoids the dead halt of the electronic equipment.

In the above embodiment, when the electronic device deforms the skinned skeleton model by using the skeleton driving coefficients in the skeleton driving coefficient set to obtain the first 3D model of the target face in the two-dimensional image, first, the electronic device determines, according to the skeleton driving coefficient of each node in the skinned skeleton model, a global rigid pose matrix of the corresponding node, and obtains the global rigid pose matrix of each node in the skinned skeleton model. And then, the electronic equipment determines the rigid pose matrix of each node in the skin skeleton model according to the global rigid pose matrix of each node in the skin skeleton model. Finally, the electronic equipment adjusts the position of the vertex in the skinned skeleton model according to the rigid pose matrix of each node in the skinned skeleton model so as to obtain a first 3D model of the target face in the two-dimensional image.

Illustratively, any one vertex i in the base skinned bone model is represented as: vertex _i The bone driving coefficient of any node j in the skin bone model of the base is expressed as localTRS _j The global rigid pose matrix for node j is represented as: currentNode _j Global, then the Global rigid pose matrix of node j can be obtained by recursion from the root node, specifically as shown in the following formula (1):

currentNode _j Global＝parentNode _j Global×localTRS _j formula (1)

Wherein, the parentNode _j Parent node representing node j, pantnode _j Global represents the now Global rigid pose matrix of the parent node of node j.

Representing a rigid transformation matrix of any node j in a skin skeleton model of a base as a form _j The determination can be made according to the following formula (2):

Derform _j ＝currentNode _j Global×iniTNode _j Global ^-1 formula (2)

Wherein the initNode _j Global represents the Global rigid pose matrix of node j in the initial state, i.e., bone binding, which is related to the skinned bone model of the base,

initNode _j Global ^-1 representing the inverse of the global rigid pose matrix in the initial state of node j.

And then, the electronic equipment adjusts the positions of all vertexes in the skinned skeleton model according to the rigid transformation matrix of the node j, so that the skinned skeleton model of the base is deformed, and a first 3D model is obtained after the deformation. In the deformation process, the positions of the vertexes are determined according to the following formula (3):

Wherein, vertex _i Representing the initial position of Vertex i in the skinned skeletal model, vertex _i ' represents the position of the vertex i after deformation, m represents the number of nodes in the skinned bone model, and alpha _)i,j) Representing the skin weight of node j to vertex i, deform _j Representing the rigid transformation matrix of node j.

By adopting the scheme, the electronic equipment deforms the skin skeleton model according to the skeleton driving coefficients in the skeleton driving coefficient set so as to obtain the first 3D model of the target face in the two-dimensional image, and the purpose of accurately generating the first 3D model is achieved.

In the foregoing embodiment, before the server generates the second 3D model according to the target face in the two-dimensional image, the server further receives a request instruction from the electronic device, where the request instruction carries the two-dimensional image, and the request instruction is used to request the server to generate the bone driving coefficient set for the two-dimensional image.

The two-dimensional image is an image taken by a user in real time or an image local to the electronic device, and the electronic device carries the two-dimensional image in a request instruction and sends the request instruction to the server so as to trigger the server to generate the skeleton driving coefficient set for a target object in the two-dimensional image.

By adopting the scheme, a user can request the server to generate the 3D model in real time and conveniently through the electronic equipment, and the method is simple and quick.

In the above embodiment, the electronic device deforms the skinned bone model by using the bone driving coefficients in the bone driving coefficient set, so as to obtain a first 3D model of the target object in the two-dimensional image, and then further obtain expression information of the user, and drive the first 3D model according to the expression information, so as to generate the animation.

After the electronic device generates the first 3D model, the electronic device obtains expression information of the user in real time in the live broadcast process by using the electronic device, and drives the first 3D model according to the expression information, so that a 3D image of the user is displayed on a live broadcast interface. The expression information is, for example, information of a change in key points of the facial features caused by facial movements or facial expressions of the user.

In addition, the electronic device can be used for other scenes in addition to the first 3D model for live broadcasting and the like. For example, the first 3D model is modified by a model designer, etc.

By adopting the scheme, the user flexibly uses the first 3D model, so that the interestingness of live broadcasting and the like is improved.

In the above embodiment, when the node performs rigid transformation, that is, when the node performs translation, rotation or scaling, the position of the vertex controlled by the node also changes, and the change amplitude is related to the weight of the node to the vertex. For example, please refer to fig. 3.

Fig. 3 is a schematic diagram of nodes and vertices in a face 3D model generating method according to an embodiment of the present application. Referring to fig. 3, the lines in the left graph are skin skeleton models, and the root nodes of the dashed subtrees are shown as rectangular black filled boxes in the graph. The right graph is a second 3D model, each triangle in the second 3D model is a patch, and the vertex (vertex) is the vertex of the triangle. In fig. 3, the skin region of the node indicated by the rectangular black filled frame in the left drawing is shown as the black rectangular frame region in the right drawing, white indicates that the skin weight of the node to the vertex is 1, and black indicates that the skin weight of the node to the vertex is 0.

In the above embodiment, the first 3D model includes at least one first vertex, the second 3D model includes at least one second vertex, the first vertex and the second vertex are in one-to-one correspondence, a position of the first vertex and a position of the corresponding second vertex are controlled by nodes in a control node set, the nodes in the control node set are nodes in the skin skeleton model, a distance between the first vertex and the corresponding second vertex is smaller than a preset distance, and the second 3D model is a 3D model generated by the server for a target object in the two-dimensional image.

For example, the first 3D model generated by the electronic device and the second 3D model generated by the server are patch models, and vertices on the two patch models are in one-to-one correspondence. That is, any one of the first vertexes on the first 3D model has a second vertex on the second 3D model, the first vertex and the second vertex are controlled by the same node, and the distance between the first vertex and the second vertex is smaller than the preset distance. That is, the first vertex on the first 3D model and the second vertex on the second 3D model form a plurality of vertex pairs, and a distance between a position of the first vertex and a position of the second vertex in each pair of vertices is less than a preset value.

By adopting the scheme, the server only transmits the skeleton driving coefficient, so that the appearance of the first 3D model generated by the electronic equipment is as close to the appearance of the second 3D model generated by the server as possible, the occupied network flow is small, and the appearance of the driving model can be very lightweight.

Fig. 4 is a schematic diagram of respective deployment conditions of a server and an electronic device in the face 3D model generating method provided in the embodiment of the present application. Referring to fig. 4, the first 3D model is unknown to the server, as indicated by the dashed box. The skin skeleton model and n BS models, namely n blendscape models, are preset on the server. The server can calculate n BS coefficients, i.e. n blendstream coefficients, from the two-dimensional image, and further obtain a second 3D model from the n BS models and the n BS coefficients. In addition, the server may also obtain the second 3D model through other manners, which is not limited in the embodiment of the present application.

After the server obtains the second 3D model, a bone driving coefficient set is determined based on the second 3D model and a preset skinned bone model. The server then transmits the set of bone drive coefficients to the electronic device. The electronic device generates a first 3D model from the set of bone drive coefficients and the skinned bone model.

Referring to fig. 4, in the process that the server determines the bone driving coefficient of each node in the skinned bone model according to the second 3D model and the preset skinned bone model to obtain the bone driving coefficient set, first, the server constructs a first equation, where the first equation is used to indicate a position of a third vertex in the skinned bone model, a skin weight of each node in the skinned bone model to a second vertex in the second 3D model, a position of the second vertex in the second 3D model, and a relationship between the rigid transformation matrix of each node in the skinned bone model, where the third vertex is a vertex corresponding to the second vertex in the skinned bone model. The server then constructs a least squares equation for determining bone drive coefficients for each node in the skinned bone model according to the first equation. And finally, the server determines the bone driving coefficient of the corresponding node according to a least square equation of the bone driving coefficient of each node in the skin bone model so as to obtain the bone driving coefficient set.

Illustratively, in order to make the appearance of the first 3D model generated by the electronic device as close as possible to the appearance of the second 3D model generated by the server, the distance between the position of the first vertex and the position of the second vertex satisfies the following formula (4):

in the formula (1), i represents the vertex i, and n represents the number of vertices in the second 3D model. Since the first 3D model and the second 3D model are close, the number of vertices in the two models is the same. VerTexB _i Representing a second vertex in a second 3D model, vertexA _i ' represents a first vertex in the first 3D model corresponding to the second vertex, vertexA _i ' VertexA in skin skeleton model _i When the vertex of the first 3D model is the vertex whose position is changed _i Second vertex VertexB in' and second 3D model _i The first 3D model and the second 3D model are most similar when the locations of the two models are closest.

In the above-mentioned formula (4),

wherein m represents the number of nodes in the skinned bone model, alpha _(i,j) Representing node j versus second vertex VertexB _i Is of the skin weight, form _j Representing the rigid transformation matrix of node j. VertexA _i And representing a third vertex which is an original vertex of a first vertex in the skin skeleton model, namely, the third vertex is a vertex corresponding to the first vertex in the skin skeleton model, namely, the third vertex is a vertex when the skin skeleton model is not deformed, and the first vertex is a vertex in a first 3D model obtained after deformation. It can be seen from this: when determining the position of each first vertex in the first 3D model, determining the sum of the rigid transformation matrix and the skin weight product of each node in m nodes, and multiplying the determined sum by the position of the third vertex.

Will beBringing into equation (1) yields equation (5):

vertex VertexA for any one of the first 3D models _i ' equation (6) is obtainable according to equation (5) above:

in order to optimize equation (6), the following equation (7) holds:

wherein m represents the number of nodes in the skin skeleton model, and VertexB _i Represents the second vertex, alpha _(i,j) Representing node j versus second vertex VertexB _i Is of the skin weight, form _j Representing the rigid transformation matrix of node j. VertexA _i Representing a third vertex which is the original vertex of a second vertex in the skinned bone model, i.e. the third vertex is the vertex in the skinned bone model corresponding to the second vertex.

The above equation (7) is the first equation. In the above formula (7), form _j Rigid transformation matrix representing node j, global rigid pose of node j (initNode _j Global) can be determined from the local rigid pose of the parent node of node j. The local rigid pose of the node j is stored on the node j, and the skeleton driving coefficient of the node j and the matrix of the local rigid pose of the node j can be mutually converted.

After obtaining the first equation, namely the equation (7), the server can construct a least square equation of the bone driving coefficient of each node according to the first equation, and then for each node, the bone driving coefficient of the node can be determined through the least square equation of the node.

By adopting the scheme, the calculated amount is small, and the consumption of the calculation resources of the server is reduced.

In the above embodiment, when the server constructs the least square equation for determining the bone driving coefficient for each node in the skinned bone model according to the first equation, first, for a first node in the skinned bone model, a set of related nodes and a set of unrelated nodes of the first node are determined, where the first node is any node in the skinned bone model, a node in the set of related nodes is a node where a skinned region changes when the first node undergoes rigid transformation, and a node in the set of unrelated nodes is a node where a skinned region does not change when the first node undergoes rigid transformation. Then, the server constructs a second equation according to the related node set, the irrelevant node set and the first equation. And finally, constructing a least square equation for determining the bone driving coefficient of the first node by the server according to the second equation and the rigid transformation matrix of each node in the related node set.

For any node, the rigid transformation matrix of the node is exemplified by _j Splitting into a transformation independent of the node and a transformation related to the node, and obtaining the following formula (8):

the first node (this node or now node) is referred to below as any node in the skinned bone model. In the above formula (8), the nodes 0-k represent the nodes of the first node correlation transformation, belong to the correlation node set, and the nodes (k+1) to m represent the nodes irrelevant to the first node, belong to the irrelevant node set. In this way, to calculate the bone driving coefficient of the first node, i.e. the node, the second vertices that need to be used include all vertices in the skin region of the first node and its associated sub-nodes. For example, please refer to fig. 5.

Fig. 5 is a schematic diagram of a related node set and an unrelated node set in the face 3D model generating method provided in the embodiment of the present application.Referring to fig. 5, assume that there are 5 nodes: root Node, node E, node F, node G, and Node H, respectively denoted as Node _Root 、Node _E 、Node _F 、Node _G 、Node _H . Of the nodes, the former Node is the parent Node of the latter Node, e.g. Node _E Is Node _F Is a parent node of (c). The 5-node skin region is shown as a diagonally filled portion of the figure.

Referring to fig. 5, when the pose of each node is affected by a parent node and the pose of the parent node is modified, the skin region of the child node of the parent node changes, and at the same time, the position of the vertex in the skin region changes. For example, even Node _F Without skin region, modify Node _F Node when the pose of (a) _G And Node _H Is changed in the skin region of (a). Thus, for Node _F In terms of the related nodes, the related nodes comprise Node _F 、Node _G And Node _H Irrelevant nodes including Node _Root And Node _E 。

In fig. 5, when the pose of the parent node is changed, the skin region of the child node of the parent node is changed, and at the same time, the position of the vertex in the skin region is changed. In addition, if any node has a skin region, when the pose of the node is changed, the position of the vertex in the skin region corresponding to the node is also changed. For example, please refer to fig. 6.

Fig. 6 is a schematic diagram of a change in the position of a vertex under a single node in the face 3D model generating method according to the embodiment of the present application.

Referring to fig. 6, the black original point indicates a vertex in the skin region corresponding to the node G, and when the node G undergoes a rigid transformation, for example, when the position of the node G moves from bottom to top, as indicated by an arrow in the figure, the shape of the skin region corresponding to the node G changes, and at the same time, the position of the vertex changes.

Since the bone drive coefficients of the first node, the now node, are calculated, the second vertices that need to be used include all vertices within the skin region of the first node and its associated child nodes. Thus, when the first Node is Node _F When the second vertex needed to be used comprises Node _F 、Node _G And Node _H All vertices within the corresponding skin region.

After the server obtains the formula (8), looking the equal sign left side of the formula (8) as a whole to obtain the following formula (9):

wherein VertexB _i "means the fourth vertex. The above equation (9) is the second equation. And then, the server constructs a least square equation for determining the bone driving coefficient of the first node according to the second equation and the rigid transformation matrix of each node in the related node set.

By adopting the scheme, the calculation amount is small by constructing the second equation, and the consumption of the calculation resources of the server is reduced.

When constructing a least square equation for determining the bone driving coefficient of the first node according to the second equation and the rigid transformation matrix of each node in the related node set, the server firstly decomposes the rigid transformation matrix of the second node to obtain a third equation, wherein the third equation is used for indicating the relation among the rigid transformation matrix of the first node, the front transformation matrix, the rear transformation matrix and the bone driving coefficient of the first node, the front transformation matrix is the global rigid pose matrix of the father node of the first node, and the rear transformation matrix is at least determined according to the inverse of the global rigid pose matrix of the first node. Then, the server constructs a least square equation for determining the bone driving coefficient of the first node according to the second equation and the third equation.

Illustratively, the server represents the rigid transformation matrix for each node in the set of related nodes as follows equation (10):

Defrom _j ＝preTransform _now ×LocalTRS _now ×PostTransform _j formula (10)

preTransform _now A pre-transformation matrix representing the first node, postTransform _j Representing the post-transform matrix of the first node. Defrom _j Representing a rigid transformation matrix of the second node j, i.e. any one of the set of related nodes. LocalTRS _now Representing the bone drive factor of the first node.

Substituting the above formula (10) into formula (9) yields the following formula (11):

the modification of formula (11) yields the following formula (12):

/>

this formula (12) satisfies the general formula b=xa of the least square method. The above formula (12) is a least square equation for determining the bone driving coefficient of the first node. And then, the server expresses the rigid transformation matrix of each node in the related node set by using a formula (10) to finally obtain a plurality of formulas (12), and further calculates the formulas (12) by using a least square method, a Pu's analysis method and other calculation methods, so that the bone driving coefficient of each node in the related node set is solved. Since each node has a set of related nodes and a set of unrelated nodes. Therefore, the server can determine the bone driving coefficient of each node by adopting the method to obtain the bone driving coefficient set.

By adopting the scheme, the server constructs a least square equation for determining the bone driving coefficient, solves the least square equation to obtain the bone driving coefficient, has less calculation amount and reduces the consumption of calculation resources of the server.

In the above embodiment, the nodes in the related node set include the first node and sub-nodes of the first node, at least one sub-node of the first node is provided, the pre-transformation matrix of the first node is the same as the pre-transformation matrix of any sub-node in the at least one sub-node, and the post-transformation matrices of the nodes in the related node set are different.

Exemplary, referring to FIG. 5, assume that the first Node is a Node _F Then the nodes in the related Node set include Node _F 、Node _G And Node _H . When Node j is Node _F 、Node _G Or Node _H When it is found according to the above formula (7):

Defrom _F ＝parent _F Global×LocalTRS _F ×initNode _F Global ^-1 ；

Defrom _G ＝parent _F Global×LocalTRS _F ×LocalTRS _G ×initNode _G Global ^-1 ；

Defrom _H ＝parent _F Global×LocalTRS _F ×LocalTRS _G ×LocalTRS _H ×initNode _H Global ^T ；

wherein Node _F 、Node _G Or Node _H The front transformation matrix of (a) is the same as that of parent _F Global. Post-transform matrix of initNode _F Global ^-1 ，Node _G Post-transform matrix of LocalTRS _G ×initNode _G Global ^-1 ，Node _H Post-transform matrix of LocalTRS _G ×LocalTRS _H ×initNode _H Global ^T 。LocalTRS _F 、LocalTRS _G 、LocalTRS _H Sequentially representing Node _F 、Node _G And Node _H Bone driving coefficients of (c).

The foregoing describes a specific implementation of the face 3D model generating method according to the embodiment of the present application, and the following is an embodiment of the apparatus of the present application, which may be used to execute the embodiment of the method of the present application. For details not disclosed in the device embodiments of the present application, please refer to the method embodiments of the present application.

Fig. 7 is a schematic structural diagram of a face 3D model generating device according to an embodiment of the present application. The apparatus may be integrated in or implemented by an electronic device. As shown in fig. 7, in the present embodiment, the face 3D model generating apparatus 100 may include: a receiving module 11 and a processing module 12.

The receiving module 11 is configured to receive a set of bone driving coefficients of a target face in a two-dimensional image, where the set of bone driving coefficients includes a bone driving coefficient of each node in a skinned bone model preset locally in the electronic device.

The processing module 12 is configured to deform the skinned bone model by using the bone driving coefficients in the bone driving coefficient set, so as to obtain a first 3D model of the target face in the two-dimensional image.

Fig. 8 is another face 3D model generating device provided in the embodiment of the present application, referring to fig. 8, in the face 3D model generating device 100 provided in the embodiment of the present application, on the basis of fig. 7, the processing module 12 includes:

the first determining unit 121 is configured to determine a global rigid pose matrix of each node in the skinned bone model according to the bone driving coefficient of each node in the skinned bone model, so as to obtain the global rigid pose matrix of each node in the skinned bone model.

A second determining unit 122, configured to determine a rigid transformation matrix of each node in the exoskeleton model according to the global rigid pose matrix of each node in the exoskeleton model.

And a transforming unit 123, configured to transform the positions of the vertices in the skinned bone model according to the rigid transformation matrix of each node in the skinned bone model, so as to obtain a first 3D model of the target face in the two-dimensional image.

In a possible design, the processing module 12 is configured to deform the skinned skeleton model by using skeleton driving coefficients in the skeleton driving coefficient set, so as to obtain a first 3D model of a target face in the two-dimensional image, and then obtain expression information of a user, where the user is a user corresponding to the target face, and drive the first 3D model according to the expression information, so as to generate an animation.

In one possible design, referring to fig. 8 again, the face 3D model generating apparatus 100 further includes: the sending module 13 is configured to send a request instruction to the server before the receiving module 11 receives the bone driving coefficient set of the target face in the two-dimensional image, where the request instruction carries the two-dimensional image, and the request instruction is used to request the server to generate the bone driving coefficient set for the two-dimensional image.

Fig. 9 is a schematic structural diagram of another face 3D model generating device according to an embodiment of the present application. The apparatus may be integrated in a server or implemented by a server. As shown in fig. 9, in the present embodiment, the face 3D model generating apparatus 200 may include: a generating module 21, a processing module 22 and a transmitting module 23.

A generating module 21, configured to generate a second 3D model according to the target face in the two-dimensional image.

The processing module 22 is configured to determine a bone driving coefficient of each node in the skinned bone model according to the second 3D model and a preset skinned bone model, so as to obtain a bone driving coefficient set.

A transmitting module 23, configured to transmit the set of bone driving coefficients to the electronic device.

Fig. 10 is another face 3D model generating device provided in the embodiment of the present application, referring to fig. 8, in the face 3D model generating device 200 provided in the embodiment of the present application, on the basis of fig. 9, the processing module 22 includes:

a first construction unit 221, configured to construct a first equation, where the first equation is used to indicate a relationship among a position of a third vertex in the skin skeleton model, the third vertex being a vertex in the first 3D model, the third vertex being a vertex in the skin skeleton model, the first vertex being a vertex in the first 3D model, the skin weighting of each node in the skin skeleton model to a second vertex in the second 3D model, the position of the second vertex in the second 3D model, and the rigid transformation matrix of each node in the skin skeleton model.

A second construction unit 222 for constructing a least squares equation for determining bone drive coefficients for each node in the skinned bone model, based on the first equation.

The processing unit 223 is configured to determine a bone driving coefficient of a corresponding node according to a least square equation of the bone driving coefficient of each node in the skinned bone model, so as to obtain the bone driving coefficient set.

In a possible design, the second building unit 222 is configured to determine, for a first node in the skinned skeleton model, a set of related nodes and a set of unrelated nodes of the first node, where the first node is any node in the skinned skeleton model, a node in the set of related nodes is a node in which a skinned region changes when the first node undergoes a rigid transformation, and a node in the set of unrelated nodes is a node in which a skinned region does not change when the first node undergoes a rigid transformation; constructing a second equation according to the related node set, the irrelevant node set and the first equation; constructing a least square equation for determining the bone driving coefficient of the first node according to the second equation and the rigid transformation matrix of each node in the related node set

In a possible design, when constructing the least square equation for determining the bone driving coefficient of the first node according to the second equation and the rigid transformation matrix of each node in the related node set, the second construction unit 222 is configured to decompose the rigid transformation matrix of the first node to obtain a third equation, where the third equation is configured to indicate the relationship among the rigid transformation matrix of the first node, the pre-transformation matrix, the post-transformation matrix and the bone driving coefficient of the first node, the pre-transformation matrix is a global rigid pose matrix of a parent node of the first node, the post-transformation matrix is determined at least according to the inverse of the global rigid pose matrix of the first node, and the least square equation for determining the bone driving coefficient of the first node is constructed according to the second equation and the third equation.

In a possible design, the nodes in the related node set include the first node and the sub-nodes of the first node, at least one sub-node of the first node is provided, the front transformation matrix of the first node is the same as the front transformation matrix of any sub-node in the at least one sub-node, and the rear transformation matrices of the nodes in the related node set are different.

In one possible design, referring to fig. 10 again, the face 3D model generating apparatus 200 further includes:

a receiving module 24, configured to receive a request instruction from the electronic device, where the request instruction carries the two-dimensional image, before the generating module 21 generates the second 3D model according to the target face in the two-dimensional image, where the request instruction is used to request the server to generate the bone driving coefficient set for the two-dimensional image.

Fig. 11 is a block diagram of an electronic device for implementing a face 3D model generation method according to an embodiment of the present application. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular telephones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be exemplary only, and are not meant to limit implementations of the application described and/or claimed herein.

As shown in fig. 11, the electronic device includes: one or more processors 31, memory 32, and interfaces for connecting the components, including high-speed interfaces and low-speed interfaces. The various components are interconnected using different buses and may be mounted on a common motherboard or in other manners as desired. The processor may process instructions executing within the electronic device, including instructions stored in or on memory to display graphical information of the GUI on an external input/output device, such as a display device coupled to the interface. In other embodiments, multiple processors and/or multiple buses may be used, if desired, along with multiple memories. Also, multiple electronic devices may be connected, each providing a portion of the necessary operations (e.g., as a server array, a set of blade servers, or a multiprocessor system). In fig. 11, a processor 31 is taken as an example.

Memory 32 is a non-transitory computer readable storage medium provided herein. The memory stores instructions executable by the at least one processor to cause the at least one processor to perform the face 3D model generation method provided herein. The non-transitory computer-readable storage medium of the present application stores computer instructions for causing a computer to execute the face 3D model generation method provided by the present application.

The memory 32 is used as a non-transitory computer readable storage medium, and may be used to store a non-transitory software program, a non-transitory computer executable program, and modules, such as program instructions/modules (e.g., the receiving module 11, the processing module 12, the transmitting module 13, the generating module 21, the processing module 22, and the transmitting module 23 shown in fig. 7, and the generating method of the face 3D model shown in the embodiments of the present application). The processor 31 executes various functional applications of the server and data processing, i.e., implements the face 3D model generation method in the above-described method embodiment, by running non-transitory software programs, instructions, and modules stored in the memory 32.

The memory 32 may include a storage program area that may store an operating system, at least one application program required for functions, and a storage data area; the storage data area may store data created according to the use of the electronic device, etc. In addition, the memory 32 may include high-speed random access memory, and may also include non-transitory memory, such as at least one magnetic disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, the memory 32 may optionally include memory remotely located with respect to the processor 31, which may be connected via a network to an electronic device for performing the face 3D model generation method. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.

The electronic device for performing the face 3D model generation method may further include: an input device 33 and an output device 34. The processor 31, the memory 32, the input device 33 and the output device 34 may be connected by a bus or otherwise, in fig. 11 by way of example.

The input device 33 may receive input numeric or character information and generate relevant key signal inputs for user settings and function control electronics, such as a touch screen, keypad, mouse, trackpad, touch pad, pointer stick, one or more mouse buttons, trackball, joystick, and the like. The output means 34 may include a display device, auxiliary lighting means (e.g., LEDs), tactile feedback means (e.g., vibration motors), and the like. The display device may include, but is not limited to, a Liquid Crystal Display (LCD), a Light Emitting Diode (LED) display, and a plasma display. In some implementations, the display device may be a touch screen.

Various implementations of the systems and techniques described here can be realized in digital electronic circuitry, integrated circuitry, application specific ASIC (application specific integrated circuit), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs, the one or more computer programs may be executed and/or interpreted on a programmable system including at least one programmable processor, which may be a special purpose or general-purpose programmable processor, that may receive data and instructions from, and transmit data and instructions to, a storage system, at least one input device, and at least one output device.

These computer programs (also known as programs, software applications, or code) include machine instructions for a programmable processor, and can be implemented in a high-level procedural and/or object-oriented programming language, and/or in assembly/machine language. As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, apparatus, and/or device (e.g., magnetic discs, optical disks, memory, programmable Logic Devices (PLDs)) used to provide machine instructions and/or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term "machine-readable signal" refers to any signal used to provide machine instructions and/or data to a programmable processor.

To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and pointing device (e.g., a mouse or trackball) by which a user can provide input to the computer. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic input, speech input, or tactile input.

The systems and techniques described here can be implemented in a computing system that includes a background component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such background, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), wide Area Networks (WANs), and the internet.

The computer system may include a client and a server. The client and server are typically remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server can be a cloud server, also called a cloud computing server or a cloud host, and is a host product in a cloud computing service system, so as to solve the defects of large management difficulty, weak service expansibility and the like in the traditional physical hosts and VPS (Virtual Private Server, VPS) service. The server may also be a server of a distributed system or a server that incorporates a blockchain.

The embodiment of the application also provides a face 3D model generation method, which comprises the following steps: acquiring a bone driving coefficient set of a target object in a two-dimensional image, wherein the bone driving coefficient set comprises bone driving coefficients of each node in a skin bone model preset in the local electronic equipment, and deforming the skin bone model by utilizing the bone driving coefficients in the bone driving coefficient set to obtain a 3D model of the target object in the two-dimensional image.

According to the face 3D model generation method, the server only needs to send the skeleton driving coefficient set to the electronic equipment and does not need to send the second 3D model, so that consumption of bandwidth between the electronic equipment and the server can be reduced. Moreover, the first 3D model is obtained based on the skinned skeleton model and the skeleton driving coefficient, is a lightweight model, occupies a small space and is easy to deploy. In addition, the electronic equipment does not need to manually adjust the skeleton driving coefficient of the skin skeleton model, but can obtain the first 3D model automatically and in a second level, does not need to carry out weighted calculation, reduces the consumption of computing resources of the electronic equipment, and avoids the dead halt of the electronic equipment.

According to an embodiment of the present application, there is also provided a computer program product comprising: a computer program stored in a readable storage medium, from which at least one processor of an electronic device can read, the at least one processor executing the computer program causing the electronic device to perform the solution provided by any one of the embodiments described above.

According to an embodiment of the present application, there is also provided a computer program product comprising: a computer program stored in a readable storage medium, from which at least one processor of a server can read the computer program, the at least one processor executing the computer program causing the server to perform the solution provided by any one of the embodiments described above.

It should be appreciated that various forms of the flows shown above may be used to reorder, add, or delete steps. For example, the steps described in the present application may be performed in parallel, sequentially, or in a different order, provided that the desired results of the technical solutions disclosed in the present application can be achieved, and are not limited herein.

The above embodiments do not limit the scope of the application. It will be apparent to those skilled in the art that various modifications, combinations, sub-combinations and alternatives are possible, depending on design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present application are intended to be included within the scope of the present application.

Claims

1. A face 3D model generation method comprises the following steps:

The method comprises the steps that an electronic device receives a bone driving coefficient set of a target face in a two-dimensional image, wherein the bone driving coefficient set comprises bone driving coefficients of each node in a skin bone model preset in the local area of the electronic device;

the electronic equipment determines a global rigid pose matrix of a corresponding node according to the bone driving coefficient of each node in the skin bone model to obtain a global rigid pose matrix of each node in the skin bone model;

the electronic equipment determines a rigid transformation matrix of each node in the skin skeleton model according to the global rigid pose matrix of each node in the skin skeleton model;

and the electronic equipment transforms the positions of the vertexes in the skinned skeleton model according to the rigid transformation matrix of each node in the skinned skeleton model so as to obtain a first 3D model of the target face in the two-dimensional image.

2. The method of claim 1, wherein the electronic device, after deforming the skinned bone model with bone drive coefficients in the set of bone drive coefficients to obtain a first 3D model of a target face in the two-dimensional image, further comprises:

The electronic equipment acquires expression information of a user, wherein the user is a user corresponding to the target face;

and the electronic equipment drives the first 3D model according to the expression information so as to generate animation.

3. The method of claim 1, wherein prior to the electronic device receiving the set of skeletal drive coefficients for the target face in the two-dimensional image, further comprising:

the electronic equipment sends a request instruction to a server, wherein the request instruction carries the two-dimensional image, and the request instruction is used for requesting the server to generate the skeleton driving coefficient set aiming at the two-dimensional image.

4. A face 3D model generation method comprises the following steps:

the server generates a second 3D model according to the target face in the two-dimensional image;

the server builds a first equation, where the first equation is used to indicate a relationship among a position of a third vertex in the skinned skeleton model, a skinning weight of each node in the skinned skeleton model to a second vertex in the second 3D model, a position of the second vertex in the second 3D model, and a rigid transformation matrix of each node in the skinned skeleton model, the third vertex is a vertex in the skinned skeleton model corresponding to the second vertex, the third vertex is a vertex in the skinned skeleton model corresponding to the first vertex, and the first vertex is a vertex in the first 3D model;

The server constructs a least square equation for determining bone driving coefficients for each node in the skinned bone model according to the first equation;

and the server determines the bone driving coefficient of the corresponding node according to a least square equation of the bone driving coefficient of each node in the skin bone model so as to obtain the bone driving coefficient set.

5. The method of claim 4, wherein the server constructs a least squares equation for determining bone drive coefficients for each node in the skinned bone model according to the first equation, comprising:

for a first node in the skinned skeleton model, the server determines a relevant node set and an irrelevant node set of the first node, wherein the first node is any node in the skinned skeleton model, the nodes in the relevant node set are nodes of which the skinned region changes when the first node is subjected to rigid transformation, and the nodes in the irrelevant node set are nodes of which the skinned region does not change when the first node is subjected to rigid transformation;

the server constructs a second equation according to the related node set, the irrelevant node set and the first equation;

And the server constructs a least square equation for determining the bone driving coefficient of the first node according to the second equation and the rigid transformation matrix of each node in the related node set.

6. The method of claim 5, wherein the server constructs a least squares equation for determining bone drive coefficients for the first node from the second equation, the rigid transformation matrix for each node in the set of related nodes, comprising:

the server decomposes a rigid transformation matrix of the first node to obtain a third equation, wherein the third equation is used for indicating the relation among the rigid transformation matrix of the first node, a front transformation matrix, a rear transformation matrix and a bone driving coefficient of the first node, the front transformation matrix is a global rigid pose matrix of a father node of the first node, and the rear transformation matrix is determined at least according to the inverse of the global rigid pose matrix of the first node;

and constructing a least square equation for determining the bone driving coefficient of the first node according to the second equation and the third equation.

7. The method of claim 6, wherein the nodes in the set of related nodes comprise the first node and sub-nodes of the first node, the first node has at least one sub-node, the pre-transform matrix of the first node and the pre-transform matrix of any sub-node in the at least one sub-node are the same, and the post-transform matrices of the nodes in the set of related nodes are different.

8. The method of any of claims 4-7, wherein prior to generating the second 3D model from the target face in the two-dimensional image, the server further comprises:

the server receives a request instruction from the electronic device, wherein the request instruction carries the two-dimensional image, and the request instruction is used for requesting the server to generate the skeleton driving coefficient set for the two-dimensional image.

9. A face 3D model generation apparatus, comprising:

the receiving module is used for receiving a skeleton driving coefficient set of a target face in the two-dimensional image, wherein the skeleton driving coefficient set comprises skeleton driving coefficients of each node in a skin skeleton model preset in the local electronic equipment;

the processing module is used for deforming the skinned bone model by utilizing the bone driving coefficients in the bone driving coefficient set so as to obtain a first 3D model of the target face in the two-dimensional image;

the processing module comprises:

the first determining unit is used for determining a global rigid pose matrix of a corresponding node according to the bone driving coefficient of each node in the skin bone model to obtain a global rigid pose matrix of each node in the skin bone model;

The second determining unit is used for determining a rigid transformation matrix of each node in the skin skeleton model according to the global rigid pose matrix of each node in the skin skeleton model;

10. The apparatus of claim 9, wherein,

the processing module deforms the skinned bone model by utilizing the bone driving coefficients in the bone driving coefficient set to obtain a first 3D model of a target face in the two-dimensional image, and is further used for obtaining expression information of a user, wherein the user is a user corresponding to the target face, and the first 3D model is driven according to the expression information to generate animation.

11. The apparatus of claim 9, wherein the apparatus further comprises:

the sending module is used for sending a request instruction to the server before the receiving module receives the skeleton driving coefficient set of the target face in the two-dimensional image, wherein the request instruction carries the two-dimensional image, and the request instruction is used for requesting the server to generate the skeleton driving coefficient set aiming at the two-dimensional image.

12. A 3D face model generation apparatus, comprising:

the generation module is used for generating a second 3D model according to the target face in the two-dimensional image;

the processing module is used for determining the bone driving coefficient of each node in the skinned bone model according to the second 3D model and a preset skinned bone model so as to obtain a bone driving coefficient set;

a transmitting module, configured to transmit the set of bone driving coefficients to an electronic device;

wherein the processing module comprises:

a first construction unit configured to construct a first equation indicating a relationship among a position of a third vertex in the skinned bone model, the third vertex being a vertex in the skinned bone model corresponding to the second vertex, the third vertex being a vertex in the skinned bone model corresponding to the first vertex, a skinned weight of each node in the skinned bone model to a second vertex in the second 3D model, a position of the second vertex in the second 3D model, and a rigid transformation matrix of each node in the skinned bone model;

a second construction unit for constructing a least square equation for determining bone driving coefficients for each node in the skinned bone model according to the first equation;

And the processing unit is used for determining the bone driving coefficient of the corresponding node according to a least square equation of the bone driving coefficient of each node in the skin bone model so as to obtain the bone driving coefficient set.

13. The apparatus of claim 12, wherein,

the second construction unit is configured to determine, for a first node in the skinned skeleton model, a relevant node set and an irrelevant node set of the first node, where the first node is any node in the skinned skeleton model, a node in the relevant node set is a node in which a skinned region changes when the first node undergoes rigid transformation, and a node in the irrelevant node set is a node in which the skinned region does not change when the first node undergoes rigid transformation; constructing a second equation according to the related node set, the irrelevant node set and the first equation; and constructing a least square equation for determining the bone driving coefficient of the first node according to the second equation and the rigid transformation matrix of each node in the related node set.

14. The apparatus of claim 13, wherein,

The second construction unit is configured to decompose the rigid transformation matrix of the first node to obtain a third equation according to the second equation and the rigid transformation matrix of each node in the related node set when constructing a least square equation for determining the bone driving coefficient of the first node, where the third equation is used to indicate the relationship among the rigid transformation matrix of the first node, the pre-transformation matrix, the post-transformation matrix and the bone driving coefficient of the first node, the pre-transformation matrix is a global rigid pose matrix of a parent node of the first node, the post-transformation matrix is determined at least according to the inverse of the global rigid pose matrix of the first node, and construct a least square equation for determining the bone driving coefficient of the first node according to the second equation and the third equation.

15. The apparatus of claim 14, wherein the nodes in the set of related nodes comprise the first node and children of the first node, the first node has at least one child node, the first node has a same pre-transform matrix and any one of the at least one child node, and the related nodes have different post-transform matrices.

16. The apparatus of any of claims 12-15, wherein the apparatus further comprises:

the receiving module is used for receiving a request instruction from the electronic equipment before the generating module generates the second 3D model according to the target face in the two-dimensional image, the request instruction carries the two-dimensional image, and the request instruction is used for requesting a server to generate the skeleton driving coefficient set aiming at the two-dimensional image.

17. An electronic device, comprising:

at least one processor; and

a memory communicatively coupled to the at least one processor; wherein,,

the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-8.

18. A non-transitory computer readable storage medium storing computer instructions for causing an electronic device to perform the method of any one of claims 1-8.

19. A 3D model generation method, comprising:

acquiring a bone driving coefficient set of a target object in a two-dimensional image, wherein the bone driving coefficient set comprises bone driving coefficients of each node in a skin bone model preset in the local electronic equipment;

Determining a global rigid pose matrix of a corresponding node according to a bone driving coefficient of each node in the skin bone model to obtain a global rigid pose matrix of each node in the skin bone model;

determining a rigid transformation matrix of each node in the skinned bone model according to the global rigid pose matrix of each node in the skinned bone model;

and transforming the positions of the vertexes in the skinned bone model according to the rigid transformation matrix of each node in the skinned bone model so as to obtain a first 3D model of the target face in the two-dimensional image.