CN110619681A - Human body geometric reconstruction method based on Euler field deformation constraint - Google Patents
Human body geometric reconstruction method based on Euler field deformation constraint Download PDFInfo
- Publication number
- CN110619681A CN110619681A CN201910605065.1A CN201910605065A CN110619681A CN 110619681 A CN110619681 A CN 110619681A CN 201910605065 A CN201910605065 A CN 201910605065A CN 110619681 A CN110619681 A CN 110619681A
- Authority
- CN
- China
- Prior art keywords
- field
- human body
- target
- local
- deformation
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
- G06T17/20—Finite element generation, e.g. wire-frame surface description, tesselation
Landscapes
- Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Computer Graphics (AREA)
- Geometry (AREA)
- Software Systems (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Processing Or Creating Images (AREA)
- Image Processing (AREA)
Abstract
The invention discloses a human body geometric reconstruction method based on Euler field deformation constraint, which comprises the following steps: extracting human body feature points by using a random forest regression method by taking a target human body grid as a data source, and calculating to obtain target human body skeleton nodes; driving the skeleton to deform by using the characteristic points, and driving the grid to deform by using an LBS (location based service) method to roughly align the postures of the template model and the target model; performing rigid transformation driven by local shape difference on the basis of a local field generated by HRBF; performing local shape drive-based region expansion and contraction; and optimizing a target energy function based on target data constraint, human body deformation constraint and level set characteristic constraint in the Euler field based on the SDF, calculating a deformation vector field to obtain an SDF deformation result, and driving the template grid to be aligned with the target model according to the shape to realize human body reconstruction. The invention improves the robustness and efficiency of three-dimensional human body structural reconstruction and meets the requirement of human body reconstruction on rough data.
Description
Technical Field
The invention particularly relates to a human body geometric reconstruction method based on Euler field deformation constraint.
Background
In recent years, three-dimensional modeling of objects in virtual scenes, particularly human body three-dimensional modeling, can be applied to not only creation of roles required in movies, games and the like, but also wide application in aspects of virtual fitting, medical assistance, vocational education and the like, but a human body modeling method has certain limitations, and accurate, rapid and vivid reconstruction of a specific human body model is difficult and complicated in calculation. In recent years, three-dimensional human reconstruction meeting engineering and scientific research requirements is receiving attention from scholars and scientific research institutions at home and abroad. Three-dimensional human reconstruction studies based on computer graphics related techniques are undoubtedly facing enormous challenges and attractive to fluid-related researchers and computer graphics researchers.
Three-dimensional human body reconstruction methods are mainly divided into two types: a template-free human body surface reconstruction method and a template-based human body structural reconstruction method. In order to perform the three-dimensional human body structural reconstruction, the general method is to establish the human body shape space or perform the internal anatomy simulation, which are relatively complex and have certain limitations.
Disclosure of Invention
In order to overcome the defects of the prior art, the invention provides a human body geometric reconstruction method based on Euler field deformation constraint, which improves the effect of grid deformation in three-dimensional human body reconstruction, improves the robustness and efficiency of three-dimensional human body structural reconstruction, and meets the requirement of human body reconstruction on rough data.
In order to achieve the purpose, the invention adopts the following technical scheme:
a human body geometric reconstruction method based on Euler field deformation constraint comprises the following three steps:
(1) roughly aligning the human body postures based on the characteristic point driven skeleton deformation: extracting human body feature points by using a random forest regression method by taking a target human body grid as a data source, and calculating through the feature points to obtain target human body skeleton nodes; driving the skeleton to deform by using the characteristic points, and driving the grid to deform by using a linear hybrid skin LBS method, so that the postures of the template model and the target model are roughly aligned;
(2) posture adjustment and region deformation based on HRBF local field constraint: performing rigid transformation driven by local shape difference on the basis of a local field generated by HRBF; and performing region expansion and contraction based on local shape driving;
(3) human body reconstruction based on Euler field target energy function drive evolution: based on the SDF generated by the template grid aligned in posture, a target energy function based on target data constraint, human body deformation constraint and level set characteristic constraint in the Euler field is optimized, a deformation vector field is calculated, an SDF deformation result is obtained, and the template grid and a target model are driven to be aligned in shape so as to realize three-dimensional human body reconstruction.
Further, in the step (1), the template model is in a T-shaped static posture.
Further, the specific process of the step (2) is as follows: the local posture adjustment adopts the rigid transformation of the estimated local HRBF field to estimate the local surface of the target human body, and carries out space discretization on the target data to obtain a symbol distance field phi of the target datatarAnd obtaining the gradient of the fieldDriving a central point set by using a spatial field, and indirectly driving the deformation of the HRBF local field to be close to the local shape of target data; after the target SDF is obtained, the drive vector for the HRBF center point is expressed as
Wherein phi istar(vc) Is indicated at point vcValue of field function of (n)cDenotes vcThe direction of the field gradient, where a particular field function value at the center point is obtained using a tri-linear interpolation of the spatial node values; with u (v)c) For vcDriving, and enabling the central point to finally fall on the surface of the target data through iteration; at the central point to the end positionThen, the normal direction is set to Thus, obtaining a surface alignment result with the HRBF field central point as a core; to make the local HRBF field align with the local part of the target as much as possible, the energy equation of the attitude adjustment is expressed as
TpRepresenting the rotation variation of the template skeleton, and when a rotation axis is estimated by using the rotation transformation of a central point motion set about local nodes, expressing the average by using quaternion, and estimating the rotation angle, iteratively solving by using a local linearization method; here the normal energy constraint of the target surface is addedAnd isRepresents and vcAfter each iteration is finished, updating the target matching point of each central point according to the normal and the position relation by an ICP method, and then performing the next iteration; the distortion vector is consistent with the rough alignment stage, and after the iteration end point is reached, the adjustment result of the human body posture is obtained;
providing a state of a local scale by utilizing expansion and contraction of an HRBF field as initialization input of a fitting stage; when the local field is used for attitude adjustment, the HRBF field is used as the estimation of the local shape, so that the central point set is driven to the target curved surface; firstly, the deformation end point state of HRBF is estimated in advance, and then the appropriate local field expansion and contraction is estimated through the central point drive fitting after each iterationThe gamma value of (2); in knowing the motion endpoint set of HRBF center pointAfter, the energy can be expressed as:
since the unknown is only a constant, the solution to the target energy function can be written directly
Wherein N iscRepresenting the number of the central points except the additional points, when gamma is used as an expansion and contraction coefficient to be applied to a local field, the central points reach new positions through movement, then the state of an Euler vector field is obtained at the new positions where the central points reach, and the central points are moved to the surface of a target through the Euler field driving in the state to obtain a new end point set; through iteration, the expansion and contraction coefficients of the region can be obtained.
Further, the specific process of the step (3) is as follows: driving evolution by using a spatially discretized Euler field target energy function, and reconstructing the shape; discretizing the space where the template and the target human body grid are located to generate an SDF, then establishing a target energy function based on an Euler space by using target data constraint, human body deformation constraint and level set constraint, and estimating a vector field aligning the template and the target SDF by using a gradient descent method; and after the SDFs are aligned, driving the template grids to a target shape by using the SDFs, and finishing the structural human body reconstruction of the target data.
Further, assume the template model SDF is φtmpThe target model SDF is phitarSDF after being driven by vector field is phitmp(Ψ), then the energy that constrains the alignment of the template model vector field to the target model vector field is represented as
The summation operation refers to summing the results of all the nodes participating in the operation in the SDF;
with deformation driving of the HRBF field, a smooth energy constraint based on body deformation is provided for the vector field Ψ, written as:
wherein, JΨI.e., the Jacobian matrix of Ψ, andthen represents ΨbodyA Jacobian matrix of; here ΨbodyMeans that phi can be adjustedtmpPhi and phiHRBFAligned vector field of phiHRBFGenerated by a combined field of local HRBF fields deformed to an end point driven by the target SDF, so psibodyAlso has the characteristics of smoothness and local deformation of human body, and the characteristics are reflected inThus when Ψ vector field local characteristics and ΨbodyWhen approaching, local deformation characteristic constraint of human body can be obtained, so that phi is driventmpThe human body deformation characteristics are better embodied during evolution;
to ensure the level set characteristics of the SDF field, satisfied after each iteration
I.e. to ensure that its iso-surface dimension is 1.
Further, while generating the SDF, the triangular grid points are recorded to correspond to φtmpTrue value psi0At phitmpWhen the driving deformation is carried out in the vector field, the grid points are also driven along with the SDF change as follows
Wherein v isiWhich represents the grid points,indicating the new position after driving of the grid points, # (v)i) Representing grid points viAfter the vector field changes, in the vector field phitmp(Ψ) and a new value inThen represents the corresponding position in the vector fieldtmpGradient of (Ψ)Driven using this formula, v is driven by the step length ηiMoving to the original iso-surface position is equivalent to following the SDF surface.
Further, in the SDF driving process, a local tangential relaxation method is used, and the grid points are moved to the weighted centers of the 1-ring neighborhood points of the grid points according to the weights in each iteration; in the initial state, a mean coordinate method is used for the grids to obtain a projection set q of 1-ring neighborhood points of the grid points on a tangent plane of the grid pointsijAnd the coordinate of the center of mass is bijIf v ═ Σ is satisfied in the initial statejbijqijAnd in iteration, the following formula is used for correction:
wherein the content of the first and second substances,representing the tangentially modified vertex position, the weight μ is expressed as a function of ψ (v)i) Varying smoothing function
Representing the initial value, ψ, after the grid points have not been driven and the SDF has been changed0(vi) Indicates that the triangular mesh points correspond to phitmpThe true value of (d); a relatively large tangential relaxation will be performed at the beginning of the drive, and a gradual tapering relaxation up to 0 is chosen as the target isosurface is approached.
Further, when some regions generate sharp distortion, Laplacian smoothing (Laplacian smoothing) processing is used at the corresponding region points.
The invention has the beneficial effects that:
(1) the invention adopts a top-down template deformation method to align the postures first and then realize local shape alignment, thereby improving the efficiency of the algorithm.
(2) The target energy functions of the space discrete nodes can be operated simultaneously in parallel, so that the operation speed in the process of optimizing alignment is improved.
(3) The method uses HRBF to fit the deformation characteristic of the local human body, realizes human body reconstruction by using a mode of constructing a local field, and reduces the calculation complexity of deformation.
(4) The invention adopts a method for optimally driving the alignment of the human body template model and the target data by using an Euler field energy function containing local field deformation constraint, on one hand, a smooth implicit surface is used for driving the template grid to ensure the reconstruction quality, and on the other hand, the calculation efficiency is improved by using the characteristic that the Euler field drive can be parallelized, thereby solving the reconstruction problem of rough human body data and reconstructing a three-dimensional human body structural geometric model which accords with the shape and the posture of the target.
Drawings
FIG. 1 is a process diagram of a human body geometric reconstruction method based on Euler field deformation constraint.
Fig. 2 is a schematic view of human body feature points.
FIG. 3 is a schematic view of a human body skeleton.
FIG. 4 is a schematic diagram of the HRBF local field of human body.
Fig. 5 is a schematic diagram of a three-dimensional human body reconstruction result when different data are input.
Detailed Description
The general process flow of the geometric reconstruction of the human body based on the Euler field deformation constraint is shown in FIG. 1, and the invention is further explained in the following with reference to the accompanying drawings.
The invention provides a human body geometric reconstruction method based on Euler field deformation constraint, in particular to skeleton deformation driven by characteristic points, local deformation driven by HRBF and shape deformation driven by Euler field, comprising the following steps:
(1) roughly aligning the human body postures based on the characteristic point driven skeleton deformation: taking a target human body grid as a data source, extracting predefined human body characteristic points by using a random forest regression method, and calculating through the characteristic points to obtain target human body skeleton nodes; driving skeleton deformation by using the characteristic points, obtaining skeleton weight distribution on template model grid points by using a Linear Blend skin Skinning (LBS) method, driving skeleton deformation, and driving the grid to deform simultaneously by using corresponding weight so as to roughly align the postures of the template model and the target model;
the method uses a target function nonlinear optimization mode to carry out skeleton alignment and posture alignment; the distribution of the human body characteristic points is shown in fig. 2, the skeleton structure is shown in fig. 3, and skeleton nodes are shown in fig. 3; use ofAndrespectively representing the human body characteristic points l marked in advance on the template modeltmpSet of (2)Extracting human body characteristic points l on a target human body grid by using a pre-trained random forest regression model expressing the position relation of shape characteristics and characteristic pointstarSet of (2)Use ofRepresenting a set of scaling ratios of the skeleton, s representing a scaling ratio of the skeleton, using(Here [. cndot.)]TRepresenting a corresponding column vector) represents a rotation state of the skeleton, whereinRepresenting the rotation and translation of rigid transformation of a human body in space, and theta represents a rotation angle set of all skeleton nodes; the rest state, i.e. the state of the framework in T-shape, is expressed by theta0And (4) showing.
By using feature point sets on a target body meshLocally calculating the average position to obtain the skeleton node set of the target human body gridjtarThe state change of theta is estimated through a target skeleton node and a feature point set; the local scale of the template and the target needs to be consistent firstly, and for a certain skeleton, the father and son nodes of the template are assumed to be j respectivelya,jb(ii) a The parent and child nodes of the target are j 'respectively'a,j′bThen the scaling ratio can be expressed as:
assigning a set of expansion ratios to a template framework, and applying framework expansion transformation to a template grid to obtain a template model consistent with a target local scale; after the scales are consistent, the bone rotation transformation estimation based on the distortion representation is carried out. Unlike continuous frame data used in animation or motion estimation scenarios, where the pose difference of the template and the target generally has a large magnitude; for iterative estimation of a rotation angle, firstly, initializing a template framework by using the obtained target framework node information, and reducing the posture difference between the template and a target; for this purpose, the template skeleton is pre-estimated from the parent to child rotation angles top-down.
The rigid transformation of human body is estimated by the state in the root node space, and for a certain bone, the rotation angle can be estimated by the rotation angle of the vector corresponding to the template and the target bone, and the rotation angle theta and the rotation axis omega can be respectively expressed as:
according toAndthe initialized skeleton rotation and translation vector can be obtainedCan change theta0Is initialized toTo substantially align the template skeleton with the target skeleton; here, it should be noted thatAndafter all the father skeletons corresponding to the current skeleton layer by layer are rotated, the new position of the current skeleton nodeAndthe applied rotation angle and the rotation axis are required. In addition, in the process of scaling and rotation angle estimation, some skeleton nodes are father nodes of a plurality of skeletons, for this purpose, the average value of the calculation result of the corresponding vector of each skeleton is used as the rotation and scaling parameters of the father nodes, wherein in order to ensure the smoothness, the average value of the rotation angle uses a double-quaternion method; at the moment, the template skeleton is approximately aligned with the target skeleton globally, but the two groups of feature pointsAndthere is also some local rotational difference between them.
For detail adjustment, the rotation adjustment of the framework driven by the characteristic points is needed; wherein the feature points at different positions are restricted to the local parts and corresponding skeleton weights are used, so that the difference energy of the two can be expressed as the whole skeleton
Where Δ Θ represents a rotation parameter of the bone, and Δ ΘkRepresenting the amount of change, ξ, of the skeleton kjRepresenting initialized skeleton transform vectors Is xijIn the form of a matrix; t isa(ΔΘk) In the form of a linear mixture of skeletal transformations, wiRepresents the weight of bone i, A (i) represents the set of adjacent bones of bone i, Δ θjThe variation of the rotation angle of the skeleton j is shown, and the initialized distortion vector is used to keep the rotation and the initialization direction consistent in the optimization process; due to the initialized alignment, fromThe state starts, changes are small in the process, so that local linearization of the warped representation can be performed, e.g. for the matrix Ta(ΔΘk) At Δ Θ — 0, the taylor expansion can be approximated as:
where I represents a unit array, so a linearized equation can be solved:
H·ΔΘ＝b
in the linear equation, H and b represent a coefficient matrix and a target value vector, respectively, which are generated by the energy-constrained objective function calculation. In order to minimize the energy function, the linear equation is solved, new linearization is performed near each iteration result, new iteration is performed, and finally the locally adjusted Δ Θ is obtained.
(2) Posture adjustment and region deformation based on HRBF local field constraint: performing rigid transformation driven by local shape difference on the basis of a local field generated by HRBF; and performing region expansion and contraction based on local shape driving;
a) the Hermite Radial Basis Function (HRBF) local field pose adjustment algorithm is as follows:
the human HRBF local field is shown in FIG. 4; the local posture adjustment is carried out in a mode of estimating the rigid transformation of a local HRBF field to enable the rigid transformation to be as close to the local posture of a target human body as possible; to estimate the local surface of the target human body, the target data is discretized spatially to obtain a Signed Distance Field (SDF) phi of the target datatarAnd obtaining the gradient of the fieldAnd driving the central point set by using the spatial field, and indirectly driving the deformation of the HRBF local field so as to be close to the local shape of the target data. After the target SDF is obtained, the drive vector for the HRBF center point may be expressed as
Wherein phi istar(vc) Is indicated at point vcValue of field function of (n)cDenotes vcThe direction of the field gradient, where a particular field function value at the center point is obtained using a tri-linear interpolation of the spatial node values; with u (v)c) For vcDriving, and finally enabling the central point to fall on the surface of the target data through proper iteration times; at the central point to the end positionThen, in order to acquire information on the target surface, the normal direction thereof is set toThus, obtaining a surface alignment result with the HRBF field central point as a core; to align the local HRBF field as closely as possible to the local part of the target, the energy equation for the pose adjustment may be expressed as
Here TpAnd T in the aboveaSimilarly, when the variation of the template skeleton rotation is expressed, and the rotation axis is estimated by using the rotation transformation of the central point motion set about the local node, the rotation angle can be estimated by using the quaternion expression average, and the iterative solution can be performed by using the similar local linearization method. Here the normal energy constraint of the target surface is addedAnd isRepresents and vcAnd the matched central point, namely after each iteration is finished, updating the target matching point of each central point according to the normal and the position relation by an ICP method, and then performing the next iteration. It should be noted that the twist vector is consistent with the rough alignment stage to ensure the continuity of rotation; and obtaining the adjustment result of the human body posture after the iteration end point is reached.
b) The HRBF local field region adjustment algorithm is as follows:
the expansion and contraction of the HRBF field are utilized to provide a more appropriate local scale state as an initialization input of the fitting stage. When the local field is used for attitude adjustment, the HRBF field is used as the estimation of the local shape, so that the central point set is driven to the target curved surface; firstly, estimating a deformation end point state of HRBF in advance, and then estimating a proper gamma value of local field expansion and contraction through center point drive fitting after each iteration; in knowing the motion endpoint set of HRBF center pointAfter, the energy can be expressed as:
it was observed that the solution to the target energy function can in fact be written directly, since the unknowns are only a constant quantity
Wherein N iscRepresenting the number of the central points (except additional points), when gamma is used as an expansion and contraction coefficient to be applied to the local field, the central points reach new positions through movement, then the state of an Euler vector field is obtained at the new positions where the central points reach, and the central points are moved to the target surface through the Euler field drive in the state to obtain a new end point setCombining; through a few iterations, the expansion and contraction coefficients of the region can be obtained.
(3) Human body reconstruction based on Euler field target energy function drive evolution: based on the SDF generated by the template grid aligned in posture, a target energy function based on target data constraint, human body deformation constraint and level set characteristic constraint in the Euler field is optimized, a deformation vector field is calculated, an SDF deformation result is obtained, and the template grid and a target model are driven to be aligned in shape so as to realize three-dimensional human body reconstruction.
Energy constraints constructed based on the Euler field SDF are divided into three parts, namely target data energy constraints, human body deformation energy constraints and energy constraints of a level set; the overall energy equation can be expressed as:
Eshape＝wdEdata+wbEbody+wlElsm
wherein E isdata、Ebody、ElsmRespectively representing the target data energy, the human body deformation energy and the energy of the level set, wd， wb，wlRespectively representing the weight coefficients of the target data energy, the human body deformation energy and the energy of the level set, and respectively introducing the three energy terms;
first, to align the template model with the target model surface as much as possible, in implicitly expressed SDF, it means that the value difference of each node of the SDF field for the two surfaces is as small as possible. The change of the node value is realized by vector field driving, so that the problem is converted into energy constraint on the vector field; assume template model SDF as phitmpThe target model SDF is phitarSDF driven by a vector field is phitmp(Ψ), then the energy that constrains the alignment of the template model vector field to the target model vector field can be expressed as
The summation operation here means that the results of all the nodes participating in the operation in the SDF are summed.
Secondly, since the target data may have noise, affecting the quality of its SDF, if only data item constraints are used, it will likely cause uncontrollable local deformation in the noisy region; the local field generated by the Hermite Radial Basis Function (HRBF) has a good constraint effect in regional deformation, and the combined field generated by the local field has the characteristics of smooth deformation and human body deformation with different regions, so that the vector field Ψ can be provided with smooth energy constraint based on human body deformation by means of deformation driving of the HRBF field, and the constraint can be written as:
here JΨI.e., the Jacobian matrix of Ψ, andthen represents ΨbodyA Jacobian matrix of; here ΨbodyMeans that phi can be adjustedtmpPhi and phiHRBFAligned vector field of phiHRBFGenerated by a combined field of local HRBF fields deformed to an end point driven by the target SDF, so psibodyAlso has the characteristics of smoothness and local deformation of human body, and the characteristics can be reflected inThus when Ψ vector field local characteristics and ΨbodyWhen approaching, local deformation characteristic restriction of human body can be obtained, so that phi is driventmpThe human body deformation characteristic is better embodied during evolution.
In addition, to ensure the level set characteristics of the SDF field, it is also necessary to satisfy the requirements after each iteration
I.e. to ensure that its iso-surface dimension is 1.
Here, a gradient descent method is used to find a solution that minimizes the energy function through a first order iterative update. Because the optimization object is a function in a three-dimensional space, a variational method is required to be used for derivation of an energy function by using an Euler-Lagrange equation; the derivative of each energy term is expressed as follows:
wherein the content of the first and second substances,represents phi after vector field drivetmpThe field gradient of (a) is determined,is indicative of phitmpHessian matrix (Hessian matrix), which is composed of its second order partial derivatives about three coordinate axis directions x, y, z; in a similar manner, the first and second substrates are, whereinThen is the hessian matrix representing the respective components u, v, w of the vector field Ψ in the x, y, z directions; note that the modulus of the field gradient at the denominator position uses a subscript e, expressed as a constant plus a small quantity e 10-5The divisor is avoided to be 0.
After respective derivative expressions are obtained, iteration is carried out on the vector field under a gradient descent framework, and the expressions are recurred
Ψnew＝Ψ-τE′shape(Ψ)
Where π represents the step size, E ', of the gradient descent iteration'shapeDerivative expression, Ψ, representing the total energy in an updatenewRepresenting a new state of the vector field after one iteration; when the update scale of Ψ is smaller than 0.01, the update stops.
Driven by vector field, corresponding phi of template modeltmpPhi can be generated with the target datatarLocal shape alignment; however, the triangular mesh of the template model needs to be aligned with the target shape, so that the template model depending on skeleton driving has a complete reconstruction form, and local shape details of a human body may be locally lost due to the spatial discretization corresponding to the SDF, which requires the deformation of the triangular mesh to migrate the details to the reconstruction result.
First, due to phitmpIt is only one shape estimate of the triangular mesh, which means that the SDF value corresponding to the spatial position of the vertex of the triangular mesh is not necessarily 0; it is therefore necessary to record that the triangular grid points correspond to phi while generating the SDFtmpTrue value psi0At phitmpWhen the driving deformation is carried out in the vector field, the grid points are also driven along with the SDF change as follows
Wherein v isiWhich represents the grid points,indicating the new position after driving of the grid points, # (v)i) Representing grid points viAfter the vector field changes, in the vector field phitmp(Ψ) and a new value inThen represents the corresponding position in the vector fieldtmpGradient of (Ψ)Driven by this formula, it can be driven by appropriateStep length η, viMoving to the original iso-surface position is equivalent to following the SDF surface.
It should be noted that in global SDF, areas of the body where different areas are closely spaced from each other tend to occur, such as gaps between the arms and the body; at this time, the iteration of the grid points should be stopped in time according to the change of the field gradient, so as to avoid the problem of self-intersection which may occur.
Secondly, when the grid points are driven by the SDF in an unconstrained state, tangential distortion can be caused, unnecessary noise is introduced, and the original grid quality is even reduced; for this purpose, a local tangential relaxation method is used, and the grid points are moved to the weighted centers of the 1-ring neighborhood points of the grid points according to the weights in each iteration; to achieve the purpose, in an initial state, a Mean value coordinate (Mean coordinate) method is used for grids to obtain a projection set q of 1-ring neighborhood points of grid points on a tangent plane of the grid pointsijAnd the coordinate of the center of mass is bijIf v ═ Σ is satisfied in the initial statejbijqijAnd in iteration, the following formula is used for correction:
denotes the position of the vertex after tangential correction, where the weight μ can be expressed as a function of ψ (v)i) Varying smoothing function
Here, theRepresenting the initial values after the SDF has been changed without the grid points being driven, as already mentioned above, ψ0(vi) Indicates that the triangular mesh points correspond to phitmpThe true value of (d). It can be seen that the driving will proceed at the beginning of the drivingAnd when the tangential relaxation is larger, the gradual reduction relaxation is selected to be 0 when the target isosurface is approached.
Furthermore, since the driving process is not a global energy constraint, some regions may generate sharp distortions, in which case Laplacian smoothing (Laplacian smoothing) is used at the corresponding region points. The triangular meshes before and after the SDF evolution are driven by proper step length, so that the triangular meshes are deformed into target shapes.
Fig. 5 shows human body reconstruction output results of different data inputs, and fig. 5(a), (b), (c), and (d) respectively show a target data model, a reconstruction result, and an error thermogram from left to right, where the range of the error thermogram is 0 to 4cm, and the errors sequentially correspond to colors with different shades from small to large, so that the size of the error can be determined by the shade of the color, and it can be known from the figure that the lighter the color is, the larger the error is. The target data in fig. 5(a) is from the student's Kinect scan fusion data, the target data in fig. 5(b) is from the cause dataset, the target data in fig. 5(c) is from the human body dataset created by Shahu et al, and fig. 5(d) is the MIT multi-view reconstruction dataset.
From the reconstruction results, fig. 5(a) and (b) show that the model only worn by underpants can better recover the shape of the human body, and the reconstruction result has less deviation when the noise of the target data is small, and of course, when the target itself is a high-precision scanning model, the accuracy after reconstruction is lost compared with the FAUST, which is because the method of the present invention depends more on the details of the human body of the template model, and thus the method focuses on dealing with rough data.
Fig. 5(c) and (d) show the dressing model, for the reconstruction of the dressing model, the HRBF local field in the shape alignment must use a smaller sampling rate, and the weight of the target energy constraint needs to be reduced to avoid causing some distortion introduced by the clothes. However, if the reconstruction result is simply compared with the target data, the grid errors in fig. 5(c) and (d) are larger, but it can be seen that fig. 5(c) and (d) have good estimation of the human body shape. Therefore, the method can reconstruct a good human body shape model for different types of human body data, particularly rough human body surface data.
Technical contents not described in detail in the present invention belong to the well-known techniques of those skilled in the art.
Although illustrative embodiments of the present invention have been described above to facilitate the understanding of the present invention by those skilled in the art, it should be understood that the present invention is not limited to the scope of the embodiments, and various changes may be apparent to those skilled in the art as long as they are within the spirit and scope of the present invention as defined and defined by the appended claims, and all inventions utilizing the inventive concept are protected.
Claims (8)
1. A human body geometric reconstruction method based on Euler field deformation constraint is characterized by comprising the following three steps:
(1) roughly aligning the human body postures based on the characteristic point driven skeleton deformation: extracting human body feature points by using a random forest regression method by taking a target human body grid as a data source, and calculating through the feature points to obtain target human body skeleton nodes; driving the skeleton to deform by using the characteristic points, and driving the grid to deform by using a linear hybrid skin LBS method, so that the postures of the template model and the target model are roughly aligned;
(2) posture adjustment and region deformation based on HRBF local field constraint: performing rigid transformation driven by local shape difference on the basis of a local field generated by HRBF; and performing region expansion and contraction based on local shape driving;
(3) human body reconstruction based on Euler field target energy function drive evolution: based on the SDF generated by the template grid aligned in posture, a target energy function based on target data constraint, human body deformation constraint and level set characteristic constraint in the Euler field is optimized, a deformation vector field is calculated, an SDF deformation result is obtained, and the template grid and a target model are driven to be aligned in shape so as to realize three-dimensional human body reconstruction.
2. The geometric reconstruction method of human body based on Euler field deformation constraint of claim 1, wherein in step (1), the template model is in T-shaped static posture.
3. The geometric reconstruction method of human body based on Euler field deformation constraint according to claim 1, wherein the specific process of step (2) is as follows: the local posture adjustment adopts the rigid transformation of the estimated local HRBF field to estimate the local surface of the target human body, and carries out space discretization on the target data to obtain a symbol distance field phi of the target datatarAnd obtaining the gradient of the fieldDriving a central point set by using a spatial field, and indirectly driving the deformation of the HRBF local field to be close to the local shape of target data; after the target SDF is obtained, the drive vector for the HRBF center point is expressed as
Wherein phi istar(vc) Is indicated at point vcValue of field function of (n)cDenotes vcThe direction of the field gradient, where a particular field function value at the center point is obtained using a tri-linear interpolation of the spatial node values; with u (v)c) For vcDriving, and enabling the central point to finally fall on the surface of the target data through iteration; at the central point to the end positionThen, the normal direction is set to Thus, obtaining a surface alignment result with the HRBF field central point as a core; to align the local HRBF field as closely as possible to the local part of the target, the energy equation for the pose adjustment is expressed as
TpRepresenting the rotation variation of the template skeleton, and when a rotation axis is estimated by using the rotation transformation of a central point motion set about local nodes, expressing the average by using quaternion, and estimating the rotation angle, iteratively solving by using a local linearization method; here the normal energy constraint of the target surface is addedAnd isRepresents and vcAfter each iteration is finished, updating the target matching point of each central point according to the normal and the position relation by an ICP method, and then performing the next iteration; the distortion vector is consistent with the rough alignment stage, and after the iteration end point is reached, the adjustment result of the human body posture is obtained;
providing a state of a local scale by utilizing expansion and contraction of an HRBF field as initialization input of a fitting stage; when the local field is used for attitude adjustment, the HRBF field is used as the estimation of the local shape, so that the central point set is driven to the target curved surface; firstly, estimating a deformation end point state of HRBF in advance, and estimating a proper gamma value of local field expansion and contraction through center point drive fitting after each iteration; in knowing the motion endpoint set of HRBF center pointAfter, the energy can be expressed as:
since the unknown is only a constant, the solution to the target energy function can be written directly
Wherein N iscRepresenting the number of the central points except the additional points, when gamma is used as an expansion and contraction coefficient to be applied to a local field, the central points reach new positions through movement, then the state of an Euler vector field is obtained at the new positions where the central points reach, and the central points are moved to the surface of a target through the Euler field driving in the state to obtain a new end point set; through iteration, the expansion and contraction coefficients of the region can be obtained.
4. The geometric reconstruction method of human body based on Euler field deformation constraint according to claim 1, wherein the specific process of step (3) is as follows: driving evolution by using a spatially discretized Euler field target energy function, and reconstructing the shape; discretizing the space where the template and the target human body grid are located to generate an SDF, then establishing a target energy function based on an Euler space by using target data constraint, human body deformation constraint and level set constraint, and estimating a vector field of the alignment template and the target SDF by using a gradient descent method; and after the SDFs are aligned, driving the template grids to a target shape by using the SDFs, and finishing the structured human body reconstruction of the target data.
5. The Euler field deformation constraint-based human geometry reconstruction method of claim 4, wherein the template model SDF is assumed to be φtmpThe target model SDF is phitarSDF after being driven by vector field is phitmp(Ψ), then the energy that constrains the alignment of the template model vector field to the target model vector field is represented as
The summation operation refers to summing the results of all the nodes participating in the operation in the SDF;
with deformation driving of the HRBF field, a smooth energy constraint based on body deformation is provided for the vector field Ψ, written as:
wherein, JΨI.e., the Jacobian matrix of Ψ, andthen represents ΨbodyA Jacobian matrix of; here ΨbodyMeans that phi can be adjustedtmpPhi and phiHRBFAligned vector field of phiHRBFGenerated by a combined field of local HRBF fields deformed to an end point driven by the target SDF, so psibodyAlso has the characteristics of smoothness and local deformation of human body, and the characteristics are reflected inThus when Ψ vector field local characteristics and ΨbodyWhen approaching, local deformation characteristic constraint of human body can be obtained, so that phi is driventmpThe human body deformation characteristics are better embodied during evolution;
to ensure the level set characteristics of the SDF field, satisfied after each iteration
I.e. to ensure that its iso-surface dimension is 1.
6. The Euler field deformation constraint-based human geometry reconstruction method of claim 4, wherein while generating the SDF, recording triangular grid points corresponding to φtmpTrue value psi0At phitmpWhen the driving deformation is carried out in the vector field, the grid points are also driven along with the SDF change as follows
Wherein v isiWhich represents the grid points,indicating the new position after driving of the grid points, # (v)i) Representing grid points viAfter the vector field changes, in the vector field phitmp(Ψ) and a new value inThen represents the corresponding position in the vector fieldtmpGradient of (Ψ)Driven using this formula, v is driven by the step length ηiMoving to the original iso-surface position is equivalent to following the SDF surface.
7. The Euler field deformation constraint-based human body geometric reconstruction method according to claim 4, wherein in the SDF driving process, a local tangential relaxation method is used, and in each iteration, a grid point is moved to a weighted center of a 1-ring neighborhood point of the grid point according to weight; in the initial state, a mean coordinate method is used for the grids to obtain a projection set q of 1-ring neighborhood points of the grid points on a tangent plane of the grid pointsijAnd the coordinate of the center of mass is bijIf v ═ Σ is satisfied in the initial statejbijqijAnd in iteration, the following formula is used for correction:
wherein the content of the first and second substances,representing the tangentially modified vertex position, the weight μ is expressed as a function of ψ (v)i) Varying smoothing function
Representing the initial value, ψ, after the grid points have not been driven and the SDF has been changed0(vi) Indicates that the triangular mesh points correspond to phitmpThe true value of (d); a relatively large tangential relaxation will be performed at the beginning of the drive, and a gradual reduction of the relaxation up to 0 is chosen as the target isosurface is approached.
8. The euler field deformation constraint-based human geometry reconstruction method according to claim 7, wherein when some regions generate sharp distortion, Laplacian smoothing (Laplacian smoothing) processing is used at the corresponding region points.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910605065.1A CN110619681B (en) | 2019-07-05 | 2019-07-05 | Human body geometric reconstruction method based on Euler field deformation constraint |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910605065.1A CN110619681B (en) | 2019-07-05 | 2019-07-05 | Human body geometric reconstruction method based on Euler field deformation constraint |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110619681A true CN110619681A (en) | 2019-12-27 |
| CN110619681B CN110619681B (en) | 2022-04-05 |
Family
ID=68921459
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910605065.1A Active CN110619681B (en) | 2019-07-05 | 2019-07-05 | Human body geometric reconstruction method based on Euler field deformation constraint |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110619681B (en) |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111311732A (en) * | 2020-04-26 | 2020-06-19 | 中国人民解放军国防科技大学 | 3D human body grid obtaining method and device |
| CN111476901A (en) * | 2020-04-10 | 2020-07-31 | 中国科学技术大学 | Three-dimensional human body shape representation method |
| CN111862318A (en) * | 2020-07-28 | 2020-10-30 | 杭州优链时代科技有限公司 | Digital human body fitting method and system |
| CN112785712A (en) * | 2021-01-25 | 2021-05-11 | 新东方教育科技集团有限公司 | Three-dimensional model generation method and device and electronic equipment |
| CN112991444A (en) * | 2021-02-10 | 2021-06-18 | 北京字跳网络技术有限公司 | Position determination method and device |
| CN116051696A (en) * | 2023-01-10 | 2023-05-02 | 之江实验室 | Reconstruction method and device of human body implicit model capable of being re-illuminated |
| CN117519150A (en) * | 2023-11-02 | 2024-02-06 | 浙江大学 | Autonomous implicit indoor scene reconstruction method combined with boundary exploration |
| CN117519150B (en) * | 2023-11-02 | 2024-06-04 | 浙江大学 | Autonomous implicit indoor scene reconstruction method combined with boundary exploration |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20170032579A1 (en) * | 2015-07-27 | 2017-02-02 | Technische Universiteit Delft | Skeletal Joint Optimization For Linear Blend Skinning Deformations Utilizing Skeletal Pose Sampling |
| CN108665537A (en) * | 2018-05-15 | 2018-10-16 | 清华大学 | The three-dimensional rebuilding method and system of combined optimization human body figure and display model |
| CN109840940A (en) * | 2019-02-11 | 2019-06-04 | 清华-伯克利深圳学院筹备办公室 | Dynamic three-dimensional reconstruction method, device, equipment, medium and system |
-
2019
- 2019-07-05 CN CN201910605065.1A patent/CN110619681B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20170032579A1 (en) * | 2015-07-27 | 2017-02-02 | Technische Universiteit Delft | Skeletal Joint Optimization For Linear Blend Skinning Deformations Utilizing Skeletal Pose Sampling |
| CN108665537A (en) * | 2018-05-15 | 2018-10-16 | 清华大学 | The three-dimensional rebuilding method and system of combined optimization human body figure and display model |
| CN109840940A (en) * | 2019-02-11 | 2019-06-04 | 清华-伯克利深圳学院筹备办公室 | Dynamic three-dimensional reconstruction method, device, equipment, medium and system |
Non-Patent Citations (2)
| Title |
|---|
| ABALAKIN I V等: "Reconstruction of body geometry on unstructured meshes when using immersed boundary method", 《MATEMATICHESKOE MODELIROVANIE》 * |
| LI H等: "Surface reconstruction from unorganized points with l0 gradient minimization", 《COMPUTER VISION AND IMAGE UNDERSTANDING》 * |
Cited By (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111476901A (en) * | 2020-04-10 | 2020-07-31 | 中国科学技术大学 | Three-dimensional human body shape representation method |
| CN111476901B (en) * | 2020-04-10 | 2023-04-07 | 中国科学技术大学 | Three-dimensional human body shape representation method |
| CN111311732A (en) * | 2020-04-26 | 2020-06-19 | 中国人民解放军国防科技大学 | 3D human body grid obtaining method and device |
| CN111862318A (en) * | 2020-07-28 | 2020-10-30 | 杭州优链时代科技有限公司 | Digital human body fitting method and system |
| CN112785712A (en) * | 2021-01-25 | 2021-05-11 | 新东方教育科技集团有限公司 | Three-dimensional model generation method and device and electronic equipment |
| CN112991444A (en) * | 2021-02-10 | 2021-06-18 | 北京字跳网络技术有限公司 | Position determination method and device |
| CN116051696A (en) * | 2023-01-10 | 2023-05-02 | 之江实验室 | Reconstruction method and device of human body implicit model capable of being re-illuminated |
| CN116051696B (en) * | 2023-01-10 | 2023-12-22 | 之江实验室 | Reconstruction method and device of human body implicit model capable of being re-illuminated |
| CN117519150A (en) * | 2023-11-02 | 2024-02-06 | 浙江大学 | Autonomous implicit indoor scene reconstruction method combined with boundary exploration |
| CN117519150B (en) * | 2023-11-02 | 2024-06-04 | 浙江大学 | Autonomous implicit indoor scene reconstruction method combined with boundary exploration |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110619681B (en) | 2022-04-05 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110619681B (en) | Human body geometric reconstruction method based on Euler field deformation constraint | |
| Jiang et al. | Simplicial complex augmentation framework for bijective maps | |
| Sumner et al. | Mesh-based inverse kinematics | |
| Tagliasacchi et al. | 3d skeletons: A state‐of‐the‐art report | |
| US10586371B2 (en) | Motion retargeting method for character animation and apparatus thererof | |
| CN102982578B (en) | Estimation method for dressed body 3D model in single character image | |
| Zhang et al. | Interactive tensor field design and visualization on surfaces | |
| Takayama et al. | Geobrush: Interactive mesh geometry cloning | |
| Mitra et al. | Shadow art | |
| US20090002376A1 (en) | Gradient Domain Editing of Animated Meshes | |
| CN114429538B (en) | Method for interactively editing nerve radiation field geometry | |
| JP2023524252A (en) | Generative nonlinear human shape model | |
| Zhang et al. | Portrait relief modeling from a single image | |
| CN105957154B (en) | A kind of the threedimensional model edit methods and system of data-driven | |
| Zhang et al. | Interactive modeling of complex geometric details based on empirical mode decomposition for multi-scale 3D shapes | |
| Cetinaslan et al. | Sketching manipulators for localized blendshape editing | |
| Dey et al. | Eigen deformation of 3d models | |
| Chen et al. | Efficient registration for human surfaces via isometric regularization on embedded deformation | |
| Cheng et al. | Design and analysis of optimization methods for subdivision surface fitting | |
| Wu et al. | Freescup: A novel platform for assisting sculpture pose design | |
| CN109801367B (en) | Grid model characteristic editing method based on compressed manifold mode | |
| Li et al. | Animating cartoon faces by multi‐view drawings | |
| Salval et al. | Multiresolution surface blending for detail reconstruction | |
| CN111274732A (en) | Grid repairing method based on 'connection relation-position' iterative optimization | |
| Pang et al. | Developing an image-based 3D model editing method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |