WO2023024395A1 - Method and apparatus for model optimization, electronic device, storage medium, computer program, and computer program product - Google Patents

Method and apparatus for model optimization, electronic device, storage medium, computer program, and computer program product Download PDF

Info

Publication number
WO2023024395A1
WO2023024395A1 PCT/CN2021/144033 CN2021144033W WO2023024395A1 WO 2023024395 A1 WO2023024395 A1 WO 2023024395A1 CN 2021144033 W CN2021144033 W CN 2021144033W WO 2023024395 A1 WO2023024395 A1 WO 2023024395A1
Authority
WO
WIPO (PCT)
Prior art keywords
model
vertex
patch
information
optimization
Prior art date
Application number
PCT/CN2021/144033
Other languages
French (fr)
Chinese (zh)
Inventor
李辰宸
项骁骏
余亦豪
Original Assignee
深圳市慧鲤科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 深圳市慧鲤科技有限公司 filed Critical 深圳市慧鲤科技有限公司
Publication of WO2023024395A1 publication Critical patent/WO2023024395A1/en

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/50—Depth or shape recovery
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/50—Image enhancement or restoration using two or more images, e.g. averaging or subtraction
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
    • G06T7/85—Stereo camera calibration
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/90—Determination of colour characteristics

Definitions

  • the present disclosure relates to the technical field of computer vision, and in particular to a model optimization method and device, electronic equipment, storage media, computer programs, and computer program products.
  • the existing detail optimization methods usually choose to optimize the indirect representation of 3D models such as Truncated Signed Distance Function (TSDF), the solution process is often complicated, and the optimized results need to go through additional network construction algorithms to reflect On the final 3D model, not only additional algorithm overhead is introduced, but also the network construction algorithm itself may lose some details again. In view of this, how to improve the quality and efficiency of model optimization has become an urgent problem to be solved.
  • TSDF Truncated Signed Distance Function
  • the disclosure provides a model optimization method and device, electronic equipment, storage media, computer programs, and computer program products.
  • An embodiment of the present disclosure provides a model optimization method, including: obtaining a model to be optimized; wherein, the model to be optimized includes at least one model face, and the model face is composed of model vertices; obtaining attribute information of each model face and each model The first normal information of the model patch, and obtain the first position information of each model vertex; optimize the first normal information of the model patch based on the attribute information of the model patch, and obtain the second normal information of the model patch; based on The first normal information of the model patch and the second normal information of the model patch optimize the first position information of the model vertices, obtain the second position information of the model vertices, and optimize the model to be optimized.
  • An embodiment of the present disclosure provides a model optimization device, including: a model acquisition module, an information acquisition module, a normal optimization module, and a vertex optimization module; a model acquisition module configured to acquire a model to be optimized; wherein the model to be optimized includes at least one The model face is composed of model vertices; the information acquisition module is configured to obtain the attribute information of each model face and the first normal information of each model face, and obtain the first position information of each model vertex; The direction optimization module is configured to optimize the first normal information of the model surface based on the attribute information of the model surface to obtain the second normal information of the model surface; the vertex optimization module is configured to be based on the first normal direction of the model surface The first position information of the vertices of the model is optimized with the information of the information model surface and the second normal direction information, and the second position information of the vertices of the model is obtained, and the model to be optimized is optimized.
  • An embodiment of the present disclosure provides an electronic device, including a memory and a processor coupled to each other, and the processor is configured to execute program instructions stored in the memory, so as to implement some or all steps of the above model optimization method.
  • An embodiment of the present disclosure provides a computer-readable storage medium, on which program instructions are stored, and when the program instructions are executed by a processor, some or all steps of the above-mentioned model optimization method are implemented.
  • An embodiment of the present disclosure provides a computer program, the computer program includes computer readable code, and when the computer readable code is read and executed by a computer, a part or part of the method in any embodiment of the present disclosure is realized. All steps.
  • An embodiment of the present disclosure provides a computer program product, the computer program product includes a non-transitory computer-readable storage medium storing a computer program, and when the computer program is read and executed by a computer, any embodiment of the present disclosure is realized Some or all of the steps in the method.
  • the model to be optimized can be obtained, and the model to be optimized includes at least one model facet, the model facet is composed of model vertices, and the attribute information of each model facet and the first method of each model facet are obtained. direction information, and obtain the first position information of each model vertex, so as to optimize the first normal information of the model surface based on the attribute information of the model surface, obtain the second normal information of the model surface, and based on the model surface
  • the first normal information and the second normal information of the model patch optimize the first position information of the vertices of the model, obtain the second position information of the vertices of the model, and optimize the model to be optimized.
  • the model optimization can be realized without the help of indirect representations of 3D models such as truncated signed distance fields, so the algorithm overhead can be greatly reduced , which is conducive to improving the efficiency of model optimization.
  • the connection relationship between each model patch and each model vertex is considered in the optimization process, it is beneficial to improve the quality of model optimization. Therefore, the present disclosure can improve the quality and efficiency of model optimization.
  • FIG. 1 is a first schematic flow diagram of a model optimization method provided by an embodiment of the present disclosure
  • FIG. 2 is a schematic diagram of a model to be optimized provided by an embodiment of the present disclosure
  • FIG. 3 is a schematic diagram of a model surface and model vertices provided by an embodiment of the present disclosure
  • FIG. 4 is a schematic flow diagram II of a model optimization method provided by an embodiment of the present disclosure.
  • Fig. 5a is a first schematic diagram of a patch to be subdivided and its adjacent patches provided by an embodiment of the present disclosure
  • Fig. 5b is a second schematic diagram of a patch to be subdivided and its adjacent patches provided by an embodiment of the present disclosure
  • Fig. 5c is a third schematic diagram of a patch to be subdivided and its adjacent patches provided by an embodiment of the present disclosure
  • FIG. 6 is a schematic diagram of a model to be optimized and a model after optimization provided by an embodiment of the present disclosure
  • FIG. 7 is a schematic framework diagram of a model optimization device provided by an embodiment of the present disclosure.
  • FIG. 8 is a schematic frame diagram of an electronic device provided by an embodiment of the present disclosure.
  • FIG. 9 is a schematic framework diagram of a computer-readable storage medium provided by an embodiment of the present disclosure.
  • system and “network” are often used interchangeably herein.
  • the term “and/or” in this article is just an association relationship describing associated objects, which means that there can be three relationships, for example, A and/or B can mean: A exists alone, A and B exist simultaneously, and there exists alone B these three situations.
  • the character "/” in this article generally indicates that the contextual objects are an “or” relationship.
  • “many” herein means two or more than two.
  • FIG. 1 is a first schematic flowchart of a model optimization method provided by an embodiment of the present disclosure.
  • the model optimization method can be executed by terminal equipment or other processing equipment, wherein the terminal equipment can be user equipment (User Equipment, UE), mobile equipment, user terminal, terminal, cellular phone, cordless phone, personal digital assistant (Personal Digital Assistant) , PDA), handheld devices, computing devices, vehicle-mounted devices, wearable devices, etc.
  • the terminal equipment can be user equipment (User Equipment, UE), mobile equipment, user terminal, terminal, cellular phone, cordless phone, personal digital assistant (Personal Digital Assistant) , PDA), handheld devices, computing devices, vehicle-mounted devices, wearable devices, etc.
  • PDA Personal Digital Assistant
  • the model optimization method provided by the present disclosure may include the following steps:
  • Step S11 Obtain the model to be optimized.
  • the model to be optimized may include at least one model facet, and the model facet may be composed of model vertices.
  • FIG. 2 is a schematic diagram of a model to be optimized provided by an embodiment of the present disclosure.
  • the model to be optimized can include at least one model facet 21, and the model facet 21 can include but not limited to a triangle, and the model facet 21 is made up of model vertices 22, for example, when the model facet 21 is a triangle, A model patch is formed by connecting three model vertices 22 .
  • Figure 2 is only an example of a possible model to be optimized in practical applications.
  • the model to be optimized can be obtained by reconstructing the model of a single object such as a table or a chair, or it can be It is obtained by reconstructing the model of a scene containing multiple objects such as a sand table and a living room, and the specific form of the optimized model is not limited here.
  • the model to be optimized may be located in the world coordinate system.
  • expressions such as position information and position coordinates may refer to position information and position coordinates in the world coordinate system.
  • multiple key frame data of the target object can be collected through the device, and the key frame data includes RGB images and depth images, and the nearest neighbor iterative algorithm is used
  • ICP Intelligent Closest Point
  • optical flow method optical flow method
  • IMU inertial sensor
  • key frame is a computer animation term that refers to the character Or the frame where the key action in the movement of the object is located, which is equivalent to the original painting in the two-dimensional animation.
  • the depth image of the target object and the reference voxel model can be fused in combination with the camera pose to obtain the reconstructed voxel model of the target object.
  • the model of the three-dimensional object is represented by orderly combination, and the voxels in the reconstructed voxel model may contain voxel information, and the voxel information may include truncated sign distances.
  • the reconstructed voxel model when a new depth image is obtained, can be used as a new reference voxel model, and the above-mentioned fusion of the depth image of the target object and the reference voxel model can be performed again to obtain The step of reconstructing the voxel model of the target object until the collection of multiple key frame data of the target object is completed.
  • the reconstructed voxel model can be processed by means such as Marching Cubes (MC) to obtain a three-dimensional grid as the model to be optimized.
  • multiple key frame data of the target object can be collected through the device, and the key frame data includes RGB images, and the motion recovery structure (Structure from Motion, SFM), solve the internal parameters of the camera and the camera pose, then restore the image depth information through the multi-view matching algorithm (Multi-view Stereo, MVS), and finally generate the 3D mesh of the target object through the Delaunay algorithm, as the Optimize the model.
  • SFM Structure from Motion
  • MVS Multi-view Stereo
  • texture mapping can also be performed on the model to be optimized, so that the model patches in the model to be optimized have image information (such as color, light and shade, etc.), and then after texture mapping The model of is used as the model to be optimized for subsequent optimization.
  • the model optimization method provided by the present disclosure can be applied to computer artificial three-dimensional modeling scenarios, and the model obtained by computer artificial three-dimensional modeling can be used as the model to be optimized, and the model patches of the model can be relatively sparse (i.e. low poly).
  • the model after texture mapping can be optimized to obtain a model with denser model patches and finer structure (ie high model).
  • the model to be optimized can be reconstructed on the basis of at least one frame image.
  • Step S12 Obtain attribute information of each model patch and first normal information of each model patch, and acquire first position information of each model vertex.
  • the first normal information of the model facet t can be represented by a normal vector, then for the model facet t ⁇ T, three points can be randomly selected on the model facet t, for the convenience of description , which can be recorded as p1, p2 and p3, then the first vector connecting p1 and p2 can be obtained according to the position coordinates of p1 and p2, and the second vector connecting p1 and p3 can be obtained according to the position coordinates of p1 and p3, and according to The position coordinates of p2 and p3, resulting in a third vector connecting p2 and p3.
  • the normal vector of the model facet t is orthogonal to the first vector, the second vector, and the third vector (that is, the result of multiplying the normal vector by any of the first vector, the second vector, and the third vector is 0)
  • a system of equations about the normal vector of the model facet t can be established, and the normal vector of the model facet t can be obtained by solving the system of equations.
  • the attribute information of the model patch may include but not limited to: brightness information, hue information, photometric information and diffuse reflection information of the model patch.
  • the luminance information of the model patch t ⁇ T can be expressed as I(t)
  • the hue information of the model patch can be expressed as C(t)
  • the luminosity information of the model patch can be expressed as B(t)
  • the model The diffuse reflection information of the patch can be expressed as A(t).
  • the camera pose P j and camera internal parameters K j can be used to reproject the model to be optimized to the j-th frame image F j In the image coordinate system, and based on the depth buffer algorithm for occlusion judgment. Then, the luminance information I j (t i ) and hue information C j ( t i ).
  • the brightness information and hue information of other model patches can be deduced by analogy, so we won't give examples one by one here.
  • the brightness information and hue information of the model patch can be obtained directly from the texture coordinates and texture images of the model patch in the model to be optimized.
  • the model patch when it is detected that the model patch does not correspond to any image region, the model patch may be skipped during subsequent processing. It should be noted that the following luminance information I j (t i ) and hue information C j (t i ) both indicate that the model patch t i has valid luminance information and hue information.
  • the surface of the target object is rough, and the illumination of the environment where the target object is located is dominated by global illumination (ie, natural light).
  • the environmental illumination coefficient I may be estimated.
  • the third-order spherical harmonic illumination model can be used to describe the ambient light , and the first normal information n( t), diffuse reflection information A(t), to obtain photometric information B(t), the calculation formula of photometric information B(t) can be expressed as:
  • t represents the model facet
  • n(t) represents the first normal information of the model facet t
  • B(t) represents the photometric information of the model facet t
  • A(t) represents the model facet
  • H m (n) represents the spherical harmonic illumination basis function.
  • the spherical harmonic illumination basis function H m (n) can be expressed as:
  • the first normal information n(t) of the model patch t can be represented by a normal vector
  • n x , n y , n z can respectively represent the first normal information n (t) a component in the direction of the x axis, a component in the direction of the y axis, and a component in the direction of the z axis
  • n represents the first normal information of the model patch
  • I represents the environmental illumination coefficient of the model patch t
  • j represents the frame number used to project the image F j of the model to be optimized, and the model to be optimized can be reprojected to the image coordinates of the jth frame image F j
  • B(t) represents the photometric information of the model patch t
  • I j (t) represents the brightness information of the model patch t corresponding to each visible light image G j .
  • the ambient illumination coefficient I is obtained by minimizing the above formula (11), which can make the deduced luminosity information closer to the luminance information of the model patch.
  • the diffuse reflection information A(t) can be initialized, for example, it can be initialized to 1, 0.5 or other values, and the initialization value of the diffuse reflection information A(t) is not here Do limited.
  • the environmental illumination coefficient I can be substituted into formula (1) to obtain the photometric information B(t) of the model patch t.
  • Step S13 Optimizing the first normal information of the model surface based on the attribute information of the model surface to obtain the second normal information of the model surface.
  • a cost function about the second normal information of the model patch can be constructed, and the cost function can be iteratively optimized to obtain the model The second normal information of the patch.
  • the cost function may include a first cost sub-term, a second cost sub-term, a third cost sub-term and a fourth cost sub-term.
  • the first cost sub-item can be used to represent the gradient residual of photometric information and brightness information in the attribute information of the model patch
  • the second cost sub-item can be used to represent the first normal information and the second The deviation between the normal information
  • the third cost sub-item can be used to smooth the second normal information of the model patch
  • the fourth cost sub-item can be used to alleviate the texture copy problem.
  • the gradient value of the photometric information B(t) of the model patch t can be calculated, and the brightness of the model patch t corresponding to each frame image can be calculated
  • the gradient value of the information I j (t) and calculate the gradient value of the photometric information B(t) of the model patch, and the gradient value of the model patch t corresponding to the brightness information I j (t) in each frame image
  • the sum of squared residuals is calculated based on the residual values of the model patch t corresponding to each frame image, and the first cost sub-item E g (t) is obtained.
  • the calculation formula of the first cost sub-item E g (t) can be expressed as:
  • E g (t) represents the first cost sub-item
  • the gradient of the photometric information can be made closer to the gradient of the brightness information.
  • the above-mentioned Formula (12) is modified to obtain the first cost sub-item
  • the first cost sub-item E g (t) can be obtained according to any of the above two methods, that is, by obtaining the gradient value of the photometric information B(t) of the model patch, and the model
  • the patch t corresponds to the residual sum of squares between the gradient values of the brightness information I j (t) in each frame image, or directly calculates the model patch t corresponding to the brightness information I j (t) in each frame image
  • the residual sum of squares with the photometric information B(t) is not limited here.
  • the first cost sub-item E is calculated by the gradient value of the photometric information B(t) of the model patch and the gradient value of the model patch t corresponding to the brightness information I j (t) in each frame image g (t) has better stability.
  • the deviation between the first normal information of the model facet t and the second normal information of the model facet t can be calculated to obtain the second cost factor
  • the term E s (t) The calculation formula of the second cost sub-item E s (t) can be expressed as:
  • E s (t) represents the second cost sub-item
  • n(t) represents the second normal information
  • n 0 (t) represents the first normal information
  • ⁇ . ⁇ represents the vector modulo function .
  • the adjacent facet t′ ⁇ N(t) adjacent to the model facet t can be obtained.
  • the adjacent facets of the model facet t The patch and the model patch t have shared edges, that is, at least one edge is shared between them.
  • the deviation between the second normal information of the model patch t and the second normal information of the adjacent patch t′ can be obtained to obtain the third cost sub-item E r (t).
  • the calculation formula of the third cost sub-item E r (t) can be expressed as:
  • t′ ⁇ N(t) represents the adjacent facets of the model facet t
  • n(t) represents the second normal information of the model facet t
  • n(t′) represents the model facet
  • the second normal information of the adjacent facet t′ of t, ⁇ . ⁇ represents the vector modulo function.
  • the hue information C j (t) of the model patch t in each frame image and the adjacent patch t′ of the model patch t in each frame can be obtained
  • the hue information C j (t′) in the image, and according to the difference between the two, the hue weight is obtained.
  • the difference between the diffuse reflection information A(t) of the model patch t and the diffuse reflection information A(t′) of the adjacent patch t′ of the model patch t can be obtained, and then according to the difference and the hue Weight, to construct the fourth cost sub-item E a (t).
  • the calculation formula of the fourth cost sub-item E a (t) can be expressed as:
  • t′ ⁇ N(t) represents the adjacent patches of the model patch t
  • C j (t) denotes the hue information of the model patch t in each frame image
  • C j (t′) Represents the hue information of the adjacent facet t′ of the model facet t in each frame image
  • A(t) represents the diffuse reflection information of the model facet t
  • A(t′) represents the adjacent facets of the model facet t Diffuse reflection information of t′
  • ⁇ (c) 1/(1+3 ⁇ c ⁇ ) 3
  • c represents the difference between hue information C j (t) and hue information C j (t′)
  • ⁇ ( c) represents the hue weight calculated according to the difference.
  • the hue weight is smaller, so the color of the model patch itself and the texture caused by lighting can be distinguished, which can help suppress the texture copy problem and improve the quality of model optimization.
  • the first cost of all model patches can be combined subitem, second cost subitem, third cost subitem, and fourth cost subitem, and the respective importance levels of first cost subitem, second cost subitem, third cost subitem, and fourth cost subitem, Build the cost function.
  • the importance degree ⁇ g of the first cost sub-item can be set to 1
  • the importance degree ⁇ s of the second cost sub-item can be set to 0.005
  • the importance degree ⁇ r of the third cost sub-item can be set to 0.003.
  • the importance degree ⁇ a of the four-cost subitem can be set to 2, which is not limited here.
  • the cost function E can be expressed as:
  • E g (t) represents the first cost sub-item, ⁇ g represents the importance of the first cost sub-item, such as the weight value;
  • E s (t) represents the second cost sub-item, and ⁇ s represents The importance of the second cost sub-item, such as the weight value;
  • E r (t) represents the third cost sub-item, ⁇ r represents the importance of the third cost sub-item, such as the weight value;
  • E a (t) represents the fourth cost Subitem, ⁇ a represents the importance of the fourth cost subitem, such as weight value.
  • a solution algorithm can be used to solve the cost function, such as L-BFGS (Limited-memory Broyden-Fletcher-Goldfarb-Shanno), DFP (Davidon-Fletcher-Powell) and other algorithms, to optimize each model surface
  • L-BFGS Lited-memory Broyden-Fletcher-Goldfarb-Shanno
  • DFP Davidon-Fletcher-Powell
  • the above-mentioned solving algorithm can be iteratively executed for a preset number of times, and the preset number of times can be set according to actual application needs.
  • the preset times can be set appropriately less, such as 10, 15, etc., or, in the case of relatively loose requirements for solution efficiency, the preset times can be It should be set appropriately larger, for example, it can be set to 30, 35, etc., which is not limited here.
  • the solution algorithm can use quasi-Newton algorithms such as the aforementioned L-BFGS and DFP. The optimal result can be obtained within a short time, and the optimization efficiency can be greatly improved while improving the quality.
  • conventional solution algorithms such as the Gauss-Newton method can also be used, and the solution algorithm used is not specifically limited here.
  • Step S14 Based on the first normal information of the model patch and the second normal information of the model patch, optimize the first position information of the model vertices, obtain the second position information of the model vertices, and optimize the model to be optimized.
  • the first position information may include the first coordinates
  • the second position information may include the second coordinates
  • each model vertex is respectively used as the first current vertex
  • a connection edge exists between the first current vertex and the first current vertex.
  • the model vertex of is used as the reference vertex. Since the endpoints of each edge in the model to be optimized are model vertices, the edge with the first current vertex as the endpoint can be selected as the above-mentioned connecting edge in the model to be optimized, and the selected edge (that is, the other end point of the connecting edge) is used as a reference vertex.
  • the two model patches sharing the connection edge can be regarded as the first patch and the second patch respectively, based on the first normal information of the first patch and the second normal direction of the first patch Information, get the first normal transformation parameters of the first patch, and based on the first normal information of the second patch and the second normal information of the second patch, get the second normal transformation of the second patch parameters, on this basis, and then based on the first normal transformation parameter and the second normal transformation parameter, construct the objective function of the first current vertex, and the objective function includes the first reference sub-item and the second reference sub-item, the first The reference sub-item is the product of the preset reference coefficient and the first coordinate of the first current vertex, the second reference sub-item is the product of the preset reference coefficient and the second coordinate of the first current vertex, and then based on the objective function of each model vertex , solve to get the second coordinates of each model vertex.
  • the product of the preset reference coefficient and the first coordinate of the model vertex, and the product of the preset reference coefficient and the second coordinate of the model vertex are taken into account, and the preset reference coefficient and The product of the first coordinates of the model vertices and the product of the preset reference coefficient and the second coordinates of the model vertices are used as sub-items of the objective function, which is conducive to improving the error tolerance rate of the objective function solution and improving the stability of the solution results.
  • FIG. 3 is a schematic diagram of a model surface and a model vertex provided by an embodiment of the present disclosure. It should be noted that, for visual presentation, FIG. 3 only exemplarily shows a partial area of the model to be optimized.
  • the model vertex A is the first current vertex
  • the edges with the model vertex A as the endpoint include: AB, AC, AD, AE, and the other endpoint of these edges can be used as a reference vertex, that is, the first current Model vertices with connecting edges between vertices may include: model vertex B, model vertex C, model vertex D, and model vertex E.
  • model vertex E may be selected as a reference vertex, so that the two shared connecting edges AE model facets, that is, model facet AED and model facet AEB, respectively as the first facet and the second facet.
  • model vertex B, model vertex C, and model vertex D it can be deduced by analogy, and no more examples will be given here. That is to say, for the first current vertex, if there are N model vertices with edges connected thereto, then N pairs of the first facet and the second facet can be obtained.
  • the second normal information of the first patch can be obtained, and the first normal transformation can be obtained by solving parameters; similarly, the second normal information of the second patch can be obtained by multiplying the first normal information of the second patch by the second normal transformation parameter, and the second normal transformation parameter can be obtained by solving .
  • connection edge may refer to a connection edge shared by the first patch and the second patch, that is, a connection edge formed by connecting the first current vertex and the reference vertex.
  • the connection edge may refer to a connection edge shared by the first patch and the second patch, that is, a connection edge formed by connecting the first current vertex and the reference vertex.
  • the second diagonal of the connecting edge in the second patch constructs the second optimizer item.
  • the objective function of the first current vertex is obtained based on the first optimization sub-item, the first reference sub-item, the second optimization sub-item and the second reference sub-item.
  • the connecting edge AE in the first facet AED can be The diagonal (ie ⁇ ADE) is taken as the first diagonal, and the diagonal (ie ⁇ ABE) connecting the edge AE in the second patch AEB is taken as the second diagonal.
  • the diagonal ie ⁇ ADE
  • the diagonal ie ⁇ ABE
  • the product of the cotangent of the first diagonal and the first normal transformation parameter, and the product of the cotangent of the second diagonal and the second normal transformation parameter can be summed to obtain the first and value, and calculate the first difference between the first coordinate of the first current vertex and the first coordinate of the reference vertex, and calculate the product of the first sum and the first difference to obtain the first optimization sub-item.
  • the cotangent value of the first diagonal angle and the cotangent value of the second diagonal angle can be summed to obtain the second sum value, and the second coordinate of the first current vertex and the second coordinate of the reference vertex can be obtained.
  • the second difference of the two coordinates and calculate the product of the second sum and the second difference to obtain the second optimization sub-item.
  • any model vertex v i ⁇ V it can be regarded as the first current vertex, its first coordinate can be recorded as v i , and its second coordinate can be recorded as v′ i .
  • the reference vertex of the model vertex can be denoted as v j , its first coordinate can be denoted as v j , and its second coordinate can be denoted as v′ j .
  • the two model patches that share the connection edge v i v j that is, the model patches v i v j v k , v i v j v l , are respectively used as the first patch and the second patch, and the first patch
  • the first pair of angles connecting sides v i v j in can be written as
  • the second opposite angle connecting edge v i v j in the second patch can be written as
  • the first normal transformation parameter of the first patch can be written as
  • the second normal transformation parameter of the second patch can be written as Therefore, the objective function of the first current vertex v i can be obtained, and the objective function of the first current vertex v i can be expressed as:
  • ⁇ represents a preset reference coefficient, and the value of ⁇ can be set according to the actual application, for example, it can be set to 0.01.
  • i represents the vertex number of the first current vertex
  • j represents the vertex number of the reference vertex
  • k represents the diagonal number of the first diagonal
  • l represents the diagonal number of the second diagonal
  • ⁇ v i represents the first reference sub-item
  • v i represents the first current vertex
  • v i represents the first coordinate of the first current vertex
  • v' i represents the second coordinate of the first current vertex
  • v j represents the reference vertex
  • v j represents the first coordinate of the reference vertex
  • v' j represents the second coordinate of the reference vertex
  • ⁇ v′ i represents the second reference sub-item
  • N objective functions can be constructed.
  • the N objective functions can be expressed as:
  • N(v i ) represents the set of model vertices connected with the first current vertex v i
  • ⁇ represents a preset reference coefficient
  • the value of ⁇ can be set according to the actual application, for example, it can be set to 0.01.
  • ⁇ v i represents the first reference sub-item
  • v i represents the first current vertex
  • v i represents the first coordinate of the first current vertex
  • v' i represents the second coordinate of the first current vertex
  • v j represents the reference vertex
  • v j represents the first coordinate of the reference vertex
  • v' j represents the second coordinate of the reference vertex
  • ⁇ v′ i represents the second reference sub-item
  • the first diagonal of the connected edges in the first patch and the second diagonal of the connected edges in the second patch may not be acquired.
  • the first optimization sub-item can be constructed directly based on the first normal transformation parameter, the second normal transformation parameter, the first coordinate of the first current vertex, and the first coordinate of the reference vertex.
  • a second optimization sub-item is constructed. Based on this, the objective function of the first current vertex is obtained based on the first optimization sub-item, the first reference sub-item, the second optimization sub-item and the second reference sub-item.
  • the objective function of the first current vertex v i can be simplified as:
  • ⁇ represents a preset reference coefficient, and the value of ⁇ can be set according to the actual application, for example, it can be set to 0.01.
  • i represents the vertex number of the first current vertex
  • j represents the vertex number of the reference vertex
  • k represents the diagonal number of the first diagonal
  • l represents the diagonal number of the second diagonal
  • ⁇ v i represents the first reference sub-item
  • v i represents the first current vertex
  • v i represents the first coordinate of the first current vertex
  • v' i represents the second coordinate of the first current vertex
  • v j represents the reference vertex
  • v j represents the first coordinate of the reference vertex
  • v' j represents the second coordinate of the reference vertex.
  • N objective functions can be constructed, and the N objective functions can also be simplified as:
  • N(v i ) represents the set of model vertices connected to the first current vertex v i
  • ⁇ represents a preset reference coefficient
  • the value of ⁇ can be set according to the actual application, for example, it can be set to 0.01.
  • i represents the vertex number of the first current vertex
  • j represents the vertex number of the reference vertex
  • k represents the diagonal number of the first diagonal
  • l represents the diagonal number of the second diagonal
  • ⁇ v i represents the first reference sub-item
  • v i represents the first current vertex
  • v i represents the first coordinate of the first current vertex
  • v' i represents the second coordinate of the first current vertex
  • v j represents the reference vertex
  • v j represents the first coordinate of the reference vertex
  • v' j represents the second coordinate of the reference vertex.
  • the equations of the second coordinates of each model vertex can be simultaneously established, and the simultaneous equations can be solved, that is, Get the second coordinates of each model vertex.
  • the second coordinates of each model vertex can be obtained by combining the equations about the second coordinates of each model vertex in one solution, which is beneficial to improve model optimization efficiency.
  • each model vertex has been updated, so the surface morphology of the model to be optimized also changes, because in the process of model optimization, each model
  • the position information of vertices is updated synchronously. Therefore, on the one hand, it can improve the optimization efficiency, and on the other hand, it can also take into account the correlation between the vertices of the model, especially the correlation between adjacent vertices with connecting edges; it is beneficial to improve the quality of the optimized mesh of the model, greatly Reduce the occurrence of low-quality patches such as narrow, degenerated, and reversed patches, improve the stability of the process, and improve the usability of the results.
  • the embodiments of the present disclosure can be applied to augmented reality (Augmented Reality, AR), virtual reality (Virtual Reality, VR) and other fields, such as application scenarios such as games, film and television, e-commerce, education, real estate, and home decoration.
  • augmented reality Augmented Reality
  • VR Virtual Reality
  • application scenarios such as games, film and television, e-commerce, education, real estate, and home decoration.
  • application scenarios of the embodiments of the present disclosure are not limited here.
  • the model to be optimized can be obtained, and the model to be optimized includes at least one model facet, the model facet is composed of model vertices, and the attribute information of each model facet and the first method of each model facet are obtained. direction information, and obtain the first position information of each model vertex, so as to optimize the first normal information of the model surface based on the attribute information of the model surface, obtain the second normal information of the model surface, and based on the model surface
  • the first normal information and the second normal information of the model patch optimize the first position information of the vertices of the model, obtain the second position information of the vertices of the model, and optimize the model to be optimized.
  • the model optimization can be realized without the help of indirect representations of 3D models such as truncated signed distance fields, so the algorithm overhead can be greatly reduced , which is conducive to improving the efficiency of model optimization.
  • the connection relationship between each model patch and each model vertex is considered in the optimization process, it is beneficial to improve the quality of model optimization. Therefore, the present disclosure can improve the quality and efficiency of model optimization.
  • FIG. 4 is a second schematic flowchart of a model optimization method provided by an embodiment of the present disclosure.
  • the model to be optimized may undergo a preset number of optimization stages.
  • the model optimization method provided by the embodiment of the present disclosure may include the following steps:
  • Step S41 Obtain the model to be optimized.
  • the model to be optimized includes at least one model facet, and the model facet is composed of model vertices.
  • the model to be optimized can be obtained by reconstructing the model of a single object such as a table or a chair, or it can also be obtained by reconstructing a model of a scene containing multiple objects such as a sand table or a living room, where The specific form of the optimization model is not limited.
  • the model to be optimized may be located in the world coordinate system.
  • Step S42 Obtain attribute information of each model patch and first normal information of each model patch, and acquire first position information of each model vertex.
  • the attribute information of the model patch may include but not limited to: brightness information, hue information, luminosity information and diffuse reflection information of the model patch, and the first normal information of the model patch may be represented by a normal vector .
  • Step S43 Optimizing the first normal information of the model patch based on the attribute information of the model patch to obtain the second normal information of the model patch.
  • a cost function about the second normal information can be constructed according to the attribute information and the first normal information, and the cost function can be iteratively optimized to obtain the second normal information of the model patch.
  • the cost function may include a first cost sub-term, a second cost sub-term, a third cost sub-term and a fourth cost sub-term.
  • the first cost sub-item can be used to represent the gradient residual of the photometric information and brightness information in the attribute information
  • the second cost sub-item can be used to represent the deviation between the first normal information and the second normal information
  • the third cost sub-term can be used to smooth the second normal information
  • the fourth cost sub-term can be used to alleviate the texture copy problem.
  • diffuse reflection information of each model patch can also be optimized and obtained.
  • Step S44 Detect whether the current optimization stage satisfies the preset condition, if so, execute step S45, otherwise execute step S49.
  • the current optimization stage meeting the preset condition may include: the current optimization stage does not reach a preset number of optimization stages.
  • a counting parameter n may be initialized at the first detection, and the counting parameter n is used to represent the current optimization stage, for example, at the first detection, the counting parameter may be set to 0.
  • the preset value can be set according to the actual situation. For example, in the case of higher requirements for model optimization efficiency, the preset value can be set appropriately smaller, such as 2, 3, etc., or, when optimizing the model When the efficiency requirement is relatively loose, the preset value can be set appropriately larger, for example, it can be set to 4, 5, etc., which is not limited here.
  • Step S45 When it is detected that the current optimization stage meets the preset conditions, based on the attribute information of each model patch, the first normal information of each model patch, and the second normal information of each model patch, obtain each Optimization loss for model patches.
  • the model to be optimized can be refined.
  • the optimization loss of the model patch can be obtained based on the attribute information of the model patch, the first normal information of the model patch, and the second normal information of the model patch.
  • the attribute information of the model patch may include but not limited to: brightness information, hue information, luminosity information and diffuse reflection information of the model patch.
  • the brightness information of the model patch can be expressed as I(t)
  • the hue information of the model patch can be expressed as C(t)
  • the photometric information of the model patch can be expressed as B(t)
  • the diffuse reflection information of the model patch can be denoted as A(t).
  • the optimization loss of the model patch can include a first cost sub-item, a second cost sub-item, a third cost sub-item and a fourth cost sub-item, wherein the first cost sub-item is used to represent the gradient residual of photometric information and brightness information difference, the second cost sub-item is used to represent the deviation between the first normal information and the second normal information, the third cost sub-item is used to smooth the second normal information, and the fourth cost sub-item is used to better Alleviate texture duplication issues.
  • the optimization loss here can be understood as the aforementioned cost function.
  • the calculation method of the first cost sub-item can refer to the formula (12) in the previous embodiment
  • the calculation method of the second cost sub-item can refer to the formula (13) in the previous embodiment
  • the third cost sub-item For the calculation method of the item, refer to the formula (14) in the foregoing disclosed embodiments
  • the calculation method of the fourth cost sub-term refer to the formula (15) in the foregoing embodiments.
  • the first cost sub-item, the second cost sub-item, the third cost sub-item and the fourth cost sub-item of each model patch can be used
  • the respective importance of the cost sub-item, the third cost sub-item and the fourth cost sub-item is constructed to obtain the optimization loss.
  • the respective importance levels of the first cost sub-item, the second cost sub-item, the third cost sub-item and the fourth cost sub-item can be used to respectively evaluate the first cost sub-item, the second cost sub-item, the The third cost sub-item and the fourth cost sub-item are weighted to obtain an optimization loss.
  • the importance degree ⁇ g of the first cost sub-item can be set to 1
  • the importance degree ⁇ s of the second cost sub-item can be set to 0.005
  • the importance degree ⁇ r of the third cost sub-item can be set to 0.003
  • the fourth cost sub-item The importance degree ⁇ a of the subitem can be set to 2, which is not limited here.
  • the optimization loss can be equivalent to the cost function E, and the optimization loss can also be expressed as: ⁇ t ⁇ T ( ⁇ g E g (t)+ ⁇ s E s (t)+ ⁇ r E r (t) + ⁇ a E a (t)).
  • the diffuse reflectance information used is the diffuse reflectance obtained by synchronous optimization in the process of updating the first normal information recently.
  • Step S46 Based on the optimization loss of each model patch, select at least one model patch as the patch to be subdivided in the current optimization stage.
  • the model patches can be sorted in descending order of the optimization loss, and the model patch at the pre-ordered position is selected as the patch to be subdivided.
  • the preset order can be set according to the actual application needs. For example, it can be set to 15%, 20%, 25%, etc. according to the percentage, that is, you can choose the model patch in the top 15%, top 20%, or top 25%.
  • the model to be optimized can be refined based on the model patch with a large optimization loss, so it can be beneficial to optimize the details of the part with poor details and improve the quality of model optimization.
  • the first threshold can be set according to actual application needs. For example, in the case of higher requirements for model optimization effects, the first threshold can be set appropriately lower, or, in the case of relatively loose requirements for model optimization effects, the first The threshold can be set appropriately higher, which is not limited here. In this way, the model to be optimized can be refined based on the model patch with a large optimization loss, so it can be beneficial to optimize the details of the part with poor details and improve the quality of model optimization.
  • no model patch with optimization loss greater than the first threshold can be selected, that is, the optimization loss of all model patches is not higher than the first threshold, it can be considered that the current detail effect of the model to be optimized is better, and it can be Skip this model subdivision operation, and directly execute the subsequent step of optimizing the first position information based on the first normal information and the second normal information to obtain the second position information of the model vertices to optimize the model to be optimized.
  • the mesh to be subdivided in order to improve the quality of model optimization, can also be selected based on the optimization loss of the model mesh and the area of the mesh. In some embodiments, after selecting at least one model patch as the patch to be subdivided in the current optimization stage based on the optimization loss, screen the patch to be subdivided whose patch area is greater than or equal to the second threshold, and then based on the screening The obtained face piece to be subdivided is subdivided into the model to be optimized, and the subdivided model to be optimized is obtained.
  • model patches with too small patch area contribute little to model optimization, it can reduce the negative impact on optimization efficiency caused by over-subdivision of model patches with too small patch area, which is beneficial to improve Model optimization efficiency.
  • the area to be subdivided cannot be selected with a patch area not lower than the second threshold, that is, the area of all the patches to be subdivided is lower than the second threshold, then these surfaces to be subdivided can be considered as
  • the slice does not need to be further subdivided, and this model subdivision operation can be skipped, and the subsequent optimization of the first position information based on the first normal information and the second normal information can be directly performed to obtain the second position information of the model vertices to be optimized. Steps to optimize the model.
  • the third threshold may not be limited, for example, it may be set to 0, 100, 500, 2000 and so on.
  • Step S47 based on the mesh to be subdivided, the model to be optimized is subdivided to obtain the subdivided model to be optimized, and the model mesh of the model to be optimized is updated.
  • At least one model vertex may be added on the edge of the mesh to be subdivided, and each newly added model vertex may be used as a second current vertex, and a set of candidate vertices for the second current vertex may be obtained.
  • the set of candidate vertices includes at least one candidate vertex, and each candidate vertex is located on each candidate facet (can be located in the candidate facet or on the edge of the candidate facet), and each candidate facet contains the second current vertex. side.
  • at least one candidate vertex is selected from the candidate vertex set, and at least one candidate vertex is connected to the second current vertex respectively to obtain a subdivided model to be optimized.
  • model patches can be subdivided at positions where the details of the model to be optimized are not good, which is conducive to achieving more refined optimization at these positions and improving the quality of model optimization.
  • At least one model vertex may be added to each edge of the mesh to be subdivided.
  • the newly added model vertex may be located at the midpoint of each edge of the mesh to be subdivided.
  • FIG. 5 a is a first schematic diagram of a face piece to be subdivided and its adjacent face pieces provided by an embodiment of the present disclosure. It should be noted that the face to be subdivided shares the connecting edge with its adjacent face. Edge AC shares connecting edge BC with its adjacent patch BCD.
  • a new model vertex can be added on each side of the surface ABC to be subdivided, that is, a model vertex G can be added on the side AB of the surface ABC to be subdivided, and A new model vertex H is added on the edge AC of ABC, and a new model vertex I is added on the edge BC of the mesh ABC to be subdivided.
  • a model vertex G can be added on the side AB of the surface ABC to be subdivided
  • a new model vertex H is added on the edge AC of ABC
  • a new model vertex I is added on the edge BC of the mesh ABC to be subdivided.
  • Fig. 5a taking the model vertex G as the second current vertex as an example, since the second current vertex G is located on the edge AB, the model facet ABF and the model facet containing the edge AB can be ABC is used as a candidate face, and the model vertices (that is, model vertex A, model vertex B, model vertex C, model vertex F, model vertex G, model vertex H, and model vertex I) located on these two candidate faces are used as candidate vertices.
  • model vertices that is, model vertex A, model vertex B, model vertex C, model vertex F, model vertex G, model vertex H, and model vertex I
  • model vertex G is still used as Taking the second current vertex as an example, model vertex C, model vertex H, model vertex I, and model vertex F may be used as candidate vertices.
  • model vertex C, model vertex H, model vertex I, and model vertex F may be used as candidate vertices.
  • more candidate vertices can be selected in the candidate vertex set, and the second current vertex and candidate vertices are connected to subdivide the model to be optimized .
  • a new model vertex can be added in the patch to be subdivided, and the newly added model vertex can be used as the second current vertex, and the vertex of the patch to be subdivided and the second current vertex can be respectively connected to obtain a detailed The model to be optimized after classification.
  • a point in the patch to be subdivided can be randomly selected as the newly added model vertex, or the center of gravity of the patch to be subdivided can be selected as the newly added model vertex, which is not limited here.
  • FIG. 5 b is a schematic diagram 2 of a patch to be subdivided and its adjacent patches provided by an embodiment of the present disclosure.
  • the center of gravity G of the patch ABC to be subdivided can be selected as the newly added model vertex, and it can be used as the second current vertex, and vertex A, vertex B, and vertex C of the patch to be subdivided are respectively connected and the second current vertex G, thereby subdividing the patch ABC to be subdivided into a patch AGC, a patch BGC and a patch AGB.
  • Other situations can be deduced by analogy, and no more examples will be given here.
  • a new model vertex can be added in the patch to be subdivided, and the vertex of the patch to be subdivided is connected to the newly added model vertex, and continues to extend to the opposite side of the vertex to intersect to obtain The new model vertex.
  • all new model vertices can be used as the second current vertex respectively, and the candidate vertex set of the second current vertex is obtained, and the candidate vertex set includes at least one candidate vertex, and each candidate vertex is located on the edge of each candidate patch , and each candidate patch contains the edge where the second current vertex is located, and at least one candidate vertex is selected in the candidate vertex set, and at least one candidate vertex and the second current vertex are respectively connected to obtain the subdivided model to be optimized .
  • FIG. 5c is a third schematic diagram of a face piece to be subdivided and its adjacent face pieces provided by an embodiment of the present disclosure.
  • the center of gravity J of the surface to be subdivided ABC can be selected as the newly added model vertex, and the vertex A, vertex B, and vertex C of the surface to be subdivided ABC are respectively connected to the newly added model vertex J , and continue to extend to the opposite side BC of vertex A, the opposite side AC of vertex B, and the opposite side AB of vertex C to intersect to obtain new model vertex I, new model vertex H, and new model vertex G respectively.
  • all the new model vertex G, the new model vertex H, and the new model vertex I can be respectively used as the second current vertex, and a set of candidate vertices for the second current vertex can be obtained.
  • the patch ABC to be subdivided can be subdivided into patches AGJ, patch AHJ, patch CHJ, patch CIJ, patch BIJ, and patch BGJ, and the patch to be subdivided
  • the adjacent facet ABF of ABC is subdivided into facet AGF and facet BGF
  • the adjacent facet BCD of the facet ABC to be subdivided is subdivided into facet BID and facet CID
  • the facet ABC to be subdivided Adjacent patch ACE is subdivided into patch AHE and patch CHE.
  • Other situations can be deduced by analogy, and no more examples will be given here.
  • the counting parameter n+1 can be set.
  • Step S48 Re-execute steps S42 to S47.
  • the re-execution of steps S42 to S47 is for the updated model to be optimized when step S47 was executed last time.
  • the re-execution of step S42 may include: obtaining the attribute information of each model patch in the updated model to be optimized and the first normal information of each model patch, and obtaining the updated model vertex information in the model to be optimized. First location information.
  • the difference from the first execution of step S42 is that when step S42 is executed for the Nth (N>1)th time, the diffuse reflection information in the attribute information of the model patch is the last time (that is, the N-1th) execution of step S42 and its The diffuse reflection information obtained in the subsequent steps is the newly obtained diffuse reflection information.
  • Step S49 When it is detected that the current optimization stage does not meet the preset conditions, optimize the first position information of the model vertices based on the first normal information of the model patch and the second normal information of the model patch, and obtain the model vertex The second position information of the to-be-optimized model is optimized.
  • FIG. 6 is a schematic diagram of a model to be optimized and a model after optimization provided by an embodiment of the present disclosure.
  • the model shown in FIG. 6 is the model after rendering. As shown in FIG. 6, it corresponds to the model 610 of the "Sculpture of David” before optimization.
  • the face details of the model 611 of the "Sculpture of David” are richer; and corresponding to the model 620 of the "Lion Statue” before optimization, the details of the mane of the optimized “Lion Statue” model 621 are richer.
  • the Optimize the loss so that based on the optimization loss, at least one model patch is selected as the patch to be subdivided in the current optimization stage, and based on the patch to be subdivided, the model to be optimized is subdivided, and the attributes of each model patch are re-executed information and first normal information steps and subsequent steps.
  • the optimization loss of each model patch can be detected, and the patch to be subdivided can be selected according to the optimization loss, so that the model to be optimized can be subdivided according to the patch to be subdivided, and the model details can be improved. It is beneficial to improve the quality of model optimization.
  • FIG. 7 is a schematic framework diagram of a model optimization device provided by an embodiment of the present disclosure.
  • the model optimization device 70 includes: a model acquisition module 71 , an information acquisition module 72 , a normal direction optimization module 73 and a vertex optimization module 74 .
  • the model obtaining module 71 is configured to obtain a model to be optimized; wherein, the model to be optimized includes at least one model patch, and the model patch is composed of model vertices.
  • the information acquiring module 72 is configured to acquire attribute information of each model patch and first normal information of each model patch, and acquire first position information of each model vertex.
  • the normal optimization module 73 is configured to optimize the first normal information of the model patch based on the attribute information of the model patch, and obtain the second normal information of the model patch.
  • the vertex optimization module 74 is configured to optimize the first position information of the model vertices based on the first normal information of the model facets and the second normal direction information of the model facets, obtain the second position information of the model vertices, and optimize the model to be optimized .
  • the present disclosure can improve the quality and efficiency of model optimization.
  • the first location information includes first coordinates
  • the second location information includes second coordinates.
  • the vertex optimization module 74 includes a reference selection submodule configured to use each model vertex as the first current vertex respectively, and select a model vertex that has a connection edge with the first current vertex as a reference vertex, and use two shared connection edges The model faces are respectively used as the first face and the second face.
  • the vertex optimization module 74 includes a normal transformation submodule configured to obtain a first normal transformation parameter of the first patch based on the first normal information of the first patch and the second normal information of the first patch, and Based on the first normal information of the second patch and the second normal information of the second patch, a second normal transformation parameter of the second patch is obtained.
  • the vertex optimization module 74 includes a function construction sub-module configured to construct an objective function of the first current vertex based on the first normal transformation parameter and the second normal transformation parameter; wherein, the objective function includes a first reference sub-item and a second reference sub-item sub-items, the first reference sub-item is the product of the preset reference coefficient and the first coordinate of the first current vertex, and the second reference sub-item is the product of the preset reference coefficient and the second coordinate of the first current vertex.
  • the vertex optimization module 74 includes a function solving submodule configured to obtain the second coordinates of each model vertex based on the objective function of each model vertex.
  • the products of the preset reference coefficients and the first coordinates of the model vertices and the products of the second coordinates of the model vertices are taken into account, and the preset reference coefficients and the model vertices
  • the product of the first coordinate and the product of the preset reference coefficient and the second coordinate of the vertex of the model are used as sub-items of the objective function, which is conducive to improving the error tolerance rate of the objective function solution and improving the stability of the solution result.
  • the function building submodule includes a diagonal obtaining unit configured to obtain a first diagonal of connected edges in the first patch, and obtain a second diagonal of connected edges in the second patch.
  • the function construction submodule includes a first subitem unit configured to be based on the first normal transformation parameter and the first diagonal, the second normal transformation parameter and the second diagonal, and the first coordinates of the first current vertex, the reference vertex The first coordinate of is used to construct the first optimization sub-item.
  • the function construction submodule includes a second subitem unit configured to construct a second optimized subitem based on the first diagonal, the second diagonal, the second coordinates of the first current vertex, and the second coordinates of the reference vertex.
  • the function construction sub-module includes a function construction unit configured to obtain the objective function of the first current vertex based on the first optimization sub-item, the first reference sub-item, the second optimization sub-item and the second reference sub-item.
  • the first optimization sub-item and based on the first diagonal of the connecting edge in the first patch, the first coordinate of the second patch Connecting the second opposite corners of the edges, and the second coordinates of the first current vertex and the second coordinates of the reference vertex, a second optimization sub-item is constructed.
  • the objective function of the first current vertex can be obtained, so in the optimization process, the relative The correlation between adjacent patches is beneficial to improve the optimization quality.
  • the function solving submodule includes an equation simultaneous unit configured to simultaneously combine a set of equations about the second coordinates of each model vertex based on the objective function of each model vertex.
  • the function solving sub-module includes an equation solving unit configured to solve the equation system to obtain the second coordinates of each model vertex.
  • the model optimization device 70 includes a loss calculation module configured to, based on the attribute information of each model patch and the first normal direction of each model patch, when it is detected that the current optimization stage meets the preset condition information and the second normal information of each model patch to obtain the optimization loss of each model patch.
  • the model optimization device 70 includes a patch selection module configured to select at least one model patch as the patch to be subdivided in the current optimization stage based on the optimization loss.
  • the model optimization device 70 includes a model subdivision module configured to subdivide the model to be optimized based on the mesh to be subdivided, obtain the subdivided model to be optimized, and update the model mesh of the model to be optimized.
  • the information acquisition module 72 is configured to obtain the attribute information of each model patch in the updated model to be optimized and the first normal information of each model patch, and obtain the first normal direction information of each model vertex in the updated model to be optimized. location information.
  • model optimization by detecting the optimization loss of each model patch, and according to the optimization loss, select the patch to be subdivided, and then subdivide the model to be optimized according to the patch to be subdivided, and obtain the subdivided
  • the model to be optimized is updated to update the model facets of the model to be optimized, thereby improving model details and improving the quality of model optimization.
  • the patch selection module includes a sorting submodule configured to sort the model patches in descending order of optimization loss.
  • the patch selection module includes a selection submodule configured to select a model patch located in a pre-ordered position as a patch to be subdivided; or, the patch selection module is configured to select a model patch with an optimization loss greater than the first threshold as Dough to be subdivided.
  • the model patches are sorted, and the model patch at the pre-ordered position is selected as the model to be optimized, or the model patch with the optimization loss greater than the first threshold is directly selected as the model to be optimized.
  • the patch to be subdivided can refine the model to be optimized based on the model patch with a large optimization loss, so it can help optimize the details of the part with poor details and improve the quality of model optimization.
  • the model subdivision module includes a vertex adding submodule configured to add at least one model vertex on an edge of the mesh patch to be subdivided.
  • the model subdivision module includes a collection acquisition submodule configured to use each newly added model vertex as a second current vertex, and obtain a set of candidate vertices for the second current vertex; wherein, the set of candidate vertices includes at least one candidate vertex, and each The candidate vertices are located on each candidate facet, and each candidate facet includes an edge where the second current vertex is located.
  • the model subdivision module includes a vertex connection submodule configured to select at least one candidate vertex from the candidate vertex set, and respectively connect at least one candidate vertex with the second current vertex to obtain a subdivided model to be optimized.
  • At least one model vertex is added on the edge of the patch to be subdivided, each newly added model vertex is used as the second current vertex, and the candidate vertex set of the second current vertex is obtained, and the candidate vertex set contains at least one Candidate vertices, each candidate vertex is located on each candidate facet, and each candidate facet contains the edge where the second current vertex is located, on this basis, select at least one candidate vertex in the candidate vertex set, and connect at least one candidate vertex respectively With the second current vertex, the refined model to be optimized can be obtained, so that the model patch can be subdivided at the positions where the details of the model to be optimized are not good, which is conducive to achieving more refined optimization in these parts and improving the quality of model optimization .
  • the vertex optimization module 74 is configured to optimize the model based on the first normal information of the model patch and the second normal information of the model patch when it is detected that the current optimization stage does not satisfy the preset condition
  • the first position information of the vertex is obtained to obtain the second position information of the vertex of the model, and the model to be optimized is optimized.
  • the first position information of the model vertex is optimized based on the first normal information of the model patch and the second normal information of the model patch, and the first position information of the model vertex is obtained.
  • the second position information is used to optimize the model to be optimized, that is, if the current optimization stage does not meet the preset conditions, the model will not be subdivided, which is beneficial to better understand the details under the limited time cost.
  • the refinement of the best parts is beneficial to the time-consuming control model optimization.
  • the model optimization device 70 further includes a mesh screening module configured to screen meshes whose area is greater than or equal to the second threshold to be subdivided.
  • the model subdivision module is configured to subdivide the model to be optimized based on the screened faces to be subdivided to obtain the subdivided model to be optimized.
  • the patch to be subdivided is selected from the patches to be subdivided according to the area of the patch.
  • Larger model patches with larger patch areas should be refined to optimize the model.
  • the negative impact on optimization efficiency caused by over-subdivision of the mesh is beneficial to improve the efficiency of model optimization. It is beneficial to improve the stability of model optimization.
  • the current optimization stage meeting the preset condition includes: the current optimization stage does not reach the preset value of optimization stages.
  • the current optimization stage to meet the preset conditions to include: the current optimization stage does not reach the preset number of optimization stages, it is possible to selectively subdivide the patches that need to be subdivided as needed, and control the increase in time overhead Under the premise of improving the quality of model optimization.
  • FIG. 8 is a schematic frame diagram of an electronic device provided by an embodiment of the present disclosure.
  • an electronic device 80 includes a memory 81 and a processor 82 coupled to each other, and the processor 82 is configured to execute the program instructions stored in the memory 81 to implement any of the above-mentioned model optimization method embodiments
  • the electronic device 80 may include, but is not limited to: a microcomputer and a server.
  • the electronic device 80 may also include mobile devices such as notebook computers and tablet computers, which are not limited here.
  • the processor 82 is used to control itself and the memory 81 to implement the steps in any one of the above embodiments of the model optimization method.
  • the processor 82 may also be called a CPU (Central Processing Unit, central processing unit).
  • the processor 82 may be an integrated circuit chip with signal processing capability.
  • the processor 82 can also be a general-purpose processor, a digital signal processor (Digital Signal Processor, DSP), an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), a field-programmable gate array (Field-Programmable Gate Array, FPGA) or other Programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.
  • a general-purpose processor may be a microprocessor, or the processor may be any conventional processor, or the like.
  • the processor 82 may be jointly realized by an integrated circuit chip.
  • the model is directly optimized by using the normal information before optimization and the normal information after optimization without resorting to indirect representations such as truncated signed distance fields, so the algorithm overhead can be greatly reduced, which is beneficial to improve Optimization efficiency, on the other hand, because the connection relationship between each model patch and each model vertex is considered in the optimization process, it is beneficial to improve the optimization quality, so the quality and efficiency of model optimization can be improved.
  • FIG. 9 is a schematic framework diagram of a computer-readable storage medium provided by an embodiment of the present disclosure.
  • a computer-readable storage medium 90 stores program instructions 91 that can be executed by a processor, and the program instructions 91 are used to implement some or all of the steps in any of the above embodiments of the model optimization method.
  • a computer readable storage medium may be a tangible device that can hold and store instructions for use by an instruction execution device, and may be a volatile storage medium or a nonvolatile storage medium.
  • a computer readable storage medium may be, for example, but is not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing.
  • Computer-readable storage media include: portable computer diskettes, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), or flash memory), static random access memory (SRAM), compact disc read only memory (CD-ROM), digital versatile disc (DVD), memory stick, floppy disk, mechanically encoded device, such as a printer with instructions stored thereon A hole card or a raised structure in a groove, and any suitable combination of the above.
  • RAM random access memory
  • ROM read-only memory
  • EPROM erasable programmable read-only memory
  • flash memory static random access memory
  • SRAM static random access memory
  • CD-ROM compact disc read only memory
  • DVD digital versatile disc
  • memory stick floppy disk
  • mechanically encoded device such as a printer with instructions stored thereon
  • a hole card or a raised structure in a groove and any suitable combination of the above.
  • computer-readable storage media are not to be construed as transient signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., pulses of light through fiber optic cables), or transmitted electrical signals.
  • An embodiment of the present disclosure also proposes a computer program, the computer program includes computer readable code, and when the computer readable code is read and executed by a computer, part of the method in any embodiment of the present disclosure is implemented or all steps.
  • An embodiment of the present disclosure also provides a computer program product, including computer-readable codes, or a non-volatile computer-readable storage medium carrying computer-readable codes, when the computer-readable codes are stored in a processor of an electronic device When running in the electronic device, the processor in the electronic device executes some or all steps of the above method.
  • the above scheme directly optimizes the model through the normal information before and after optimization, without resorting to indirect representations such as truncated signed distance fields, so it can greatly reduce the algorithm overhead and help improve optimization efficiency.
  • the connection relationship between each model patch and each model vertex is considered, so it is beneficial to improve the optimization quality, so the quality and efficiency of model optimization can be improved.
  • This disclosure relates to the field of augmented reality.
  • acquiring the image information of the target object in the real environment and then using various visual related algorithms to detect or identify the relevant features, states and attributes of the target object, and obtain a virtual reality that matches the application.
  • AR effects combined with reality may involve faces, limbs, gestures, actions, etc. related to the human body, or markers and markers related to objects, or sand tables, display areas or display items related to venues or places.
  • Vision-related algorithms can involve visual positioning, SLAM, 3D reconstruction, image registration, background segmentation, object key point extraction and tracking, object pose or depth detection, etc.
  • the application can not only involve interactive scenes such as tour, navigation, explanation, reconstruction, virtual effect overlay display and other interactive scenes related to real scenes or objects, but also can involve special effects processing related to people, such as makeup beautification, body beautification , special effects display, virtual model display and other interactive scenes.
  • interactive scenes such as tour, navigation, explanation, reconstruction, virtual effect overlay display and other interactive scenes related to real scenes or objects
  • special effects processing related to people such as makeup beautification, body beautification , special effects display, virtual model display and other interactive scenes.
  • the relevant features, states and attributes of the target object can be detected or identified through the convolutional neural network.
  • the above-mentioned convolutional neural network is a network model obtained by performing model training based on a deep learning framework.
  • the disclosed methods and devices may be implemented in other ways.
  • the device implementations described above are only illustrative.
  • the division of modules or units is only a logical function division. In actual implementation, there may be other division methods.
  • units or components can be combined or integrated. to another system, or some features may be ignored, or not implemented.
  • the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces, and the indirect coupling or communication connection of devices or units may be in electrical, mechanical or other forms.
  • a unit described as a separate component may or may not be physically separated, and a component shown as a unit may or may not be a physical unit, that is, it may be located in one place, or may also be distributed to network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.
  • each functional unit in each embodiment of the present disclosure may be integrated into one processing unit, each unit may exist separately physically, or two or more units may be integrated into one unit.
  • the above-mentioned integrated units can be implemented in the form of hardware or in the form of software functional units.
  • the integrated unit is realized in the form of a software function unit and sold or used as an independent product, it can be stored in a computer-readable storage medium.
  • the technical solution of the present disclosure is essentially or part of the contribution to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium , including several instructions to make a computer device (which may be a personal computer, a server, or a network device, etc.) or a processor (processor) execute all or part of the steps of the methods in various embodiments of the present disclosure.
  • the aforementioned storage media include: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), magnetic disk or optical disc, etc., which can store program codes. .

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Computer Graphics (AREA)
  • Geometry (AREA)
  • Software Systems (AREA)
  • Image Generation (AREA)
  • Image Processing (AREA)

Abstract

The present disclosure provides a method and apparatus for model optimization, an electronic device, and a storage medium. The model optimization method comprises: acquiring a model to be optimized, the model to be optimized comprising at least one model patch, and the model patch consisting of model vertices; acquiring attribute information of each model patch and first normal information of each model patch, and acquiring first location information of each model vertex; on the basis of the attribute information of a model patch, optimizing first normal information of the model patch to obtain second normal information of the model patch; on the basis of the first normal information of a model patch and the second normal information of a model patch, optimizing the first position information of a vertex of the model to obtain second position information of the vertex of the model, and optimizing the model to be optimized. The described solution can improve model optimization quality and efficiency.

Description

模型优化方法及装置、电子设备、存储介质、计算机程序、计算机程序产品Model optimization method and device, electronic device, storage medium, computer program, computer program product
相关申请的交叉引用Cross References to Related Applications
本公开基于申请号为202110988927.0、申请日为2021年08月26日、申请名称为“模型优化方法及相关装置、电子设备和存储介质”中国专利申请提出，并要求该中国专利申请的优先权，该中国专利申请的全部内容在此引入本公开作为参考。This disclosure is based on the Chinese patent application with the application number 202110988927.0, the application date is August 26, 2021, and the application name is "Model Optimization Method and Related Devices, Electronic Equipment and Storage Media", and claims the priority of the Chinese patent application. The entire content of this Chinese patent application is hereby incorporated by reference into this disclosure.
技术领域technical field
本公开涉及计算机视觉技术领域，特别是涉及一种模型优化方法及装置、电子设备、存储介质、计算机程序、计算机程序产品。The present disclosure relates to the technical field of computer vision, and in particular to a model optimization method and device, electronic equipment, storage media, computer programs, and computer program products.
背景技术Background technique
在三维重建应用中，由于重建算法等软件层面原因、或传感器精度等硬件层面原因，最终生成的重建模型往往缺失大量几何细节。因此，有必要对重建模型进行细节优化，以提升模型质量。In 3D reconstruction applications, due to software-level reasons such as reconstruction algorithms, or hardware-level reasons such as sensor accuracy, the final reconstruction model often lacks a lot of geometric details. Therefore, it is necessary to optimize the details of the reconstructed model to improve the quality of the model.
目前，现有细节优化方式通常选择优化截断符号距离场(Truncated Signed Distance Function，TSDF)等三维模型的间接表示形式，求解过程往往较为复杂，且优化的结果需要经过额外的构网算法才能体现到最终的三维模型上，不仅引入额外的算法开销，同时构网算法本身可能会再次丢失一部分细节。有鉴于此，如何提升模型优化质量和效率成为亟待解决的问题。At present, the existing detail optimization methods usually choose to optimize the indirect representation of 3D models such as Truncated Signed Distance Function (TSDF), the solution process is often complicated, and the optimized results need to go through additional network construction algorithms to reflect On the final 3D model, not only additional algorithm overhead is introduced, but also the network construction algorithm itself may lose some details again. In view of this, how to improve the quality and efficiency of model optimization has become an urgent problem to be solved.
发明内容Contents of the invention
本公开提供一种模型优化方法及装置、电子设备、存储介质、计算机程序、计算机程序产品。The disclosure provides a model optimization method and device, electronic equipment, storage media, computer programs, and computer program products.
本公开实施例提供了一种模型优化方法，包括：获取待优化模型；其中，待优化模型包括至少一个模型面片，模型面片由模型顶点构成；获取各个模型面片的属性信息和各个模型面片的第一法向信息，并获取各个模型顶点的第一位置信息；基于模型面片的属性信息优化模型面片的第一法向信息，得到模型面片的第二法向信息；基于模型面片的第一法向信息和模型面片的第二法向信息优化模型顶点的第一位置信息，得到模型顶点的第二位置信息，对待优化模型进行优化。An embodiment of the present disclosure provides a model optimization method, including: obtaining a model to be optimized; wherein, the model to be optimized includes at least one model face, and the model face is composed of model vertices; obtaining attribute information of each model face and each model The first normal information of the model patch, and obtain the first position information of each model vertex; optimize the first normal information of the model patch based on the attribute information of the model patch, and obtain the second normal information of the model patch; based on The first normal information of the model patch and the second normal information of the model patch optimize the first position information of the model vertices, obtain the second position information of the model vertices, and optimize the model to be optimized.
本公开实施例提供了一种模型优化装置，包括：模型获取模块、信息获取模块、法向优化模块和顶点优化模块；模型获取模块，配置为获取待优化模型；其中，待优化模型包括至少一个模型面片，模型面片由模型顶点构成；信息获取模块，配置为获取各个模型面片的属性信息和各个模型面片的第一法向信息，并获取各个模型顶点的第一位置信息；法向优化模块，配置为基于模型面片的属性信息优化模型面片的第一法向信息，得到模型面片的第二法向信息；顶点优化模块，配置为基于模型面片的第一法向信息模型面片的和第二法向信息优化模型顶点的第一位置信息，得到模型顶点的第二位置信息，对待优化模型进行优化。An embodiment of the present disclosure provides a model optimization device, including: a model acquisition module, an information acquisition module, a normal optimization module, and a vertex optimization module; a model acquisition module configured to acquire a model to be optimized; wherein the model to be optimized includes at least one The model face is composed of model vertices; the information acquisition module is configured to obtain the attribute information of each model face and the first normal information of each model face, and obtain the first position information of each model vertex; The direction optimization module is configured to optimize the first normal information of the model surface based on the attribute information of the model surface to obtain the second normal information of the model surface; the vertex optimization module is configured to be based on the first normal direction of the model surface The first position information of the vertices of the model is optimized with the information of the information model surface and the second normal direction information, and the second position information of the vertices of the model is obtained, and the model to be optimized is optimized.
本公开实施例提供了一种电子设备，包括，相互耦接的存储器和处理器，处理器配置为执行存储器中存储的程序指令，以实现上述模型优化方法的部分或全部步骤。An embodiment of the present disclosure provides an electronic device, including a memory and a processor coupled to each other, and the processor is configured to execute program instructions stored in the memory, so as to implement some or all steps of the above model optimization method.
本公开实施例提供了一种计算机可读存储介质，其上存储有程序指令，程序指令被处理器执行时实现上述模型优化方法的部分或全部步骤。An embodiment of the present disclosure provides a computer-readable storage medium, on which program instructions are stored, and when the program instructions are executed by a processor, some or all steps of the above-mentioned model optimization method are implemented.
本公开实施例提供一种计算机程序，所述计算机程序包括计算机可读代码，在所述计算机可读代码被计算机读取并执行的情况下，实现本公开任一实施例中的方法的部分或全部步骤。An embodiment of the present disclosure provides a computer program, the computer program includes computer readable code, and when the computer readable code is read and executed by a computer, a part or part of the method in any embodiment of the present disclosure is realized. All steps.
本公开实施例提供一种计算机程序产品，所述计算机程序产品包括存储了计算机程序的非瞬时性计算机可读存储介质，所述计算机程序被计算机读取并执行时，实现本公开任一实施例中的方法的部分或全部步骤。An embodiment of the present disclosure provides a computer program product, the computer program product includes a non-transitory computer-readable storage medium storing a computer program, and when the computer program is read and executed by a computer, any embodiment of the present disclosure is realized Some or all of the steps in the method.
在本公开实施例中，可以获取待优化模型，且待优化模型包括至少一个模型面片，模型面片由模型顶点构成，并获取各个模型面片的属性信息和各个模型面片的第一法向信息，以及获取各个模型顶点的第一位置信息，从而基于模型面片的属性信息优化模型面片的第一法向信息，得到模型面片的第二法向信息，并基于模型面片的第一法向信息和模型面片的第二法向信息优化模型顶点的第一位置信息，得到模型顶点的第二位置信息，对待优化模型进行优化。一方面，由于通过优化前的法向信息和优化后的法向信息直接对模型进行优化，而无需借助截断符号距离场等三维模型的间接表示形式来实现模型的优化，故而能够大大减少算法开销，有利于提升模型优化的效率。另一方面，由于在优化过程中考虑到了各个模型面片和各个模型顶点的连接关系，故有利于提升模型优化的质量。因此，本公开能够提高模型优化的质量和效率。In the embodiment of the present disclosure, the model to be optimized can be obtained, and the model to be optimized includes at least one model facet, the model facet is composed of model vertices, and the attribute information of each model facet and the first method of each model facet are obtained. direction information, and obtain the first position information of each model vertex, so as to optimize the first normal information of the model surface based on the attribute information of the model surface, obtain the second normal information of the model surface, and based on the model surface The first normal information and the second normal information of the model patch optimize the first position information of the vertices of the model, obtain the second position information of the vertices of the model, and optimize the model to be optimized. On the one hand, since the model is optimized directly through the normal information before optimization and the normal information after optimization, the model optimization can be realized without the help of indirect representations of 3D models such as truncated signed distance fields, so the algorithm overhead can be greatly reduced , which is conducive to improving the efficiency of model optimization. On the other hand, since the connection relationship between each model patch and each model vertex is considered in the optimization process, it is beneficial to improve the quality of model optimization. Therefore, the present disclosure can improve the quality and efficiency of model optimization.
附图说明Description of drawings
图1为本公开实施例提供的一种模型优化方法的流程示意图一；FIG. 1 is a first schematic flow diagram of a model optimization method provided by an embodiment of the present disclosure;
图2为本公开实施例提供的一种待优化模型的示意图；FIG. 2 is a schematic diagram of a model to be optimized provided by an embodiment of the present disclosure;
图3为本公开实施例提供的一种模型面片和模型顶点的示意图；FIG. 3 is a schematic diagram of a model surface and model vertices provided by an embodiment of the present disclosure;
图4为本公开实施例提供的一种模型优化方法的流程示意图二；FIG. 4 is a schematic flow diagram II of a model optimization method provided by an embodiment of the present disclosure;
图5a为本公开实施例提供的一种待细分面片及其相邻面片的示意图一；Fig. 5a is a first schematic diagram of a patch to be subdivided and its adjacent patches provided by an embodiment of the present disclosure;
图5b为本公开实施例提供的一种待细分面片及其相邻面片的示意图二；Fig. 5b is a second schematic diagram of a patch to be subdivided and its adjacent patches provided by an embodiment of the present disclosure;
图5c为本公开实施例提供的一种待细分面片及其相邻面片的示意图三；Fig. 5c is a third schematic diagram of a patch to be subdivided and its adjacent patches provided by an embodiment of the present disclosure;
图6为本公开实施例提供的一种待优化模型和优化之后的模型的示意图；FIG. 6 is a schematic diagram of a model to be optimized and a model after optimization provided by an embodiment of the present disclosure;
图7为本公开实施例提供的一种模型优化装置的框架示意图；FIG. 7 is a schematic framework diagram of a model optimization device provided by an embodiment of the present disclosure;
图8为本公开实施例提供的一种电子设备的框架示意图；FIG. 8 is a schematic frame diagram of an electronic device provided by an embodiment of the present disclosure;
图9为本公开实施例提供的一种计算机可读存储介质的框架示意图。FIG. 9 is a schematic framework diagram of a computer-readable storage medium provided by an embodiment of the present disclosure.
具体实施方式Detailed ways
下面结合说明书附图，对本公开实施例的方案进行详细说明。The solutions of the embodiments of the present disclosure will be described in detail below in conjunction with the accompanying drawings.
以下描述中，为了说明而不是为了限定，提出了诸如特定***结构、接口、技术之类的具体细节，以便透彻理解本公开。In the following description, for purposes of illustration rather than limitation, specific details, such as specific system architectures, interfaces, techniques, are set forth in order to provide a thorough understanding of the present disclosure.
本文中术语“***”和“网络”在本文中常被可互换使用。本文中术语“和/或”，仅仅是一种描述关联对象的关联关系，表示可以存在三种关系，例如，A和/或B，可以表示：单独存在A，同时存在A和B，单独存在B这三种情况。另外，本文中字符“/”，一般表示前后关联对象是一种“或”的关系。此外，本文中的“多”表示两个或者多于两个。The terms "system" and "network" are often used interchangeably herein. The term "and/or" in this article is just an association relationship describing associated objects, which means that there can be three relationships, for example, A and/or B can mean: A exists alone, A and B exist simultaneously, and there exists alone B these three situations. In addition, the character "/" in this article generally indicates that the contextual objects are an "or" relationship. In addition, "many" herein means two or more than two.
请参阅图1，图1为本公开实施例提供的一种模型优化方法的流程示意图一。该模 型优化方法可以由终端设备或其它处理设备执行，其中，终端设备可以为用户设备(User Equipment，UE)、移动设备、用户终端、终端、蜂窝电话、无绳电话、个人数字处理(Personal Digital Assistant，PDA)、手持设备、计算设备、车载设备、可穿戴设备等。Please refer to FIG. 1 . FIG. 1 is a first schematic flowchart of a model optimization method provided by an embodiment of the present disclosure. The model optimization method can be executed by terminal equipment or other processing equipment, wherein the terminal equipment can be user equipment (User Equipment, UE), mobile equipment, user terminal, terminal, cellular phone, cordless phone, personal digital assistant (Personal Digital Assistant) , PDA), handheld devices, computing devices, vehicle-mounted devices, wearable devices, etc.
如图1所示，在一些实施例中，本公开提供的模型优化方法可以包括如下步骤：As shown in Figure 1, in some embodiments, the model optimization method provided by the present disclosure may include the following steps:
步骤S11：获取待优化模型。Step S11: Obtain the model to be optimized.
在本公开实施例中，待优化模型可以包括至少一个模型面片，模型面片可以由模型顶点构成。In the embodiment of the present disclosure, the model to be optimized may include at least one model facet, and the model facet may be composed of model vertices.
请结合参阅图2，图2为本公开实施例提供的一种待优化模型的示意图。如图2所示，待优化模型可以包含至少一个模型面片21，模型面片21可以包括但不限于三角形，模型面片21由模型顶点22构成，例如，在模型面片21为三角形时，模型面片由三个模型顶点22连接而构成。需要说明的是，图2仅仅是示例性地给出了实际应用时一种可能的待优化模型，待优化模型可以是对桌子、椅子等单个物体进行模型重建而得到的，或者，也可以是对沙盘、客厅等包含多个物体的场景进行模型重建而得到的，在此不对待优化模型的具体形式做限定。为了便于在模型优化过程中进行计算，在一些实施例中，待优化模型可以位于世界坐标系中。此外，在一些实施例中，如无特别说明，位置信息、位置坐标等表述可以指在世界坐标系中的位置信息、位置坐标。Please refer to FIG. 2 in conjunction with FIG. 2 , which is a schematic diagram of a model to be optimized provided by an embodiment of the present disclosure. As shown in Figure 2, the model to be optimized can include at least one model facet 21, and the model facet 21 can include but not limited to a triangle, and the model facet 21 is made up of model vertices 22, for example, when the model facet 21 is a triangle, A model patch is formed by connecting three model vertices 22 . It should be noted that Figure 2 is only an example of a possible model to be optimized in practical applications. The model to be optimized can be obtained by reconstructing the model of a single object such as a table or a chair, or it can be It is obtained by reconstructing the model of a scene containing multiple objects such as a sand table and a living room, and the specific form of the optimized model is not limited here. In order to facilitate calculation during model optimization, in some embodiments, the model to be optimized may be located in the world coordinate system. In addition, in some embodiments, unless otherwise specified, expressions such as position information and position coordinates may refer to position information and position coordinates in the world coordinate system.
在一些实施例中，对于集成有RGB相机和深度相机的设备而言，可以通过该设备采集目标对象的多个关键帧数据，且关键帧数据包含RGB图像和深度图像，并使用最近邻迭代算法(Iterative Closest Point，ICP)、光流法、惯性传感器(Inertial Measurement Unit，IMU)中的一种或多种组合，求解得到相机位姿，其中，关键帧，是一种计算机动画术语，指角色或者物体运动变化中关键动作所处的那一帧，相当于二维动画中的原画。在此基础上，可以结合相机位姿将目标对象的深度图像和参考体素模型进行融合，得到目标对象的重建体素模型，其中，体素模型是指采用大量规则体素(如立方体)的有序组合来表示三维物体的模型，且重建体素模型中的体素可以包含体素信息，体素信息可以包括截断符号距离。在一些实施例中，可以在获取到新的深度图像的情况下，将重建体素模型作为新的参考体素模型，并重新执行上述将目标对象的深度图像和参考体素模型进行融合，得到目标对象的重建体素模型的步骤，直至结束采集目标对象的多个关键帧数据为止。在一些实施例中，可以采用诸如移动立方体算法(Marching Cubes，MC)等方式对重建体素模型进行处理，得到三维网格，作为待优化模型。In some embodiments, for a device integrated with an RGB camera and a depth camera, multiple key frame data of the target object can be collected through the device, and the key frame data includes RGB images and depth images, and the nearest neighbor iterative algorithm is used One or more combinations of (Iterative Closest Point, ICP), optical flow method, and inertial sensor (Inertial Measurement Unit, IMU) are solved to obtain the camera pose. Among them, key frame is a computer animation term that refers to the character Or the frame where the key action in the movement of the object is located, which is equivalent to the original painting in the two-dimensional animation. On this basis, the depth image of the target object and the reference voxel model can be fused in combination with the camera pose to obtain the reconstructed voxel model of the target object. The model of the three-dimensional object is represented by orderly combination, and the voxels in the reconstructed voxel model may contain voxel information, and the voxel information may include truncated sign distances. In some embodiments, when a new depth image is obtained, the reconstructed voxel model can be used as a new reference voxel model, and the above-mentioned fusion of the depth image of the target object and the reference voxel model can be performed again to obtain The step of reconstructing the voxel model of the target object until the collection of multiple key frame data of the target object is completed. In some embodiments, the reconstructed voxel model can be processed by means such as Marching Cubes (MC) to obtain a three-dimensional grid as the model to be optimized.
在一些实施场例中，对于仅集成RGB相机的设备而言，可以通过该设备采集到目标对象的多个关键帧数据，且关键帧数据包含RGB图像，并使用运动恢复结构(Structure from Motion，SFM)，求解得到相机内参和相机位姿，然后通过多视图匹配算法(Multi-view Stereo，MVS)恢复图像深度信息，最后通过狄洛尼(Delaunay)算法生成目标对象的三维网格，作为待优化模型。In some implementation examples, for a device that only integrates an RGB camera, multiple key frame data of the target object can be collected through the device, and the key frame data includes RGB images, and the motion recovery structure (Structure from Motion, SFM), solve the internal parameters of the camera and the camera pose, then restore the image depth information through the multi-view matching algorithm (Multi-view Stereo, MVS), and finally generate the 3D mesh of the target object through the Delaunay algorithm, as the Optimize the model.
在一些实施例中，在得到待优化模型之后，还可以对待优化模型进行纹理贴图，从而使得待优化模型中的模型面片具有图像信息(如，色彩、明暗等)，再将进行纹理贴图之后的模型，作为后续优化的待优化模型。In some embodiments, after the model to be optimized is obtained, texture mapping can also be performed on the model to be optimized, so that the model patches in the model to be optimized have image information (such as color, light and shade, etc.), and then after texture mapping The model of is used as the model to be optimized for subsequent optimization.
在一些实施例中，本公开提供的模型优化方法，可以应用于计算机人工三维建模场景中，可以将计算机人工三维建模所得到的模型作为待优化模型，该模型的模型面片可以较为稀疏(即低模)。例如，在对模型进行纹理贴图之后，为了丰富细节，可以对进行纹理贴图之后的模型进行优化，以得到模型面片更加稠密、结构更为精细的模型(即高模)。In some embodiments, the model optimization method provided by the present disclosure can be applied to computer artificial three-dimensional modeling scenarios, and the model obtained by computer artificial three-dimensional modeling can be used as the model to be optimized, and the model patches of the model can be relatively sparse (i.e. low poly). For example, after texture mapping is performed on the model, in order to enrich the details, the model after texture mapping can be optimized to obtain a model with denser model patches and finer structure (ie high model).
在一些实施例中，为了便于描述，待优化模型可以记为M＝{T,V}，其中，T＝{t 0,t 1,…}表示模型面片，V＝{v 0,v 1,…}表示模型顶点。需要说明的是，如前所述， 待优化模型可以是在至少一个帧图像基础上重建得到的，为了便于描述，上述至少一个帧图像可以记为F＝{F 0,F 1,…,F n-1}，其中每帧图像F j∈F可以包含一帧深度图像和一帧可见光图像G j(如，RGB图像)。如前所述，深度图像可以用于，在采用集成RGB相机和深度相机的设备进行模型重建过程中，融入深度信息。此外，需要说明的是，每帧图像可以具有两个参数，分别为相机位姿P j和相机内参K j，故每帧图像F j可以描述为F j＝{G j,P j,K j}。 In some embodiments, for the convenience of description, the model to be optimized can be recorded as M={T,V}, where T={t 0 ,t 1 ,…} represents a model patch, and V={v 0 ,v 1 ,…} represent model vertices. It should be noted that, as mentioned above, the model to be optimized can be reconstructed on the basis of at least one frame image. For the convenience of description, the above-mentioned at least one frame image can be recorded as F={F 0 , F 1 ,...,F n-1 }, where each frame of image F j ∈ F may include a frame of depth image and a frame of visible light image G j (eg, RGB image). As mentioned earlier, depth images can be used to incorporate depth information during model reconstruction using devices integrating RGB cameras and depth cameras. In addition, it should be noted that each frame of image can have two parameters, which are camera pose P j and camera intrinsic parameter K j , so each frame of image F j can be described as F j ={G j ,P j ,K j }.
步骤S12：获取各个模型面片的属性信息和各个模型面片的第一法向信息，并获取各个模型顶点的第一位置信息。Step S12: Obtain attribute information of each model patch and first normal information of each model patch, and acquire first position information of each model vertex.
在一些实施场例中，模型面片t的第一法向信息可以采用法向量来表示，则对于模型面片t∈T而言，可以在模型面片t上任取三个点，为了便于描述，可以记为p1、p2和p3，则可以根据p1和p2的位置坐标，得到连接p1和p2的第一向量，根据p1和p3的位置坐标，得到连接p1和p3的第二向量，并根据p2和p3的位置坐标，得到连接p2和p3的第三向量。由于模型面片t的法向量与第一向量、第二向量、第三向量均正交(即法向量与第一向量、第二向量、第三向量中任一者相乘结果为0)，则根据该关系，可以建立关于模型面片t的法向量的方程组，并求解该方程组得到模型面片t的法向量。此外，也可以基于向量外积，快速求解模型面片t的法向量，即模型面片的法向量
Figure PCTCN2021144033-appb-000001
Figure PCTCN2021144033-appb-000002
其中，
Figure PCTCN2021144033-appb-000003
表示连接p1和p2的第一向量，
Figure PCTCN2021144033-appb-000004
表示连接p2和p3的第三向量，‖.‖表示向量取模运算。 In some embodiments, the first normal information of the model facet t can be represented by a normal vector, then for the model facet t∈T, three points can be randomly selected on the model facet t, for the convenience of description , which can be recorded as p1, p2 and p3, then the first vector connecting p1 and p2 can be obtained according to the position coordinates of p1 and p2, and the second vector connecting p1 and p3 can be obtained according to the position coordinates of p1 and p3, and according to The position coordinates of p2 and p3, resulting in a third vector connecting p2 and p3. Since the normal vector of the model facet t is orthogonal to the first vector, the second vector, and the third vector (that is, the result of multiplying the normal vector by any of the first vector, the second vector, and the third vector is 0), Then, according to this relationship, a system of equations about the normal vector of the model facet t can be established, and the normal vector of the model facet t can be obtained by solving the system of equations. In addition, it is also possible to quickly solve the normal vector of the model facet t based on the vector outer product, that is, the normal vector of the model facet
Figure PCTCN2021144033-appb-000001
Figure PCTCN2021144033-appb-000002
in,
Figure PCTCN2021144033-appb-000003
represents the first vector connecting p1 and p2,
Figure PCTCN2021144033-appb-000004
Indicates the third vector connecting p2 and p3, ‖.‖ indicates vector modulo operation.
在一些实施例中，模型面片的属性信息可以包括但不限于：模型面片的亮度信息、色相信息、光度信息和漫反射信息。为了便于描述，模型面片t∈T的亮度信息可以表示为I(t)，模型面片的色相信息可以表示为C(t)，模型面片的光度信息可以表示为B(t)，模型面片的漫反射信息可以表示为A(t)。In some embodiments, the attribute information of the model patch may include but not limited to: brightness information, hue information, photometric information and diffuse reflection information of the model patch. For the convenience of description, the luminance information of the model patch t ∈ T can be expressed as I(t), the hue information of the model patch can be expressed as C(t), the luminosity information of the model patch can be expressed as B(t), and the model The diffuse reflection information of the patch can be expressed as A(t).
在一些实施例中，对于待优化模型中的第i个模型面片t i∈T而言，可以使用相机位姿P j和相机内参K j将待优化模型重投影至第j帧图像F j的图像坐标系中，并基于深度缓冲算法进行遮挡判断。然后，可以根据模型面片t i在可见光图像G j上覆盖的像素颜色均值计算出该模型面片t i对应于可见光图像G j的亮度信息I j(t i)和色相信息C j(t i)。其他模型面片的亮度信息和色相信息可以以此类推，在此不再一一举例。 In some embodiments, for the i-th model patch t i ∈ T in the model to be optimized, the camera pose P j and camera internal parameters K j can be used to reproject the model to be optimized to the j-th frame image F j In the image coordinate system, and based on the depth buffer algorithm for occlusion judgment. Then, the luminance information I j (t i ) and hue information C j ( t i ). The brightness information and hue information of other model patches can be deduced by analogy, so we won't give examples one by one here.
在一些实施例中，在待优化模型经过纹理贴图的情况下，可以直接由待优化模型中的模型面片的纹理坐标和纹理图像，得到模型面片的亮度信息和色相信息。In some embodiments, when the model to be optimized has undergone texture mapping, the brightness information and hue information of the model patch can be obtained directly from the texture coordinates and texture images of the model patch in the model to be optimized.
在一些实施例中，在检测到模型面片未对应任何图像区域的情况下，可以在后续处理过程中对该模型面片进行跳过处理。需要说明的是，下述亮度信息I j(t i)和色相信息C j(t i)均表示，模型面片t i具有有效的亮度信息和色相信息。 In some embodiments, when it is detected that the model patch does not correspond to any image region, the model patch may be skipped during subsequent processing. It should be noted that the following luminance information I j (t i ) and hue information C j (t i ) both indicate that the model patch t i has valid luminance information and hue information.
在一些实施例中，可以假定目标对象表面粗糙，目标对象所在环境的光照以全局光(即自然光)为主，在此基础上，可以估算环境光照系数I。在一些实施例中，可以采用三阶球谐光照模型描述环境光，并根据球谐光照基函数H m(n)、环境光照系数I和模型面片t∈T的第一法向信息n(t)、漫反射信息A(t)，得到光度信息B(t)，光度信息B(t)的计算公式可以表示为： In some embodiments, it may be assumed that the surface of the target object is rough, and the illumination of the environment where the target object is located is dominated by global illumination (ie, natural light). On this basis, the environmental illumination coefficient I may be estimated. In some embodiments, the third-order spherical harmonic illumination model can be used to describe the ambient light , and the first normal information n( t), diffuse reflection information A(t), to obtain photometric information B(t), the calculation formula of photometric information B(t) can be expressed as:
Figure PCTCN2021144033-appb-000005
Figure PCTCN2021144033-appb-000005
上述公式(1)中，t表示模型面片，n(t)表示模型面片t的第一法向信息，B(t)表示模型面片t的光度信息，A(t)表示模型面片t的漫反射信息，l m和I＝{l 0,l 1,…,l 8}均表示模型面片t的环境光照系数，H m(n)表示球谐光照基函数。 In the above formula (1), t represents the model facet, n(t) represents the first normal information of the model facet t, B(t) represents the photometric information of the model facet t, and A(t) represents the model facet The diffuse reflection information of t, l m and I={l 0 ,l 1 ,…,l 8 } both represent the ambient illumination coefficient of the model patch t, and H m (n) represents the spherical harmonic illumination basis function.
在一些实施例中，球谐光照基函数H m(n)可以表示为： In some embodiments, the spherical harmonic illumination basis function H m (n) can be expressed as:
H 0(n)＝1.0          (2)； H 0 (n) = 1.0 (2);
H 1(n)＝n y          (3)； H 1 (n) = n y (3);
H 2(n)＝n z                       (4)； H 2 (n)= nz (4);
H 3(n)＝n x                       (5)； H 3 (n)=n x (5);
H 4(n)＝n xn y                     (6)； H 4 (n) = n x n y (6);
H 5(n)＝n yn z                     (7)； H 5 (n) = n y n z (7);
H 6(n)＝-n xn x-n yn y+2n zn z     (8)； H 6 (n) = -n x n x -n y n y +2n z n z (8);
H 7(n)＝n zn x                     (9)； H 7 (n)= nz n x (9);
H 8(n)＝n xn x-n yn y              (10)； H 8 (n) = n x n x - n y n y (10);
上述公式(2)至(10)中，模型面片t的第一法向信息n(t)可以由法向量进行表示，且n x,n y,n z可以分别表示第一法向信息n(t)在x轴方向上的分量、y轴方向上的分量、z轴方向上的分量，n表示模型面片的第一法向信息。在此基础上，可以根据光度信息B(t)和模型面片t对应于各个可见光图像G j的亮度信息I j(t)，构建得到关于环境光照系数I的优化函数，并最小化求解该优化函数，得到环境光照系数I，环境光照系数I的计算公式可以表示为： In the above formulas (2) to (10), the first normal information n(t) of the model patch t can be represented by a normal vector, and n x , n y , n z can respectively represent the first normal information n (t) a component in the direction of the x axis, a component in the direction of the y axis, and a component in the direction of the z axis, and n represents the first normal information of the model patch. On this basis, according to the photometric information B(t) and the luminance information I j (t) of the model patch t corresponding to each visible light image G j , an optimization function for the environmental illumination coefficient I can be constructed, and the solution to this problem can be minimized Optimize the function to obtain the environmental illumination coefficient I, and the calculation formula of the environmental illumination coefficient I can be expressed as:
Figure PCTCN2021144033-appb-000006
Figure PCTCN2021144033-appb-000006
上述公式(11)中I表示模型面片t的环境光照系数，j表示用于投影待优化模型的图像F j的帧序号，可以将待优化模型重投影至第j帧图像F j的图像坐标系中，B(t)表示模型面片t的光度信息，I j(t)表示模型面片t对应于各个可见光图像G j的亮度信息。 In the above formula (11), I represents the environmental illumination coefficient of the model patch t, and j represents the frame number used to project the image F j of the model to be optimized, and the model to be optimized can be reprojected to the image coordinates of the jth frame image F j In the system, B(t) represents the photometric information of the model patch t, and I j (t) represents the brightness information of the model patch t corresponding to each visible light image G j .
通过上述公式(11)进行最小化求解，得到环境光照系数I，能够使得推导出的光度信息更加贴近于模型面片的亮度信息。此外，在首次执行对环境光照系数的求解过程中，可以将漫反射信息A(t)进行初始化，如可以初始化为1、0.5或其他值，在此不对漫反射信息A(t)的初始化值做限定。需要说明的是，除了采用三阶球谐光照模型描述环境光，也可以采用其他形式的光照模型，如可以采用更高阶的球谐光照模型，在此不对描述环境光的光照模型进行限定。在得到环境光照系数I之后，即可将环境光照系数I代入公式(1)，得到模型面片t的光度信息B(t)。The ambient illumination coefficient I is obtained by minimizing the above formula (11), which can make the deduced luminosity information closer to the luminance information of the model patch. In addition, in the process of solving the environmental illumination coefficient for the first time, the diffuse reflection information A(t) can be initialized, for example, it can be initialized to 1, 0.5 or other values, and the initialization value of the diffuse reflection information A(t) is not here Do limited. It should be noted that, in addition to using the third-order spherical harmonic illumination model to describe ambient light, other forms of illumination models can also be used, such as a higher-order spherical harmonic illumination model, and the illumination model for describing ambient light is not limited here. After obtaining the environmental illumination coefficient I, the environmental illumination coefficient I can be substituted into formula (1) to obtain the photometric information B(t) of the model patch t.
步骤S13：基于模型面片的属性信息优化模型面片的第一法向信息，得到模型面片的第二法向信息。Step S13: Optimizing the first normal information of the model surface based on the attribute information of the model surface to obtain the second normal information of the model surface.
在一些实施例中，可以根据模型面片的属性信息和模型面片的第一法向信息，构建关于模型面片的第二法向信息的代价函数，并对代价函数进行迭代优化，得到模型面片的第二法向信息。代价函数可以包括第一代价子项、第二代价子项、第三代价子项和***价子项。其中，第一代价子项可以用于表示模型面片的属性信息中的光度信息和亮度信息的梯度残差，第二代价子项可以用于表示模型面片的第一法向信息和第二法向信息之间的偏差，第三代价子项可以用于平滑模型面片的第二法向信息，***价子项可以用于缓解纹理复制问题。In some embodiments, according to the attribute information of the model patch and the first normal information of the model patch, a cost function about the second normal information of the model patch can be constructed, and the cost function can be iteratively optimized to obtain the model The second normal information of the patch. The cost function may include a first cost sub-term, a second cost sub-term, a third cost sub-term and a fourth cost sub-term. Among them, the first cost sub-item can be used to represent the gradient residual of photometric information and brightness information in the attribute information of the model patch, and the second cost sub-item can be used to represent the first normal information and the second The deviation between the normal information, the third cost sub-item can be used to smooth the second normal information of the model patch, and the fourth cost sub-item can be used to alleviate the texture copy problem.
在一些实施例中，对于模型面片t∈T而言，可以求取模型面片t的光度信息B(t)的梯度值，并求取该模型面片t对应于各帧图像中的亮度信息I j(t)的梯度值，并求取模型面片的光度信息B(t)的梯度值，和模型面片t对应于各帧图像中的亮度信息I j(t)的梯度值之间的残差值，在此基础上，再基于该模型面片t对应于各帧图像的残差值计算残差平方和，得到第一代价子项E g(t)。第一代价子项E g(t)的计算公式可以表示为： In some embodiments, for the model patch t∈T, the gradient value of the photometric information B(t) of the model patch t can be calculated, and the brightness of the model patch t corresponding to each frame image can be calculated The gradient value of the information I j (t), and calculate the gradient value of the photometric information B(t) of the model patch, and the gradient value of the model patch t corresponding to the brightness information I j (t) in each frame image On this basis, the sum of squared residuals is calculated based on the residual values of the model patch t corresponding to each frame image, and the first cost sub-item E g (t) is obtained. The calculation formula of the first cost sub-item E g (t) can be expressed as:
Figure PCTCN2021144033-appb-000007
Figure PCTCN2021144033-appb-000007
在上述公式(12)中，E g(t)表示第一代价子项，
Figure PCTCN2021144033-appb-000008
可以表示梯度算子。通过约束第一代价子项E g(t)，能够使得光度信息的梯度更加贴近于亮度信息的梯度。需要说明的是，对于模型面片t∈T而言，也可以直接求取该模型面片t对应于各帧图像中亮度信息I j(t)与光度信息B(t)之间的残差值。在此基础上，再基于该模型面片t对应于各帧图像的 残差值计算残差平方和，得到第一代价子项E g(t)，在此情况下，可以适应性地对上述公式(12)修改，得到第一代价子项
Figure PCTCN2021144033-appb-000009
在实际应用过程中，可以根据上述两种方式中任一者求取第一代价子项E g(t)，即，通过求取模型面片的光度信息B(t)的梯度值，和模型面片t对应于各帧图像中的亮度信息I j(t)的梯度值之间的残差平方和，或者直接通过求取模型面片t对应于各帧图像中亮度信息I j(t)与光度信息B(t)之间的残差平方和，在此不做限定。需要说明的是，通过模型面片的光度信息B(t)的梯度值，和模型面片t对应于各帧图像中的亮度信息I j(t)的梯度值来计算第一代价子项E g(t)具有更优的稳定性。 In the above formula (12), E g (t) represents the first cost sub-item,
Figure PCTCN2021144033-appb-000008
Can represent gradient operators. By constraining the first cost sub-item E g (t), the gradient of the photometric information can be made closer to the gradient of the brightness information. It should be noted that, for the model patch t∈T, it is also possible to directly calculate the residual difference between the brightness information I j (t) and the photometric information B(t) corresponding to the model patch t in each frame image value. On this basis, calculate the sum of residual squares based on the residual values of the model patch t corresponding to each frame image, and obtain the first cost sub-item E g (t). In this case, the above-mentioned Formula (12) is modified to obtain the first cost sub-item
Figure PCTCN2021144033-appb-000009
In the actual application process, the first cost sub-item E g (t) can be obtained according to any of the above two methods, that is, by obtaining the gradient value of the photometric information B(t) of the model patch, and the model The patch t corresponds to the residual sum of squares between the gradient values of the brightness information I j (t) in each frame image, or directly calculates the model patch t corresponding to the brightness information I j (t) in each frame image The residual sum of squares with the photometric information B(t) is not limited here. It should be noted that the first cost sub-item E is calculated by the gradient value of the photometric information B(t) of the model patch and the gradient value of the model patch t corresponding to the brightness information I j (t) in each frame image g (t) has better stability.
在一些实施例中，对于模型面片t∈T而言，可以求取模型面片t的第一法向信息和模型面片t的第二法向信息之间的偏差，得到第二代价子项E s(t)。第二代价子项E s(t)的计算公式可以表示为： In some embodiments, for the model facet t∈T, the deviation between the first normal information of the model facet t and the second normal information of the model facet t can be calculated to obtain the second cost factor The term E s (t). The calculation formula of the second cost sub-item E s (t) can be expressed as:
E s(t)＝‖n(t)-n 0(t)‖ 2          (13)； E s (t) = ‖ n (t) - n 0 (t) ‖ 2 (13);
上述公式(13)中，E s(t)表示第二代价子项，n(t)表示第二法向信息，n 0(t)表示第一法向信息，‖.‖表示向量取模函数。通过约束该子项，能够使得第二法向信息接近于第一法向信息，而不至于过于偏离第一法向信息。 In the above formula (13), E s (t) represents the second cost sub-item, n(t) represents the second normal information, n 0 (t) represents the first normal information, ‖.‖ represents the vector modulo function . By constraining this sub-item, the second normal information can be made close to the first normal information without being too far away from the first normal information.
在一些实施例中，对于模型面片t∈T而言，可以获取与模型面片t相邻的相邻面片t′∈N(t)，需要说明的是，模型面片t的相邻面片与模型面片t具有共享边，即两者之间至少共享一条边。在此基础上，可以获取模型面片t的第二法向信息与相邻面片t′的第二法向信息之间的偏差，得到第三代价子项E r(t)。第三代价子项E r(t)的计算公式可以表示为： In some embodiments, for the model facet t∈T, the adjacent facet t′∈N(t) adjacent to the model facet t can be obtained. It should be noted that the adjacent facets of the model facet t The patch and the model patch t have shared edges, that is, at least one edge is shared between them. On this basis, the deviation between the second normal information of the model patch t and the second normal information of the adjacent patch t′ can be obtained to obtain the third cost sub-item E r (t). The calculation formula of the third cost sub-item E r (t) can be expressed as:
E r(t)＝∑ t′∈N(t)‖n(t)-n(t′)‖ 2        (14)； E r (t) = ∑ t′∈N(t) ‖n(t)-n(t′)‖ 2 (14);
上述公式(14)中，t′∈N(t)表示模型面片t的相邻面片，n(t)表示模型面片t的第二法向信息，n(t′)表示模型面片t的相邻面片t′的第二法向信息，‖.‖表示向量取模函数。通过约束该子项，能够使得模型面片t的第二法向信息接近于其相邻面片t′的第二法向信息，而不至于过于偏离其相邻面片t′的第二法向信息，从而能够使得模型面片t与相邻面片t′之间具有一定的平滑性。In the above formula (14), t′∈N(t) represents the adjacent facets of the model facet t, n(t) represents the second normal information of the model facet t, and n(t′) represents the model facet The second normal information of the adjacent facet t′ of t, ‖.‖ represents the vector modulo function. By constraining this sub-item, the second normal information of the model facet t can be made close to the second normal information of its adjacent facet t′, and will not deviate too much from the second normal information of its adjacent facet t′. Orientation information, so that there is a certain smoothness between the model patch t and the adjacent patch t′.
在一些实施例中，对于模型面片t∈T而言，可以获取模型面片t在各帧图像中的色相信息C j(t)及模型面片t的相邻面片t′在各帧图像中的色相信息C j(t′)，并根据两者之间的差异，得到色相权重。与此同时，可以获取模型面片t的漫反射信息A(t)及模型面片t的相邻面片t′的漫反射信息A(t′)之间的差异，再根据该差异与色相权重，构建***价子项E a(t)。***价子项E a(t)的计算公式可以表示为： In some embodiments, for the model patch t∈T, the hue information C j (t) of the model patch t in each frame image and the adjacent patch t′ of the model patch t in each frame can be obtained The hue information C j (t′) in the image, and according to the difference between the two, the hue weight is obtained. At the same time, the difference between the diffuse reflection information A(t) of the model patch t and the diffuse reflection information A(t′) of the adjacent patch t′ of the model patch t can be obtained, and then according to the difference and the hue Weight, to construct the fourth cost sub-item E a (t). The calculation formula of the fourth cost sub-item E a (t) can be expressed as:
Figure PCTCN2021144033-appb-000010
Figure PCTCN2021144033-appb-000010
上述公式(15)中，t′∈N(t)表示模型面片t的相邻面片，C j(t)表示模型面片t在各帧图像中的色相信息，C j(t′)表示模型面片t的相邻面片t′在各帧图像中的色相信息，A(t)表示模型面片t的漫反射信息，A(t′)表示模型面片t的相邻面片t′的漫反射信息；φ(c)＝1/(1+3‖c‖) 3，其中c表示色相信息C j(t)与色相信息C j(t′)之间的差异，φ(c)表示根据该差异计算得到的色相权重。由于模型面片t在各帧图像中的色相信息C j(t)和其相邻面片t′在各帧图像中的色相信息C j(t′)差异越大，色相权重越小，故通过约束该子项，能够区分模型面片本身的颜色以及光照所引起的纹理，从而能够有利于抑制纹理复制问题，提升模型优化质量。 In the above formula (15), t′∈N(t) represents the adjacent patches of the model patch t, C j (t) denotes the hue information of the model patch t in each frame image, and C j (t′) Represents the hue information of the adjacent facet t′ of the model facet t in each frame image, A(t) represents the diffuse reflection information of the model facet t, and A(t′) represents the adjacent facets of the model facet t Diffuse reflection information of t′; φ(c)=1/(1+3∥c‖) 3 , where c represents the difference between hue information C j (t) and hue information C j (t′), φ( c) represents the hue weight calculated according to the difference. Since the difference between the hue information C j (t) of the model patch t in each frame image and the hue information C j (t′) of its adjacent patch t′ in each frame image is greater, the hue weight is smaller, so By constraining this sub-item, the color of the model patch itself and the texture caused by lighting can be distinguished, which can help suppress the texture copy problem and improve the quality of model optimization.
在一些实施例中，在构建得到各个模型面片的第一代价子项、第二代价子项、第三代价子项和***价子项之后，即可以结合所有模型面片的第一代价子项、第二代价子项、第三代价子项和***价子项，以及第一代价子项、第二代价子项、第三代价子项 和***价子项各自的重要程度，构建得到代价函数。示例性地，第一代价子项的重要程度λ g可以设置为1，第二代价子项的重要程度λ s可以设置为0.005，第三代价子项的重要程度λ r可以设置为0.003，***价子项的重要程度λ a可以设置为2，在此不做限定。在一些实施例中，代价函数E可以表示为： In some embodiments, after constructing the first cost sub-item, the second cost sub-item, the third cost sub-item and the fourth cost sub-item of each model patch, the first cost of all model patches can be combined subitem, second cost subitem, third cost subitem, and fourth cost subitem, and the respective importance levels of first cost subitem, second cost subitem, third cost subitem, and fourth cost subitem, Build the cost function. Exemplarily, the importance degree λ g of the first cost sub-item can be set to 1, the importance degree λ s of the second cost sub-item can be set to 0.005, and the importance degree λ r of the third cost sub-item can be set to 0.003. The importance degree λ a of the four-cost subitem can be set to 2, which is not limited here. In some embodiments, the cost function E can be expressed as:
E＝∑ t∈T(λ gE g(t)+λ sE s(t)+λ rE r(t)+λ aE a(t))      (16)； E＝ ∑t∈T ( λgEg ( t )+ λsEs ( t )+ λrEr ( t ) + λaEa (t)) (16);
上述公式(16)中，E g(t)表示第一代价子项，λ g表示第一代价子项的重要程度，如权重值；E s(t)表示第二代价子项，λ s表示第二代价子项的重要程度，如权重值；E r(t)表示第三代价子项，λ r表示第三代价子项的重要程度，如权重值；E a(t)表示***价子项，λ a表示***价子项的重要程度，如权重值。 In the above formula (16), E g (t) represents the first cost sub-item, λ g represents the importance of the first cost sub-item, such as the weight value; E s (t) represents the second cost sub-item, and λ s represents The importance of the second cost sub-item, such as the weight value; E r (t) represents the third cost sub-item, λ r represents the importance of the third cost sub-item, such as the weight value; E a (t) represents the fourth cost Subitem, λ a represents the importance of the fourth cost subitem, such as weight value.
在一些实施例中，可以采用求解算法对代价函数进行求解，诸如L-BFGS(Limited-memory Broyden-Fletcher-Goldfarb-Shanno)、DFP(Davidon-Fletcher-Powell)等算法，以优化得到各个模型面片的第二法向信息以及漫反射信息。在一些实施例中，在求解过程中，利用可以迭代执行预设次数上述求解算法，预设次数可以根据实际应用需要进行设置。例如，在对求解效率要求较高的情况下，预设次数可以设置的适当少一些，如可以设置为10、15等等，或者，在对求解效率要求相对宽松的情况下，预设次数可以设置的适当大一些，如可以设置为30、35等等，在此不做限定。需要说明的是，求解算法可以采用诸如前述L-BFGS、DFP等拟牛顿算法，相较于其他常规求解算法，采用拟牛顿算法能够在较短时间内求解得到较优结果，从而能够在较短时间内求解得到较优结果，在提升质量的同时，极大程度提高优化效率。此外，在对求解效率要求较为宽松的情况下，也可以采用高斯-牛顿法等常规求解算法，在此不对采用的求解算法做具体限定。In some embodiments, a solution algorithm can be used to solve the cost function, such as L-BFGS (Limited-memory Broyden-Fletcher-Goldfarb-Shanno), DFP (Davidon-Fletcher-Powell) and other algorithms, to optimize each model surface The second normal information of the patch and the diffuse information. In some embodiments, during the solving process, the above-mentioned solving algorithm can be iteratively executed for a preset number of times, and the preset number of times can be set according to actual application needs. For example, in the case of higher requirements for solution efficiency, the preset times can be set appropriately less, such as 10, 15, etc., or, in the case of relatively loose requirements for solution efficiency, the preset times can be It should be set appropriately larger, for example, it can be set to 30, 35, etc., which is not limited here. It should be noted that the solution algorithm can use quasi-Newton algorithms such as the aforementioned L-BFGS and DFP. The optimal result can be obtained within a short time, and the optimization efficiency can be greatly improved while improving the quality. In addition, in the case of relatively loose requirements on the solution efficiency, conventional solution algorithms such as the Gauss-Newton method can also be used, and the solution algorithm used is not specifically limited here.
步骤S14：基于模型面片的第一法向信息和模型面片的第二法向信息优化模型顶点的第一位置信息，得到模型顶点的第二位置信息，对待优化模型进行优化。Step S14: Based on the first normal information of the model patch and the second normal information of the model patch, optimize the first position information of the model vertices, obtain the second position information of the model vertices, and optimize the model to be optimized.
在一些实施例中，第一位置信息可以包括第一坐标，第二位置信息可以包括第二坐标，将各个模型顶点分别作为第一当前顶点，并选择一个与第一当前顶点之间存在连接边的模型顶点作为参考顶点，由于待优化模型中各条边的端点为模型顶点，故可以在待优化模型中选择以第一当前顶点为端点的边作为上述连接边，并将所选择到的边(即连接边)的另一端点作为参考顶点。在此基础上，可以将共享该连接边的两个模型面片分别作为第一面片和第二面片，基于第一面片的第一法向信息和第一面片的第二法向信息，得到第一面片的第一法向变换参数，并基于第二面片的第一法向信息和第二面片的第二法向信息，得到第二面片的第二法向变换参数，在此基础上，再基于第一法向变换参数和第二法向变换参数，构建第一当前顶点的目标函数，且目标函数包含第一参考子项和第二参考子项，第一参考子项为预设参考系数与第一当前顶点的第一坐标的乘积，第二参考子项为预设参考系数与第一当前顶点的第二坐标的乘积，再基于各个模型顶点的目标函数，求解得到各个模型顶点的第二坐标。In some embodiments, the first position information may include the first coordinates, the second position information may include the second coordinates, each model vertex is respectively used as the first current vertex, and a connection edge exists between the first current vertex and the first current vertex. The model vertex of is used as the reference vertex. Since the endpoints of each edge in the model to be optimized are model vertices, the edge with the first current vertex as the endpoint can be selected as the above-mentioned connecting edge in the model to be optimized, and the selected edge (that is, the other end point of the connecting edge) is used as a reference vertex. On this basis, the two model patches sharing the connection edge can be regarded as the first patch and the second patch respectively, based on the first normal information of the first patch and the second normal direction of the first patch Information, get the first normal transformation parameters of the first patch, and based on the first normal information of the second patch and the second normal information of the second patch, get the second normal transformation of the second patch parameters, on this basis, and then based on the first normal transformation parameter and the second normal transformation parameter, construct the objective function of the first current vertex, and the objective function includes the first reference sub-item and the second reference sub-item, the first The reference sub-item is the product of the preset reference coefficient and the first coordinate of the first current vertex, the second reference sub-item is the product of the preset reference coefficient and the second coordinate of the first current vertex, and then based on the objective function of each model vertex , solve to get the second coordinates of each model vertex.
这里，在模型顶点的位置坐标优化过程中，考虑到了预设参考系数与模型顶点的第一坐标的乘积，以及预设参考系数与模型顶点的第二坐标的乘积，并将预设参考系数与模型顶点的第一坐标的乘积，以及预设参考系数与模型顶点的第二坐标的乘积作为目标函数的子项，有利于提升目标函数求解的容错率，提高求解结果的稳定性。Here, in the process of optimizing the position coordinates of the model vertex, the product of the preset reference coefficient and the first coordinate of the model vertex, and the product of the preset reference coefficient and the second coordinate of the model vertex are taken into account, and the preset reference coefficient and The product of the first coordinates of the model vertices and the product of the preset reference coefficient and the second coordinates of the model vertices are used as sub-items of the objective function, which is conducive to improving the error tolerance rate of the objective function solution and improving the stability of the solution results.
在一些实施例中，请结合参阅图3，图3为本公开实施例提供的一种模型面片和模型顶点的示意图。需要说明的是，为了直观展示，图3仅示例性地表示待优化模型的部分区域。如图3所示，模型顶点A为第一当前顶点，以模型顶点A为端点的边包括：AB、AC、AD、AE，这些边的另一端点即可作为参考顶点，即与第一当前顶点之间存在连接边的模型顶点可以包括：模型顶点B、模型顶点C、模型顶点D、模型顶点E， 示例性地，可以选择模型顶点E作为参考顶点，从而可以将共享连接边AE的两个模型面片，即模型面片AED和模型面片AEB，分别作为第一面片和第二面片。在选择模型顶点B、模型顶点C、模型顶点D的情况下，可以以此类推，在此不再一一举例。也就是说，对于第一当前顶点而言，若与其具有连接边的模型顶点有N个，则可以得到N对第一面片和第二面片。In some embodiments, please refer to FIG. 3 . FIG. 3 is a schematic diagram of a model surface and a model vertex provided by an embodiment of the present disclosure. It should be noted that, for visual presentation, FIG. 3 only exemplarily shows a partial area of the model to be optimized. As shown in Figure 3, the model vertex A is the first current vertex, and the edges with the model vertex A as the endpoint include: AB, AC, AD, AE, and the other endpoint of these edges can be used as a reference vertex, that is, the first current Model vertices with connecting edges between vertices may include: model vertex B, model vertex C, model vertex D, and model vertex E. Exemplarily, model vertex E may be selected as a reference vertex, so that the two shared connecting edges AE model facets, that is, model facet AED and model facet AEB, respectively as the first facet and the second facet. In the case of selecting model vertex B, model vertex C, and model vertex D, it can be deduced by analogy, and no more examples will be given here. That is to say, for the first current vertex, if there are N model vertices with edges connected thereto, then N pairs of the first facet and the second facet can be obtained.
在一些实施例中，第一面片的第一法向信息乘以第一法向变换参数之后，可以求解得到第一面片的第二法向信息，据此可以求解得到第一法向变换参数；与之类似地，第二面片的第一法向信息乘以第二法向变换参数，可以得到第二面片的第二法向信息，据此可以求解得到第二法向变换参数。In some embodiments, after the first normal information of the first patch is multiplied by the first normal transformation parameter, the second normal information of the first patch can be obtained, and the first normal transformation can be obtained by solving parameters; similarly, the second normal information of the second patch can be obtained by multiplying the first normal information of the second patch by the second normal transformation parameter, and the second normal transformation parameter can be obtained by solving .
在一些实施例中，还可以获取第一面片中连接边的第一对角，并获取第二面片中连接边的第二对角。需要说明的是，该连接边可以是指第一面片和第二面片共享的连接边，即连接第一当前顶点和参考顶点所形成的连接边。在此基础上，可以基于第一法向变换参数和第一对角、第二法向变换参数和第二对角，以及第一当前顶点第一坐标、参考顶点的第一坐标，构建第一优化子项。并基于第一面片中连接边的第一对角、第二面片中连接边的第二对角，以及第一当前顶点的第二坐标、参考顶点的第二坐标，构建第二优化子项。基于此，再基于第一优化子项、第一参考子项、第二优化子项和第二参考子项，得到第一当前顶点的目标函数。这样，能够在优化过程中，充分考虑相邻面片之间的关联关系，有利于提升优化质量。In some embodiments, it is also possible to obtain the first pair of corners of the connected edges in the first patch, and obtain the second pair of corners of the connected edges in the second patch. It should be noted that the connection edge may refer to a connection edge shared by the first patch and the second patch, that is, a connection edge formed by connecting the first current vertex and the reference vertex. On this basis, based on the first normal transformation parameter and the first diagonal, the second normal transformation parameter and the second diagonal, as well as the first coordinate of the first current vertex and the first coordinate of the reference vertex, the first Optimize children. And based on the first diagonal of the connecting edge in the first patch, the second diagonal of the connecting edge in the second patch, the second coordinate of the first current vertex, and the second coordinate of the reference vertex, construct the second optimizer item. Based on this, the objective function of the first current vertex is obtained based on the first optimization sub-item, the first reference sub-item, the second optimization sub-item and the second reference sub-item. In this way, during the optimization process, the association relationship between adjacent patches can be fully considered, which is conducive to improving the optimization quality.
在一些实施例中，请继续结合参阅图3，在将模型面片AED和模型面片AEB，分别作为第一面片和第二面片之后，可以将第一面片AED中连接边AE的对角(即∠ADE)作为第一对角，并将第二面片AEB中连接边AE的对角(即∠ABE)作为第二对角。其他情况可以以此类推，在此不再一一举例。In some embodiments, please continue to refer to FIG. 3 , after using the model facet AED and the model facet AEB as the first facet and the second facet respectively, the connecting edge AE in the first facet AED can be The diagonal (ie ∠ADE) is taken as the first diagonal, and the diagonal (ie ∠ABE) connecting the edge AE in the second patch AEB is taken as the second diagonal. Other situations can be deduced by analogy, and no more examples will be given here.
在一些实施例中，可以将第一对角的余切值与第一法向变换参数的乘积、第二对角的余切值与第二法向变换参数的乘积求和，得到第一和值，并求取第一当前顶点的第一坐标、参考顶点的第一坐标的第一差值，并求取第一和值与第一差值的乘积，得到第一优化子项。In some embodiments, the product of the cotangent of the first diagonal and the first normal transformation parameter, and the product of the cotangent of the second diagonal and the second normal transformation parameter can be summed to obtain the first and value, and calculate the first difference between the first coordinate of the first current vertex and the first coordinate of the reference vertex, and calculate the product of the first sum and the first difference to obtain the first optimization sub-item.
在一些实施例中，可以将第一对角的余切值、第二对角的余切值求和，得到第二和值，以及求取第一当前顶点的第二坐标、参考顶点的第二坐标的第二差值，并求取第二和值与第二差值的乘积，得到第二优化子项。In some embodiments, the cotangent value of the first diagonal angle and the cotangent value of the second diagonal angle can be summed to obtain the second sum value, and the second coordinate of the first current vertex and the second coordinate of the reference vertex can be obtained. The second difference of the two coordinates, and calculate the product of the second sum and the second difference to obtain the second optimization sub-item.
在一些实施例中，为了便于描述，对于任意模型顶点v i∈V而言，可以将其作为第一当前顶点，其第一坐标可以记为v i，其第二坐标可以记为v′ i。模型顶点的参考顶点可以记为v j，其第一坐标可以记为v j，其第二坐标可以记为v′ j。并将共享连接边v iv j的两个模型面片，即模型面片v iv jv k、v iv jv l，分别作为第一面片和第二面片，第一面片中连接边v iv j的第一对角可以记为
Figure PCTCN2021144033-appb-000011
第二面片中连接边v iv j的第二对角可以记为
Figure PCTCN2021144033-appb-000012
此外，第一面片的第一法向变换参数可以记为
Figure PCTCN2021144033-appb-000013
第二面片的第二法向变换参数可以记为
Figure PCTCN2021144033-appb-000014
故可以得到第一当前顶点v i的目标函数，第一当前顶点v i的目标函数可以表示为： In some embodiments, for the convenience of description, for any model vertex v i ∈ V, it can be regarded as the first current vertex, its first coordinate can be recorded as v i , and its second coordinate can be recorded as v′ i . The reference vertex of the model vertex can be denoted as v j , its first coordinate can be denoted as v j , and its second coordinate can be denoted as v′ j . And the two model patches that share the connection edge v i v j , that is, the model patches v i v j v k , v i v j v l , are respectively used as the first patch and the second patch, and the first patch The first pair of angles connecting sides v i v j in can be written as
Figure PCTCN2021144033-appb-000011
The second opposite angle connecting edge v i v j in the second patch can be written as
Figure PCTCN2021144033-appb-000012
In addition, the first normal transformation parameter of the first patch can be written as
Figure PCTCN2021144033-appb-000013
The second normal transformation parameter of the second patch can be written as
Figure PCTCN2021144033-appb-000014
Therefore, the objective function of the first current vertex v i can be obtained, and the objective function of the first current vertex v i can be expressed as:
Figure PCTCN2021144033-appb-000015
Figure PCTCN2021144033-appb-000015
上述公式(17)中，μ表示预设参考系数，μ的数值可以根据实际应用设置，如可以设置为0.01。i表示第一当前顶点的顶点序号，j表示参考顶点的顶点序号，k表示第一对角的对角序号，l表示第二对角的对角序号，μv i表示第一参考子项，v i表示第一当前顶点，v i表示第一当前顶点的第一坐标，v′ i表示第一当前顶点的第二坐标，v j表示参考顶点，v j表示参考顶点的第一坐标，v′ j表示参考顶点的第二坐标，
Figure PCTCN2021144033-appb-000016
Figure PCTCN2021144033-appb-000017
表示第一优化子项，μv′ i表示第二参考子项，
Figure PCTCN2021144033-appb-000018
Figure PCTCN2021144033-appb-000019
表示第二优化子项。 In the above formula (17), μ represents a preset reference coefficient, and the value of μ can be set according to the actual application, for example, it can be set to 0.01. i represents the vertex number of the first current vertex, j represents the vertex number of the reference vertex, k represents the diagonal number of the first diagonal, l represents the diagonal number of the second diagonal, μv i represents the first reference sub-item, v i represents the first current vertex, v i represents the first coordinate of the first current vertex, v' i represents the second coordinate of the first current vertex, v j represents the reference vertex, v j represents the first coordinate of the reference vertex, v' j represents the second coordinate of the reference vertex,
Figure PCTCN2021144033-appb-000016
Figure PCTCN2021144033-appb-000017
represents the first optimization sub-item, μv′ i represents the second reference sub-item,
Figure PCTCN2021144033-appb-000018
Figure PCTCN2021144033-appb-000019
Indicates the second optimization subterm.
在一些实施例中，如前所述，对于第一当前顶点而言，若与其具有连接边的模型顶点有N个，则可以得到N对第一面片和第二面片。则相应地，对于第一当前顶点而言，可以构建得到N个目标函数，为了简化表述，可以将N个目标函数表示为：In some embodiments, as mentioned above, for the first current vertex, if there are N model vertices with edges connected thereto, then N pairs of the first facet and the second facet can be obtained. Correspondingly, for the first current vertex, N objective functions can be constructed. In order to simplify the expression, the N objective functions can be expressed as:
Figure PCTCN2021144033-appb-000020
Figure PCTCN2021144033-appb-000020
上述公式(18)中，N(v i)表示与第一当前顶点v i存在连接边模型顶点集合，μ表示预设参考系数，μ的数值可以根据实际应用设置，如可以设置为0.01。i表示第一当前顶点的顶点序号，j表示参考顶点的顶点序号，k表示第一对角的对角序号，l表示第二对角的对角序号，μv i表示第一参考子项，v i表示第一当前顶点，v i表示第一当前顶点的第一坐标，v′ i表示第一当前顶点的第二坐标，v j表示参考顶点，v j表示参考顶点的第一坐标，v′ j表示参考顶点的第二坐标，
Figure PCTCN2021144033-appb-000021
表示第一优化子项，μv′ i表示第二参考子项，
Figure PCTCN2021144033-appb-000022
表示第二优化子项。 In the above formula (18), N(v i ) represents the set of model vertices connected with the first current vertex v i , μ represents a preset reference coefficient, and the value of μ can be set according to the actual application, for example, it can be set to 0.01. i represents the vertex number of the first current vertex, j represents the vertex number of the reference vertex, k represents the diagonal number of the first diagonal, l represents the diagonal number of the second diagonal, μv i represents the first reference sub-item, v i represents the first current vertex, v i represents the first coordinate of the first current vertex, v' i represents the second coordinate of the first current vertex, v j represents the reference vertex, v j represents the first coordinate of the reference vertex, v' j represents the second coordinate of the reference vertex,
Figure PCTCN2021144033-appb-000021
represents the first optimization sub-item, μv′ i represents the second reference sub-item,
Figure PCTCN2021144033-appb-000022
Indicates the second optimization subterm.
在一些实施例中，为了降低目标函数的构建复杂度，区别于前述方式，也可以不获取第一面片中连接边的第一对角以及第二面片中连接边的第二对角。在此基础上，可以直接基于第一法向变换参数、第二法向变换参数以及第一当前顶点第一坐标、参考顶点的第一坐标，构建第一优化子项。并基于第一当前顶点的第二坐标、参考顶点的第二坐标，构建第二优化子项。基于此，再基于第一优化子项、第一参考子项、第二优化子项和第二参考子项，得到第一当前顶点的目标函数。在此情况下，第一当前顶点v i的目标函数可以简化表示为： In some embodiments, in order to reduce the complexity of constructing the objective function, different from the foregoing method, the first diagonal of the connected edges in the first patch and the second diagonal of the connected edges in the second patch may not be acquired. On this basis, the first optimization sub-item can be constructed directly based on the first normal transformation parameter, the second normal transformation parameter, the first coordinate of the first current vertex, and the first coordinate of the reference vertex. And based on the second coordinates of the first current vertex and the second coordinates of the reference vertex, a second optimization sub-item is constructed. Based on this, the objective function of the first current vertex is obtained based on the first optimization sub-item, the first reference sub-item, the second optimization sub-item and the second reference sub-item. In this case, the objective function of the first current vertex v i can be simplified as:
Figure PCTCN2021144033-appb-000023
Figure PCTCN2021144033-appb-000023
上述公式(19)中，μ表示预设参考系数，μ的数值可以根据实际应用设置，如可以设置为0.01。i表示第一当前顶点的顶点序号，j表示参考顶点的顶点序号，k表示第一对角的对角序号，l表示第二对角的对角序号，μv i表示第一参考子项，v i表示第一当前顶点，v i表示第一当前顶点的第一坐标，v′ i表示第一当前顶点的第二坐标，v j表示参考顶点，v j表示参考顶点的第一坐标，v′ j表示参考顶点的第二坐标。 In the above formula (19), μ represents a preset reference coefficient, and the value of μ can be set according to the actual application, for example, it can be set to 0.01. i represents the vertex number of the first current vertex, j represents the vertex number of the reference vertex, k represents the diagonal number of the first diagonal, l represents the diagonal number of the second diagonal, μv i represents the first reference sub-item, v i represents the first current vertex, v i represents the first coordinate of the first current vertex, v' i represents the second coordinate of the first current vertex, v j represents the reference vertex, v j represents the first coordinate of the reference vertex, v' j represents the second coordinate of the reference vertex.
与之类似地，对于第一当前顶点而言，可以构建得到N个目标函数，N个目标函数也可以简化表示为：Similarly, for the first current vertex, N objective functions can be constructed, and the N objective functions can also be simplified as:
Figure PCTCN2021144033-appb-000024
Figure PCTCN2021144033-appb-000024
上述公式(20)中，N(v i)表示与第一当前顶点v i存在连接边模型顶点集合,μ表示预设参考系数，μ的数值可以根据实际应用设置，如可以设置为0.01。i表示第一当前顶点的顶点序号，j表示参考顶点的顶点序号，k表示第一对角的对角序号，l表示第二对角的对角序号，μv i表示第一参考子项，v i表示第一当前顶点，v i表示第一当前顶点的第一坐标，v′ i表示第一当前顶点的第二坐标，v j表示参考顶点，v j表示参考顶点的第一坐标，v′ j表示参考顶点的第二坐标。 In the above formula (20), N(v i ) represents the set of model vertices connected to the first current vertex v i , μ represents a preset reference coefficient, and the value of μ can be set according to the actual application, for example, it can be set to 0.01. i represents the vertex number of the first current vertex, j represents the vertex number of the reference vertex, k represents the diagonal number of the first diagonal, l represents the diagonal number of the second diagonal, μv i represents the first reference sub-item, v i represents the first current vertex, v i represents the first coordinate of the first current vertex, v' i represents the second coordinate of the first current vertex, v j represents the reference vertex, v j represents the first coordinate of the reference vertex, v' j represents the second coordinate of the reference vertex.
需要说明的是，本实施例中仅描述与前述方案不同之处，相同之处可以参阅前述相关描述。It should be noted that this embodiment only describes the differences from the foregoing solutions, and for the similarities, refer to the foregoing related descriptions.
在一些实施例中，在得到各个模型顶点的目标函数之后，可以基于各个模型顶点的目标函数，联立关于各个模型顶点的第二坐标的方程组，并对联立的方程组进行求解，即可得到各个模型顶点的第二坐标。这样，能够通过联合关于各个模型顶点的第二坐标的方程组，从而一次求解即可得到各个模型顶点的第二坐标，有利于提升模型优化效率。In some embodiments, after obtaining the objective function of each model vertex, based on the objective function of each model vertex, the equations of the second coordinates of each model vertex can be simultaneously established, and the simultaneous equations can be solved, that is, Get the second coordinates of each model vertex. In this way, the second coordinates of each model vertex can be obtained by combining the equations about the second coordinates of each model vertex in one solution, which is beneficial to improve model optimization efficiency.
需要说明的是，在得到各个模型顶点的第二坐标之后，即可认为已经更新待优化模型中各个模型顶点中位置，故待优化模型表面形态也发生变化，由于在模型优化过程中，各个模型顶点的位置信息同步更新。故一方面能够提升优化效率，另一方面也能够兼顾模型顶点之间的关联性，特别是本身存在连接边的相邻顶点之间的关联性；有利于提升 模型优化后的面片质量，大大减少狭长、退化、反向等低质量面片的出现几率，提高流程的稳定性，提升结果的可用性。It should be noted that after obtaining the second coordinates of each model vertex, it can be considered that the position of each model vertex in the model to be optimized has been updated, so the surface morphology of the model to be optimized also changes, because in the process of model optimization, each model The position information of vertices is updated synchronously. Therefore, on the one hand, it can improve the optimization efficiency, and on the other hand, it can also take into account the correlation between the vertices of the model, especially the correlation between adjacent vertices with connecting edges; it is beneficial to improve the quality of the optimized mesh of the model, greatly Reduce the occurrence of low-quality patches such as narrow, degenerated, and reversed patches, improve the stability of the process, and improve the usability of the results.
此外，本公开实施例可以应用于增强现实(Augmented Reality，AR)、虚拟现实(Virtual Reality，VR)等领域，如可以应用于游戏、影视、电子商务、教育、房产、家居装修等应用场景，在此不对本公开实施例的具体应用场景做限定。In addition, the embodiments of the present disclosure can be applied to augmented reality (Augmented Reality, AR), virtual reality (Virtual Reality, VR) and other fields, such as application scenarios such as games, film and television, e-commerce, education, real estate, and home decoration. The specific application scenarios of the embodiments of the present disclosure are not limited here.
在本公开实施例中，可以获取待优化模型，且待优化模型包括至少一个模型面片，模型面片由模型顶点构成，并获取各个模型面片的属性信息和各个模型面片的第一法向信息，以及获取各个模型顶点的第一位置信息，从而基于模型面片的属性信息优化模型面片的第一法向信息，得到模型面片的第二法向信息，并基于模型面片的第一法向信息和模型面片的第二法向信息优化模型顶点的第一位置信息，得到模型顶点的第二位置信息，对待优化模型进行优化。一方面，由于通过优化前的法向信息和优化后的法向信息直接对模型进行优化，而无需借助截断符号距离场等三维模型的间接表示形式来实现模型的优化，故而能够大大减少算法开销，有利于提升模型优化的效率。另一方面，由于在优化过程中考虑到了各个模型面片和各个模型顶点的连接关系，故有利于提升模型优化的质量。因此，本公开能够提高模型优化的质量和效率。In the embodiment of the present disclosure, the model to be optimized can be obtained, and the model to be optimized includes at least one model facet, the model facet is composed of model vertices, and the attribute information of each model facet and the first method of each model facet are obtained. direction information, and obtain the first position information of each model vertex, so as to optimize the first normal information of the model surface based on the attribute information of the model surface, obtain the second normal information of the model surface, and based on the model surface The first normal information and the second normal information of the model patch optimize the first position information of the vertices of the model, obtain the second position information of the vertices of the model, and optimize the model to be optimized. On the one hand, since the model is optimized directly through the normal information before optimization and the normal information after optimization, the model optimization can be realized without the help of indirect representations of 3D models such as truncated signed distance fields, so the algorithm overhead can be greatly reduced , which is conducive to improving the efficiency of model optimization. On the other hand, since the connection relationship between each model patch and each model vertex is considered in the optimization process, it is beneficial to improve the quality of model optimization. Therefore, the present disclosure can improve the quality and efficiency of model optimization.
请参阅图4，图4为本公开实施例提供的一种模型优化方法的流程示意图二。在本公开实施例中，为了提升模型细节，在模型优化过程中，待优化模型可以进行预设数值个优化阶段。如图4所示，本公开实施例提供的模型优化方法可以包括如下步骤：Please refer to FIG. 4 . FIG. 4 is a second schematic flowchart of a model optimization method provided by an embodiment of the present disclosure. In the embodiment of the present disclosure, in order to improve the details of the model, during the model optimization process, the model to be optimized may undergo a preset number of optimization stages. As shown in Figure 4, the model optimization method provided by the embodiment of the present disclosure may include the following steps:
步骤S41：获取待优化模型。Step S41: Obtain the model to be optimized.
在本公开实施例中，待优化模型包括至少一个模型面片，模型面片由模型顶点构成。在一些实施例中，待优化模型可以是对桌子、椅子等单个物体进行模型重建而得到的，或者，也可以是对沙盘、客厅等包含多个物体的场景进行模型重建而得到的，在此不对待优化模型的具体形式做限定。为了便于在模型优化过程中进行计算，在一些实施例中，待优化模型可以位于世界坐标系中。In an embodiment of the present disclosure, the model to be optimized includes at least one model facet, and the model facet is composed of model vertices. In some embodiments, the model to be optimized can be obtained by reconstructing the model of a single object such as a table or a chair, or it can also be obtained by reconstructing a model of a scene containing multiple objects such as a sand table or a living room, where The specific form of the optimization model is not limited. In order to facilitate calculation during model optimization, in some embodiments, the model to be optimized may be located in the world coordinate system.
步骤S42：获取各个模型面片的属性信息和各个模型面片的第一法向信息，并获取各个模型顶点的第一位置信息。Step S42: Obtain attribute information of each model patch and first normal information of each model patch, and acquire first position information of each model vertex.
在一些实施例中，模型面片的属性信息可以包括但不限于：模型面片的亮度信息、色相信息、光度信息和漫反射信息，模型面片的第一法向信息可以采用法向量来表示。In some embodiments, the attribute information of the model patch may include but not limited to: brightness information, hue information, luminosity information and diffuse reflection information of the model patch, and the first normal information of the model patch may be represented by a normal vector .
步骤S43：基于模型面片的属性信息优化模型面片的第一法向信息，得到模型面片的第二法向信息。Step S43: Optimizing the first normal information of the model patch based on the attribute information of the model patch to obtain the second normal information of the model patch.
在一些实施例中，可以根据属性信息和第一法向信息，构建关于第二法向信息的代价函数，并对代价函数进行迭代优化，得到模型面片的第二法向信息。代价函数可以包括第一代价子项、第二代价子项、第三代价子项和***价子项。其中，第一代价子项可以用于表示属性信息中的光度信息和亮度信息的梯度残差，第二代价子项可以用于表示第一法向信息和第二法向信息之间的偏差，第三代价子项可以用于平滑第二法向信息，***价子项可以用于缓解纹理复制问题。In some embodiments, a cost function about the second normal information can be constructed according to the attribute information and the first normal information, and the cost function can be iteratively optimized to obtain the second normal information of the model patch. The cost function may include a first cost sub-term, a second cost sub-term, a third cost sub-term and a fourth cost sub-term. Among them, the first cost sub-item can be used to represent the gradient residual of the photometric information and brightness information in the attribute information, and the second cost sub-item can be used to represent the deviation between the first normal information and the second normal information, The third cost sub-term can be used to smooth the second normal information, and the fourth cost sub-term can be used to alleviate the texture copy problem.
需要说明的是，如前述公开实施例所述，在优化第一法向信息，得到第二法向信息过程中，还能够优化得到各个模型面片的漫反射信息。It should be noted that, as described in the aforementioned disclosed embodiments, in the process of optimizing the first normal information and obtaining the second normal information, diffuse reflection information of each model patch can also be optimized and obtained.
步骤S44：检测当前优化阶段是否满足预设条件，若是则执行步骤S45，否则执行步骤S49。Step S44: Detect whether the current optimization stage satisfies the preset condition, if so, execute step S45, otherwise execute step S49.
在一些实施例中，当前优化阶段满足预设条件可以包括：当前优化阶段未达到预设数值个优化阶段。为了便于计数，在首次检测时，可以初始化一个计数参数n，该计数参数n用于表示当前优化阶段，例如，在首次检测时，该计数参数可以设置为0。此外，预设数值可以根据实际情况进行设置，如在对模型优化效率要求较高的情况下，预设数 值可以设置地适当小一些，如可以设置为2、3等，或者，在对模型优化效率要求相对宽松的情况下，预设数值可以设置地适当大一些，如可以设置为4、5等，在此不做限定。In some embodiments, the current optimization stage meeting the preset condition may include: the current optimization stage does not reach a preset number of optimization stages. In order to facilitate counting, a counting parameter n may be initialized at the first detection, and the counting parameter n is used to represent the current optimization stage, for example, at the first detection, the counting parameter may be set to 0. In addition, the preset value can be set according to the actual situation. For example, in the case of higher requirements for model optimization efficiency, the preset value can be set appropriately smaller, such as 2, 3, etc., or, when optimizing the model When the efficiency requirement is relatively loose, the preset value can be set appropriately larger, for example, it can be set to 4, 5, etc., which is not limited here.
步骤S45：在检测到当前优化阶段满足预设条件的情况下，基于各个模型面片的属性信息、各个模型面片的第一法向信息和各个模型面片的第二法向信息，获取各个模型面片的优化损失。Step S45: When it is detected that the current optimization stage meets the preset conditions, based on the attribute information of each model patch, the first normal information of each model patch, and the second normal information of each model patch, obtain each Optimization loss for model patches.
在当前优化阶段满足预设条件的情况下，可以对待优化模型进行细化。在一些实施例中，可以基于模型面片的属性信息、模型面片的第一法向信息和模型面片的第二法向信息，获取模型面片的优化损失。如前述公开实施例所述，模型面片的属性信息可以包括但不限于：模型面片的亮度信息、色相信息、光度信息和漫反射信息。此外，为了便于描述，模型面片t∈T而言，模型面片的亮度信息可以表示为I(t)，模型面片的色相信息可以表示为C(t)，模型面片的光度信息可以表示为B(t)，模型面片的漫反射信息可以表示为A(t)。模型面片的优化损失可以包括第一代价子项、第二代价子项、第三代价子项和***价子项，其中，第一代价子项用于表示光度信息和亮度信息的梯度残差，第二代价子项用于表示第一法向信息和第二法向信息之间的偏差，第三代价子项用于平滑第二法向信息，***价子项用于更好地缓解纹理复制问题。这里的优化损失，可以理解为前述的代价函数。When the current optimization stage meets the preset conditions, the model to be optimized can be refined. In some embodiments, the optimization loss of the model patch can be obtained based on the attribute information of the model patch, the first normal information of the model patch, and the second normal information of the model patch. As described in the aforementioned disclosed embodiments, the attribute information of the model patch may include but not limited to: brightness information, hue information, luminosity information and diffuse reflection information of the model patch. In addition, for the convenience of description, for the model patch t ∈ T, the brightness information of the model patch can be expressed as I(t), the hue information of the model patch can be expressed as C(t), and the photometric information of the model patch can be expressed as Denoted as B(t), the diffuse reflection information of the model patch can be denoted as A(t). The optimization loss of the model patch can include a first cost sub-item, a second cost sub-item, a third cost sub-item and a fourth cost sub-item, wherein the first cost sub-item is used to represent the gradient residual of photometric information and brightness information difference, the second cost sub-item is used to represent the deviation between the first normal information and the second normal information, the third cost sub-item is used to smooth the second normal information, and the fourth cost sub-item is used to better Alleviate texture duplication issues. The optimization loss here can be understood as the aforementioned cost function.
在一些实施例中，第一代价子项的计算方式可以参阅前述实施例中的公式(12)，第二代价子项的计算方式可以参阅前述实施例中的公式(13)，第三代价子项的计算方式可以参阅前述公开实施例中的公式(14)，***价子项的计算方式可以参阅前述实施例中的公式(15)。In some embodiments, the calculation method of the first cost sub-item can refer to the formula (12) in the previous embodiment, the calculation method of the second cost sub-item can refer to the formula (13) in the previous embodiment, and the third cost sub-item For the calculation method of the item, refer to the formula (14) in the foregoing disclosed embodiments, and for the calculation method of the fourth cost sub-term, refer to the formula (15) in the foregoing embodiments.
在一些实施例中，在构建得到各个模型面片的第一代价子项、第二代价子项、第三代价子项和***价子项之后，即可利用第一代价子项、第二代价子项、第三代价子项和***价子项各自的重要程度，构建得到优化损失。在一些实施例中，可以利用第一代价子项、第二代价子项、第三代价子项和***价子项各自的重要程度，分别对第一代价子项、第二代价子项、第三代价子项和***价子项进行加权处理，得到优化损失。In some embodiments, after constructing the first cost sub-item, the second cost sub-item, the third cost sub-item and the fourth cost sub-item of each model patch, the first cost sub-item, the second cost sub-item can be used The respective importance of the cost sub-item, the third cost sub-item and the fourth cost sub-item is constructed to obtain the optimization loss. In some embodiments, the respective importance levels of the first cost sub-item, the second cost sub-item, the third cost sub-item and the fourth cost sub-item can be used to respectively evaluate the first cost sub-item, the second cost sub-item, the The third cost sub-item and the fourth cost sub-item are weighted to obtain an optimization loss.
例如，第一代价子项的重要程度λ g可以设置为1，第二代价子项的重要程度λ s可以设置为0.005，第三代价子项的重要程度λ r可以设置为0.003，***价子项的重要程度λ a可以设置为2，在此不做限定。在一些实施例中，优化损失可以等同于代价函数E，优化损失也可以表示为：∑ t∈T(λ gE g(t)+λ sE s(t)+λ rE r(t)+λ aE a(t))。 For example, the importance degree λ g of the first cost sub-item can be set to 1, the importance degree λ s of the second cost sub-item can be set to 0.005, the importance degree λ r of the third cost sub-item can be set to 0.003, and the fourth cost sub-item The importance degree λ a of the subitem can be set to 2, which is not limited here. In some embodiments, the optimization loss can be equivalent to the cost function E, and the optimization loss can also be expressed as: ∑ t∈T (λ g E g (t)+λ s E s (t)+λ r E r (t) +λ a E a (t)).
需要说明的是，与前述公开实施例中各个子项过程不同的是，在计算优化损失时，所采用的漫反射信息为在最近更新第一法向信息的过程中，同步优化得到的漫反射信息，即在最新一次执行步骤S43时同步更新得到的漫反射信息。It should be noted that, unlike the sub-item processes in the aforementioned disclosed embodiments, when calculating the optimization loss, the diffuse reflectance information used is the diffuse reflectance obtained by synchronous optimization in the process of updating the first normal information recently. Information, that is, the diffuse reflectance information that is synchronously updated when step S43 is executed for the latest time.
步骤S46：基于各个模型面片的优化损失，选择至少一个模型面片作为当前优化阶段的待细分面片。Step S46: Based on the optimization loss of each model patch, select at least one model patch as the patch to be subdivided in the current optimization stage.
在一些实施例中，可以按照优化损失从大到小的顺序，将模型面片进行排序，并选择位于前预设序位的模型面片，作为待细分面片。预设序位可以根据实际应用需要进行设置，例如，可以按照百分比设置为15％、20％、25％等，即可以选择位于前15％、前20％，或者前25％的模型面片，作为待细分面片，或者，也可以按照数值设置为1000、5000、20000等，即可以选择前1000位、前5000位，或者前20000位的模型面片，作为待细分面片，在此不做限定。这样，能够基于优化损失较大的模型面片对待优化模型进行细化，故能够有利于对细节欠佳的部分进行细节优化，有利于提升模型优化质量。In some embodiments, the model patches can be sorted in descending order of the optimization loss, and the model patch at the pre-ordered position is selected as the patch to be subdivided. The preset order can be set according to the actual application needs. For example, it can be set to 15%, 20%, 25%, etc. according to the percentage, that is, you can choose the model patch in the top 15%, top 20%, or top 25%. As the patch to be subdivided, or, it can also be set to 1000, 5000, 20000, etc. according to the value, that is, the model patch of the first 1000, the first 5000, or the first 20000 can be selected as the patch to be subdivided. This is not limited. In this way, the model to be optimized can be refined based on the model patch with a large optimization loss, so it can be beneficial to optimize the details of the part with poor details and improve the quality of model optimization.
在一些实施例中，也可以直接选择优化损失大于第一阈值的模型面片，作为待细分面片。第一阈值可以根据实际应用需要进行设置，如在对模型优化效果要求较高的情况 下，第一阈值可以设置地适当低一些，或者，在对模型优化效果要求相对宽松的情况下，第一阈值可以设置地适当高一些，在此不做限定。这样，能够基于优化损失较大的模型面片对待优化模型进行细化，故能够有利于对细节欠佳的部分进行细节优化，有利于提升模型优化质量。需要说明的是，若选择不到优化损失大于第一阈值的模型面片，即所有模型面片的优化损失均不高于第一阈值，则可以认为待优化模型当前的细节效果较好，可以跳过本次模型细分操作，而直接执行后续基于第一法向信息和第二法向信息优化第一位置信息，得到模型顶点的第二位置信息，以对待优化模型进行优化的步骤。In some embodiments, it is also possible to directly select a model patch whose optimization loss is greater than the first threshold as a patch to be subdivided. The first threshold can be set according to actual application needs. For example, in the case of higher requirements for model optimization effects, the first threshold can be set appropriately lower, or, in the case of relatively loose requirements for model optimization effects, the first The threshold can be set appropriately higher, which is not limited here. In this way, the model to be optimized can be refined based on the model patch with a large optimization loss, so it can be beneficial to optimize the details of the part with poor details and improve the quality of model optimization. It should be noted that if no model patch with optimization loss greater than the first threshold can be selected, that is, the optimization loss of all model patches is not higher than the first threshold, it can be considered that the current detail effect of the model to be optimized is better, and it can be Skip this model subdivision operation, and directly execute the subsequent step of optimizing the first position information based on the first normal information and the second normal information to obtain the second position information of the model vertices to optimize the model to be optimized.
在一些实施例中，为了提升模型优化质量，还可以基于模型面片的优化损失和面片面积，选择待细分面片。在一些实施例中，可以在基于优化损失，选择至少一个模型面片作为当前优化阶段的待细分面片之后，筛选面片面积大于或等于第二阈值的待细分面片，然后基于筛选得到的待细分面片，对待优化模型进行细分，得到细分后的待优化模型。这样，一方面，由于面片面积过小的模型面片对于模型优化的贡献微乎其微，故能够减少由于对面片面积过小的模型面片过度细分而对优化效率造成的负面影响，有利于提升模型优化效率，另一方面，由于面片面积过小的模型面片其模型顶点的位置坐标区分难度也较大，故也有利于提升模型优化的稳定性。In some embodiments, in order to improve the quality of model optimization, the mesh to be subdivided can also be selected based on the optimization loss of the model mesh and the area of the mesh. In some embodiments, after selecting at least one model patch as the patch to be subdivided in the current optimization stage based on the optimization loss, screen the patch to be subdivided whose patch area is greater than or equal to the second threshold, and then based on the screening The obtained face piece to be subdivided is subdivided into the model to be optimized, and the subdivided model to be optimized is obtained. In this way, on the one hand, since the model patches with too small patch area contribute little to model optimization, it can reduce the negative impact on optimization efficiency caused by over-subdivision of model patches with too small patch area, which is beneficial to improve Model optimization efficiency. On the other hand, it is also difficult to distinguish the position coordinates of model vertices for model patches with too small a patch area, so it is also conducive to improving the stability of model optimization.
需要说明的是，若选择不到面片面积不低于第二阈值的待细分面积，即所有待细分面片的面片面积均低于第二阈值，则可以认为这些待细分面片无需再细分，可以跳过本次模型细分操作，而直接执行后续基于第一法向信息和第二法向信息优化第一位置信息，得到模型顶点的第二位置信息，以对待优化模型进行优化的步骤。It should be noted that if the area to be subdivided cannot be selected with a patch area not lower than the second threshold, that is, the area of all the patches to be subdivided is lower than the second threshold, then these surfaces to be subdivided can be considered as The slice does not need to be further subdivided, and this model subdivision operation can be skipped, and the subsequent optimization of the first position information based on the first normal information and the second normal information can be directly performed to obtain the second position information of the model vertices to be optimized. Steps to optimize the model.
在一些实施例中，在基于优化损失选择得到待细分面片，或者结合优化损失和面片面积共同选择得到待细分面片之后，还可以检测选择得到的待细分面片的总数是否多于第三阈值。若是，则可以执行后续基于待细分面片对待优化模型进行细分的步骤，否则可以跳过本次模型细分操作，而直接执行后续基于第一法向信息和第二法向信息优化第一位置信息，得到模型顶点的第二位置信息，以对待优化模型进行优化的步骤。此外，第三阈值的具体数值可以不做限定，例如，可以设置为0、100、500、2000等等。In some embodiments, after selecting the mesh to be subdivided based on the optimization loss, or selecting the mesh to be subdivided by combining the optimization loss and the area of the mesh, it is also possible to check whether the total number of selected meshes to be subdivided is more than the third threshold. If yes, the subsequent step of subdividing the model to be optimized based on the face to be subdivided can be performed, otherwise, this model subdivision operation can be skipped, and the subsequent step of optimizing the first normal direction information and the second normal direction information can be directly performed. The first position information is to obtain the second position information of the vertices of the model, so as to optimize the model to be optimized. In addition, the specific numerical value of the third threshold may not be limited, for example, it may be set to 0, 100, 500, 2000 and so on.
步骤S47：基于待细分面片，对待优化模型进行细分，得到细分后的待优化模型，更新待优化模型的模型面片。Step S47: based on the mesh to be subdivided, the model to be optimized is subdivided to obtain the subdivided model to be optimized, and the model mesh of the model to be optimized is updated.
在一些实施例中，可以在待细分面片的边上新增至少一个模型顶点，分别将各个新增的模型顶点作为第二当前顶点，并获取第二当前顶点的候选顶点集合。其中，候选顶点集合包括至少一个候选顶点，各个候选顶点位于各个候选面片上(可以位于候选面片内，也可以位于候选面片的边上)，且各个候选面片均包含第二当前顶点所在的边。然后在候选顶点集合中选择至少一个候选顶点，并分别连接至少一个候选顶点和第二当前顶点，得到细分后的待优化模型。这样，能够在待优化模型细节欠佳的位置处进行模型面片进行细分，有利于在这些部位实现更加精细的优化，提升模型优化质量。In some embodiments, at least one model vertex may be added on the edge of the mesh to be subdivided, and each newly added model vertex may be used as a second current vertex, and a set of candidate vertices for the second current vertex may be obtained. Wherein, the set of candidate vertices includes at least one candidate vertex, and each candidate vertex is located on each candidate facet (can be located in the candidate facet or on the edge of the candidate facet), and each candidate facet contains the second current vertex. side. Then at least one candidate vertex is selected from the candidate vertex set, and at least one candidate vertex is connected to the second current vertex respectively to obtain a subdivided model to be optimized. In this way, model patches can be subdivided at positions where the details of the model to be optimized are not good, which is conducive to achieving more refined optimization at these positions and improving the quality of model optimization.
在一些实施例中，为了在一个优化阶段对待优化模型进行细致细分，可以在待细分面片的各个边上新增至少一个模型顶点。在一些实施例中，在待细分面片的各个边上新增一个模型顶点的情况下，新增的模型顶点可以位于待细分面片的各个边的中点。请结合参阅图5a，图5a为本公开实施例提供的一种待细分面片及其相邻面片的示意图一。需要说明的是，待细分面片与其相邻面片共享连接边，如图5a所示，待细分面片ABC与其相邻面片ABF共享连接边AB，与其相邻面片ACE共享连接边AC，与其相邻面片BCD共享连接边BC。在此情况下，可以在待细分面片ABC的各个边上分别新增一个模型顶点，即可以在待细分面片ABC的边AB上新增模型顶点G，并在待细分面片ABC的边AC上新增模型顶点H，以及在待细分面片ABC的边BC上新增模型顶点I。其他情况可以以此类推，在此不再一一举例。In some embodiments, in order to finely subdivide the model to be optimized in an optimization stage, at least one model vertex may be added to each edge of the mesh to be subdivided. In some embodiments, when a model vertex is added on each edge of the mesh to be subdivided, the newly added model vertex may be located at the midpoint of each edge of the mesh to be subdivided. Please refer to FIG. 5 a in conjunction with FIG. 5 a , which is a first schematic diagram of a face piece to be subdivided and its adjacent face pieces provided by an embodiment of the present disclosure. It should be noted that the face to be subdivided shares the connecting edge with its adjacent face. Edge AC shares connecting edge BC with its adjacent patch BCD. In this case, a new model vertex can be added on each side of the surface ABC to be subdivided, that is, a model vertex G can be added on the side AB of the surface ABC to be subdivided, and A new model vertex H is added on the edge AC of ABC, and a new model vertex I is added on the edge BC of the mesh ABC to be subdivided. Other situations can be deduced by analogy, and no more examples will be given here.
在一些实施例中，请继续结合参阅图5a，以模型顶点G作为第二当前顶点为例，由于第二当前顶点G位于边AB，故可以将包含边AB的模型面片ABF和模型面片ABC作为候选面片，并将位于这两个候选面片上的模型顶点(即，模型顶点A、模型顶点B、模型顶点C、模型顶点F、模型顶点G、模型顶点H、模型顶点I)作为候选顶点。其他情况可以以此类推，在此不再一一举例。此外，在一些实施例中，在待优化模型中，候选顶点可以与第二当前顶点不存在连接边，且候选顶点也可以不是第二当前顶点本身，在此情况下，仍以模型顶点G作为第二当前顶点为例，可以将模型顶点C、模型顶点H、模型顶点I、模型顶点F作为候选顶点。其他情况可以以此类推，在此不再一一举例。In some embodiments, please continue to refer to Fig. 5a, taking the model vertex G as the second current vertex as an example, since the second current vertex G is located on the edge AB, the model facet ABF and the model facet containing the edge AB can be ABC is used as a candidate face, and the model vertices (that is, model vertex A, model vertex B, model vertex C, model vertex F, model vertex G, model vertex H, and model vertex I) located on these two candidate faces are used as candidate vertices. Other situations can be deduced by analogy, and no more examples will be given here. In addition, in some embodiments, in the model to be optimized, there may be no connecting edge between the candidate vertex and the second current vertex, and the candidate vertex may not be the second current vertex itself. In this case, the model vertex G is still used as Taking the second current vertex as an example, model vertex C, model vertex H, model vertex I, and model vertex F may be used as candidate vertices. Other situations can be deduced by analogy, and no more examples will be given here.
在一些实施例中，为了在优化阶段更精细地对待优化模型进行细分，可以在候选顶点集合中选择更多的候选顶点，并连接第二当前顶点和候选顶点，以对待优化模型进行细分。请继续结合参阅图5a，仍以模型顶点G作为第二当前顶点为例，可以选择候选顶点H、候选顶点I和候选顶点F，并连接第二当前顶点G与候选顶点H(如图中虚线所示)，以将待细分面片ABC一分为二，并连接第二当前顶点G与候选顶点I(如图中虚线所示)，以将待细分面片ABC二分为三，并连接第二当前顶点G与候选顶点F(如图中虚线所示)，以将候选面片ABF一分为二。其他情况可以以此类推，在此不再一一举例。由此可见，通过细分待优化模型，模型面片和模型顶点的数量也会增加，从而能够对原先细节欠佳的部位，进行优化。In some embodiments, in order to subdivide the model to be optimized more finely in the optimization stage, more candidate vertices can be selected in the candidate vertex set, and the second current vertex and candidate vertices are connected to subdivide the model to be optimized . Please continue to refer to Figure 5a, still taking the model vertex G as the second current vertex as an example, you can select candidate vertex H, candidate vertex I and candidate vertex F, and connect the second current vertex G and candidate vertex H (the dotted line in the figure As shown), to divide the face sheet ABC to be subdivided into two, and connect the second current vertex G and the candidate vertex I (as shown by the dotted line in the figure), to divide the face sheet ABC to be subdivided into three, and Connect the second current vertex G with the candidate vertex F (as shown by the dotted line in the figure), so as to divide the candidate patch ABF into two. Other situations can be deduced by analogy, and no more examples will be given here. It can be seen that by subdividing the model to be optimized, the number of model patches and model vertices will also increase, so that the original parts with poor details can be optimized.
在一些实施例中，可以在待细分面片内新增一个模型顶点，将新增的模型顶点作为第二当前顶点，并分别连接待细分面片的顶点和第二当前顶点，得到细分后的待优化模型。需要说明的是，可以在待细分面片中随机选择一个点作为新增的模型顶点，也可以选择待细分面片的重心作为新增的模型顶点，在此不做限定。请结合参阅图5b，图5b为本公开实施例提供的一种待细分面片及其相邻面片的示意图二。如图5b所示，可以选择待细分面片ABC的重心G作为新增的模型顶点，并将其作为第二当前顶点，以及分别连接待细分面片的顶点A、顶点B、顶点C和第二当前顶点G，从而将待细分面片ABC细分为面片AGC、面片BGC和面片AGB。其他情况可以以此类推，在此不再一一举例。In some embodiments, a new model vertex can be added in the patch to be subdivided, and the newly added model vertex can be used as the second current vertex, and the vertex of the patch to be subdivided and the second current vertex can be respectively connected to obtain a detailed The model to be optimized after classification. It should be noted that a point in the patch to be subdivided can be randomly selected as the newly added model vertex, or the center of gravity of the patch to be subdivided can be selected as the newly added model vertex, which is not limited here. Please refer to FIG. 5 b in conjunction with FIG. 5 b , which is a schematic diagram 2 of a patch to be subdivided and its adjacent patches provided by an embodiment of the present disclosure. As shown in Figure 5b, the center of gravity G of the patch ABC to be subdivided can be selected as the newly added model vertex, and it can be used as the second current vertex, and vertex A, vertex B, and vertex C of the patch to be subdivided are respectively connected and the second current vertex G, thereby subdividing the patch ABC to be subdivided into a patch AGC, a patch BGC and a patch AGB. Other situations can be deduced by analogy, and no more examples will be given here.
在一些实施例中，可以在待细分面片内新增一个模型顶点，从待细分面片的顶点开始连接至新增的模型顶点，并继续延伸至该顶点的对边，以相交得到新的模型顶点。在此基础上，可以将所有新的模型顶点分别作为第二当前顶点，并获取第二当前顶点的候选顶点集合，且候选顶点集合包括至少一个候选顶点，各个候选顶点位于各个候选面片的边上，且各个候选面片均包含第二当前顶点所在的边，以及在候选顶点集合中选择至少一个候选顶点，并分别连接至少一个候选顶点和第二当前顶点，得到细分后的待优化模型。请结合参阅图5c，图5c为本公开实施例提供的一种待细分面片及其相邻面片的示意图三。如图5c所示，可以选择待细分面片ABC的重心J作为新增的模型顶点，分别从待细分面片ABC的顶点A、顶点B、顶点C开始连接至新增的模型顶点J，并继续分别延伸至顶点A的对边BC、顶点B的对边AC、顶点C的对边AB，以分别相交得到新的模型顶点I、新的模型顶点H、新的模型顶点G。在此基础上，可以将所有新的模型顶点G、新的模型顶点H和新的模型顶点I分别作为第二当前顶点，并获取第二当前顶点的候选顶点集合。故此，通过上述细分方式，可以将待细分面片ABC细分为面片AGJ、面片AHJ、面片CHJ、面片CIJ、面片BIJ和面片BGJ，以及将待细分面片ABC的相邻面片ABF细分为面片AGF和面片BGF，将待细分面片ABC的相邻面片BCD细分为面片BID和面片CID，将待细分面片ABC的相邻面片ACE细分为面片AHE和面片CHE。其他情况可以以此类推，在此不再一一举例。In some embodiments, a new model vertex can be added in the patch to be subdivided, and the vertex of the patch to be subdivided is connected to the newly added model vertex, and continues to extend to the opposite side of the vertex to intersect to obtain The new model vertex. On this basis, all new model vertices can be used as the second current vertex respectively, and the candidate vertex set of the second current vertex is obtained, and the candidate vertex set includes at least one candidate vertex, and each candidate vertex is located on the edge of each candidate patch , and each candidate patch contains the edge where the second current vertex is located, and at least one candidate vertex is selected in the candidate vertex set, and at least one candidate vertex and the second current vertex are respectively connected to obtain the subdivided model to be optimized . Please refer to FIG. 5c in conjunction with FIG. 5c , which is a third schematic diagram of a face piece to be subdivided and its adjacent face pieces provided by an embodiment of the present disclosure. As shown in Figure 5c, the center of gravity J of the surface to be subdivided ABC can be selected as the newly added model vertex, and the vertex A, vertex B, and vertex C of the surface to be subdivided ABC are respectively connected to the newly added model vertex J , and continue to extend to the opposite side BC of vertex A, the opposite side AC of vertex B, and the opposite side AB of vertex C to intersect to obtain new model vertex I, new model vertex H, and new model vertex G respectively. On this basis, all the new model vertex G, the new model vertex H, and the new model vertex I can be respectively used as the second current vertex, and a set of candidate vertices for the second current vertex can be obtained. Therefore, through the above subdivision method, the patch ABC to be subdivided can be subdivided into patches AGJ, patch AHJ, patch CHJ, patch CIJ, patch BIJ, and patch BGJ, and the patch to be subdivided The adjacent facet ABF of ABC is subdivided into facet AGF and facet BGF, the adjacent facet BCD of the facet ABC to be subdivided is subdivided into facet BID and facet CID, and the facet ABC to be subdivided is Adjacent patch ACE is subdivided into patch AHE and patch CHE. Other situations can be deduced by analogy, and no more examples will be given here.
需要说明的是，上述几种细分方式仅仅是实际应用时可能使用的细分方式，本公开 实施例并不因此而限定实际细分时所采用的细分方式，即在此不对待细分面片的细分方式做限定。It should be noted that the above subdivision methods are only the subdivision methods that may be used in actual applications, and the embodiment of the present disclosure does not limit the subdivision methods used in actual subdivision, that is, the subdivision methods are not treated here. The subdivision method of the patch is limited.
在一些实施例中，在对所有新增的模型顶点，均完成与其候选顶点连接之后，可以认为本轮优化阶段完成，则可以将计数参数n+1。In some embodiments, after all newly added model vertices are connected to their candidate vertices, it can be considered that the current round of optimization is completed, and the counting parameter n+1 can be set.
步骤S48：重新执行步骤S42至步骤S47。Step S48: Re-execute steps S42 to S47.
需要说明的是，重新执行的步骤S42至步骤S47，均是针对上一次执行步骤S47时更新后的待优化模型。例如，重新执行的步骤S42可以包括：获取更新后的待优化模型中各个模型面片的属性信息和各个模型面片的第一法向信息，并获取更新后的待优化模型中各个模型顶点的第一位置信息。与首次执行步骤S42时不同的是，在第N(N>1)次执行步骤S42时，模型面片的属性信息中漫反射信息为上一次(即第N-1次)执行步骤S42及其后续步骤所得到的漫反射信息，即最新得到的漫反射信息。It should be noted that the re-execution of steps S42 to S47 is for the updated model to be optimized when step S47 was executed last time. For example, the re-execution of step S42 may include: obtaining the attribute information of each model patch in the updated model to be optimized and the first normal information of each model patch, and obtaining the updated model vertex information in the model to be optimized. First location information. The difference from the first execution of step S42 is that when step S42 is executed for the Nth (N>1)th time, the diffuse reflection information in the attribute information of the model patch is the last time (that is, the N-1th) execution of step S42 and its The diffuse reflection information obtained in the subsequent steps is the newly obtained diffuse reflection information.
步骤S49：在检测到当前优化阶段不满足预设条件的情况下，基于模型面片的第一法向信息和模型面片的第二法向信息优化模型顶点的第一位置信息，得到模型顶点的第二位置信息，对待优化模型进行优化。Step S49: When it is detected that the current optimization stage does not meet the preset conditions, optimize the first position information of the model vertices based on the first normal information of the model patch and the second normal information of the model patch, and obtain the model vertex The second position information of the to-be-optimized model is optimized.
这样，有利于在有限的时间开销下，更好地对细节欠佳的部位进行精细化，有利于控制模型优化耗时。In this way, it is beneficial to better refine the parts with poor details under the limited time cost, and it is beneficial to control the time-consuming of model optimization.
请参阅图6，图6为本公开实施例提供的一种待优化模型和优化之后的模型的示意图。需要说明的是，为了便于展示优化前后的对比效果，图6所示的模型为经过渲染之后的模型，如图6所示，对应于优化前的“大卫雕塑”的模型610，优化后的“大卫雕塑”的模型611的面部细节更加丰富；而对应于优化前的“狮子塑像”的模型620，优化后的“狮子塑像”的模型621的鬃毛部位细节更加丰富。Please refer to FIG. 6 . FIG. 6 is a schematic diagram of a model to be optimized and a model after optimization provided by an embodiment of the present disclosure. It should be noted that, in order to facilitate the comparison before and after optimization, the model shown in FIG. 6 is the model after rendering. As shown in FIG. 6, it corresponds to the model 610 of the "Sculpture of David" before optimization. The face details of the model 611 of the "Sculpture of David" are richer; and corresponding to the model 620 of the "Lion Statue" before optimization, the details of the mane of the optimized "Lion Statue" model 621 are richer.
在本公开实施例中，在进行优化之前，在检测到当前优化阶段满足预设条件时，可以基于模型面片的属性信息、第一法向信息和第二法向信息，获取模型面片的优化损失，从而基于优化损失，选择至少一个模型面片作为当前优化阶段的待细分面片，并基于待细分面片，对待优化模型进行细分，并重新执行获取各个模型面片的属性信息和第一法向信息的步骤以及后续步骤。即在模型优化过程中，能够检测各个模型面片的优化损失，并根据优化损失，选择得到待细分面片，从而根据待细分面片对待优化模型进行细分，进而能够提升模型细节，有利于提升模型优化质量。In the embodiment of the present disclosure, before optimization, when it is detected that the current optimization stage satisfies the preset condition, based on the attribute information, the first normal direction information and the second normal direction information of the model surface, the Optimize the loss, so that based on the optimization loss, at least one model patch is selected as the patch to be subdivided in the current optimization stage, and based on the patch to be subdivided, the model to be optimized is subdivided, and the attributes of each model patch are re-executed information and first normal information steps and subsequent steps. That is, in the process of model optimization, the optimization loss of each model patch can be detected, and the patch to be subdivided can be selected according to the optimization loss, so that the model to be optimized can be subdivided according to the patch to be subdivided, and the model details can be improved. It is beneficial to improve the quality of model optimization.
请参阅图7，图7为本公开实施例提供的一种模型优化装置的框架示意图。如图7所示，模型优化装置70包括：模型获取模块71、信息获取模块72、法向优化模块73和顶点优化模块74。模型获取模块71，配置为获取待优化模型；其中，待优化模型包括至少一个模型面片，模型面片由模型顶点构成。信息获取模块72，配置为获取各个模型面片的属性信息和各个模型面片的第一法向信息，并获取各个模型顶点的第一位置信息。法向优化模块73，配置为基于模型面片的属性信息优化模型面片的第一法向信息，得到模型面片的第二法向信息。顶点优化模块74，配置为基于模型面片的第一法向信息和模型面片的第二法向信息优化模型顶点的第一位置信息，得到模型顶点的第二位置信息，对待优化模型进行优化。Please refer to FIG. 7 . FIG. 7 is a schematic framework diagram of a model optimization device provided by an embodiment of the present disclosure. As shown in FIG. 7 , the model optimization device 70 includes: a model acquisition module 71 , an information acquisition module 72 , a normal direction optimization module 73 and a vertex optimization module 74 . The model obtaining module 71 is configured to obtain a model to be optimized; wherein, the model to be optimized includes at least one model patch, and the model patch is composed of model vertices. The information acquiring module 72 is configured to acquire attribute information of each model patch and first normal information of each model patch, and acquire first position information of each model vertex. The normal optimization module 73 is configured to optimize the first normal information of the model patch based on the attribute information of the model patch, and obtain the second normal information of the model patch. The vertex optimization module 74 is configured to optimize the first position information of the model vertices based on the first normal information of the model facets and the second normal direction information of the model facets, obtain the second position information of the model vertices, and optimize the model to be optimized .
在本公开实施例中，一方面，由于通过优化前的法向信息和优化后的法向信息直接对模型进行优化，而无需借助截断符号距离场等三维模型的间接表示形式来实现模型的优化，故而能够大大减少算法开销，有利于提升模型优化的效率。另一方面，由于在优化过程中考虑到了各个模型面片和各个模型顶点的连接关系，故有利于提升模型优化的质量。因此，本公开能够提高模型优化的质量和效率。In the embodiment of the present disclosure, on the one hand, since the model is directly optimized through the normal information before optimization and the normal information after optimization, it is not necessary to use the indirect representation of the 3D model such as the truncated signed distance field to realize the optimization of the model , so the algorithm overhead can be greatly reduced, which is conducive to improving the efficiency of model optimization. On the other hand, since the connection relationship between each model patch and each model vertex is considered in the optimization process, it is beneficial to improve the quality of model optimization. Therefore, the present disclosure can improve the quality and efficiency of model optimization.
在一些公开实施例中，第一位置信息包括第一坐标，第二位置信息包括第二坐标。顶点优化模块74包括参考选择子模块，配置为将各个模型顶点分别作为第一当前顶点， 并选择一个与第一当前顶点之间存在连接边的模型顶点作为参考顶点，以及将共享连接边的两个模型面片分别作为第一面片和第二面片。顶点优化模块74包括法向变换子模块，配置为基于第一面片的第一法向信息和第一面片的第二法向信息，得到第一面片的第一法向变换参数，并基于第二面片的第一法向信息和第二面片的第二法向信息，得到第二面片的第二法向变换参数。顶点优化模块74包括函数构建子模块，配置为基于第一法向变换参数和第二法向变换参数，构建第一当前顶点的目标函数；其中，目标函数包含第一参考子项和第二参考子项，第一参考子项为预设参考系数与第一当前顶点的第一坐标的乘积，且第二参考子项为预设参考系数与第一当前顶点的第二坐标的乘积。顶点优化模块74包括函数求解子模块，配置为基于各个模型顶点的目标函数，求解得到各个模型顶点的第二坐标。In some disclosed embodiments, the first location information includes first coordinates, and the second location information includes second coordinates. The vertex optimization module 74 includes a reference selection submodule configured to use each model vertex as the first current vertex respectively, and select a model vertex that has a connection edge with the first current vertex as a reference vertex, and use two shared connection edges The model faces are respectively used as the first face and the second face. The vertex optimization module 74 includes a normal transformation submodule configured to obtain a first normal transformation parameter of the first patch based on the first normal information of the first patch and the second normal information of the first patch, and Based on the first normal information of the second patch and the second normal information of the second patch, a second normal transformation parameter of the second patch is obtained. The vertex optimization module 74 includes a function construction sub-module configured to construct an objective function of the first current vertex based on the first normal transformation parameter and the second normal transformation parameter; wherein, the objective function includes a first reference sub-item and a second reference sub-item sub-items, the first reference sub-item is the product of the preset reference coefficient and the first coordinate of the first current vertex, and the second reference sub-item is the product of the preset reference coefficient and the second coordinate of the first current vertex. The vertex optimization module 74 includes a function solving submodule configured to obtain the second coordinates of each model vertex based on the objective function of each model vertex.
这里，在模型顶点的位置坐标优化过程中，考虑到了预设参考系数分别与模型顶点的第一坐标的乘积，以及与模型顶点的第二坐标的乘积，并将预设参考系数与模型顶点的第一坐标的乘积，以及预设参考系数与模型顶点的第二坐标的乘积作为目标函数的子项，有利于提升目标函数求解的容错率，提高求解结果的稳定性。Here, in the process of optimizing the position coordinates of the model vertices, the products of the preset reference coefficients and the first coordinates of the model vertices and the products of the second coordinates of the model vertices are taken into account, and the preset reference coefficients and the model vertices The product of the first coordinate and the product of the preset reference coefficient and the second coordinate of the vertex of the model are used as sub-items of the objective function, which is conducive to improving the error tolerance rate of the objective function solution and improving the stability of the solution result.
在一些公开实施例中，函数构建子模块包括对角获取单元，配置为获取第一面片中连接边的第一对角，并获取第二面片中连接边的第二对角。函数构建子模块包括第一子项单元，配置为基于第一法向变换参数和第一对角、第二法向变换参数和第二对角，以及第一当前顶点的第一坐标、参考顶点的第一坐标，构建第一优化子项。函数构建子模块包括第二子项单元，配置为基于第一对角、第二对角，以及第一当前顶点的第二坐标、参考顶点的第二坐标，构建第二优化子项。函数构建子模块包括函数构建单元，配置为基于第一优化子项、第一参考子项、第二优化子项和第二参考子项，得到第一当前顶点的目标函数。In some disclosed embodiments, the function building submodule includes a diagonal obtaining unit configured to obtain a first diagonal of connected edges in the first patch, and obtain a second diagonal of connected edges in the second patch. The function construction submodule includes a first subitem unit configured to be based on the first normal transformation parameter and the first diagonal, the second normal transformation parameter and the second diagonal, and the first coordinates of the first current vertex, the reference vertex The first coordinate of is used to construct the first optimization sub-item. The function construction submodule includes a second subitem unit configured to construct a second optimized subitem based on the first diagonal, the second diagonal, the second coordinates of the first current vertex, and the second coordinates of the reference vertex. The function construction sub-module includes a function construction unit configured to obtain the objective function of the first current vertex based on the first optimization sub-item, the first reference sub-item, the second optimization sub-item and the second reference sub-item.
这里，获取第一面片中连接边的第一对角，并获取第二面片中连接边的第二对角，基于第一法向变换参数和第一对角、第二法向变换参数和第二对角，以及第一当前顶点的第一坐标、参考顶点的第一坐标，构建第一优化子项，并基于第一面片中连接边的第一对角、第二面片中连接边的第二对角，以及第一当前顶点的第二坐标、参考顶点的第二坐标，构建第二优化子项。在此基础上，再基于第一优化子项、第一参考子项、第二优化子项和第二参考子项，得到第一当前顶点的目标函数，故能够在优化过程中，充分考虑相邻面片之间的关联关系，有利于提升优化质量。Here, get the first diagonal of the connected edge in the first patch, and get the second diagonal of the connected edge in the second patch, based on the first normal transformation parameter and the first diagonal and second normal transformation parameters and the second diagonal, as well as the first coordinates of the first current vertex and the first coordinates of the reference vertex, construct the first optimization sub-item, and based on the first diagonal of the connecting edge in the first patch, the first coordinate of the second patch Connecting the second opposite corners of the edges, and the second coordinates of the first current vertex and the second coordinates of the reference vertex, a second optimization sub-item is constructed. On this basis, based on the first optimization sub-item, the first reference sub-item, the second optimization sub-item and the second reference sub-item, the objective function of the first current vertex can be obtained, so in the optimization process, the relative The correlation between adjacent patches is beneficial to improve the optimization quality.
在一些公开实施例中，函数求解子模块包括方程联立单元，配置为基于各个模型顶点的目标函数，联立关于各个模型顶点的第二坐标的方程组。函数求解子模块包括方程求解单元，配置为对方程组进行求解，得到各个模型顶点的第二坐标。In some disclosed embodiments, the function solving submodule includes an equation simultaneous unit configured to simultaneously combine a set of equations about the second coordinates of each model vertex based on the objective function of each model vertex. The function solving sub-module includes an equation solving unit configured to solve the equation system to obtain the second coordinates of each model vertex.
因此，基于各个模型顶点的目标函数，联立关于各个模型顶点的第二坐标的方程组，并对方程组进行求解，得到各个模型顶点的第二坐标。故能够通过联合关于各个模型顶点的第二坐标的方程组，从而一次求解即可得到各个模型顶点的第二坐标，有利于提升模型优化效率。Therefore, based on the objective function of each model vertex, a system of equations about the second coordinates of each model vertex is simultaneously established, and the equations are solved to obtain the second coordinates of each model vertex. Therefore, by combining the equations about the second coordinates of each model vertex, the second coordinates of each model vertex can be obtained by one solution, which is beneficial to improve the efficiency of model optimization.
在一些公开实施例中，模型优化装置70包括损失计算模块，配置为在检测到当前优化阶段满足预设条件的情况下，基于各个模型面片的属性信息、各个模型面片的第一法向信息和各个模型面片的第二法向信息，获取各个模型面片的优化损失。模型优化装置70包括面片选择模块，配置为基于优化损失，选择至少一个模型面片作为当前优化阶段的待细分面片。模型优化装置70包括模型细分模块，配置为基于待细分面片，对待优化模型进行细分，得到细分后的待优化模型，更新待优化模型的模型面片。这里信息获取模块72，配置为获取更新后的待优化模型中各个模型面片的属性信息和各个模型面片的第一法向信息，并获取更新后的待优化模型中各个模型顶点的第一位置信息。In some disclosed embodiments, the model optimization device 70 includes a loss calculation module configured to, based on the attribute information of each model patch and the first normal direction of each model patch, when it is detected that the current optimization stage meets the preset condition information and the second normal information of each model patch to obtain the optimization loss of each model patch. The model optimization device 70 includes a patch selection module configured to select at least one model patch as the patch to be subdivided in the current optimization stage based on the optimization loss. The model optimization device 70 includes a model subdivision module configured to subdivide the model to be optimized based on the mesh to be subdivided, obtain the subdivided model to be optimized, and update the model mesh of the model to be optimized. Here, the information acquisition module 72 is configured to obtain the attribute information of each model patch in the updated model to be optimized and the first normal information of each model patch, and obtain the first normal direction information of each model vertex in the updated model to be optimized. location information.
因此，在模型优化过程中，能够通过检测各个模型面片的优化损失，并根据优化损失，选择得到待细分面片，从而根据待细分面片对待优化模型进行细分，得到细分后的待优化模型，以更新待优化模型的模型面片，进而能够提升模型细节，有利于提升模型优化质量。Therefore, in the process of model optimization, by detecting the optimization loss of each model patch, and according to the optimization loss, select the patch to be subdivided, and then subdivide the model to be optimized according to the patch to be subdivided, and obtain the subdivided The model to be optimized is updated to update the model facets of the model to be optimized, thereby improving model details and improving the quality of model optimization.
在一些公开实施例中，面片选择模块包括排序子模块，配置为按照优化损失从大到小的顺序，将模型面片进行排序。面片选择模块包括选择子模块，配置为选择位于前预设序位的模型面片，作为待细分面片；或者，面片选择模块配置为选择优化损失大于第一阈值的模型面片作为待细分面片。In some disclosed embodiments, the patch selection module includes a sorting submodule configured to sort the model patches in descending order of optimization loss. The patch selection module includes a selection submodule configured to select a model patch located in a pre-ordered position as a patch to be subdivided; or, the patch selection module is configured to select a model patch with an optimization loss greater than the first threshold as Dough to be subdivided.
因此，按照优化损失由大到小的顺序，将模型面片进行排序，并选择位于前预设序位的模型面片作为待优化模型，或者直接选择优化损失大于第一阈值的模型面片作为待细分面片，能够基于优化损失较大的模型面片对待优化模型进行细化，故能够有利于对细节欠佳的部分进行细节优化，有利于提升模型优化质量。Therefore, according to the order of optimization loss from large to small, the model patches are sorted, and the model patch at the pre-ordered position is selected as the model to be optimized, or the model patch with the optimization loss greater than the first threshold is directly selected as the model to be optimized. The patch to be subdivided can refine the model to be optimized based on the model patch with a large optimization loss, so it can help optimize the details of the part with poor details and improve the quality of model optimization.
在一些公开实施例中，模型细分模块包括顶点新增子模块，配置为在待细分面片的边上新增至少一个模型顶点。模型细分模块包括集合获取子模块，配置为分别将各个新增的模型顶点作为第二当前顶点，并获取第二当前顶点的候选顶点集合；其中，候选顶点集合包括至少一个候选顶点，且各个候选顶点位于各个候选面片上，且各个候选面片均包含第二当前顶点所在的边。模型细分模块包括顶点连接子模块，配置为在候选顶点集合中选择至少一个候选顶点，并分别连接至少一个候选顶点与第二当前顶点，得到细分后的待优化模型。In some disclosed embodiments, the model subdivision module includes a vertex adding submodule configured to add at least one model vertex on an edge of the mesh patch to be subdivided. The model subdivision module includes a collection acquisition submodule configured to use each newly added model vertex as a second current vertex, and obtain a set of candidate vertices for the second current vertex; wherein, the set of candidate vertices includes at least one candidate vertex, and each The candidate vertices are located on each candidate facet, and each candidate facet includes an edge where the second current vertex is located. The model subdivision module includes a vertex connection submodule configured to select at least one candidate vertex from the candidate vertex set, and respectively connect at least one candidate vertex with the second current vertex to obtain a subdivided model to be optimized.
因此，在待细分面片的边上新增至少一个模型顶点，分别将各个新增的模型顶点作为第二当前顶点，并获取第二当前顶点的候选顶点集合，且候选顶点集合包含至少一个候选顶点，各个候选顶点位于各个候选面片上，且各个候选面片均包含第二当前顶点所在的边，在此基础上，在候选顶点集合中选择至少一个候选顶点，并分别连接至少一个候选顶点与第二当前顶点，得到细化后的待优化模型，从而能够在待优化模型细节欠佳的位置处进行模型面片进行细分，有利于在这些部位实现更加精细的优化，提升模型优化质量。Therefore, at least one model vertex is added on the edge of the patch to be subdivided, each newly added model vertex is used as the second current vertex, and the candidate vertex set of the second current vertex is obtained, and the candidate vertex set contains at least one Candidate vertices, each candidate vertex is located on each candidate facet, and each candidate facet contains the edge where the second current vertex is located, on this basis, select at least one candidate vertex in the candidate vertex set, and connect at least one candidate vertex respectively With the second current vertex, the refined model to be optimized can be obtained, so that the model patch can be subdivided at the positions where the details of the model to be optimized are not good, which is conducive to achieving more refined optimization in these parts and improving the quality of model optimization .
在一些公开实施例中，顶点优化模块74配置为在检测到当前优化阶段不满足预设条件的情况下，基于模型面片的第一法向信息和模型面片的第二法向信息优化模型顶点的第一位置信息，得到模型顶点的第二位置信息，对待优化模型进行优化。In some disclosed embodiments, the vertex optimization module 74 is configured to optimize the model based on the first normal information of the model patch and the second normal information of the model patch when it is detected that the current optimization stage does not satisfy the preset condition The first position information of the vertex is obtained to obtain the second position information of the vertex of the model, and the model to be optimized is optimized.
因此，在检测到当前优化阶段不满足预设条件时，执行基于模型面片的第一法向信息和模型面片的第二法向信息优化模型顶点的第一位置信息，得到模型顶点的第二位置信息，以对待优化模型进行优化的步骤，即在当前优化阶段不满足预设条件的情况下，不再对模型进行细分，有利于在有限的时间开销下，更好地对细节欠佳的部位进行精细化，有利于控制模型优化耗时。Therefore, when it is detected that the current optimization stage does not meet the preset conditions, the first position information of the model vertex is optimized based on the first normal information of the model patch and the second normal information of the model patch, and the first position information of the model vertex is obtained. The second position information is used to optimize the model to be optimized, that is, if the current optimization stage does not meet the preset conditions, the model will not be subdivided, which is beneficial to better understand the details under the limited time cost. The refinement of the best parts is beneficial to the time-consuming control model optimization.
在一些公开实施例中，模型优化装置70还包括面片筛选模块，配置为筛选面片面积大于或等于第二阈值的待细分面片。这里，模型细分模块配置为基于筛选得到的待细分面片，对待优化模型进行细分，得到细分后的待优化模型。In some disclosed embodiments, the model optimization device 70 further includes a mesh screening module configured to screen meshes whose area is greater than or equal to the second threshold to be subdivided. Here, the model subdivision module is configured to subdivide the model to be optimized based on the screened faces to be subdivided to obtain the subdivided model to be optimized.
因此，在基于优化损失选择得到待细分面片的情况下，根据面片面积从待细分面片中筛选出面片面积大于或等于第二阈值的待细分面片，故能够基于优化损失较大且面片面积较大的模型面片对待优化模型进行细化，一方面，由于面片面积过小的模型面片对于模型优化的贡献微乎其微，故能够减少由于对面片面积过小的模型面片过度细分而对优化效率造成的负面影响，有利于提升模型优化效率，另一方面，由于面片面积过小的模型面片其模型顶点的位置坐标区分难度也较大，故也有利于提升模型优化的稳定性。Therefore, in the case of selecting the patch to be subdivided based on the optimization loss, the patch to be subdivided is selected from the patches to be subdivided according to the area of the patch Larger model patches with larger patch areas should be refined to optimize the model. On the one hand, since the contribution of model patches with too small patch areas to model optimization is negligible, it can reduce the problem of models with too small patch areas. The negative impact on optimization efficiency caused by over-subdivision of the mesh is beneficial to improve the efficiency of model optimization. It is beneficial to improve the stability of model optimization.
在一些公开实施例中，当前优化阶段满足预设条件包括：当前优化阶段未达到预设 数值个优化阶段。In some disclosed embodiments, the current optimization stage meeting the preset condition includes: the current optimization stage does not reach the preset value of optimization stages.
因此，通过将当前优化阶段满足预设条件设置为包括：当前优化阶段未达到预设数值个优化阶段，故能够按需对需要细分的面片进行选择性的细分，在控制时间开销增长的前提下，提升模型优化质量。Therefore, by setting the current optimization stage to meet the preset conditions to include: the current optimization stage does not reach the preset number of optimization stages, it is possible to selectively subdivide the patches that need to be subdivided as needed, and control the increase in time overhead Under the premise of improving the quality of model optimization.
请参阅图8，图8为本公开实施例提供的一种电子设备的框架示意图。如图8所示，电子设备80包括相互耦接的存储器81和处理器82，处理器82配置为执行存储器81中存储的程序指令，以实现上述任一模型优化方法实施例Please refer to FIG. 8 . FIG. 8 is a schematic frame diagram of an electronic device provided by an embodiment of the present disclosure. As shown in FIG. 8 , an electronic device 80 includes a memory 81 and a processor 82 coupled to each other, and the processor 82 is configured to execute the program instructions stored in the memory 81 to implement any of the above-mentioned model optimization method embodiments
的部分或全部步骤。在一些实施例中，电子设备80可以包括但不限于：微型计算机、服务器，此外，电子设备80还可以包括笔记本电脑、平板电脑等移动设备，在此不做限定。some or all of the steps. In some embodiments, the electronic device 80 may include, but is not limited to: a microcomputer and a server. In addition, the electronic device 80 may also include mobile devices such as notebook computers and tablet computers, which are not limited here.
在一些实施例中，处理器82用于控制其自身以及存储器81以实现上述任一模型优化方法实施例的步骤。处理器82还可以称为CPU(Central Processing Unit，中央处理单元)。处理器82可能是一种集成电路芯片，具有信号的处理能力。处理器82还可以是通用处理器、数字信号处理器(Digital Signal Processor,DSP)、专用集成电路(Application Specific Integrated Circuit,ASIC)、现场可编程门阵列(Field-Programmable Gate Array,FPGA)或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件。通用处理器可以是微处理器或者该处理器也可以是任何常规的处理器等。另外，处理器82可以由集成电路芯片共同实现。In some embodiments, the processor 82 is used to control itself and the memory 81 to implement the steps in any one of the above embodiments of the model optimization method. The processor 82 may also be called a CPU (Central Processing Unit, central processing unit). The processor 82 may be an integrated circuit chip with signal processing capability. The processor 82 can also be a general-purpose processor, a digital signal processor (Digital Signal Processor, DSP), an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), a field-programmable gate array (Field-Programmable Gate Array, FPGA) or other Programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. A general-purpose processor may be a microprocessor, or the processor may be any conventional processor, or the like. In addition, the processor 82 may be jointly realized by an integrated circuit chip.
本公开实施例，一方面由于通过优化前的法向信息和优化后的法向信息直接对模型进行优化，而无需借助截断符号距离场等间接表示形式，故能够大大减少算法开销，有利于提升优化效率，另一方面由于在优化过程中考虑到各个模型面片和各个模型顶点的连接关系，故有利于提升优化质量，故此能够提高模型优化的质量和效率。In the embodiments of the present disclosure, on the one hand, the model is directly optimized by using the normal information before optimization and the normal information after optimization without resorting to indirect representations such as truncated signed distance fields, so the algorithm overhead can be greatly reduced, which is beneficial to improve Optimization efficiency, on the other hand, because the connection relationship between each model patch and each model vertex is considered in the optimization process, it is beneficial to improve the optimization quality, so the quality and efficiency of model optimization can be improved.
请参阅图9，图9为本公开实施例提供的一种计算机可读存储介质的框架示意图。如图9所示，计算机可读存储介质90存储有能够被处理器运行的程序指令91，程序指令91用于实现上述任一模型优化方法实施例的部分或全部步骤。Please refer to FIG. 9 . FIG. 9 is a schematic framework diagram of a computer-readable storage medium provided by an embodiment of the present disclosure. As shown in FIG. 9 , a computer-readable storage medium 90 stores program instructions 91 that can be executed by a processor, and the program instructions 91 are used to implement some or all of the steps in any of the above embodiments of the model optimization method.
计算机可读存储介质可以是可以保持和存储由指令执行设备使用的指令的有形设备，可为易失性存储介质或者非易失性存储介质。计算机可读存储介质例如可以是(但不限于)电存储设备、磁存储设备、光存储设备、电磁存储设备、半导体存储设备或者上述的任意合适的组合。计算机可读存储介质的更具体的例子(非穷举的列表)包括：便携式计算机盘、硬盘、随机存取存储器(RAM)、只读存储器(ROM)、可擦式可编程只读存储器(EPROM或闪存)、静态随机存取存储器(SRAM)、便携式压缩盘只读存储器(CD-ROM)、数字多功能盘(DVD)、记忆棒、软盘、机械编码设备、例如其上存储有指令的打孔卡或凹槽内凸起结构、以及上述的任意合适的组合。这里所使用的计算机可读存储介质不被解释为瞬时信号本身，诸如无线电波或者其他自由传播的电磁波、通过波导或其他传输媒介传播的电磁波(例如，通过光纤电缆的光脉冲)、或者通过电线传输的电信号。A computer readable storage medium may be a tangible device that can hold and store instructions for use by an instruction execution device, and may be a volatile storage medium or a nonvolatile storage medium. A computer readable storage medium may be, for example, but is not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer diskettes, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), or flash memory), static random access memory (SRAM), compact disc read only memory (CD-ROM), digital versatile disc (DVD), memory stick, floppy disk, mechanically encoded device, such as a printer with instructions stored thereon A hole card or a raised structure in a groove, and any suitable combination of the above. As used herein, computer-readable storage media are not to be construed as transient signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., pulses of light through fiber optic cables), or transmitted electrical signals.
本公开实施例还提出一种计算机程序，所述计算机程序包括计算机可读代码，在所述计算机可读代码被计算机读取并执行的情况下，实现本公开任一实施例中的方法的部分或全部步骤。An embodiment of the present disclosure also proposes a computer program, the computer program includes computer readable code, and when the computer readable code is read and executed by a computer, part of the method in any embodiment of the present disclosure is implemented or all steps.
本公开实施例还提供了一种计算机程序产品，包括计算机可读代码，或者承载有计算机可读代码的非易失性计算机可读存储介质，当所述计算机可读代码在电子设备的处理器中运行时，所述电子设备中的处理器执行上述方法的部分或全部步骤。An embodiment of the present disclosure also provides a computer program product, including computer-readable codes, or a non-volatile computer-readable storage medium carrying computer-readable codes, when the computer-readable codes are stored in a processor of an electronic device When running in the electronic device, the processor in the electronic device executes some or all steps of the above method.
上述方案，一方面由于通过优化前后的法向信息直接对模型进行优化，而无需借助截断符号距离场等间接表示形式，故能够大大减少算法开销，有利于提升优化效率，另 一方面由于在优化过程中考虑到各个模型面片和各个模型顶点的连接关系，故有利于提升优化质量，故此能够提高模型优化的质量和效率。The above scheme, on the one hand, directly optimizes the model through the normal information before and after optimization, without resorting to indirect representations such as truncated signed distance fields, so it can greatly reduce the algorithm overhead and help improve optimization efficiency. In the process, the connection relationship between each model patch and each model vertex is considered, so it is beneficial to improve the optimization quality, so the quality and efficiency of model optimization can be improved.
本公开涉及增强现实领域，通过获取现实环境中的目标对象的图像信息，进而借助各类视觉相关算法实现对目标对象的相关特征、状态及属性进行检测或识别处理，从而得到与应用匹配的虚拟与现实相结合的AR效果。示例性的，目标对象可涉及与人体相关的脸部、肢体、手势、动作等，或者与物体相关的标识物、标志物，或者与场馆或场所相关的沙盘、展示区域或展示物品等。视觉相关算法可涉及视觉定位、SLAM、三维重建、图像注册、背景分割、对象的关键点提取及跟踪、对象的位姿或深度检测等。在一些实施例中，应用不仅可以涉及跟真实场景或物品相关的导览、导航、讲解、重建、虚拟效果叠加展示等交互场景，还可以涉及与人相关的特效处理，比如妆容美化、肢体美化、特效展示、虚拟模型展示等交互场景。This disclosure relates to the field of augmented reality. By acquiring the image information of the target object in the real environment, and then using various visual related algorithms to detect or identify the relevant features, states and attributes of the target object, and obtain a virtual reality that matches the application. AR effects combined with reality. Exemplarily, the target object may involve faces, limbs, gestures, actions, etc. related to the human body, or markers and markers related to objects, or sand tables, display areas or display items related to venues or places. Vision-related algorithms can involve visual positioning, SLAM, 3D reconstruction, image registration, background segmentation, object key point extraction and tracking, object pose or depth detection, etc. In some embodiments, the application can not only involve interactive scenes such as tour, navigation, explanation, reconstruction, virtual effect overlay display and other interactive scenes related to real scenes or objects, but also can involve special effects processing related to people, such as makeup beautification, body beautification , special effects display, virtual model display and other interactive scenes.
可通过卷积神经网络，实现对目标对象的相关特征、状态及属性进行检测或识别处理。上述卷积神经网络是基于深度学习框架进行模型训练而得到的网络模型。The relevant features, states and attributes of the target object can be detected or identified through the convolutional neural network. The above-mentioned convolutional neural network is a network model obtained by performing model training based on a deep learning framework.
在本公开所提供的几个实施例中，应该理解到，所揭露的方法和装置，可以通过其它的方式实现。例如，以上所描述的装置实施方式仅仅是示意性的，例如，模块或单元的划分，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式，例如单元或组件可以结合或者可以集成到另一个***，或一些特征可以忽略，或不执行。另一点，所显示或讨论的相互之间的耦合或直接耦合或通信连接可以是通过一些接口，装置或单元的间接耦合或通信连接，可以是电性、机械或其它的形式。In the several embodiments provided in the present disclosure, it should be understood that the disclosed methods and devices may be implemented in other ways. For example, the device implementations described above are only illustrative. For example, the division of modules or units is only a logical function division. In actual implementation, there may be other division methods. For example, units or components can be combined or integrated. to another system, or some features may be ignored, or not implemented. In another point, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces, and the indirect coupling or communication connection of devices or units may be in electrical, mechanical or other forms.
作为分离部件说明的单元可以是或者也可以不是物理上分开的，作为单元显示的部件可以是或者也可以不是物理单元，即可以位于一个地方，或者也可以分布到网络单元上。可以根据实际的需要选择其中的部分或者全部单元来实现本实施方式方案的目的。A unit described as a separate component may or may not be physically separated, and a component shown as a unit may or may not be a physical unit, that is, it may be located in one place, or may also be distributed to network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.
另外，在本公开各个实施例中的各功能单元可以集成在一个处理单元中，也可以是各个单元单独物理存在，也可以两个或两个以上单元集成在一个单元中。上述集成的单元既可以采用硬件的形式实现，也可以采用软件功能单元的形式实现。In addition, each functional unit in each embodiment of the present disclosure may be integrated into one processing unit, each unit may exist separately physically, or two or more units may be integrated into one unit. The above-mentioned integrated units can be implemented in the form of hardware or in the form of software functional units.
集成的单元如果以软件功能单元的形式实现并作为独立的产品销售或使用时，可以存储在一个计算机可读取存储介质中。基于这样的理解，本公开的技术方案本质上或者说对现有技术做出贡献的部分或者该技术方案的全部或部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质中，包括若干指令用以使得一台计算机设备(可以是个人计算机，服务器，或者网络设备等)或处理器(processor)执行本公开各个实施方式方法的全部或部分步骤。而前述的存储介质包括：U盘、移动硬盘、只读存储器(ROM，Read-Only Memory)、随机存取存储器(RAM，Random Access Memory)、磁碟或者光盘等各种可以存储程序代码的介质。If the integrated unit is realized in the form of a software function unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present disclosure is essentially or part of the contribution to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium , including several instructions to make a computer device (which may be a personal computer, a server, or a network device, etc.) or a processor (processor) execute all or part of the steps of the methods in various embodiments of the present disclosure. The aforementioned storage media include: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), magnetic disk or optical disc, etc., which can store program codes. .

Claims (24)

  1. 一种模型优化方法，包括：A method for model optimization, comprising:
    获取待优化模型；其中，所述待优化模型包括至少一个模型面片，所述模型面片由模型顶点构成；Obtaining a model to be optimized; wherein, the model to be optimized includes at least one model patch, and the model patch is composed of model vertices;
    获取各个所述模型面片的属性信息和各个所述模型面片的第一法向信息，并获取各个所述模型顶点的第一位置信息；Obtain attribute information of each of the model faces and first normal information of each of the model faces, and obtain first position information of each of the model vertices;
    基于所述模型面片的属性信息优化所述模型面片的第一法向信息，得到所述模型面片的第二法向信息；Optimizing the first normal information of the model surface based on the attribute information of the model surface to obtain the second normal information of the model surface;
    基于所述模型面片的第一法向信息和所述模型面片的第二法向信息优化所述模型顶点的第一位置信息，得到所述模型顶点的第二位置信息，对所述待优化模型进行优化。Optimizing the first position information of the model vertices based on the first normal information of the model facet and the second normal direction information of the model facet, to obtain the second position information of the model vertices, for the to-be-treated The optimization model is optimized.
  2. 根据权利要求1所述的方法，其中，所述第一位置信息包括第一坐标，所述第二位置信息包括第二坐标；所述基于所述模型面片的第一法向信息和所述模型面片的第二法向信息优化所述模型顶点的第一位置信息，得到所述模型顶点的第二位置信息，包括：The method according to claim 1, wherein the first position information includes first coordinates, and the second position information includes second coordinates; the first normal information based on the model patch and the The second normal information of the model patch optimizes the first position information of the model vertices to obtain the second position information of the model vertices, including:
    将各个所述模型顶点分别作为第一当前顶点，并选择一个与所述第一当前顶点之间存在连接边的模型顶点作为参考顶点，以及将共享所述连接边的两个所述模型面片分别作为第一面片和第二面片；Using each of the model vertices as the first current vertex, and selecting a model vertex that has a connection edge with the first current vertex as a reference vertex, and two model patches that share the connection edge Respectively as the first surface and the second surface;
    基于所述第一面片的第一法向信息和所述第一面片的第二法向信息，得到所述第一面片的第一法向变换参数，并基于所述第二面片的第一法向信息和所述第二面片的第二法向信息，得到所述第二面片的第二法向变换参数；Based on the first normal information of the first patch and the second normal information of the first patch, the first normal transformation parameter of the first patch is obtained, and based on the second patch The first normal information and the second normal information of the second patch obtain the second normal transformation parameters of the second patch;
    基于所述第一法向变换参数和所述第二法向变换参数，构建所述第一当前顶点的目标函数；其中，所述目标函数包含第一参考子项和第二参考子项，所述第一参考子项为预设参考系数与所述第一当前顶点的第一坐标的乘积，且所述第二参考子项为所述预设参考系数与所述第一当前顶点的第二坐标的乘积；Based on the first normal transformation parameter and the second normal transformation parameter, construct the objective function of the first current vertex; wherein, the objective function includes a first reference sub-item and a second reference sub-item, so The first reference subitem is the product of the preset reference coefficient and the first coordinate of the first current vertex, and the second reference subitem is the second coordinate of the preset reference coefficient and the first current vertex. the product of coordinates;
    基于各个所述模型顶点的目标函数，求解得到各个所述模型顶点的第二坐标。Based on the objective function of each of the model vertices, the second coordinates of each of the model vertices are obtained by solving.
  3. 根据权利要求2所述的方法，其中，所述基于所述第一法向变换参数和所述第二法向变换参数，构建所述第一当前顶点的目标函数，包括：The method according to claim 2, wherein said constructing the objective function of the first current vertex based on the first normal transformation parameter and the second normal transformation parameter comprises:
    获取所述第一面片中所述连接边的第一对角，并获取所述第二面片中所述连接边的第二对角；obtaining a first diagonal of the connected edges in the first patch, and obtaining a second diagonal of the connected edges in the second patch;
    基于所述第一法向变换参数和所述第一对角、所述第二法向变换参数和所述第二对角，以及所述第一当前顶点的第一坐标、所述参考顶点的第一坐标，构建第一优化子项；Based on the first normal transformation parameter and the first diagonal, the second normal transformation parameter and the second diagonal, the first coordinate of the first current vertex, the reference vertex The first coordinate, constructing the first optimization sub-item;
    基于所述第一对角、所述第二对角，以及所述第一当前顶点的第二坐标、所述参考顶点的第二坐标，构建第二优化子项；Constructing a second optimization sub-item based on the first diagonal, the second diagonal, the second coordinates of the first current vertex, and the second coordinates of the reference vertex;
    基于所述第一优化子项、所述第一参考子项、所述第二优化子项和所述第二参考子项，得到所述第一当前顶点的目标函数。Based on the first optimization sub-item, the first reference sub-item, the second optimization sub-item and the second reference sub-item, an objective function of the first current vertex is obtained.
  4. 根据权利要求2所述的方法，其中，所述基于各个所述模型顶点的目标函数，求解得到各个所述模型顶点的第二坐标，包括：The method according to claim 2, wherein the second coordinates of each of the model vertices are obtained by solving the objective function based on each of the model vertices, comprising:
    基于各个所述模型顶点的目标函数，联立关于各个所述模型顶点的第二坐标的方程组；Simultaneously establishing a system of equations about the second coordinates of each of the model vertices based on the objective function of each of the model vertices;
    对所述方程组进行求解，得到各个所述模型顶点的第二坐标。Solving the equations to obtain the second coordinates of each vertex of the model.
  5. 根据权利要求1至4中任一项所述的方法，其中，在所述基于所述模型面片的第一法向信息和所述模型面片的第二法向信息优化所述模型顶点的第一位置信息，得到 所述模型顶点的第二位置信息，对所述待优化模型进行优化之前，所述方法还包括：The method according to any one of claims 1 to 4, wherein in the optimization of the model vertices based on the first normal information of the model patch and the second normal information of the model patch, The first position information is to obtain the second position information of the vertices of the model. Before optimizing the model to be optimized, the method further includes:
    在检测到当前优化阶段满足预设条件的情况下，基于各个所述模型面片的属性信息、各个所述模型面片的第一法向信息和各个所述模型面片的第二法向信息，获取各个所述模型面片的优化损失；When it is detected that the current optimization stage satisfies the preset condition, based on the attribute information of each of the model patches, the first normal information of each of the model patches, and the second normal information of each of the model patches , to obtain the optimization loss of each model patch;
    基于所述优化损失，选择至少一个所述模型面片作为所述当前优化阶段的待细分面片；Based on the optimization loss, selecting at least one model patch as a patch to be subdivided in the current optimization stage;
    基于所述待细分面片，对所述待优化模型进行细分，得到细分后的待优化模型，更新所述待优化模型的所述模型面片；Subdividing the model to be optimized based on the mesh to be subdivided to obtain the subdivided model to be optimized, and updating the model mesh of the model to be optimized;
    所述获取各个所述模型面片的属性信息和各个所述模型面片的第一法向信息，并获取各个所述模型顶点的第一位置信息，包括：The acquiring the attribute information of each of the model faces and the first normal information of each of the model faces, and acquiring the first position information of each of the model vertices includes:
    获取更新后的所述待优化模型中各个所述模型面片的属性信息和各个所述模型面片的第一法向信息，并获取更新后的所述待优化模型中各个所述模型顶点的第一位置信息。Obtain the attribute information of each of the model patches in the updated model to be optimized and the first normal information of each of the model patches, and obtain the updated First location information.
  6. 根据权利要求5所述的方法，其中，所述基于所述优化损失，选择至少一个所述模型面片作为所述当前优化阶段的待细分面片，包括：The method according to claim 5, wherein said selecting at least one model facet as the facet to be subdivided in the current optimization stage based on the optimization loss comprises:
    按照所述优化损失从大到小的顺序，将所述模型面片进行排序，并选择位于前预设序位的所述模型面片，作为所述待细分面片；Sorting the model patches according to the order of the optimization loss from large to small, and selecting the model patch located in the previous pre-ordered position as the patch to be subdivided;
    或者，选择所述优化损失大于第一阈值的所述模型面片，作为所述待细分面片。Alternatively, select the model patch whose optimization loss is greater than a first threshold as the patch to be subdivided.
  7. 根据权利要求5所述的方法，其中，所述基于所述待细分面片，对所述待优化模型进行细分，得到细分后的待优化模型，包括：The method according to claim 5, wherein said subdividing the model to be optimized based on the face piece to be subdivided to obtain the subdivided model to be optimized comprises:
    在所述待细分面片的边上新增至少一个所述模型顶点；Adding at least one model vertex on the edge of the mesh to be subdivided;
    分别将各个新增的模型顶点作为第二当前顶点，并获取所述第二当前顶点的候选顶点集合；其中，所述候选顶点集合包括至少一个候选顶点，且各个所述候选顶点位于各个候选面片上，各个所述候选面片均包含所述第二当前顶点所在的边；Each newly added model vertex is used as a second current vertex, and a candidate vertex set of the second current vertex is obtained; wherein, the candidate vertex set includes at least one candidate vertex, and each of the candidate vertexes is located on each candidate surface On the slice, each of the candidate faces includes an edge where the second current vertex is located;
    在所述候选顶点集合中选择至少一个所述候选顶点，并分别连接所述至少一个所述候选顶点与所述第二当前顶点，得到细分后的待优化模型。Selecting at least one candidate vertex in the candidate vertex set, and respectively connecting the at least one candidate vertex and the second current vertex to obtain a subdivided model to be optimized.
  8. 根据权利要求5所述的方法，其中，所述方法还包括：The method according to claim 5, wherein the method further comprises:
    在检测到所述当前优化阶段不满足所述预设条件的情况下，基于所述模型面片的第一法向信息和所述模型面片的第二法向信息优化所述模型顶点的第一位置信息，得到所述模型顶点的所述第二位置信息，对所述待优化模型进行优化。When it is detected that the current optimization stage does not meet the preset condition, optimize the first normal direction information of the model surface based on the first normal information of the model surface and the second normal information of the model surface. a position information, obtain the second position information of the vertices of the model, and optimize the model to be optimized.
  9. 根据权利要求5所述的方法，其中，在所述基于所述待细分面片，对所述待优化模型进行细分，得到细分后的待优化模型之前，所述方法还包括：The method according to claim 5, wherein, before said subdividing the model to be optimized based on the surface patch to be subdivided to obtain the subdivided model to be optimized, the method further comprises:
    筛选面片面积大于或等于第二阈值的待细分面片；Screening patches to be subdivided whose patch area is greater than or equal to the second threshold;
    所述基于所述待细分面片，对所述待优化模型进行细分，得到细分后的待优化模型，包括：The said subdividing the model to be optimized based on the surface to be subdivided to obtain the subdivided model to be optimized includes:
    基于筛选得到的所述待细分面片，对所述待优化模型进行细分，得到所述细分后的待优化模型。The model to be optimized is subdivided based on the screened face pieces to be subdivided to obtain the subdivided model to be optimized.
  10. 根据权利要求5所述的方法，其中，所述当前优化阶段满足预设条件包括：所述当前优化阶段未达到预设数值个优化阶段。The method according to claim 5, wherein the current optimization stage meeting a preset condition comprises: the current optimization stage does not reach a preset number of optimization stages.
  11. 一种模型优化装置，包括：A model optimization device, comprising:
    模型获取模块，配置为获取待优化模型；其中，所述待优化模型包括至少一个模型面片，所述模型面片由模型顶点构成；A model acquisition module configured to acquire a model to be optimized; wherein, the model to be optimized includes at least one model patch, and the model patch is composed of model vertices;
    信息获取模块，配置为获取各个所述模型面片的属性信息和各个所述模型面片的第一法向信息，并获取各个所述模型顶点的第一位置信息；An information acquisition module configured to acquire the attribute information of each of the model faces and the first normal information of each of the model faces, and acquire the first position information of each of the model vertices;
    法向优化模块，配置为基于所述模型面片的属性信息优化所述模型面片的第一法向信息，得到所述模型面片的第二法向信息；The normal optimization module is configured to optimize the first normal information of the model surface based on the attribute information of the model surface, and obtain the second normal information of the model surface;
    顶点优化模块，配置为基于所述模型面片的第一法向信息和所述模型面片的第二法向信息优化所述模型顶点的第一位置信息，得到所述模型顶点的第二位置信息，对所述待优化模型进行优化。Vertex optimization module, configured to optimize the first position information of the model vertices based on the first normal information of the model facet and the second normal direction information of the model facet, to obtain the second position of the model vertices information to optimize the model to be optimized.
  12. 根据权利要求11所述的装置，其中，所述第一位置信息包括第一坐标，所述第二位置信息包括第二坐标；所述顶点优化模块，包括：The device according to claim 11, wherein the first location information includes first coordinates, and the second location information includes second coordinates; the vertex optimization module includes:
    参考选择子模块，配置为将各个所述模型顶点分别作为第一当前顶点，并选择一个与所述第一当前顶点之间存在连接边的模型顶点作为参考顶点，以及将共享所述连接边的两个模型面片分别作为第一面片和第二面片；The reference selection submodule is configured to use each of the model vertices as the first current vertex, select a model vertex that has a connection edge with the first current vertex as a reference vertex, and use the model vertices that share the connection edge The two model faces are respectively used as the first face and the second face;
    法向变换子模块，配置为基于所述第一面片的第一法向信息和所述第一面片的第二法向信息，得到所述第一面片的第一法向变换参数，并基于所述第二面片的第一法向信息和所述第二面片的第二法向信息，得到所述第二面片的第二法向变换参数；a normal transformation submodule configured to obtain a first normal transformation parameter of the first patch based on the first normal information of the first patch and the second normal information of the first patch, And based on the first normal information of the second patch and the second normal information of the second patch, obtain the second normal transformation parameter of the second patch;
    函数构建子模块，配置为基于所述第一法向变换参数和所述第二法向变换参数，构建所述第一当前顶点的目标函数；其中，所述目标函数包含第一参考子项和第二参考子项，所述第一参考子项为预设参考系数与所述第一当前顶点的第一坐标的乘积，且所述第二参考子项为预设参考系数与所述第一当前顶点的第二坐标的乘积；A function construction submodule configured to construct an objective function of the first current vertex based on the first normal transformation parameter and the second normal transformation parameter; wherein the objective function includes a first reference subitem and The second reference sub-item, the first reference sub-item is the product of the preset reference coefficient and the first coordinate of the first current vertex, and the second reference sub-item is the preset reference coefficient and the first the product of the second coordinates of the current vertex;
    函数求解子模块，配置为基于各个所述模型顶点的目标函数，求解得到各个所述模型顶点的第二坐标。The function solving submodule is configured to obtain the second coordinates of each of the model vertices based on the objective function of each of the model vertices.
  13. 根据权利要求12所述的装置，其中，所述函数构建子模块，包括：The apparatus according to claim 12, wherein said function building sub-module comprises:
    对角获取单元，配置为获取所述第一面片中所述连接边的第一对角，并获取所述第二面片中所述连接边的第二对角；a diagonal obtaining unit configured to obtain a first diagonal of the connected edges in the first patch, and obtain a second diagonal of the connected edges in the second patch;
    第一子项单元，配置为基于所述第一法向变换参数和所述第一对角、所述第二法向变换参数和所述第二对角，以及所述第一当前顶点的第一坐标、所述参考顶点的第一坐标，构建第一优化子项；The first sub-item unit is configured to be based on the first normal transformation parameter and the first diagonal, the second normal transformation parameter and the second diagonal, and the first current vertex One coordinate, the first coordinate of the reference vertex, construct the first optimization sub-item;
    第二子项单元，配置为基于所述第一对角、所述第二对角，以及所述第一当前顶点的第二坐标、所述参考顶点的第二坐标，构建第二优化子项；The second sub-item unit is configured to construct a second optimization sub-item based on the first diagonal, the second diagonal, the second coordinates of the first current vertex, and the second coordinates of the reference vertex ;
    函数构建单元，配置为基于所述第一优化子项、所述第一参考子项、所述第二优化子项和所述第二参考子项，得到所述第一当前顶点的目标函数。A function construction unit configured to obtain the objective function of the first current vertex based on the first optimization sub-item, the first reference sub-item, the second optimization sub-item and the second reference sub-item.
  14. 根据权利要求12所述的装置，其中，所述函数求解子模块，包括：The device according to claim 12, wherein the function solving submodule comprises:
    方程联立单元，配置为基于各个所述模型顶点的目标函数，联立关于各个所述模型顶点的第二坐标的方程组；An equation simultaneous unit configured to simultaneously establish a system of equations about the second coordinates of each of the model vertices based on the objective function of each of the model vertices;
    方程求解单元，配置为对所述方程组进行求解，得到各个所述模型顶点的第二坐标。The equation solving unit is configured to solve the equation system to obtain the second coordinates of each vertex of the model.
  15. 根据权利要求11至14中任一项所述的装置，其中，所述模型优化装置，包括：The device according to any one of claims 11 to 14, wherein the model optimization device comprises:
    损失计算模块，配置为在检测到当前优化阶段满足预设条件的情况下，基于各个所述模型面片的属性信息、各个所述模型面片的第一法向信息和各个所述模型面片的第二法向信息，获取各个所述模型面片的优化损失；The loss calculation module is configured to, based on the attribute information of each of the model patches, the first normal information of each of the model patches, and the The second normal information of each of the model faces is obtained to obtain the optimization loss;
    面片选择模块，配置为基于所述优化损失，选择至少一个所述模型面片作为当前优化阶段的待细分面片；A patch selection module configured to select at least one model patch as a patch to be subdivided in the current optimization stage based on the optimization loss;
    模型细分模块，配置为基于所述待细分面片，对所述待优化模型进行细分，得到细分后的待优化模型，更新所述待优化模型的所述模型面片；The model subdivision module is configured to subdivide the model to be optimized based on the mesh to be subdivided, obtain the subdivided model to be optimized, and update the model mesh of the model to be optimized;
    所述信息获取模块，配置为获取更新后的所述待优化模型中各个所述模型面片的属性信息和各个所述模型面片的第一法向信息，并获取更新后的所述待优化模型中各个所述模型顶点的第一位置信息。The information acquisition module is configured to acquire the attribute information of each of the model patches in the updated model to be optimized and the first normal information of each of the model patches, and acquire the updated The first position information of each model vertex in the model.
  16. 根据权利要求15所述的装置，其中，所述面片选择模块，包括：The device according to claim 15, wherein the patch selection module comprises:
    排序子模块，配置为按照所述优化损失从大到小的顺序，将所述模型面片进行排序；The sorting submodule is configured to sort the model patches in descending order of the optimization loss;
    选择子模块，配置为选择位于前预设序位的所述模型面片，作为所述待细分面片；或者，选择所述优化损失大于第一阈值的所述模型面片作为所述待细分面片。The selection submodule is configured to select the model patch located in the previous pre-ordered position as the patch to be subdivided; or select the model patch whose optimization loss is greater than a first threshold as the patch to be subdivided. Subdivide the dough.
  17. 根据权利要求15所述的装置，其中，所述模型细分模块，包括：The device according to claim 15, wherein the model subdivision module comprises:
    顶点新增子模块，配置为在所述待细分面片的边上新增至少一个所述模型顶点；The vertex adding submodule is configured to add at least one model vertex on the edge of the mesh to be subdivided;
    集合获取子模块，配置为分别将各个新增的模型顶点作为第二当前顶点，并获取所述第二当前顶点的候选顶点集合；其中，所述候选顶点集合包括至少一个候选顶点，且各个所述候选顶点位于各个候选面片上，各个所述候选面片均包含所述第二当前顶点所在的边；The collection acquisition sub-module is configured to use each newly added model vertex as a second current vertex, and obtain a set of candidate vertices of the second current vertex; wherein, the set of candidate vertices includes at least one candidate vertex, and each of the The candidate vertex is located on each candidate face, and each of the candidate faces includes the edge where the second current vertex is located;
    顶点连接子模块，配置为在所述候选顶点集合中选择至少一个所述候选顶点，并分别连接所述至少一个所述候选顶点与所述第二当前顶点，得到细分后的待优化模型。The vertex connection sub-module is configured to select at least one candidate vertex in the candidate vertex set, and respectively connect the at least one candidate vertex and the second current vertex to obtain a subdivided model to be optimized.
  18. 根据权利要求15所述的装置，其中，所述顶点优化模块，配置为：The device according to claim 15, wherein the vertex optimization module is configured to:
    在检测到当前优化阶段不满足预设条件的情况下，基于所述模型面片的第一法向信息和所述模型面片的第二法向信息优化所述模型顶点的第一位置信息，得到所述模型顶点的所述第二位置信息，对所述待优化模型进行优化。When it is detected that the current optimization stage does not satisfy the preset condition, optimizing the first position information of the model vertices based on the first normal information of the model patch and the second normal information of the model patch, Obtain the second position information of the vertices of the model, and optimize the model to be optimized.
  19. 根据权利要求15所述的装置，其中，所述模型优化装置，包括:The device according to claim 15, wherein the model optimization device comprises:
    面片筛选模块，配置为筛选面片面积大于或等于第二阈值的待细分面片；A patch screening module configured to filter patches whose patch area is greater than or equal to the second threshold;
    所述模型细分模块，配置为基于筛选得到的所述待细分面片，对所述待优化模型进行细分，得到所述细分后的待优化模型。The model subdivision module is configured to subdivide the model to be optimized based on the face pieces to be subdivided obtained through screening, to obtain the subdivided model to be optimized.
  20. 根据权利要求15所述的装置，其中，所述当前优化阶段满足预设条件包括：所述当前优化阶段未达到预设数值个优化阶段。The device according to claim 15, wherein the current optimization stage meeting a preset condition comprises: the current optimization stage does not reach a preset number of optimization stages.
  21. 一种电子设备，包括：相互耦接的存储器和处理器，所述处理器配置为执行所述存储器中存储的程序指令，以实现权利要求1至10中任一项所述的模型优化方法。An electronic device, comprising: a memory coupled to each other and a processor, the processor configured to execute program instructions stored in the memory, so as to implement the model optimization method according to any one of claims 1 to 10.
  22. 一种计算机可读存储介质，其上存储有程序指令，所述程序指令被处理器执行时实现权利要求1至10中任一项所述的模型优化方法。A computer-readable storage medium, on which program instructions are stored, and when the program instructions are executed by a processor, the model optimization method according to any one of claims 1 to 10 is implemented.
  23. 一种计算机程序，包括计算机可读代码，在计算机可读代码在设备上运行的情况下，设备中的处理器执行用于实现权利要求1至10中任一所述的方法。A computer program comprising computer readable codes, in case the computer readable codes run on the device, the processor in the device executes to implement the method according to any one of claims 1 to 10.
  24. 一种计算机程序产品，配置为存储计算机可读指令，所述计算机可读指令被执行时使得计算机执行权利要求1至10中任一所述的方法。A computer program product configured to store computer-readable instructions that, when executed, cause a computer to perform the method of any one of claims 1-10.
PCT/CN2021/144033 2021-08-26 2021-12-31 Method and apparatus for model optimization, electronic device, storage medium, computer program, and computer program product WO2023024395A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202110988927.0A CN113706431B (en) 2021-08-26 2021-08-26 Model optimization method and related device, electronic equipment and storage medium
CN202110988927.0 2021-08-26

Publications (1)

Publication Number Publication Date
WO2023024395A1 true WO2023024395A1 (en) 2023-03-02

Family

ID=78655282

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/144033 WO2023024395A1 (en) 2021-08-26 2021-12-31 Method and apparatus for model optimization, electronic device, storage medium, computer program, and computer program product

Country Status (2)

Country Link
CN (1) CN113706431B (en)
WO (1) WO2023024395A1 (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117197000A (en) * 2023-11-06 2023-12-08 武汉中观自动化科技有限公司 Quick grid denoising method and device and electronic equipment

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113706431B (en) * 2021-08-26 2022-10-21 深圳市慧鲤科技有限公司 Model optimization method and related device, electronic equipment and storage medium

Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108053484A (en) * 2017-12-12 2018-05-18 武汉大学 A kind of constant mean curvature Surface Construction method average based on normal direction
CN108492370A (en) * 2018-03-02 2018-09-04 中国地质大学（武汉） Triangle gridding filtering method based on TV and anisotropy Laplacian regular terms
CN109242972A (en) * 2018-08-14 2019-01-18 重庆大学 A kind of double normal mesh model method for fairing based on apex feature
CN111028356A (en) * 2019-11-25 2020-04-17 中国地质大学（武汉） Optimization method based on non-convex non-smooth second-order regular term and sparse fidelity term
CN111640188A (en) * 2020-05-29 2020-09-08 中国地质大学（武汉） Anti-noise three-dimensional grid optimization method based on Mumford-Shah algorithm framework
US20210097703A1 (en) * 2019-09-26 2021-04-01 Medit Corp. Apparatus and method for aligning 3-dimensional data
CN112862736A (en) * 2021-02-05 2021-05-28 浙江大学 Real-time three-dimensional reconstruction and optimization method based on points
CN113178013A (en) * 2021-05-24 2021-07-27 广东奥普特科技股份有限公司 Triangular mesh filtering method and device, electronic equipment and storage medium
CN113706431A (en) * 2021-08-26 2021-11-26 深圳市慧鲤科技有限公司 Model optimization method and related device, electronic equipment and storage medium

Family Cites Families (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111369659B (en) * 2018-12-26 2023-05-02 杭州海康威视数字技术股份有限公司 Texture mapping method, device and equipment based on three-dimensional model
US20200334890A1 (en) * 2019-04-22 2020-10-22 University Of Kentucky Research Foundation Systems and Methods for Fast Texture Mapping of 3D Models
CN111243071A (en) * 2020-01-08 2020-06-05 叠境数字科技（上海）有限公司 Texture rendering method, system, chip, device and medium for real-time three-dimensional human body reconstruction
CN111882666B (en) * 2020-07-20 2022-06-21 浙江商汤科技开发有限公司 Method, device and equipment for reconstructing three-dimensional grid model and storage medium
CN112489230B (en) * 2020-12-09 2023-03-24 贝壳技术有限公司 Color matching method and device for patch model and storage medium
CN112862933B (en) * 2021-02-04 2023-06-27 北京百度网讯科技有限公司 Method, apparatus, device and storage medium for optimizing model
CN113052774B (en) * 2021-03-30 2022-10-21 上海商汤智能科技有限公司 Image optimization method, training method of related model, related device and equipment

Patent Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108053484A (en) * 2017-12-12 2018-05-18 武汉大学 A kind of constant mean curvature Surface Construction method average based on normal direction
CN108492370A (en) * 2018-03-02 2018-09-04 中国地质大学（武汉） Triangle gridding filtering method based on TV and anisotropy Laplacian regular terms
CN109242972A (en) * 2018-08-14 2019-01-18 重庆大学 A kind of double normal mesh model method for fairing based on apex feature
US20210097703A1 (en) * 2019-09-26 2021-04-01 Medit Corp. Apparatus and method for aligning 3-dimensional data
CN111028356A (en) * 2019-11-25 2020-04-17 中国地质大学（武汉） Optimization method based on non-convex non-smooth second-order regular term and sparse fidelity term
CN111640188A (en) * 2020-05-29 2020-09-08 中国地质大学（武汉） Anti-noise three-dimensional grid optimization method based on Mumford-Shah algorithm framework
CN112862736A (en) * 2021-02-05 2021-05-28 浙江大学 Real-time three-dimensional reconstruction and optimization method based on points
CN113178013A (en) * 2021-05-24 2021-07-27 广东奥普特科技股份有限公司 Triangular mesh filtering method and device, electronic equipment and storage medium
CN113706431A (en) * 2021-08-26 2021-11-26 深圳市慧鲤科技有限公司 Model optimization method and related device, electronic equipment and storage medium

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
ZHANG ZHANG CHUNSEN CHUNSEN, ZHANG MENGMENG, GUO BINGXUAN: "Refinement of the 3D Mesh Model Driven by the Image Information", ACTA GEODAETICA ET CARTOGRAPHICA SINICA, vol. 47, no. 7, 1 July 2018 (2018-07-01), pages 959 - 967, XP009543804, ISSN: 1001-1595, DOI: 10.11947/j.AGCS.2018.20170733 *

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117197000A (en) * 2023-11-06 2023-12-08 武汉中观自动化科技有限公司 Quick grid denoising method and device and electronic equipment
CN117197000B (en) * 2023-11-06 2024-03-19 武汉中观自动化科技有限公司 Quick grid denoising method and device and electronic equipment

Also Published As

Publication number Publication date
CN113706431B (en) 2022-10-21
CN113706431A (en) 2021-11-26

Similar Documents

Publication Publication Date Title
US10679061B2 (en) Tagging virtualized content
US11210838B2 (en) Fusing, texturing, and rendering views of dynamic three-dimensional models
Du et al. Montage4D: interactive seamless fusion of multiview video textures.
US11688145B2 (en) Virtualizing content
Li et al. Detail-preserving and content-aware variational multi-view stereo reconstruction
US9330500B2 (en) Inserting objects into content
KR20190129985A (en) Robust mesh tracking and fusion using part-based key frames and a priori model
WO2023024395A1 (en) Method and apparatus for model optimization, electronic device, storage medium, computer program, and computer program product
Vicente et al. Balloon shapes: Reconstructing and deforming objects with volume from images
WO2023024441A1 (en) Model reconstruction method and related apparatus, and electronic device and storage medium
Boom et al. Interactive light source position estimation for augmented reality with an RGB‐D camera
Wang et al. Understanding of wheelchair ramp scenes for disabled people with visual impairments
CN109785429A (en) A kind of method and apparatus of three-dimensional reconstruction
Han et al. Real-time globally consistent dense 3D reconstruction with online texturing
Nguyen et al. High resolution 3d content creation using unconstrained and uncalibrated cameras
Aleksandrova et al. 3D face model reconstructing from its 2D images using neural networks
Kim et al. Planar Abstraction and Inverse Rendering of 3D Indoor Environments
US11830140B2 (en) Methods and systems for 3D modeling of an object by merging voxelized representations of the object
Sahloul et al. Foreground segmentation with efficient selection from icp outliers in 3d scene
Cooper Robust generation of 3D models from video footage of urban scenes
Worthington Reillumination-driven shape from shading
Kang et al. Near-real-time stereo matching method using temporal and spatial propagation of reliable disparity
Graber Realtime 3D reconstruction
Huang Surface Texture Processing
Malleson Dynamic scene modelling and representation from video and depth

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21954909

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE