CN110021069B - Three-dimensional model reconstruction method based on grid deformation - Google Patents
Three-dimensional model reconstruction method based on grid deformation Download PDFInfo
- Publication number
- CN110021069B CN110021069B CN201910298089.7A CN201910298089A CN110021069B CN 110021069 B CN110021069 B CN 110021069B CN 201910298089 A CN201910298089 A CN 201910298089A CN 110021069 B CN110021069 B CN 110021069B
- Authority
- CN
- China
- Prior art keywords
- dimensional
- visual angle
- discrete
- loss function
- grid
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
- G06T17/20—Finite element generation, e.g. wire-frame surface description, tesselation
- G06T17/205—Re-meshing
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Software Systems (AREA)
- General Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Biomedical Technology (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Biophysics (AREA)
- Mathematical Physics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Health & Medical Sciences (AREA)
- Computer Graphics (AREA)
- Geometry (AREA)
- Image Analysis (AREA)
- Image Processing (AREA)
Abstract
The invention provides a three-dimensional model reconstruction method based on grid deformation, which is used for constructing a training sample set and comprises the steps of manufacturing discrete visual angle pictures and corresponding three-dimensional point cloud data of a plurality of models; setting a deep learning network model based on a graph convolution neural network, wherein the deep learning network model based on the graph convolution neural network comprises a discrete visual angle feature fusion module and a grid deformation module, and the output of the discrete visual angle feature fusion module is connected with the input of the matched grid deformation module; setting a loss function, and training a deep learning network model based on a graph convolution neural network based on a training sample set; and inputting the discrete visual angle picture of the object to be reconstructed to the network model obtained by training, automatically reconstructing the three-dimensional network model and evaluating the precision. The method can support the stable and accurate automatic three-dimensional mesh model reconstruction of objects of different types and sizes by learning and training the discrete visual angle picture and the three-dimensional point cloud data set of the object.
Description
Technical Field
The invention belongs to the field of three-dimensional reconstruction, and particularly relates to a three-dimensional model reconstruction method based on grid deformation.
Background
Three-dimensional reconstruction is a very challenging problem that has been studied in the field of computer vision for decades. Traditional multi-view geometry-based methods, such as many SFM and SLAM algorithms, involve a complex series of processes including feature extraction, feature matching, matching point triangulation, and the like. Thus, this approach is not robust for challenging scenarios where feature extraction or matching cannot be done efficiently. Furthermore, their three-dimensional reconstruction results are typically a sparse reconstruction that cannot be used directly. To overcome these limitations, some learning-based approaches have emerged. And performing depth regression estimation on the scene by using the visual image characteristics of the depth to obtain a dense reconstruction result. However, similar to the earlier conventional methods, these methods still require that the input images have sufficient overlap area, which is somewhat awkward and inconvenient in practical applications.
With the rapid development of deep learning, many neural networks that infer a three-dimensional shape from only a single image have been proposed in recent years. Voxels are one of the most common methods for expressing three-dimensional models. Many methods propose extending the traditional two-dimensional convolutional neural network to a three-dimensional version and estimating a voxelized three-dimensional model from one RGB image. Unfortunately, these methods are generally limited by the high computational complexity and memory overhead of the three-dimensional convolutional neural network, and therefore the limited resolution of point cloud voxelization cannot reconstruct the structural details of the object. Another way to express a three-dimensional model is to use directly the disordered point cloud. In this expression, the network usually adopts an encoder-decoder architecture, and acquires geometric information from the RGB image at the encoder stage, and then regresses point cloud coordinates at the decoder stage. Usually, the point cloud is introduced into the network in the form of a set of unordered vectors, but the conventional convolutional neural network can only process ordered inputs, which results in the output point cloud lacking the geometric information of neighboring three-dimensional points. Recently, some methods of deep learning have proposed classification and analysis of three-dimensional models in non-euclidean space. This idea is naturally applied to reconstruct a three-dimensional model of an object directly from a single RGB image. The three-dimensional mesh model output by the method is generated by the progressive deformation of an initial mesh ellipsoid under a graph convolution based neural network (GCN) framework. This kind of inferring a three-dimensional model of an object from only one RGB image is certainly very attractive, but the above-described approaches are inherently limited to a single view. Since the geometry prior obtained from an image from only one angle is often too limited to accurately reconstruct the geometry of the object not seen in the picture.
Disclosure of Invention
To solve the problems of the above methods, the present invention finds a compromise, and proposes a discrete view mesh generation network (DMGNet) that can reconstruct a three-dimensional mesh model from a discrete view RGB image. On the one hand, the invention is more flexible than the traditional multi-view geometry approach based on multiple views, since it does not require any overlap of the input images. On the other hand, the invention simultaneously utilizes the fusion information of the discrete views, and can obtain more accurate model reconstruction results compared with the method based on the single view.
The invention provides a three-dimensional model reconstruction method based on grid deformation, which comprises the following steps:
the discrete perspective feature fusion module comprises 6 convolution calculation blocks Block1, Block2, Block3, Block4, Block5 and Block6, each Block having 2-3 convolution layers, respectively denoted as Block1_ conv1, Block1_ conv2, Block2_ conv1, Block2_ conv2, Block2_ conv3, Block3_ conv1, Block3_ conv2, Block2_ conv 2; respectively projecting the vertexes of the initial three-dimensional shape mesh in 4 layers of image feature graphs, namely block3_ conv3, block4_ conv3, block5_ conv3 and block6_ conv3, obtained from each visual angle by using camera external parameters of each discrete visual angle, and respectively obtaining image features corresponding to the vertex projection positions in each layer of image feature graph; fusing the image characteristics obtained under all the visual angles, adding the three-dimensional shape characteristics of the vertex as the multi-visual angle fusion characteristics of the vertex, and inputting the multi-visual angle fusion characteristics into a mesh deformation module;
the grid deformation module is formed by stacking 14 graph volume layers, and short connection is added between every two graph volume layers;
and 4, inputting the discrete visual angle picture of the object to be reconstructed to the deep learning network model based on the graph convolution neural network obtained by training in the step 3, automatically reconstructing the three-dimensional grid model and evaluating the precision.
In the discrete view angle feature fusion module, when the vertex of the initial three-dimensional shape mesh is respectively projected in each view angle to obtain the image feature map, the vertex can fall in a certain mesh on each layer of image feature map, and in order to ensure the accuracy of the image feature obtained by projection, a bilinear interpolation method is used for obtaining the image feature corresponding to the projection position of the vertex in each layer of image feature map.
In step 3, a discrete view back projection error is defined, and an error loss function is established as follows:
wherein, IiRepresenting an input picture of an ith visual angle, wherein p 'and q' are a vertex p on an output three-dimensional grid and a point q on sample real point cloud data of the nearest neighbor of the vertex p, and a two-dimensional projection point is obtained by back projection on the picture by utilizing corresponding camera parameters; i isi(p') and Ii(q ') is the corresponding pixel value of the two-dimensional projection point p ', q ' on the input picture of the ith view angle, and the formula is to count the error of the corresponding pixel value under all discrete view angles
Accumulated as total back projection error.
Furthermore, in step 3, the Chamfer loss function is defined as follows,
wherein S is1And S2Respectively representing the set of vertices of the reconstructed mesh and the set of real point clouds of the model, for the set S1At each point x of the otherA set S2To find the spatially closest point y, for the set S2Each point y of (a) in another set S1Finding the nearest point x in the space, and summing the squares of the distances to obtain the Chamfer distance error dCD(S1,S2) As a function of Chamfer loss.
And in the step 3, using a random gradient descent algorithm to iteratively solve the network parameters of the minimized error loss function and the Chamfer loss function to obtain a trained network model.
And in step 4, quantitative evaluation is carried out on the three-dimensional grid model reconstruction result by using the Charfer distance error and the F-score.
According to the invention, by utilizing a deep learning technology, through learning and training discrete visual angle pictures and three-dimensional point cloud data of objects, the deep learning model obtained through final training can accurately reconstruct three-dimensional models of objects of different types and sizes. Compared with the prior art, the invention has the following advantages:
(1) the learning-based method avoids the complex and time-consuming process of the traditional multi-vision geometry;
the invention uses a deep learning method to automatically learn the discrete visual angle picture of the object and the training sample formed by the corresponding three-dimensional model, thereby obtaining a network which can directly generate the three-dimensional grid model from the picture. The method does not need a series of operation processes such as feature extraction, feature matching, matching point triangularization and the like, and has the characteristics of convenience and rapidness. And the input pictures are not required to have certain overlapping degree, so the number of the required pictures is small, and the data acquisition has higher flexibility.
(2) The problem that the three-dimensional model format output by the traditional convolution network cannot be directly put into production and use is effectively solved;
the invention utilizes a new graph convolution neural network structure to directly deform on the three-dimensional grid, the finally output model format is triangular grid, compared with the traditional convolution network reconstruction result which is disordered point cloud or voxel grid, the three-dimensional model expression mode of the invention is close to the actual three-dimensional model production operation, and can be directly put into use.
(3) The problem that an invisible structure cannot be well reconstructed from a single picture is solved;
according to the invention, by designing a discrete visual angle feature fusion module, more geometric prior information can be obtained from a limited number of discrete pictures compared with a single-view picture, so that a part of object structure which cannot be seen in a single picture can be better reconstructed, the applicability of the invention is stronger, and the geometric structure of an object to be reconstructed is not required to have strong symmetry.
Drawings
Fig. 1 is a three-dimensional network model structure diagram based on a graph convolution neural network according to an embodiment of the present invention.
Fig. 2 is a structural diagram of a discrete view feature fusion module according to an embodiment of the present invention.
FIG. 3 is a schematic diagram of a discrete-view back projection error according to an embodiment of the present invention.
Fig. 4 is a structural diagram of a mesh deformation module according to an embodiment of the present invention.
Detailed Description
In order to facilitate the understanding and implementation of the present invention for those of ordinary skill in the art, the present invention is further described in detail with reference to the accompanying drawings and examples, it is to be understood that the embodiments described herein are merely illustrative and explanatory of the present invention and are not restrictive thereof.
The deep learning network only takes images of a plurality of visual angles as input and outputs a reconstructed three-dimensional grid model. As shown in fig. 1, the basic module of the network of the present invention mainly comprises 2 parts: (1) a grid deformation module based on a graph convolution neural network, and (2) a discrete visual angle image feature fusion module. With the continuous learning process of the network, the grid deformation module based on graph convolution gradually deforms the initial three-dimensional shape grid into the three-dimensional shape of the target object. The discrete visual angle image feature fusion module obtains two-dimensional image features from the discrete visual angle images respectively, then uses the camera external parameters of all visual angles to obtain the corresponding features of each grid vertex on the two-dimensional feature graphs of different visual angles through projection, and fuses multi-visual angle information as the shape features of the vertex in the grid deformation module. According to the invention, the basic network is stacked, and graph-based pooling operation is added among modules to increase the number of grid vertexes, so that the coarse-to-fine refinement process of the deformed grid is realized, and the reconstructed three-dimensional grid model is finally obtained.
The embodiment of the invention provides a three-dimensional model reconstruction method based on grid deformation, which comprises the following steps:
In an embodiment, the following sub-steps are included:
step 1.1, simulating shooting of an object on a Shapelet data set at different angles, and obtaining picture data of discrete visual angles.
The open source data set ShapeNet contains 3D models and class labels for 3D models, covering the commonly used 3D data set. Embodiments create a training sample set based on a ShapeNet data set.
And step 1.2, acquiring corresponding original three-dimensional point cloud data by uniformly sampling on the model.
And 1.3, packing the discrete visual angle picture and the point cloud data of the same model to construct a training sample set.
And 2, setting a deep learning network model based on the graph convolution neural network.
In the embodiment, a three-dimensional mesh model generation network based on a graph convolution neural network as shown in fig. 1 is designed, the network mainly comprises a discrete visual angle feature fusion module and a mesh deformation module, and the output of the discrete visual angle feature fusion module is connected with the input of the mesh deformation module.
The method specifically comprises the following substeps:
and 2.1, uniformly adjusting the sizes of the discrete view angle pictures to 224 multiplied by 224, and carrying out normalization processing on pixel values. They are input to a discrete perspective feature fusion module.
Step 2.2, setting a discrete visual angle feature fusion module: as shown in the discrete view feature fusion module in fig. 2, 6 convolution calculation blocks Block1, Block2, Block3, Block4, Block5 and Block6 are included, each Block having 2-3 convolution layers, respectively denoted as Block1_ conv1, Block1_ conv2, Block2_ conv1, Block2_ conv2, Block2_ conv3, Block3_ conv1, Block3_ conv2, Block2_ conv 2_ 2. The initial three-dimensional shape mesh of the object in the embodiment employs an initial ellipsoid mesh. The initial ellipsoid grid in the figure is generated by Meshlab software, and has 156 vertexes, and the three axial lengths are 0.2 meter, 0.2 meter and 0.4 meter respectively. The cameras for view 1, view 2 and view 3 can be placed at the vertices of an equilateral triangular support centered on the object (initial ellipsoid grid). The vertices of the initial three-dimensional shape mesh are respectively projected in the 4-layer image feature maps of Block3_ conv3, Block4_ conv3, Block5_ conv3 and Block6_ conv3 obtained from each view angle by using camera external references of each discrete view angle. On each layer of the feature map, the vertices fall within a certain mesh. In order to ensure the accuracy of the image features obtained by projection, a bilinear interpolation method is used to obtain the corresponding image features at the vertex projection. After the 4 layers of image feature vectors with different dimensions obtained by the vertex projection are obtained, the image features of the vertices with total channel number 960 under the view angle are obtained through cascading. And fusing the image characteristics obtained under all the visual angles, adding the three-dimensional shape characteristics of the vertex, taking the three-dimensional shape characteristics as the multi-visual angle fusion characteristics of the vertex, and inputting the multi-visual angle fusion characteristics into the mesh deformation module.
Step 2.3, setting a grid deformation module: stacked from 14 map convolutional layers, short connections are added between every other two map convolutional layers in order to alleviate the difficult training problem with the deepening of the network depth and to make the information exchange more efficient. And updating the vertex coordinates of the mesh to achieve mesh deformation through the stacked graph convolution operation, so that the shape of the mesh is gradually fitted to the target model.
Referring to fig. 4, 14 of the graph convolutional layers, the output of layer 1 is connected to the output of layer 3, the output of layer 3 is connected to the output of layer 5, the output of layer 5 is connected to the output of layer 7, the output of layer 7 is connected to the output of layer 9, the output of layer 9 is connected to the output of layer 11, and the output of layer 11 is connected to the output of layer 13.
Namely:
L3＝(L1+L3)/2
L5＝(L3+L5)/2
L7＝(L5+L7)/2
…
L13＝(L11+L13)/2
the right Lc of the equation is the output of the graph convolution layer c before the short connection is increased, the left Lc of the equation is the output after the short connection is increased, and c is 3, 5, 7, … 13.
And 3, setting a loss function, and training the deep learning network model based on the graph convolution neural network based on the training sample set.
In an embodiment, the loss function is composed of an error loss function and a Chamfer loss function, and the step 3 implementation includes the following sub-steps:
step 3.1, defining a discrete view angle back projection error, and establishing an error loss function as follows:
as shown in FIG. 3, wherein IiAnd representing an input picture of the ith visual angle, wherein p 'and q' are a vertex p on the output three-dimensional grid and a point q on sample real point cloud data of the nearest neighbor, and a two-dimensional projection point is obtained by back projection on the picture by using corresponding camera parameters. E.g. with a vertex p in fig. 31,p2,p3,p4,p5Corresponding to the cloud data q1,q2,q3,q4,q5。Ii(p') and Ii(q ') is the corresponding pixel value of the two-dimensional projection point p ', q ' on the input picture of the ith view angle, and the formula is to count the error of the corresponding pixel value under all discrete view angles
Accumulated as total back projection error.
Defining: charfer loss function
Wherein S is1And S2Respectively representing the set of vertices of the reconstructed mesh and the set of real point clouds of the model, for the set S1Each point x of (2) in another set S2To find the spatially closest point y, for the set S2Each point y of (a) in another set S1Finding the nearest point x in the space, and summing the squares of the distances to obtain the Chamfer distance error dCD(S1,S2) This is taken as a function of the Chamfer loss.
The sum of the error loss function and the Chamfer loss function as the total loss.
And 3.2, iteratively solving network parameters of the minimized error loss function and the Chamfer loss function by using a random gradient descent algorithm to obtain a trained network model. The stochastic gradient descent algorithm is prior art and is not described in detail herein.
step 4.1, unifying the size of the discrete visual angle picture data to be reconstructed and normalizing the pixel value;
step 4.2, inputting the test data into the trained network model to obtain a three-dimensional grid model reconstruction result;
and 4.3, carrying out quantitative evaluation on the final reconstruction result by using the Chamfer distance error and the F-fraction. The calculation formula of the Chamfer distance error is the same as the Chamfer loss function.
The formula for calculating the F-fraction is as follows:
f (d) represents an F-score, d represents a threshold of an euclidean distance between two points in space, and a value can be preset in specific implementation, for example, 0.0001 is taken, p (d) represents a percentage of points with a distance smaller than d in a real point cloud set in a reconstructed mesh vertex set, and r (d) represents a percentage of points with a distance smaller than d in a reconstructed mesh vertex set in a real point cloud set. The F-score measures how close the reconstructed mesh vertex is to the real point cloud and also measures the integrity of the reconstructed mesh vertex.
According to the method and the device for reconstructing the three-dimensional model of the picture of the actual object, the three-dimensional model of the picture of the actual object can be reconstructed stably and accurately.
In specific implementation, the above technical solutions can adopt computer software technology to implement automatic operation process, and a hardware device for operating the method should also be within the protection scope of the present invention.
It should be understood that the above description of the preferred embodiments is given for clarity and not for any purpose of limitation, and that various changes, substitutions and alterations can be made herein without departing from the spirit and scope of the invention as defined by the appended claims.
Claims (6)
1. A three-dimensional model reconstruction method based on grid deformation is characterized by comprising the following steps:
step 1, constructing a training sample set, including making discrete visual angle pictures of a plurality of models and corresponding three-dimensional point cloud data;
step 2, setting a deep learning network model based on a graph convolution neural network, wherein the deep learning network model based on the graph convolution neural network comprises a discrete visual angle feature fusion module and a grid deformation module, and the output of the discrete visual angle feature fusion module is connected with the input of the matched grid deformation module;
the discrete perspective feature fusion module comprises 6 convolution calculation blocks Block1, Block2, Block3, Block4, Block5 and Block6, each Block having 2-3 convolution layers, respectively denoted as Block1_ conv1, Block1_ conv2, Block2_ conv1, Block2_ conv2, Block2_ conv3, Block3_ conv1, Block3_ conv2, Block2_ conv 2; respectively projecting the vertexes of the initial three-dimensional shape mesh in 4 layers of image feature graphs, namely block3_ conv3, block4_ conv3, block5_ conv3 and block6_ conv3, obtained from each visual angle by using camera external parameters of each discrete visual angle, and respectively obtaining image features corresponding to the vertex projection positions in each layer of image feature graph; fusing the image characteristics obtained under all the visual angles, adding the three-dimensional shape characteristics of the vertex as the multi-visual angle fusion characteristics of the vertex, and inputting the multi-visual angle fusion characteristics into a mesh deformation module;
the grid deformation module is formed by stacking 14 graph volume layers, and short connection is added between every two graph volume layers;
step 3, setting a loss function, and training a deep learning network model based on a graph convolution neural network based on a training sample set; the loss function consists of an error loss function and a Chamfer loss function, the error loss function is established according to the back projection error of the discrete view angle, and the Chamfer loss function is established according to the Chamfer distance error between the reconstructed grid vertex set and the real point cloud set;
and 4, inputting the discrete visual angle picture of the object to be reconstructed to the deep learning network model based on the graph convolution neural network obtained by training in the step 3, automatically reconstructing the three-dimensional grid model and evaluating the precision.
2. The mesh deformation-based three-dimensional model reconstruction method according to claim 1, wherein: in the discrete visual angle feature fusion module, when the vertexes of the initial three-dimensional shape mesh are respectively projected in each visual angle to obtain an image feature map, the vertexes all fall into a certain mesh on each layer of the image feature map, and in order to ensure the accuracy of the image features obtained by projection, a bilinear interpolation method is used for obtaining the image features corresponding to the vertex projection positions in each layer of the image feature map.
3. The mesh deformation-based three-dimensional model reconstruction method according to claim 1 or 2, characterized in that: in step 3, defining a discrete view back projection error, and establishing an error loss function as follows:
wherein Ii represents an input picture of the ith view angle, p 'and q' are a vertex p on the output three-dimensional grid and a point q on sample real point cloud data nearest to the vertex p, and a two-dimensional projection point is obtained by back projection on the picture by using corresponding camera parameters; ii (p ') and Ii (q') are corresponding pixel values of the two-dimensional projection points p ', q' on the input picture of the ith view angle, and the formula calculates the error of the corresponding pixel values at all discrete view angles
Accumulated as total back projection error.
4. The mesh deformation-based three-dimensional model reconstruction method according to claim 1 or 2, characterized in that: in step 3, the Chamfer loss function is defined as follows,
wherein S is1And S2Respectively representing the set of vertices of the reconstructed mesh and the set of real point clouds of the model, for the set S1Each point x of (2) in another set S2To find the spatially closest point y, for the set S2Each point y of (a) in another set S1In finding a nullThe nearest adjacent point x is positioned between the adjacent points, and the squares of the distances are summed to obtain the Chamfer distance error dCD(S1，S2) As a function of Chamfer loss.
5. The mesh deformation-based three-dimensional model reconstruction method according to claim 1 or 2, characterized in that: in the step 3, network parameters of the minimized error loss function and the Chamfer loss function are solved iteratively by using a random gradient descent algorithm to obtain a trained network model.
6. The mesh deformation-based three-dimensional model reconstruction method according to claim 1 or 2, characterized in that: and 4, carrying out quantitative evaluation on the three-dimensional grid model reconstruction result by using the Charfer distance error and the F-fraction.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910298089.7A CN110021069B (en) | 2019-04-15 | 2019-04-15 | Three-dimensional model reconstruction method based on grid deformation |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910298089.7A CN110021069B (en) | 2019-04-15 | 2019-04-15 | Three-dimensional model reconstruction method based on grid deformation |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110021069A CN110021069A (en) | 2019-07-16 |
| CN110021069B true CN110021069B (en) | 2022-04-15 |
Family
ID=67191317
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910298089.7A Active CN110021069B (en) | 2019-04-15 | 2019-04-15 | Three-dimensional model reconstruction method based on grid deformation |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110021069B (en) |
Families Citing this family (23)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113096234B (en) * | 2019-12-23 | 2022-09-06 | 复旦大学 | Method and device for generating three-dimensional grid model by using multiple color pictures |
| CN110751730B (en) * | 2019-07-24 | 2023-02-24 | 叠境数字科技（上海）有限公司 | Dressing human body shape estimation method based on deep neural network |
| CN110443842B (en) * | 2019-07-24 | 2022-02-15 | 大连理工大学 | Depth map prediction method based on visual angle fusion |
| CN110443892B (en) * | 2019-07-25 | 2021-06-04 | 北京大学 | Three-dimensional grid model generation method and device based on single image |
| CN110458957B (en) * | 2019-07-31 | 2023-03-10 | 浙江工业大学 | Image three-dimensional model construction method and device based on neural network |
| CN110570522B (en) * | 2019-08-22 | 2023-04-07 | 天津大学 | Multi-view three-dimensional reconstruction method |
| CN111340866B (en) * | 2020-02-26 | 2024-03-01 | 腾讯科技（深圳）有限公司 | Depth image generation method, device and storage medium |
| CN111612891B (en) * | 2020-05-22 | 2023-08-08 | 北京京东乾石科技有限公司 | Model generation method, point cloud data processing method, device, equipment and medium |
| CN113822982B (en) * | 2020-06-19 | 2023-10-27 | 北京达佳互联信息技术有限公司 | Human body three-dimensional model construction method and device, electronic equipment and storage medium |
| CN112396703B (en) * | 2020-11-18 | 2024-01-12 | 北京工商大学 | Reconstruction method of single-image three-dimensional point cloud model |
| CN112489216B (en) * | 2020-11-27 | 2023-07-28 | 北京百度网讯科技有限公司 | Evaluation method, device and equipment of facial reconstruction model and readable storage medium |
| CN112489202A (en) * | 2020-12-08 | 2021-03-12 | 甘肃智通科技工程检测咨询有限公司 | Pavement macroscopic texture reconstruction method based on multi-view deep learning |
| CN112634281A (en) * | 2020-12-10 | 2021-04-09 | 浙江大学 | Grid segmentation method based on graph convolution network |
| CN112862949B (en) * | 2021-01-18 | 2022-08-19 | 北京航空航天大学 | Object 3D shape reconstruction method based on multiple views |
| CN113096249B (en) * | 2021-03-30 | 2023-02-17 | Oppo广东移动通信有限公司 | Method for training vertex reconstruction model, image reconstruction method and electronic equipment |
| CN113077554A (en) * | 2021-04-08 | 2021-07-06 | 华南理工大学 | Three-dimensional structured model reconstruction method based on any visual angle picture |
| CN113610711B (en) * | 2021-08-02 | 2023-05-23 | 南京信息工程大学 | Single-image-guided three-dimensional surface reconstruction method and device |
| CN113808006B (en) * | 2021-09-01 | 2023-05-23 | 南京信息工程大学 | Method and device for reconstructing three-dimensional grid model based on two-dimensional image |
| CN113808275B (en) * | 2021-09-24 | 2023-10-13 | 南京信息工程大学 | Single image three-dimensional reconstruction method based on GCN and topology modification |
| CN114373056A (en) * | 2021-12-17 | 2022-04-19 | 云南联合视觉科技有限公司 | Three-dimensional reconstruction method and device, terminal equipment and storage medium |
| CN116109799B (en) * | 2023-04-13 | 2023-08-04 | 深圳思谋信息科技有限公司 | Method, device, computer equipment and storage medium for training adjustment model |
| CN116168137B (en) * | 2023-04-21 | 2023-07-11 | 湖南马栏山视频先进技术研究院有限公司 | New view angle synthesis method, device and memory based on nerve radiation field |
| CN117953544A (en) * | 2024-03-26 | 2024-04-30 | 安徽农业大学 | Target behavior monitoring method and system |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107133960A (en) * | 2017-04-21 | 2017-09-05 | 武汉大学 | Image crack dividing method based on depth convolutional neural networks |
| WO2018126396A1 (en) * | 2017-01-05 | 2018-07-12 | General Electric Company | Deep learning based estimation of data for use in tomographic reconstruction |
| CN108510573A (en) * | 2018-04-03 | 2018-09-07 | 南京大学 | A method of the multiple views human face three-dimensional model based on deep learning is rebuild |
| CN109389671A (en) * | 2018-09-25 | 2019-02-26 | 南京大学 | A kind of single image three-dimensional rebuilding method based on multistage neural network |
| CN109410321A (en) * | 2018-10-17 | 2019-03-01 | 大连理工大学 | Three-dimensional rebuilding method based on convolutional neural networks |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US11074495B2 (en) * | 2013-02-28 | 2021-07-27 | Z Advanced Computing, Inc. (Zac) | System and method for extremely efficient image and pattern recognition and artificial intelligence platform |
-
2019
- 2019-04-15 CN CN201910298089.7A patent/CN110021069B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2018126396A1 (en) * | 2017-01-05 | 2018-07-12 | General Electric Company | Deep learning based estimation of data for use in tomographic reconstruction |
| CN107133960A (en) * | 2017-04-21 | 2017-09-05 | 武汉大学 | Image crack dividing method based on depth convolutional neural networks |
| CN108510573A (en) * | 2018-04-03 | 2018-09-07 | 南京大学 | A method of the multiple views human face three-dimensional model based on deep learning is rebuild |
| CN109389671A (en) * | 2018-09-25 | 2019-02-26 | 南京大学 | A kind of single image three-dimensional rebuilding method based on multistage neural network |
| CN109410321A (en) * | 2018-10-17 | 2019-03-01 | 大连理工大学 | Three-dimensional rebuilding method based on convolutional neural networks |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110021069A (en) | 2019-07-16 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110021069B (en) | Three-dimensional model reconstruction method based on grid deformation | |
| CN110458939B (en) | Indoor scene modeling method based on visual angle generation | |
| CN110458957B (en) | Image three-dimensional model construction method and device based on neural network | |
| JP5778237B2 (en) | Backfill points in point cloud | |
| Whelan et al. | Deformation-based loop closure for large scale dense RGB-D SLAM | |
| US9715761B2 (en) | Real-time 3D computer vision processing engine for object recognition, reconstruction, and analysis | |
| US8452081B2 (en) | Forming 3D models using multiple images | |
| CN110009691B (en) | Parallax image generation method and system based on binocular stereo vision matching | |
| CN110176032B (en) | Three-dimensional reconstruction method and device | |
| EP3905194A1 (en) | Pose estimation method and apparatus | |
| JP2016075637A (en) | Information processing apparatus and method for the same | |
| Ramalingam et al. | A theory of minimal 3D point to 3D plane registration and its generalization | |
| JP5106375B2 (en) | 3D shape restoration device and program thereof | |
| Saini et al. | NURBS-based geometric inverse reconstruction of free-form shapes | |
| El Hazzat et al. | 3D reconstruction system based on incremental structure from motion using a camera with varying parameters | |
| CN115035235A (en) | Three-dimensional reconstruction method and device | |
| CN114219890A (en) | Three-dimensional reconstruction method, device and equipment and computer storage medium | |
| Gadasin et al. | Reconstruction of a Three-Dimensional Scene from its Projections in Computer Vision Systems | |
| Ly et al. | Extrinsic calibration of heterogeneous cameras by line images | |
| Vong et al. | How to build a 2d and 3d aerial multispectral map?—all steps deeply explained | |
| CN104796624A (en) | Method for editing and propagating light fields | |
| CN117132737B (en) | Three-dimensional building model construction method, system and equipment | |
| Hu et al. | Learning structural graph layouts and 3D shapes for long span bridges 3D reconstruction | |
| Yuan et al. | Incremental 3D reconstruction using Bayesian learning | |
| CN113066165B (en) | Three-dimensional reconstruction method and device for multi-stage unsupervised learning and electronic equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |