CN114898028A - Scene reconstruction and rendering method based on point cloud, storage medium and electronic equipment - Google Patents
Scene reconstruction and rendering method based on point cloud, storage medium and electronic equipment Download PDFInfo
- Publication number
- CN114898028A CN114898028A CN202210473540.6A CN202210473540A CN114898028A CN 114898028 A CN114898028 A CN 114898028A CN 202210473540 A CN202210473540 A CN 202210473540A CN 114898028 A CN114898028 A CN 114898028A
- Authority
- CN
- China
- Prior art keywords
- point
- rendering
- point cloud
- feature
- scene
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000009877 rendering Methods 0.000 title claims abstract description 80
- 238000000034 method Methods 0.000 title claims abstract description 33
- 230000005855 radiation Effects 0.000 claims abstract description 15
- 238000005070 sampling Methods 0.000 claims abstract description 12
- 230000000007 visual effect Effects 0.000 claims abstract description 7
- 238000013528 artificial neural network Methods 0.000 claims abstract description 4
- 238000004590 computer program Methods 0.000 claims description 13
- 238000002834 transmittance Methods 0.000 claims description 6
- 230000001537 neural effect Effects 0.000 abstract description 6
- 238000013527 convolutional neural network Methods 0.000 description 3
- 230000000694 effects Effects 0.000 description 3
- 230000015572 biosynthetic process Effects 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000003786 synthesis reaction Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T15/00—3D [Three Dimensional] image rendering
- G06T15/10—Geometric effects
- G06T15/20—Perspective computation
- G06T15/205—Image-based rendering
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T15/00—3D [Three Dimensional] image rendering
- G06T15/08—Volume rendering
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computing Systems (AREA)
- General Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Biomedical Technology (AREA)
- Molecular Biology (AREA)
- Biophysics (AREA)
- General Engineering & Computer Science (AREA)
- Life Sciences & Earth Sciences (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Computer Graphics (AREA)
- Geometry (AREA)
- Image Generation (AREA)
Abstract
The invention relates to a scene reconstruction and rendering method based on point cloud, a storage medium and an electronic device, wherein a data set is constructed by acquiring RGB (red, green and blue) pictures, camera parameters and depth map data of a plurality of different visual angles in a single scene; carrying out multi-view three-dimensional reconstruction on the point cloud with the neural network characteristics to obtain a characteristic-enhanced point cloud; the method has the advantages that the traditional volume radiation field is simulated by using the feature-enhanced point cloud for rendering, the target scene is processed into the point cloud with the neural feature vector to simulate the neural radiation field, sampling in an open scene is avoided, the scene can be represented by less memory, and faster reconstruction of the radiation field is realized.
Description
Technical Field
The invention relates to the technical field of computer vision, computer graphic processing and three-dimensional reconstruction, in particular to a scene reconstruction and rendering method based on point cloud, a storage medium and electronic equipment.
Background
In recent years, modeling a real scene from image data and rendering a photo-realistic image are important topics in computer vision and computer graphics. For three-dimensional reconstruction, "MVSNet: Depth reference for Unstructured Multi-view Stereo" by Y Yao et al has shown great success. This is because the learning-based method can learn and utilize the global semantic information of the scene, including object material, mirror surface, and ambient lighting, so as to obtain a stronger matching degree and a more complete reconstruction effect. However, most of the currently used MVS methods use dense multi-scale three-dimensional CNNs (Convolutional Neural Networks) to predict depth maps or voxel occupancy. Three-dimensional CNN requires significant memory associated with the resolution of the model, which is a potential hindrance to the current technology.
In addition, for the scene reconstruction and rendering aspect, Mildenhall et al propose the concept of "reconstructing as a neural radial fields for view synthesis", which makes a great contribution to the reconstruction and rendering of three-dimensional scenes. This method reconstructs the radiation field using global MLPs for the entire space by way of ray propagation, but also results in a lengthy reconstruction time.
Both "pixel NeRF: Neural radial Fields from One or Few Images" by Alex Yu et al and "IBRNet: Learning Multi-View Image-Based Rendering" by Qianqian Wang et al use a relatively small set of inputs and gather Multi-View two-dimensional features at each sampled ray point, regressing volume Rendering attributes for subsequent Rendering from the radiation field. However, due to the limitations of the data set, they are not well suited for multi-view rendering of new scenes, nor do they avoid sampling in the open space of the scene.
Disclosure of Invention
In view of the above existing problems, the present invention provides a method, a storage medium, and an electronic device for scene reconstruction and rendering based on point cloud, so as to achieve faster and higher-quality multi-view scene reconstruction and rendering.
The invention relates to a scene reconstruction and rendering method based on point cloud, which comprises the following steps:
step 1, acquiring RGB (red, green and blue) pictures of a plurality of different visual angles in a certain single scene, corresponding camera parameters and depth map data of real ground, and constructing a data set;
step 2, performing multi-view three-dimensional reconstruction on the point cloud with the neural network characteristics to obtain a characteristic-enhanced point cloud, wherein each point has three attributes: coordinates, eigenvalues, and confidence:
step 3, simulating a traditional volume radiation field to render by using the feature enhanced point cloud
According to three attributes of each three-dimensional point in the feature-enhanced point cloud P, three MLP networks are constructed to return the volume density of any rendering position and the radiometric degree which changes along with the view angle, the MLP networks are used for simulating a radiation field, K points in a sphere with the target rendering position x as the center of circle and the radius of R are obtained when the volume density and the radiometric degree are obtained, then the final result is gathered from the K points, and finally a discrete volume rendering mode based on NeRF is used for obtaining a final high-precision rendering picture.
Each point in the feature enhanced point cloud in the step 2 has three attributes: the method comprises the following steps of specifically acquiring coordinates, characteristic values and confidence coefficients:
inputting the RGB pictures in the data set and corresponding camera parameters into a pre-trained MVSNet together to obtain the depth probability, the feature map and the depth map of each RGB picture;
carrying out trilinear sampling on the depth probability of each RGB picture and two adjacent RGB pictures in a data set, acquiring a value with the size between 0 and 1, and taking the value as the confidence coefficient of a point to evaluate whether the point is on a surface or not, wherein the larger the value is, the more the probability is that the point is on the surface;
projecting the obtained depth map to a three-dimensional space to obtain coordinates of each point;
and (3) performing down-sampling on the obtained feature map by using a two-dimensional convolution network with the step length of 2, extracting feature layers before each down-sampling, and combining the feature layers to form a three-layer feature pyramid F:
the subscript i represents the ith image, and the superscript represents different layers in the three-layer characteristic pyramid;
for the input N RGB pictures, the characteristic value C of each three-dimensional point j The following formula is used to obtain from the three-level feature pyramid:
where j represents the different levels in the three-level feature pyramid.
The feature enhanced point cloud P is formulated as follows:
P＝{(p i ，f i ，γ i )|i＝1，...，T} (3)
wherein i represents the ith point in the point cloud, p represents the coordinate of the point, f represents the characteristic value of the point, gamma represents the confidence coefficient, and T is the total number of the point cloud;
according to the three attributes of each three-dimensional point in the feature-enhanced point cloud P, three MLP networks are constructed to back and forth return the volume density of any rendering position and the radiometric degree which changes along with the view angle, the MLP networks are used for simulating a radiation field, K points in a sphere with a target rendering position x as the circle center and a radius of R are obtained when the volume density and the radiometric degree are obtained, then a final result is gathered from the K points, and finally a final high-precision rendering picture is obtained by using a discrete volume rendering mode based on NeRF, wherein the three attributes are specifically as follows:
the first MLP network is used to predict a feature vector f for a target rendering position x, of the form: f. of i，x ＝F(f i ，x-p i ). (4)
Where i represents the ith point of proximity of the target rendering location x, (x-p) i ) Denotes x and p i The relative position of (a);
the second MLP network is used for obtaining the radiance r of the target rendering position x: adding the feature values of the adjacent points of the target rendering position x by using an inverse distance weighting method to obtain a single feature describing the target rendering position x, wherein the single feature is defined as:
wherein, W i Is defined as follows:
the third MLP network regresses from the single feature the radiance r associated with the line of sight d:
r＝R(f x ，d) (7)
and (3) performing regression on the volume density of the point close to the target rendering position x by using the third MLP network, and performing reverse distance weighting to obtain the volume density sigma of the target rendering position x:
σ i ＝T(f i，x ) (8)
and (3) finishing multi-view high-precision scene rendering by using the obtained radiometric degree r and the volume density sigma by using a discrete volume rendering method proposed in NeRF, wherein the formula is as follows:
where c is the color of the final rendering position, M is a point at which a color is required to be calculated altogether, j denotes this where j is the jth point, τ represents the volume transmittance, σ and r are the bulk density and radiance obtained in the above steps, Δ is the distance between adjacent points, and t is the tth point in the rendering range when the volume transmittance is calculated.
A storage medium is also included having stored thereon a computer program executable by a processor, the computer program when executed implementing the steps of any of the above-described point cloud based scene reconstruction and rendering methods.
The electronic equipment comprises a processor, a memory, an input unit and a display unit, wherein the memory stores a computer program capable of being executed by the processor, and the computer program realizes the steps of any one method for reconstructing and rendering a scene based on point cloud when being executed; the input unit is used for inputting and appointing a new visual angle; the display unit is used for displaying the new view angle image.
The invention processes the target scene into the point cloud with the neural characteristic vector to simulate the neural radiation field, thereby avoiding sampling in an open scene and realizing faster reconstruction of the radiation field. On the one hand, the scene can be represented with less memory, and on the other hand, pre-training can be performed across scenes. Compared with other methods, the method has great advantages in reconstruction and rendering. In addition, the invention uses the confidence coefficient of the point to judge whether the point is on the surface, so that the noise, discrete points and the like in the point cloud can be ignored, and a better rendering effect is obtained.
Detailed Description
In order that the above objects, features and advantages of the present invention may be more clearly understood, a detailed description of the present invention will be given below by way of specific embodiments. It should be noted that the embodiments of the present invention and features of the embodiments may be combined with each other without conflict.
The invention relates to a scene reconstruction and rendering method based on point cloud, which comprises the following steps:
step 1, acquiring RGB (red, green and blue) pictures of a plurality of different visual angles in a certain single scene, corresponding camera parameters (including pose parameters) and depth map data of real ground (ground route) to construct a data set. The number of images and the shooting angle are not limited, but the finally constructed point cloud tends to be complete. The present invention uses an open source DTU data set.
Step 2, performing multi-view three-dimensional reconstruction on the point cloud with the neural network characteristics to obtain a characteristic-enhanced point cloud, wherein each point has three attributes: coordinates, eigenvalues, and confidence:
(1) obtaining coordinates and confidence of points
And inputting the RGB pictures in the data set and corresponding camera parameters into a pre-trained MVSNet together to obtain the depth probability, the feature map and the depth map of each RGB picture. The present invention uses the existing MVSNet.
The depth probability describes the likelihood of each point on the surface. Performing trilinear sampling (tri-linear sample) on the depth probability of each RGB picture and two adjacent RGB pictures in the data set, acquiring a value with the size between 0 and 1, and using the value as the confidence level of a point to evaluate whether the point is on a surface, wherein the larger the value is, the more likely the point is on the surface;
projecting the obtained depth map to a three-dimensional space to obtain coordinates of each point;
(2) obtaining characteristic value of point
Firstly, a three-layer feature pyramid is used to extract the image features of multiple perspectives: and (2) for the feature map obtained in the step (1), performing down-sampling by using a two-dimensional convolution network with the step length of 2, extracting feature layers before each down-sampling, and combining the feature layers to form a three-layer feature pyramid F:
wherein, the subscript i represents the ith image, and the superscript represents different layers in the three-layer characteristic pyramid.
Then, for the input N RGB pictures, the characteristic value C of each three-dimensional point j The following formula can be used to derive from the three-level feature pyramid:
wherein j represents different layers in the three-layer characteristic pyramid;
(3) combining the characteristic value, the coordinate and the confidence coefficient of each point to be used as the attribute of each three-dimensional point in the point cloud, thereby constructing the point cloud with enhanced characteristics;
step 3, simulating a traditional volume radiation field to render by using the feature enhanced point cloud
(1) The feature enhanced point cloud P is formulated as follows:
P＝{(p i ，f i ，γ i )|i＝1，...，T} (3)
wherein i represents the ith point in the point cloud, p represents the coordinate of the point, f represents the characteristic value of the point, gamma represents the confidence coefficient, and T is the total number of the point cloud;
(2) according to three attributes of each three-dimensional point in the feature-enhanced point cloud P, three MLP networks are constructed to return the volume density of any rendering position and the radiometric degree which changes along with the view angle, the MLP networks are used for simulating a radiation field, K points in a sphere with the target rendering position x as the center of circle and the radius of R are obtained when the volume density and the radiometric degree are obtained, then the final result is gathered from the K points, and finally a discrete volume rendering mode based on NeRF is used for obtaining a final high-precision rendering picture.
The first MLP network is used to predict a feature vector f for a target rendering position x, of the form: f. of i，x ＝F(f i ，x-p i ). (4)
Where i represents the ith proximity point of target rendering position x, (x-p) i ) Denotes x and p i The relative position of (a). The method can describe the relation between the target rendering position x and the adjacent point i, and can achieve better effect when the radiance is obtained in the next step.
The second MLP network is used for obtaining the radiance r of the target rendering position x: first, the feature values of the nearby points of the target rendering position x are added by using an inverse distance weighting method to obtain a single feature describing the target rendering position x (the closer points can be made to contribute more to the single feature by using the inverse distance weighting method), and the single feature is defined as:
wherein, W i Is defined as:
then, the radiance r associated with the line of sight d is regressed from this single feature using a third MLP network: r ═ R (f) x ，d) (7)
The third MLP network is used to regress bulk density: firstly, the third MLP network is used for carrying out regression on the volume density of the point close to the target rendering position x, and then reverse distance weighting is carried out to obtain the volume density sigma of the target rendering position x:
σ i ＝T(f i ，x) (8)
(3) the obtained radiance r and the volume density sigma are subjected to a discrete volume rendering method proposed in NeRF, so that multi-view high-precision scene rendering can be completed, and the formula is as follows:
where c is the color of the final rendering position, M is a point at which a color is required to be calculated altogether, j denotes this where j is the jth point, τ represents the volume transmittance, σ and r are the bulk density and radiance obtained in the above steps, Δ is the distance between adjacent colored points, and t is the tth point within the colored range when the volume transmittance is calculated.
The present embodiment also provides a storage medium having a computer program stored thereon, the computer program being executable by a processor, and the computer program being executed to implement the steps of the point cloud based scene reconstruction and rendering method in the present embodiment.
The embodiment also provides an electronic device, which comprises a processor, a memory, an input unit and a display unit, wherein the memory stores a computer program capable of being executed by the processor, and the computer program realizes the steps of the method for reconstructing and rendering a scene based on point cloud in the embodiment when executed; the input unit is used for inputting and appointing a new visual angle; the display unit is used for displaying the new view angle image.
The above embodiments are only for illustrating the invention and are not to be construed as limiting the invention, and those skilled in the art can make various changes and modifications without departing from the spirit and scope of the invention, therefore, all equivalent technical solutions also fall into the scope of the invention, and the scope of the invention is defined by the claims.
Claims (5)
1. The scene reconstruction and rendering method based on the point cloud is characterized by comprising the following steps:
step 1, acquiring RGB (red, green and blue) pictures of a plurality of different visual angles in a certain single scene, corresponding camera parameters and depth map data of real ground, and constructing a data set;
step 2, performing multi-view three-dimensional reconstruction on the point cloud with the neural network characteristics to obtain a characteristic-enhanced point cloud, wherein each point has three attributes: coordinates, eigenvalues, and confidence;
step 3, simulating a traditional volume radiation field to render by using the feature enhanced point cloud
According to three attributes of each three-dimensional point in the feature-enhanced point cloud P, three MLP networks are constructed to return the volume density of any rendering position and the radiometric degree which changes along with the view angle, the MLP networks are used for simulating a radiation field, K points in a sphere with the target rendering position x as the center of circle and the radius of R are obtained when the volume density and the radiometric degree are obtained, then the final result is gathered from the K points, and finally a discrete volume rendering mode based on NeRF is used for obtaining a final high-precision rendering picture.
2. The method for point cloud based scene reconstruction and rendering of claim 1, wherein each point in the feature enhanced point cloud in step 2 has three attributes: the method comprises the following steps of specifically acquiring coordinates, characteristic values and confidence coefficients:
inputting RGB pictures in a data set and corresponding camera parameters into a pre-trained MVSNet together to obtain the depth probability, the feature map and the depth map of each RGB picture;
carrying out trilinear sampling on the depth probability of each RGB picture and two adjacent RGB pictures in a data set, acquiring a value with the size between 0 and 1, and taking the value as the confidence coefficient of a point to evaluate whether the point is on a surface or not, wherein the larger the value is, the more the probability is that the point is on the surface;
projecting the obtained depth map to a three-dimensional space to obtain coordinates of each point;
and (3) performing down-sampling on the obtained feature map by using a two-dimensional convolution network with the step length of 2, extracting feature layers before each down-sampling, and combining the feature layers to form a three-layer feature pyramid F:
the subscript i represents the ith image, and the superscript represents different layers in the three-layer characteristic pyramid;
for the input N RGB pictures, the characteristic value C of each three-dimensional point j The following formula is used to obtain from the three-level feature pyramid:
where j represents the different levels in the three-level feature pyramid.
3. The method for point cloud-based scene reconstruction and rendering according to any one of claims 1 and 2, wherein:
the feature enhanced point cloud P is formulated as follows:
P＝{(p i ，f i ，γ i )|i＝1，...，T} (3)
wherein i represents the ith point in the point cloud, p represents the coordinate of the point, f represents the characteristic value of the point, gamma represents the confidence coefficient, and T is the total number of the point cloud;
according to the three attributes of each three-dimensional point in the feature-enhanced point cloud P, three MLP networks are constructed to back and forth return the volume density of any rendering position and the radiometric degree which changes along with the view angle, the MLP networks are used for simulating a radiation field, K points in a sphere with a target rendering position x as the circle center and a radius of R are obtained when the volume density and the radiometric degree are obtained, then a final result is gathered from the K points, and finally a final high-precision rendering picture is obtained by using a discrete volume rendering mode based on NeRF, wherein the three attributes are specifically as follows:
the first MLP network is used to predict a feature vector f for a target rendering position x, of the form: f. of i，x ＝F(f i，x -p i ). (4)
Where i represents the ith point of proximity of the target rendering location x, (x-p) i ) Denotes x and p i The relative position of (a);
the second MLP network is used for obtaining the radiance r of the target rendering position x: adding the feature values of the adjacent points of the target rendering position x by using an inverse distance weighting method to obtain a single feature describing the target rendering position x, wherein the single feature is defined as:
wherein, W i Is defined as:
the third MLP network regresses from the single feature the radiance r associated with the line of sight d:
r＝R(f x ，d) (7)
and (3) performing regression on the volume density of the point close to the target rendering position x by using the third MLP network, and performing reverse distance weighting to obtain the volume density sigma of the target rendering position x:
σ i ＝T(f i，x ) (8)
and (3) finishing multi-view high-precision scene rendering by using the obtained radiometric degree r and the volume density sigma by using a discrete volume rendering method proposed in NeRF, wherein the formula is as follows:
where c is the color of the final rendering position, M is a point at which a color is required to be calculated altogether, j denotes this where j is the jth point, τ represents the volume transmittance, σ and r are the bulk density and radiance obtained in the above steps, Δ is the distance between adjacent points, and t is the tth point in the rendering range when the volume transmittance is calculated.
4. A storage medium having stored thereon a computer program executable by a processor, the computer program when executed implementing the steps of any of the point cloud based methods of scene reconstruction and rendering of claims 1-3.
5. An electronic device comprising a processor, a memory, an input unit and a display unit, wherein the memory has stored thereon a computer program executable by the processor, the computer program when executed implementing the steps of any one of the point cloud based scene reconstruction and rendering methods of claims 1-3; the input unit is used for inputting and appointing a new visual angle; the display unit is used for displaying the new view angle image.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210473540.6A CN114898028A (en) | 2022-04-29 | 2022-04-29 | Scene reconstruction and rendering method based on point cloud, storage medium and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210473540.6A CN114898028A (en) | 2022-04-29 | 2022-04-29 | Scene reconstruction and rendering method based on point cloud, storage medium and electronic equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114898028A true CN114898028A (en) | 2022-08-12 |
Family
ID=82719839
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210473540.6A Pending CN114898028A (en) | 2022-04-29 | 2022-04-29 | Scene reconstruction and rendering method based on point cloud, storage medium and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114898028A (en) |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115147577A (en) * | 2022-09-06 | 2022-10-04 | 深圳市明源云科技有限公司 | VR scene generation method, device, equipment and storage medium |
| CN115243025A (en) * | 2022-09-21 | 2022-10-25 | 深圳市明源云科技有限公司 | Three-dimensional rendering method and device, terminal equipment and storage medium |
| CN115409931A (en) * | 2022-10-31 | 2022-11-29 | 苏州立创致恒电子科技有限公司 | Three-dimensional reconstruction method based on image and point cloud data fusion |
| CN115731355A (en) * | 2022-11-29 | 2023-03-03 | 湖北大学 | SuperPoint-NeRF-based three-dimensional building reconstruction method |
| CN115965749A (en) * | 2023-03-16 | 2023-04-14 | 联易云科(北京)科技有限公司 | Three-dimensional reconstruction equipment based on radar vision fusion |
| CN116129082A (en) * | 2023-03-06 | 2023-05-16 | 中南大学 | Unmanned container-oriented TIN-NeRF new visual angle image labeling method |
| CN116452758A (en) * | 2023-06-20 | 2023-07-18 | 擎翌(上海)智能科技有限公司 | Neural radiation field model acceleration training method, device, equipment and medium |
| CN116681818A (en) * | 2022-10-28 | 2023-09-01 | 荣耀终端有限公司 | New view angle reconstruction method, training method and device of new view angle reconstruction network |
| CN117994444A (en) * | 2024-04-03 | 2024-05-07 | 浙江华创视讯科技有限公司 | Reconstruction method, device and storage medium of complex scene |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20180205941A1 (en) * | 2017-01-17 | 2018-07-19 | Facebook, Inc. | Three-dimensional scene reconstruction from set of two dimensional images for consumption in virtual reality |
| CN113538664A (en) * | 2021-07-14 | 2021-10-22 | 清华大学 | Vehicle de-illumination three-dimensional reconstruction method and device, electronic equipment and storage medium |
| CN113706714A (en) * | 2021-09-03 | 2021-11-26 | 中科计算技术创新研究院 | New visual angle synthesis method based on depth image and nerve radiation field |
| CN113822977A (en) * | 2021-06-28 | 2021-12-21 | 腾讯科技（深圳）有限公司 | Image rendering method, device, equipment and storage medium |
| CN114004941A (en) * | 2022-01-04 | 2022-02-01 | 苏州浪潮智能科技有限公司 | Indoor scene three-dimensional reconstruction system and method based on nerve radiation field |
-
2022
- 2022-04-29 CN CN202210473540.6A patent/CN114898028A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20180205941A1 (en) * | 2017-01-17 | 2018-07-19 | Facebook, Inc. | Three-dimensional scene reconstruction from set of two dimensional images for consumption in virtual reality |
| CN113822977A (en) * | 2021-06-28 | 2021-12-21 | 腾讯科技（深圳）有限公司 | Image rendering method, device, equipment and storage medium |
| CN113538664A (en) * | 2021-07-14 | 2021-10-22 | 清华大学 | Vehicle de-illumination three-dimensional reconstruction method and device, electronic equipment and storage medium |
| CN113706714A (en) * | 2021-09-03 | 2021-11-26 | 中科计算技术创新研究院 | New visual angle synthesis method based on depth image and nerve radiation field |
| CN114004941A (en) * | 2022-01-04 | 2022-02-01 | 苏州浪潮智能科技有限公司 | Indoor scene three-dimensional reconstruction system and method based on nerve radiation field |
Non-Patent Citations (1)
| Title |
|---|
| 张卫龙: "局部信息约束的三维重建方法研究", 中国博士学位论文电子期刊网, 15 June 2020 (2020-06-15) * |
Cited By (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115147577A (en) * | 2022-09-06 | 2022-10-04 | 深圳市明源云科技有限公司 | VR scene generation method, device, equipment and storage medium |
| CN115243025A (en) * | 2022-09-21 | 2022-10-25 | 深圳市明源云科技有限公司 | Three-dimensional rendering method and device, terminal equipment and storage medium |
| CN116681818A (en) * | 2022-10-28 | 2023-09-01 | 荣耀终端有限公司 | New view angle reconstruction method, training method and device of new view angle reconstruction network |
| CN116681818B (en) * | 2022-10-28 | 2024-04-09 | 荣耀终端有限公司 | New view angle reconstruction method, training method and device of new view angle reconstruction network |
| CN115409931A (en) * | 2022-10-31 | 2022-11-29 | 苏州立创致恒电子科技有限公司 | Three-dimensional reconstruction method based on image and point cloud data fusion |
| CN115731355A (en) * | 2022-11-29 | 2023-03-03 | 湖北大学 | SuperPoint-NeRF-based three-dimensional building reconstruction method |
| CN115731355B (en) * | 2022-11-29 | 2024-06-04 | 湖北大学 | SuperPoint-NeRF-based three-dimensional building reconstruction method |
| CN116129082B (en) * | 2023-03-06 | 2024-01-23 | 中南大学 | Unmanned container-oriented TIN-NeRF new visual angle image labeling method |
| CN116129082A (en) * | 2023-03-06 | 2023-05-16 | 中南大学 | Unmanned container-oriented TIN-NeRF new visual angle image labeling method |
| CN115965749A (en) * | 2023-03-16 | 2023-04-14 | 联易云科(北京)科技有限公司 | Three-dimensional reconstruction equipment based on radar vision fusion |
| CN116452758B (en) * | 2023-06-20 | 2023-10-20 | 擎翌(上海)智能科技有限公司 | Neural radiation field model acceleration training method, device, equipment and medium |
| CN116452758A (en) * | 2023-06-20 | 2023-07-18 | 擎翌(上海)智能科技有限公司 | Neural radiation field model acceleration training method, device, equipment and medium |
| CN117994444A (en) * | 2024-04-03 | 2024-05-07 | 浙江华创视讯科技有限公司 | Reconstruction method, device and storage medium of complex scene |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN114898028A (en) | Scene reconstruction and rendering method based on point cloud, storage medium and electronic equipment | |
| CN109003325B (en) | Three-dimensional reconstruction method, medium, device and computing equipment | |
| CN108564527B (en) | Panoramic image content completion and restoration method and device based on neural network | |
| WO2022165809A1 (en) | Method and apparatus for training deep learning model | |
| CN108805979A (en) | A kind of dynamic model three-dimensional rebuilding method, device, equipment and storage medium | |
| CN110490917A (en) | Three-dimensional rebuilding method and device | |
| US11887256B2 (en) | Deferred neural rendering for view extrapolation | |
| Sun et al. | Underwater image enhancement with reinforcement learning | |
| CN116977522A (en) | Rendering method and device of three-dimensional model, computer equipment and storage medium | |
| US20220351463A1 (en) | Method, computer device and storage medium for real-time urban scene reconstruction | |
| CN111754622B (en) | Face three-dimensional image generation method and related equipment | |
| CN113822965A (en) | Image rendering processing method, device and equipment and computer storage medium | |
| CN114677479A (en) | Natural landscape multi-view three-dimensional reconstruction method based on deep learning | |
| CN113963117A (en) | Multi-view three-dimensional reconstruction method and device based on variable convolution depth network | |
| CN115797561A (en) | Three-dimensional reconstruction method, device and readable storage medium | |
| CN115222917A (en) | Training method, device and equipment for three-dimensional reconstruction model and storage medium | |
| CN116228962A (en) | Large scene neuroview synthesis | |
| CN114170290A (en) | Image processing method and related equipment | |
| CN117274501B (en) | Drivable digital person modeling method, device, equipment and medium | |
| WO2021151380A1 (en) | Method for rendering virtual object based on illumination estimation, method for training neural network, and related products | |
| WO2022217470A1 (en) | Hair rendering system based on deep neural network | |
| US11830204B2 (en) | Systems and methods for performing motion transfer using a learning model | |
| CN108765549A (en) | A kind of product three-dimensional display method and device based on artificial intelligence | |
| CN116452757B (en) | Human body surface reconstruction method and system under complex scene | |
| CN116958393A (en) | Incremental image rendering method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |