CN117058287A - Three-dimensional model mapping method, system, storage medium and terminal based on nerve radiation field - Google Patents
Three-dimensional model mapping method, system, storage medium and terminal based on nerve radiation field Download PDFInfo
- Publication number
- CN117058287A CN117058287A CN202210483487.8A CN202210483487A CN117058287A CN 117058287 A CN117058287 A CN 117058287A CN 202210483487 A CN202210483487 A CN 202210483487A CN 117058287 A CN117058287 A CN 117058287A
- Authority
- CN
- China
- Prior art keywords
- color
- specular
- dimensional model
- radiation field
- observation
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000005855 radiation Effects 0.000 title claims abstract description 81
- 238000013507 mapping Methods 0.000 title claims abstract description 80
- 238000000034 method Methods 0.000 title claims abstract description 35
- 210000005036 nerve Anatomy 0.000 title claims abstract description 35
- 238000005070 sampling Methods 0.000 claims abstract description 64
- 230000002194 synthesizing effect Effects 0.000 claims abstract description 24
- 230000001537 neural effect Effects 0.000 claims description 47
- 239000003086 colorant Substances 0.000 claims description 8
- 238000004590 computer program Methods 0.000 claims description 6
- 238000012935 Averaging Methods 0.000 claims description 4
- 230000001174 ascending effect Effects 0.000 claims description 4
- 238000012216 screening Methods 0.000 claims description 4
- 239000000463 material Substances 0.000 abstract description 11
- 238000009877 rendering Methods 0.000 abstract description 8
- 239000000919 ceramic Substances 0.000 abstract description 5
- 230000000694 effects Effects 0.000 abstract description 5
- 239000002184 metal Substances 0.000 abstract description 5
- 238000012545 processing Methods 0.000 description 12
- 238000010586 diagram Methods 0.000 description 5
- 230000006870 function Effects 0.000 description 4
- 238000005286 illumination Methods 0.000 description 4
- 230000000007 visual effect Effects 0.000 description 3
- 238000003491 array Methods 0.000 description 2
- 230000002596 correlated effect Effects 0.000 description 2
- 239000006185 dispersion Substances 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000008094 contradictory effect Effects 0.000 description 1
- 230000001276 controlling effect Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000035772 mutation Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T15/00—3D [Three Dimensional] image rendering
- G06T15/005—General purpose rendering architectures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T15/00—3D [Three Dimensional] image rendering
- G06T15/50—Lighting effects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T19/00—Manipulating 3D models or images for computer graphics
- G06T19/20—Editing of 3D images, e.g. changing shapes or colours, aligning objects or positioning parts
Landscapes
- Engineering & Computer Science (AREA)
- Computer Graphics (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Architecture (AREA)
- Computer Hardware Design (AREA)
- General Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Image Generation (AREA)
Abstract
The invention provides a three-dimensional model mapping method, a system, a storage medium and a terminal based on a nerve radiation field, which comprise the following steps: expanding the surface of the three-dimensional model to a UV coordinate system; acquiring surface sampling points of the three-dimensional model from the UV coordinate system; calculating a specular reflection color, a diffuse reflection color, a specular level, and a gloss based on the surface sampling points; and synthesizing a high-light reflection map based on the high-light reflection color, synthesizing a diffuse reflection map based on the diffuse reflection color, synthesizing a high-light level map based on the high-light level, and synthesizing a gloss map based on the glossiness. The three-dimensional model mapping method, the system, the storage medium and the terminal based on the nerve radiation field render the model by using diffuse reflection mapping, highlight level mapping and gloss mapping; the generated map is not only suitable for purely diffuse reflection objects, but also suitable for objects with high-light materials such as metal, ceramic tiles and the like; the rendering effect of the three-dimensional model is improved, and the three-dimensional model is more attached to the real model material.
Description
Technical Field
The invention relates to the technical field of texture mapping, in particular to a three-dimensional model mapping method, a three-dimensional model mapping system, a three-dimensional model mapping storage medium and a three-dimensional model mapping terminal based on a nerve radiation field.
Background
Three-dimensional models are essentially representations of polygons of objects, typically made up of two parts, mesh and texture. The mesh comprises triangles, quadrilaterals or other simple convex polygons connected by a plurality of point clouds, and the texture comprises corrugations and a map of the object surface. When the texture is mapped onto the surface of the object in a specific way, the object can be made to look more realistic and conform to the actual material of the model.
The existing three-dimensional model mapping method can automatically select mapping images for each triangular surface of the model surface, but because each triangular surface can only select one image from multi-view images for mapping, when mapping sources of adjacent triangular surfaces are changed, color mutation easily occurs, so that the visual effect of the three-dimensional model is reduced. In addition, the existing three-dimensional model mapping method is only applicable to the object surface with diffuse reflection, and cannot perform highlight mapping on the object surface with highlight reflection.
The nerve radiation field uses a nerve network formed by a plurality of layers of perceptrons to implicitly model a complex three-dimensional scene, and only uses multi-view images and camera pose to train a network model, so that clear scene images at any view point and in any observation direction can be rendered. The neural radiation field hidden function model essentially models the density and color of any point in three-dimensional space in different directions of view, so that the neural radiation field hidden function model can be used to calculate the integral along the direction of view to obtain the color at any point of the model surface, and the mapping of the model surface is synthesized based on the color.
Disclosure of Invention
In view of the above-mentioned drawbacks of the prior art, an object of the present invention is to provide a three-dimensional model mapping method, system, storage medium and terminal based on a nerve radiation field, which uses diffuse reflection mapping, highlight level mapping and gloss mapping to render a model, so that the generated mapping is applicable to not only purely diffuse reflection objects, but also objects with highlight materials such as metal and ceramic tiles, thereby improving the rendering effect of the three-dimensional model and enabling the three-dimensional model to be more attached to real model materials.
To achieve the above and other related objects, the present invention provides a three-dimensional model mapping method based on a neural radiation field, comprising the steps of: expanding the surface of the three-dimensional model to a UV coordinate system; acquiring surface sampling points of the three-dimensional model from the UV coordinate system; calculating a specular reflection color, a diffuse reflection color, a specular level, and a gloss based on the surface sampling points; and synthesizing a high-light reflection map based on the high-light reflection color, synthesizing a diffuse reflection map based on the diffuse reflection color, synthesizing a high-light level map based on the high-light level, and synthesizing a gloss map based on the glossiness.
In an embodiment of the present invention, obtaining the surface sampling points of the three-dimensional model from the UV coordinate system includes the following steps: rasterizing all grids on the three-dimensional model surface; and uniformly sampling all grids which are rasterized.
In an embodiment of the present invention, calculating the specular reflection color, diffuse reflection color, specular level, and gloss based on the surface sampling points includes the steps of:
determining a rough normal direction based on the surface sampling points;
determining a rough specular direction based on the rough normal direction;
calculating a specular color using a neural radiation field based on the rough specular direction and determining a specular direction;
and calculating diffuse reflection color, high light level and glossiness by using the nerve radiation field based on the high light reflection direction.
In an embodiment of the present invention, determining the rough normal direction based on the surface sampling point includes the steps of:
selecting a first viewing direction based on the surface sampling points;
calculating color and density along each of the first observation directions using a neural radiation field to determine first feature points in each of the first observation directions reaching a color threshold and reaching a density threshold;
calculating the distance between the first characteristic point and the surface sampling point;
removing a first observation direction in which the first feature point larger than a distance threshold is located;
and vector sum is carried out on the rest first observation directions, and the opposite directions are taken as rough normal directions.
In an embodiment of the invention, determining the rough specular direction based on the rough normal direction includes the steps of:
selecting a second observation direction based on the rough normal direction, wherein an included angle between the second observation direction and a direction opposite to the rough normal direction is smaller than a first included angle threshold;
calculating color and density along each of the second observation directions using the nerve radiation field to determine second feature points in each of the second observation directions reaching a color threshold and reaching a density threshold;
calculating the distance between the second characteristic point and the surface sampling point;
removing a second observation direction in which the second feature points larger than a distance threshold are located;
and selecting the observation direction with highest brightness from the rest second observation directions, and taking the opposite direction as a rough high light reflection direction.
In one embodiment of the present invention, calculating a specular color using a neural radiation field based on the rough specular direction and determining the specular direction includes the steps of:
selecting a third viewing direction based on the rough specular direction, the third viewing direction having an angle with the direction opposite the rough specular direction that is less than a second angle threshold;
calculating colors along each third observation direction by using the nerve radiation field, taking the color with the highest brightness as a specular reflection color, and taking the opposite direction of the third observation direction in which the specular reflection color is positioned as a specular reflection direction.
In one embodiment of the present invention, calculating diffuse reflection color, high light level and glossiness using a neural radiation field based on the high light reflection direction includes the steps of:
screening the second observation direction from the second observation direction, wherein an included angle between the second observation direction and the specular reflection direction is more than or equal to 90 degrees, so as to obtain an observation direction without specular reflection;
calculating a specular free color along the specular free viewing direction using a neural radiation field;
arranging the non-specular reflection colors in descending order or ascending order according to brightness to form a color brightness sequence;
averaging the color brightness of a preset percentage in the middle of the color brightness sequence to obtain diffuse reflection color;
calculating a specular level based on the diffuse reflectance color and the specular reflectance color;
the glossiness is calculated based on the diffuse reflection color, the specular reflection color, and the specular level.
Correspondingly, the invention provides a three-dimensional model mapping system based on a nerve radiation field, which comprises the following components:
the sampling module is used for expanding the surface of the three-dimensional model to a UV coordinate system and acquiring surface sampling points of the three-dimensional model from the UV coordinate system;
the nerve radiation field module is used for calculating high-light reflection color, diffuse reflection color, high-light level and glossiness based on the surface sampling points;
and the mapping module is used for synthesizing a highlight reflection mapping based on the highlight reflection color, synthesizing a diffuse reflection mapping based on the diffuse reflection color, synthesizing a highlight level mapping based on the highlight level and synthesizing a gloss mapping based on the glossiness.
Correspondingly, the invention provides an electronic device comprising at least one memory and a processor:
the memory is used for storing a computer program;
the processor is configured to execute the computer program stored in the memory, so that the electronic device performs any one of the three-dimensional model mapping method based on the nerve radiation field.
Correspondingly, the invention provides a computer readable storage medium, wherein computer execution instructions are stored in the computer readable storage medium, and when a processor executes the computer execution instructions, the three-dimensional model mapping method based on the nerve radiation field is realized.
As described above, the three-dimensional model mapping method, system, storage medium and terminal based on the nerve radiation field have the following beneficial effects:
(1) Rendering a model using the diffuse reflection map, the highlight level map, and the gloss map;
(2) The generated map is not only suitable for purely diffuse reflection objects, but also suitable for objects with high-light materials such as metal, ceramic tiles and the like;
(3) The rendering effect of the three-dimensional model is improved, and the three-dimensional model is more attached to the real model material.
Drawings
FIG. 1 is a flow chart of a three-dimensional model mapping method based on neural radiation field according to an embodiment of the invention.
FIG. 2 is a schematic diagram showing a rough normal direction of a three-dimensional model mapping method based on a neural radiation field according to an embodiment of the invention.
FIG. 3 is a schematic diagram showing the rough specular direction of the neural radiation field-based three-dimensional model mapping method according to an embodiment of the invention.
FIG. 4 is a schematic diagram showing the specular reflection direction of the neural radiation field-based three-dimensional model mapping method according to an embodiment of the invention.
FIG. 5 is a schematic diagram of a three-dimensional model mapping system based on neural radiation field according to an embodiment of the present invention.
Fig. 6 is a schematic structural diagram of a three-dimensional model mapping terminal based on a neural radiation field according to an embodiment of the present invention.
Description of element reference numerals
S1-S4 method steps
1. Surface sampling point
2. Outside the model
21. First viewing direction outside the model
211. Rough normal
22. Second viewing direction
221. High light incidence direction
222. Rough specular direction
223. High light reflection direction
23. Second viewing direction outside the model
3. In the model
31. First viewing direction in a model
51. Sampling module
52. Neural radiation field module
53. Mapping module
61. Processor and method for controlling the same
62. Memory device
Detailed Description
The following description of the embodiments of the present invention will be made clearly and fully with reference to the accompanying drawings, in which it is evident that the embodiments described are only some, but not all embodiments of the invention. All other embodiments, which can be made by one of ordinary skill in the art without creative efforts, are within the scope of the present invention based on the embodiments of the present invention.
It should be noted that, the illustrations provided in the present embodiment merely illustrate the basic concept of the present invention by way of illustration, and only the components related to the present invention are shown in the drawings and are not drawn according to the number, shape and size of the components in actual implementation, and the form, number and proportion of the components in actual implementation may be arbitrarily changed, and the layout of the components may be more complex.
All directional indications (such as up, down, left, right, front, back … …) in the embodiments of the present invention are merely used to explain the relative positional relationship, movement, etc. between the components in a particular gesture (as shown in the drawings), and if the particular gesture changes, the directional indication changes accordingly.
In the present invention, the description as relating to "first", "second", etc. is for descriptive purposes only and is not to be construed as indicating or implying a relative importance thereof or implicitly indicating the number of technical features indicated. Thus, a feature defining "a first" or "a second" may explicitly or implicitly include at least one such feature. In addition, the technical solutions of the embodiments may be combined with each other, but it is necessary to base that the technical solutions can be realized by those skilled in the art, and when the technical solutions are contradictory or cannot be realized, the combination of the technical solutions should be considered to be absent and not within the scope of protection claimed in the present invention.
The three-dimensional model mapping method, the system, the storage medium and the terminal based on the nerve radiation field render the model by utilizing the diffuse reflection mapping, the highlight level mapping and the gloss mapping, so that the generated mapping is not only suitable for purely diffuse reflection objects, but also suitable for objects with highlight materials such as metal, ceramic tiles and the like, the rendering effect of the three-dimensional model is improved, and the three-dimensional model is more attached to real model materials.
As shown in fig. 1, in an embodiment, the three-dimensional model mapping method based on the neural radiation field of the present invention includes the following steps:
and S1, expanding the surface of the three-dimensional model to a UV coordinate system.
Specifically, the UV coordinate system is a texture map coordinate system, and has two coordinate axes, i.e., U and V, where the U axis represents the pixel distribution on the abscissa and the V axis represents the pixel distribution on the ordinate. The UV texture map coordinates define the positional information of each point on the image from which each point on the image can be precisely mapped to the model surface since each point on the image is in turn correlated to the three-dimensional model.
And S2, acquiring surface sampling points of the three-dimensional model from the UV coordinate system.
In one embodiment, acquiring the surface sampling points of the three-dimensional model from the UV coordinate system includes the steps of:
rasterizing all grids on the three-dimensional model surface;
and uniformly sampling all grids which are rasterized.
And step S3, calculating the specular reflection color, the diffuse reflection color, the specular level and the glossiness based on the surface sampling points.
In one embodiment, calculating the specular reflection color, diffuse reflection color, specular level, and gloss based on the surface sampling points comprises the steps of:
(1) A rough normal direction is determined based on the surface sampling points.
In one embodiment, determining a rough normal direction based on the surface sampling points comprises the steps of:
selecting a first viewing direction based on the surface sampling points;
calculating color and density along each of the first observation directions using a neural radiation field to determine first feature points in each of the first observation directions reaching a color threshold and reaching a density threshold;
calculating the distance between the first characteristic point and the surface sampling point;
removing a first observation direction in which the first feature point larger than a distance threshold is located;
and vector sum is carried out on the rest first observation directions, and the opposite directions are taken as rough normal directions.
In an embodiment, the color threshold is fixed to 1, the density threshold ranges from 0.5 to 0.8, and the distance threshold is a preset parameter, and the size of the distance threshold is related to the grid size of the rasterization.
As shown in fig. 2, in an embodiment, the surface of the three-dimensional model includes a surface sampling point 1, a first observation direction 21 outside the model 2 and a first observation direction 31 inside the model 3, and the first observation directions 21 outside the model and 31 inside the model are respectively distributed at equal intervals on the outer side and the inner side of the model. Color and density are calculated using the nerve radiation field along each of the outside-model first viewing directions 21 and each of the inside-model first viewing directions 31 to determine first feature points that reach a color threshold and reach a density threshold, and distances between the first feature points and the surface sampling points are calculated. Because the opaque object in the model is blocked, the observed first feature point is not the sampling point 1 on the surface of the actual object, that is, a certain distance exists between the first feature point and the surface sampling point 1, so that the first observation direction in which the first feature point greater than the distance threshold is located can be regarded as the first observation direction in the model. To avoid judging the normal direction due to the occlusion of the object, the first observation direction 31 inside the model needs to be removed, and then the first observation direction 21 outside the model needs to be vector-summed to determine the observation direction 211 most perpendicular to the surface of the model, and the opposite direction is taken as the rough normal direction.
(2) A coarse specular direction is determined based on the coarse normal direction.
In one embodiment, determining the rough specular direction based on the rough normal direction comprises the steps of:
selecting a second observation direction based on the rough normal direction, wherein an included angle between the second observation direction and a direction opposite to the rough normal direction is smaller than a first included angle threshold;
calculating color and density along each of the second observation directions using the nerve radiation field to determine second feature points in each of the second observation directions reaching a color threshold and reaching a density threshold;
calculating the distance between the second characteristic point and the surface sampling point;
removing a second observation direction in which the second feature points larger than a distance threshold are located;
and selecting the observation direction with highest brightness from the rest second observation directions, and taking the opposite direction as a rough high light reflection direction.
In one embodiment, as shown in fig. 3, the surface of the three-dimensional model includes a surface sampling point 1 and a second viewing direction 22 outside the model 2. Specifically, the included angle between all the second directions 22 outside the model and the rough normal direction 211 (i.e. the first included angle threshold value) is not more than 90 °, and the second directions 22 outside the model are equally spaced outside the model surface, and the number is far greater than the number of the first directions 21 outside the model. Since the selected second viewing direction may come from the model 3, the rough normal direction 211 may be tilted so that it does not completely overlap with the true normal. Therefore, it is necessary to remove the second observation direction of the inner part 3 of the model by using the nerve radiation field, and then determine the observation direction with the highest brightness from the second observation directions 22 outside the remaining models, and take the opposite direction as the rough specular direction 222, and the direction in which the rough specular direction 222 is symmetrical with respect to the normal line is the specular direction 221.
In an embodiment, when the rough specular direction 222 is not the same, the vector sum of all specular directions 222 needs to be solved to obtain the mean vector as the specular direction.
(3) And calculating a specular color by using the neural radiation field based on the rough specular direction, and determining the specular direction.
In one embodiment, calculating a specular color using a neural radiation field based on the rough specular direction and determining the specular direction includes the steps of:
selecting a third viewing direction based on the rough specular direction, the third viewing direction having an angle with the direction opposite the rough specular direction that is less than a second angle threshold;
calculating colors along each third observation direction by using the nerve radiation field, taking the color with the highest brightness as a specular reflection color, and taking the opposite direction of the third observation direction in which the specular reflection color is positioned as a specular reflection direction.
In one embodiment, as shown in fig. 4, the surface of the three-dimensional model includes a surface sampling point 1 and a third viewing direction 23 outside the model 2. In particular, all the off-model third viewing directions 23 are equally spaced about the rough specular direction by an angle of no more than 10 ° from the rough specular direction 222 (i.e., the second angle threshold). The neural radiation field is used to calculate the color along each of the third viewing directions, determine the third viewing direction in which the color with the highest brightness is located, and take the opposite direction as the more accurate specular direction 223.
(4) And calculating diffuse reflection color, high light level and glossiness by using the nerve radiation field based on the high light reflection direction.
In one embodiment, calculating diffuse reflectance color, high light level, and gloss using a neural radiation field based on the high light reflectance direction comprises the steps of:
screening the second observation direction from the second observation direction, wherein an included angle between the second observation direction and the specular reflection direction is more than or equal to 90 degrees, so as to obtain an observation direction without specular reflection;
calculating a specular free color along the specular free viewing direction using a neural radiation field;
arranging the non-specular reflection colors in descending order or ascending order according to brightness to form a color brightness sequence;
averaging the color brightness of a preset percentage in the middle of the color brightness sequence to obtain diffuse reflection color;
calculating a specular level based on the diffuse reflectance color and the specular reflectance color;
the glossiness is calculated based on the diffuse reflection color, the specular reflection color, and the specular level.
In one embodiment, the predetermined percentage has a value of 70%.
In one embodiment, the high light level is calculated using a Phong illumination model based on the diffuse reflectance color and the high light reflectance color using the formula:
where Ks represents the high light level,Light specular indicating high Light reflection color, light diffuse Representing diffuse reflection color.
In one embodiment, based on the diffuse reflectance color, the specular reflectance color, and the specular level, the gloss is calculated using a Phong illumination model using the formula:
wherein alpha is gloss, light specular Indicating high Light reflection color, light diffuse Indicating diffuse reflection color, ks indicating high light level,indicating the direction of high light reflection, ">Indicating the direction of the line of sight. The intensity of the specular reflection is greater if the angle between the specular reflection direction and the line of sight direction is smaller.
And S4, synthesizing a highlight reflection map based on the highlight reflection color, synthesizing a diffuse reflection map based on the diffuse reflection color, synthesizing a highlight level map based on the highlight level, and synthesizing a gloss map based on the glossiness.
Specifically, the essence of UV texture mapping is the mapping of an image, which is the rendering of an image onto the surface of an object by means of UV coordinate mapping. The characteristics of each pixel point in the image comprise a specular reflection color, a diffuse reflection color, a specular level and glossiness, wherein the highest value of the specular reflection color and the diffuse reflection color is white, and the lowest value is black; the high light level reflects the intensity of the high light; the glossiness can control the dispersion degree of the highlight, and indirectly reflects the smoothness or roughness of the surface of the object. The high light reflection color, diffuse reflection color, high light level, and glossiness are key factors affecting the visual effect of the object surface.
As shown in fig. 5, in one embodiment, the three-dimensional model mapping system based on neural radiation field of the present invention comprises:
the sampling module 51 is configured to expand the surface of the three-dimensional model to a UV coordinate system, and acquire a surface sampling point of the three-dimensional model from the UV coordinate system.
Specifically, the UV coordinate system is a texture map coordinate system, and has two coordinate axes, i.e., U and V, where the U axis represents the pixel distribution on the abscissa and the V axis represents the pixel distribution on the ordinate. The UV texture map coordinates define the positional information of each point on the image from which each point on the image can be precisely mapped to the model surface since each point on the image is in turn correlated to the three-dimensional model.
In one embodiment, acquiring the surface sampling points of the three-dimensional model from the UV coordinate system includes the steps of:
rasterizing all grids on the three-dimensional model surface;
and uniformly sampling all grids which are rasterized.
The neural radiation field module 52 is connected to the sampling module 51 for calculating the specular color, diffuse color, specular level, and gloss based on the surface sampling points.
In one embodiment, calculating the specular reflection color, diffuse reflection color, specular level, and gloss based on the surface sampling points comprises the steps of:
(1) A rough normal direction is determined based on the surface sampling points.
In one embodiment, determining a rough normal direction based on the surface sampling points comprises the steps of:
selecting a first viewing direction based on the surface sampling points;
calculating color and density along each of the first observation directions using a neural radiation field to determine first feature points in each of the first observation directions reaching a color threshold and reaching a density threshold;
calculating the distance between the first characteristic point and the surface sampling point;
removing a first observation direction in which the first feature point larger than a distance threshold is located;
and vector sum is carried out on the rest first observation directions, and the opposite directions are taken as rough normal directions.
In an embodiment, the color threshold is fixed to 1, the density threshold ranges from 0.5 to 0.8, and the distance threshold is a preset parameter, and the size of the distance threshold is related to the grid size of the rasterization.
As shown in fig. 2, in an embodiment, the three-dimensional model includes a sampling point 1 of a model surface, a first observation direction 21 outside the model 2, and a first observation direction 31 inside the model 3, and the first observation directions 21 outside the model and 31 inside the model are respectively distributed at equal intervals on the outer side and the inner side of the model. Color and density are calculated using the nerve radiation field along each of the outside-model first viewing directions 21 and each of the inside-model first viewing directions 31 to determine first feature points that reach a color threshold and reach a density threshold, and distances between the first feature points and the surface sampling points are calculated. Because the opaque object in the model is blocked, the observed first feature point is not the sampling point 1 on the surface of the actual object, that is, a certain distance exists between the first feature point and the surface sampling point 1, so that the first observation direction in which the first feature point greater than the distance threshold is located can be regarded as the first observation direction in the model. In order to avoid judging the normal direction due to the occlusion of the object, the first observation direction 31 in the model needs to be removed first, and then the first observation direction 21 outside the model is vector-summed to determine the observation direction 211 most perpendicular to the surface of the model, and the opposite direction is taken as the rough normal direction.
(2) A coarse specular direction is determined based on the coarse normal direction.
In one embodiment, determining the rough specular direction based on the rough normal direction comprises the steps of:
selecting a second observation direction based on the rough normal direction, wherein an included angle between the second observation direction and a direction opposite to the rough normal direction is smaller than a first included angle threshold;
calculating color and density along each of the second observation directions using the nerve radiation field to determine second feature points in each of the second observation directions reaching a color threshold and reaching a density threshold;
calculating the distance between the second characteristic point and the surface sampling point;
removing a second observation direction in which the second feature points larger than a distance threshold are located;
and selecting the observation direction with highest brightness from the rest second observation directions, and taking the opposite direction as a rough high light reflection direction.
In one embodiment, as shown in fig. 3, the surface of the three-dimensional model includes a surface sampling point 1 and a second viewing direction 22 outside the model 2. Specifically, the included angle between all the second directions 22 outside the model and the rough normal direction 211 (i.e. the first included angle threshold value) is not more than 90 °, and the second directions 22 outside the model are equally spaced outside the model surface, and the number is far greater than the number of the first directions 21 outside the model. Since the selected second viewing direction may come from the model 3, the rough normal direction 211 may be tilted so that it does not completely overlap with the true normal. Therefore, it is necessary to remove the second observation direction of the inner part 3 of the model by using the nerve radiation field, and then determine the observation direction with the highest brightness from the second observation directions 22 outside the remaining models, and take the opposite direction as the rough specular direction 222, and the direction in which the rough specular direction 222 is symmetrical with respect to the normal line is the specular direction 221.
In one embodiment, when the rough specular direction is not the same, the vector sum of all specular directions 222 is solved to obtain a mean vector.
(3) And calculating a specular color by using the neural radiation field based on the rough specular direction, and determining the specular direction.
In one embodiment, calculating a specular color using a neural radiation field based on the rough specular direction and determining the specular direction includes the steps of:
selecting a third viewing direction based on the rough specular direction, the third viewing direction having an angle with the direction opposite the rough specular direction that is less than a second angle threshold;
calculating colors along each third observation direction by using the nerve radiation field, taking the color with the highest brightness as a specular reflection color, and taking the opposite direction of the third observation direction in which the specular reflection color is positioned as a specular reflection direction.
In one embodiment, as shown in fig. 4, the surface of the three-dimensional model includes a surface sampling point 1 and a third viewing direction 23 outside the model 2. In particular, all the off-model third viewing directions 23 are equally spaced about the rough specular direction by an angle of no more than 10 ° from the rough specular direction 222 (i.e., the second angle threshold). The neural radiation field is used to calculate the color along each of the third viewing directions, determine the third viewing direction in which the color with the highest brightness is located, and take the opposite direction as the more accurate specular direction 223.
(4) And calculating diffuse reflection color, high light level and glossiness by using the nerve radiation field based on the high light reflection direction.
In one embodiment, calculating diffuse reflectance color, high light level, and gloss using a neural radiation field based on the high light reflectance direction comprises the steps of:
screening the second observation direction from the second observation direction, wherein an included angle between the second observation direction and the specular reflection direction is more than or equal to 90 degrees, so as to obtain an observation direction without specular reflection;
calculating a specular free color along the specular free viewing direction using a neural radiation field;
arranging the non-specular reflection colors in descending order or ascending order according to brightness to form a color brightness sequence;
averaging the color brightness of a preset percentage in the middle of the color brightness sequence to obtain diffuse reflection color;
calculating a specular level based on the diffuse reflectance color and the specular reflectance color;
the glossiness is calculated based on the diffuse reflection color, the specular reflection color, and the specular level.
In one embodiment, the predetermined percentage has a value of 70%.
In one embodiment, the high light level is calculated using a Phong illumination model based on the diffuse reflectance color and the high light reflectance color using the formula:
where Ks represents the high Light level, light specular Indicating high Light reflection color, light diffuse Representing diffuse reflection color.
In one embodiment, based on the diffuse reflectance color, the specular reflectance color, and the specular level, the gloss is calculated using a Phong illumination model using the formula:
wherein alpha is gloss, light specular Indicating high Light reflection color, light diffuse Indicating diffuse reflection color, ks indicating high light level,indicating the direction of high light reflection, ">Indicating the direction of the line of sight. The intensity of the specular reflection is greater if the angle between the specular reflection direction and the line of sight direction is smaller.
The mapping module 53 is connected to the neural radiation field module 52, and is configured to synthesize a specular reflection map based on the specular reflection color, synthesize a diffuse reflection map based on the diffuse reflection color, synthesize a specular level map based on the specular level, and synthesize a gloss map based on the gloss level.
Specifically, the essence of UV texture mapping is the mapping of an image, which is the rendering of an image onto the surface of an object by means of UV coordinate mapping. The characteristics of each pixel point in the image comprise a specular reflection color, a diffuse reflection color, a specular level and glossiness, wherein the highest value of the specular reflection color and the diffuse reflection color is white, and the lowest value is black; the high light level reflects the intensity of the high light; the glossiness can control the dispersion degree of the highlight, and indirectly reflects the smoothness or roughness of the surface of the object. The high light reflection color, diffuse reflection color, high light level, and glossiness are key factors affecting the visual effect of the object surface.
It should be noted that, it should be understood that the division of the modules of the above apparatus is merely a division of a logic function, and may be fully or partially integrated into a physical entity or may be physically separated. And these modules may all be implemented in software in the form of calls by the processing element; or can be realized in hardware; the method can also be realized in a form of calling software by a processing element, and the method can be realized in a form of hardware by a part of modules. For example, the x module may be a processing element that is set up separately, may be implemented in a chip of the apparatus, or may be stored in a memory of the apparatus in the form of program code, and the function of the x module may be called and executed by a processing element of the apparatus. The implementation of the other modules is similar. In addition, all or part of the modules can be integrated together or can be independently implemented. The processing element described herein may be an integrated circuit having signal processing capabilities. In implementation, each step of the above method or each module above may be implemented by an integrated logic circuit of hardware in a processor element or an instruction in a software form.
For example, the modules above may be one or more integrated circuits configured to implement the methods above, such as: one or more application specific integrated circuits (Application Specific Integrated Circuit, abbreviated as ASIC), or one or more microprocessors (Digital Signal Processor, abbreviated as DSP), or one or more field programmable gate arrays (Field Programmable Gate Array, abbreviated as FPGA), or the like. For another example, when a module above is implemented in the form of a processing element scheduler code, the processing element may be a general-purpose processor, such as a central processing unit (Central Processing Unit, CPU) or other processor that may invoke the program code. For another example, the modules may be integrated together and implemented in the form of a system-on-a-chip (SOC).
As shown in fig. 6, in an embodiment, the three-dimensional model mapping terminal based on a neural radiation field of the present invention includes: a processor 61 and a memory 62.
The memory 62 is used for storing a computer program.
The memory 62 includes: various media capable of storing program codes, such as ROM, RAM, magnetic disk, U-disk, memory card, or optical disk.
The processor 61 is connected to the memory 62 and is configured to execute a computer program stored in the memory 62, so that the three-dimensional model mapping terminal based on the neural radiation field performs the three-dimensional model mapping method based on the neural radiation field.
Preferably, the processor 61 may be a general-purpose processor, including a central processing unit (Central Processing Unit, abbreviated as CPU), a Network Processor (NP), etc.; but also digital signal processors (Digital Signal Processor, DSP for short), application specific integrated circuits (Application Specific Integrated Circuit, ASIC for short), field programmable gate arrays (Field Programmable Gate Array, FPGA for short) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.
In summary, in the three-dimensional model mapping method, system, storage medium and terminal based on the nerve radiation field, the diffuse reflection mapping, the highlight mapping, the high-light level mapping and the gloss mapping are utilized to render the model, so that the generated mapping is not only suitable for purely diffuse reflection objects, but also suitable for objects with highlight materials such as metal, ceramic tiles and the like, the rendering effect of the three-dimensional model is improved, and the three-dimensional model is more attached to real model materials. Therefore, the invention effectively overcomes various defects in the prior art and has high industrial utilization value.
The above embodiments are merely illustrative of the principles of the present invention and its effectiveness, and are not intended to limit the invention. Modifications and variations may be made to the above-described embodiments by those skilled in the art without departing from the spirit and scope of the invention. Accordingly, it is intended that all equivalent modifications and variations of the invention be covered by the claims, which are within the ordinary skill of the art, be within the spirit and scope of the present disclosure.
Claims (10)
1. A three-dimensional model mapping method based on a nerve radiation field, which is characterized by comprising the following steps:
expanding the surface of the three-dimensional model to a UV coordinate system;
acquiring surface sampling points of the three-dimensional model from the UV coordinate system;
calculating a specular reflection color, a diffuse reflection color, a specular level, and a gloss based on the surface sampling points;
and synthesizing a high-light reflection map based on the high-light reflection color, synthesizing a diffuse reflection map based on the diffuse reflection color, synthesizing a high-light level map based on the high-light level, and synthesizing a gloss map based on the glossiness.
2. The neural radiation field based three-dimensional model mapping method of claim 1, wherein obtaining surface sampling points of the three-dimensional model from the UV coordinate system comprises the steps of:
rasterizing all grids on the three-dimensional model surface;
and uniformly sampling all grids which are rasterized.
3. The neural radiation field based three-dimensional model mapping method of claim 1, wherein calculating specular color, diffuse color, specular level, and gloss based on the surface sampling points comprises the steps of:
determining a rough normal direction based on the surface sampling points;
determining a rough specular direction based on the rough normal direction;
calculating a specular color using a neural radiation field based on the rough specular direction and determining a specular direction;
and calculating diffuse reflection color, high light level and glossiness by using the nerve radiation field based on the high light reflection direction.
4. A three-dimensional model mapping method based on neural radiation fields according to claim 3, characterized in that determining a rough normal direction based on the surface sampling points comprises the steps of:
selecting a first viewing direction based on the surface sampling points;
calculating color and density along each of the first observation directions using a neural radiation field to determine first feature points in each of the first observation directions reaching a color threshold and reaching a density threshold;
calculating the distance between the first characteristic point and the surface sampling point;
removing a first observation direction in which the first feature point larger than a distance threshold is located;
and vector sum is carried out on the rest first observation directions, and the opposite directions are taken as rough normal directions.
5. A three-dimensional model mapping method based on neural radiation fields according to claim 3, characterized in that determining a rough specular reflection direction based on the rough normal direction comprises the steps of:
selecting a second observation direction based on the rough normal direction, wherein an included angle between the second observation direction and a direction opposite to the rough normal direction is smaller than a first included angle threshold;
calculating color and density along each of the second observation directions using the nerve radiation field to determine second feature points in each of the second observation directions reaching a color threshold and reaching a density threshold;
calculating the distance between the second characteristic point and the surface sampling point;
removing a second observation direction in which the second feature points larger than a distance threshold are located;
and selecting the observation direction with highest brightness from the rest second observation directions, and taking the opposite direction as a rough high light reflection direction.
6. A three-dimensional model mapping method based on neural radiation fields according to claim 3, characterized in that, based on the rough specular direction, specular color is calculated using the neural radiation fields and specular direction is determined, comprising the steps of:
selecting a third viewing direction based on the rough specular direction, the third viewing direction having an angle with the direction opposite the rough specular direction that is less than a second angle threshold;
calculating colors along each third observation direction by using the nerve radiation field, taking the color with the highest brightness as a specular reflection color, and taking the opposite direction of the third observation direction in which the specular reflection color is positioned as a specular reflection direction.
7. The neural radiation field based three-dimensional model mapping method of claim 5, wherein calculating diffuse reflectance color, high light level, and gloss using the neural radiation field based on the high light reflectance direction comprises the steps of:
screening the second observation direction from the second observation direction, wherein an included angle between the second observation direction and the specular reflection direction is more than or equal to 90 degrees, so as to obtain an observation direction without specular reflection;
calculating a specular free color along the specular free viewing direction using a neural radiation field;
arranging the non-specular reflection colors in descending order or ascending order according to brightness to form a color brightness sequence;
averaging the color brightness of a preset percentage in the middle of the color brightness sequence to obtain diffuse reflection color;
calculating a specular level based on the diffuse reflectance color and the specular reflectance color;
the glossiness is calculated based on the diffuse reflection color, the specular reflection color, and the specular level.
8. A three-dimensional model mapping system based on a neural radiation field, comprising:
the sampling module is used for expanding the surface of the three-dimensional model to a UV coordinate system and acquiring surface sampling points of the three-dimensional model from the UV coordinate system;
the nerve radiation field module is used for calculating high-light reflection color, diffuse reflection color, high-light level and glossiness based on the surface sampling points;
and the mapping module is used for synthesizing a highlight reflection mapping based on the highlight reflection color, synthesizing a diffuse reflection mapping based on the diffuse reflection color, synthesizing a highlight level mapping based on the highlight level and synthesizing a gloss mapping based on the glossiness.
9. A three-dimensional model mapping terminal based on a neural radiation field, characterized by comprising at least one memory and a processor:
the memory is used for storing a computer program;
the processor is configured to execute the computer program stored in the memory, so that the three-dimensional model mapping terminal based on the nerve radiation field executes the three-dimensional model mapping method based on the nerve radiation field according to any one of claims 1 to 7.
10. A computer readable storage medium having stored therein computer executable instructions which when executed by a processor implement the method of any of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210483487.8A CN117058287A (en) | 2022-05-05 | 2022-05-05 | Three-dimensional model mapping method, system, storage medium and terminal based on nerve radiation field |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210483487.8A CN117058287A (en) | 2022-05-05 | 2022-05-05 | Three-dimensional model mapping method, system, storage medium and terminal based on nerve radiation field |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117058287A true CN117058287A (en) | 2023-11-14 |
Family
ID=88661322
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210483487.8A Pending CN117058287A (en) | 2022-05-05 | 2022-05-05 | Three-dimensional model mapping method, system, storage medium and terminal based on nerve radiation field |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117058287A (en) |
-
2022
- 2022-05-05 CN CN202210483487.8A patent/CN117058287A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Krivanek et al. | Practical global illumination with irradiance caching | |
| US12014457B2 (en) | Importance sampling for determining a light map | |
| US8896602B2 (en) | Apparatus and method for finding visible points in a point cloud | |
| US7091973B1 (en) | Apparatus and method for estimating reflected radiance under complex distant illumination | |
| US7362332B2 (en) | System and method of simulating motion blur efficiently | |
| JP2006053911A (en) | Shell texture function | |
| CN111583381B (en) | Game resource map rendering method and device and electronic equipment | |
| CN114119853B (en) | Image rendering method, device, equipment and medium | |
| CN112819941B (en) | Method, apparatus, device and computer readable storage medium for rendering water surface | |
| CN112419460B (en) | Method, apparatus, computer device and storage medium for baking model map | |
| EP0856815B1 (en) | Method and system for determining and/or using illumination maps in rendering images | |
| CN116091684B (en) | WebGL-based image rendering method, device, equipment and storage medium | |
| JP3629243B2 (en) | Image processing apparatus and method for rendering shading process using distance component in modeling | |
| CN117058287A (en) | Three-dimensional model mapping method, system, storage medium and terminal based on nerve radiation field | |
| AU2017228700A1 (en) | System and method of rendering a surface | |
| JP4764963B2 (en) | Image processing device | |
| KR20100075351A (en) | Method and system for rendering mobile computer graphic | |
| KR100372901B1 (en) | Ray-tracing acceleration method using ZF-buffer | |
| JP5321385B2 (en) | Image processing program and computer-readable recording medium | |
| JP2016119041A (en) | Image processing system, image processor, image processing method, program and recording medium | |
| CN117333603A (en) | Virtual model rendering method, device, equipment and storage medium | |
| Brill et al. | Immersive surface interrogation | |
| CN115131298A (en) | Viewpoint visibility evaluation method for two-dimensional image shooting | |
| JP2006195938A (en) | Glare rendering circuit | |
| CN111986303A (en) | Fluid rendering method and device, storage medium and terminal equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |