CN106558076B - The method and apparatus of three-dimensional reconstruction object - Google Patents
The method and apparatus of three-dimensional reconstruction object Download PDFInfo
- Publication number
- CN106558076B CN106558076B CN201510590009.7A CN201510590009A CN106558076B CN 106558076 B CN106558076 B CN 106558076B CN 201510590009 A CN201510590009 A CN 201510590009A CN 106558076 B CN106558076 B CN 106558076B
- Authority
- CN
- China
- Prior art keywords
- voxel
- value
- tsdf value
- tsdf
- initial local
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2200/00—Indexing scheme for image data processing or generation, in general
- G06T2200/28—Indexing scheme for image data processing or generation, in general involving image processing hardware
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10028—Range image; Depth image; 3D point clouds
Landscapes
- Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Computer Graphics (AREA)
- Geometry (AREA)
- Software Systems (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Image Generation (AREA)
Abstract
The invention discloses a kind of methods and apparatus of three-dimensional reconstruction object.This method comprises: obtaining the initial local TSDF value of voxel in three-dimensional space, each initial local TSDF value corresponds to a depth map in multiple depth maps;By corresponding depth map, initial local TSDF value is grouped, at least one set of initial local TSDF value corresponds to more than one depth map;For every group of initial local TSDF value, the global TSDF value of the group is obtained;Using the global TSDF value of obtained each group as initial local TSDF value, optimization problem is solved, to obtain final overall situation TSDF value;And it is based on obtained final overall situation TSDF value, object described in three-dimensional reconstruction;Wherein, in the optimization problem, variable is the parameter of global the TSDF value and transformation of voxel, cost function and following factors are related: the global TSDF value of particular voxel and the difference of the initial local TSDF value of the transformed corresponding voxel of the voxel square weighted sum, weight is equal to the weight of the corresponding group of the transformed corresponding voxel of particular voxel.
Description
Technical field
This invention relates generally to three-dimensional imaging fields.Specifically, can accurately be carried out pair the present invention relates to one kind
The method and apparatus of the three-dimensional reconstruction of elephant.
Background technique
In recent years, with the development of 3 dimension imaging technology, many relevant applications have been emerged in large numbers, as augmented reality, number are rich
Object shop, 3 D-printing etc..The importance of 3 dimension imaging technology is three-dimensional reconstruction.The three-dimensional rebuilding method of mainstream is based on
One group of depth image rebuilds three dimensional object.
Traditional three-dimensional rebuilding method focuses mainly on how more accurately estimating the parameter of camera, such as draws
Distortion of the deformation information to estimate camera is entered.But traditional three-dimensional rebuilding method, calculating overall situation TSDF, (truncation has
When being characterized to distance function, Truncated Signed Distance Function), only simply to all parts
TSDF characterization carries out arithmetic average, without sufficiently excavating the relationship between local T SDF characterization and overall situation TSDF characterization.Due to only
The effect that three-dimensional reconstruction is promoted by the estimation improved to camera parameter, so traditional three-dimensional rebuilding method is effect improved
Limited, the accuracy of three-dimensional reconstruction object needs to be further increased.
Therefore, it is desirable to which a kind of method and apparatus of three-dimensional reconstruction object, can accurately carry out the three-dimensional reconstruction of object.
Summary of the invention
It has been given below about brief overview of the invention, in order to provide about the basic of certain aspects of the invention
Understand.It should be appreciated that this summary is not an exhaustive overview of the invention.It is not intended to determine pass of the invention
Key or pith, nor is it intended to limit the scope of the present invention.Its purpose only provides certain concepts in simplified form,
Taking this as a prelude to a more detailed description discussed later.
The purpose of the present invention is in view of the above problems in the prior art, propose one kind to lay particular emphasis on excavation local T SDF characterization
The method and apparatus of the three-dimensional reconstruction object of relationship between global TSDF characterization.
To achieve the goals above, according to an aspect of the invention, there is provided a kind of three-dimensional rebuilding method of object, is somebody's turn to do
Method includes: to obtain the initial local truncation signed distance function TSDF value of voxel in three-dimensional space, each initial local TSDF
Value corresponds to a depth map in multiple depth maps;By corresponding depth map, initial local TSDF value is grouped, it is at least one set of
Initial local TSDF value corresponds to more than one depth map;For every group of initial local TSDF value, the global TSDF of the group is obtained
Value;Using the global TSDF value of obtained each group as initial local TSDF value, optimization problem is solved, to obtain the final overall situation
TSDF value;And it is based on obtained final overall situation TSDF value, object described in three-dimensional reconstruction;Wherein, in the optimization problem
In, the global TSDF value of a voxel is obtained based on the final local T SDF value of the voxel, the final local T SDF of a voxel
Value is equal to the initial local TSDF value of the transformed corresponding voxel of the voxel, and variable is the ginseng of global the TSDF value and transformation of voxel
Number, cost function and following factors are related: the global TSDF value of particular voxel is initial with the transformed corresponding voxel of the voxel
The difference of local T SDF value square weighted sum, the weight of weighted sum is equal to the correspondence group of the transformed corresponding voxel of particular voxel
Weight.
According to another aspect of the present invention, a kind of three-dimensional reconstruction equipment of object is provided, which includes: to be filled
It sets, is configured as: obtaining the initial local truncation signed distance function TSDF value of voxel in three-dimensional space, each initial local
TSDF value corresponds to a depth map in multiple depth maps;Apparatus for grouping is configured as:, will be initial by corresponding depth map
The grouping of local T SDF value, at least one set of initial local TSDF value correspond to more than one depth map;Group overall situation TSDF value obtains dress
It sets, is configured as: for every group of initial local TSDF value, obtaining the global TSDF value of the group;Solving device is configured as: will
The global TSDF value of obtained each group solves optimization problem as initial local TSDF value, to obtain final overall situation TSDF
Value;And reconstructing device, it is configured as: being based on obtained final overall situation TSDF value, object described in three-dimensional reconstruction;Wherein, exist
In the optimization problem, the global TSDF value of a voxel is obtained based on the final local T SDF value of the voxel, a voxel
Final local T SDF value be equal to the transformed corresponding voxel of the voxel initial local TSDF value, variable is the overall situation of voxel
The parameter of TSDF value and transformation, cost function are related to following factors: the global TSDF value of particular voxel and the voxel are transformed
The difference of the initial local TSDF value of corresponding voxel square weighted sum, it is transformed right that the weight of weighted sum is equal to particular voxel
The weight of the corresponding group for the voxel answered.
In addition, according to another aspect of the present invention, additionally providing a kind of storage medium.The storage medium includes that machine can
The program code of reading, when executing said program code on information processing equipment, said program code makes at the information
Equipment is managed to execute according to the above method of the present invention.
In addition, in accordance with a further aspect of the present invention, additionally providing a kind of program product.Described program product includes that machine can
The instruction of execution, when executing described instruction on information processing equipment, described instruction executes the information processing equipment
According to the above method of the present invention.
Detailed description of the invention
Referring to reference to the accompanying drawing to the explanation of the embodiment of the present invention, the invention will be more easily understood it is above and
Other objects, features and advantages.Component in attached drawing is intended merely to show the principle of the present invention.In the accompanying drawings, identical or class
As technical characteristic or component will be indicated using same or similar appended drawing reference.In attached drawing:
Fig. 1 shows the flow chart of the three-dimensional reconstruction object method of embodiment according to the present invention;
Fig. 2 shows the flow charts of the acquisition initial local TSDF method of embodiment according to the present invention;
Fig. 3 shows the flow chart of the calculating initial local TSDF method of embodiment according to the present invention;
Fig. 4 shows the structural block diagram of the equipment of the three-dimensional reconstruction object of embodiment according to the present invention;And
Fig. 5 shows the schematic frame for the computer that can be used for implementing the method and apparatus of embodiment according to the present invention
Figure.
Specific embodiment
Exemplary embodiment of the invention is described in detail hereinafter in connection with attached drawing.It rises for clarity and conciseness
See, does not describe all features of actual implementation mode in the description.It should be understood, however, that developing any this reality
Much decisions specific to embodiment must be made during embodiment, to realize the objectives of developer,
For example, meeting restrictive condition those of related to system and business, and these restrictive conditions may be with embodiment
It is different and change.In addition, it will also be appreciated that although development is likely to be extremely complex and time-consuming, to benefit
For those skilled in the art of present disclosure, this development is only routine task.
Here, and also it should be noted is that, in order to avoid having obscured the present invention because of unnecessary details, in the accompanying drawings
Illustrate only with closely related apparatus structure and/or processing step according to the solution of the present invention, and be omitted and the present invention
The little other details of relationship.In addition, it may also be noted that being described in an attached drawing of the invention or a kind of embodiment
Elements and features can be combined with elements and features shown in one or more other attached drawings or embodiment.
The process of the three-dimensional rebuilding method of the object of embodiment according to the present invention is described below with reference to Fig. 1.
Fig. 1 shows the flow chart of the three-dimensional reconstruction object method of embodiment according to the present invention.As shown in Figure 1, according to
The three-dimensional reconstruction object method of the embodiment of the present invention includes the following steps: the initial local truncation for obtaining voxel in three-dimensional space
Signed distance function TSDF value, each initial local TSDF value correspond to a depth map (step S1) in multiple depth maps;
By corresponding depth map, initial local TSDF value is grouped, at least one set of initial local TSDF value corresponds to more than one depth
Scheme (step S2);For every group of initial local TSDF value, the global TSDF value (step S3) of the group is obtained;By obtained each group
Global TSDF value as initial local TSDF value, optimization problem is solved, to obtain final overall situation TSDF value (step S4);With
And it is based on obtained final overall situation TSDF value, object (step S5) described in three-dimensional reconstruction;Wherein, in the optimization problem
In, the global TSDF value of a voxel is obtained based on the final local T SDF value of the voxel, the final local T SDF of a voxel
Value is equal to the initial local TSDF value of the transformed corresponding voxel of the voxel, and variable is the ginseng of global the TSDF value and transformation of voxel
Number, cost function and following factors are related: the global TSDF value of particular voxel is initial with the transformed corresponding voxel of the voxel
The difference of local T SDF value square weighted sum, the weight of weighted sum is equal to the correspondence group of the transformed corresponding voxel of particular voxel
Weight.
In step sl, signed distance function TSDF value, Mei Gechu is truncated in the initial local for obtaining voxel in three-dimensional space
Beginning local T SDF value corresponds to a depth map in multiple depth maps.
Specifically, three-dimensional space is evenly divided into several voxels, it is believed that voxel is the basic of three-dimensional space
Unit.Due to rebuilding three dimensional object based on one group of depth map, so the input of three-dimensional reconstruction object method is one group of depth map.
This group of depth map is shot by multiple cameras, and each camera corresponds to a depth map.Again since input does not include phase
The parameter of machine so needing to estimate based on depth map the parameter of corresponding camera, and obtains the initial office of associated voxels in turn
Portion's TSDF value.
In addition, it should be noted here that " the multiple cameras " mentioned herein includes the physics in different position and orientation
On multiple cameras the case where, also include the same camera multiple phases in logic are formed by adjusting position and/or direction
The case where machine, include thes case where that above-mentioned two situations mix.
The initial local TSDF value of voxel be defined as based on camera parameter the calculated voxel of three-dimensional space relative to
Result of the directed distance after being truncated in the depth and depth map of camera between the depth of the voxel corresponding position.It will be such
TSDF value is known as initial local TSDF value.From defined above it is found that a voxel can calculate one relative to a camera
Directed distance defined above, a voxel can calculate multiple directed distances defined above relative to multiple cameras, often
A directed distance corresponds to a camera.Directed distance is due to break-in operation to be passed through, so a voxel likely corresponds to one
A or multiple initial local TSDF values, the upper limit of the corresponding initial local TSDF value of a voxel are camera/depth map numbers.
After being truncated, some voxels can not have initial local TSDF value, and such voxel is no longer considered.That is, counting in step sl
The voxel that calculation obtains initial local TSDF value can just continue in subsequent step S2, S3.In addition, can from above-mentioned definition
Know, each initial local TSDF value corresponds to multiple magazine cameras, namely corresponding to one in multiple depth maps
Depth map.
Due to camera capture depth map when, acquisition be subject surface information, it is possible to understand be based on phase
Machine parameter the calculated voxel of three-dimensional space relative to the depth of camera be between imaging point of the voxel with camera at a distance from
Projection in the direction of the optical axis, and the depth of the voxel corresponding position is between the voxel and the imaging point of camera in depth map
The projection of the distance between imaging point of point and camera in the subject surface that light is passed through in the direction of the optical axis.Due to the body
Element may be farther or closer apart from camera relative to the point of this in subject surface, so the difference of the projection of distance in the direction of the optical axis
For directed distance.Distance in the direction of the optical axis be projected as zero, show that the voxel is overlapped with the point in subject surface, namely this
Voxel is the point in subject surface.
Theoretically, the null all voxels of initial local TSDF value constitute subject surface.But due to initial local
The calculating of the value of TSDF depends on the estimation of camera parameter, and the estimated value of camera parameter and the actual value of camera parameter might not
It is completely the same, so initial local TSDF value can not be used directly to rebuild the surface of three dimensional object.A kind of possible method is
Arithmetic mean of instantaneous value is asked to all initial local TSDF values of each voxel with initial local TSDF value, arithmetic mean of instantaneous value is made
For the global TSDF value of the voxel, overall situation TSDF value and voxel with the value are then based on to rebuild the table of three dimensional object
Face.However, generating the problem that the relationship having cured between initial local TSDF value and overall situation TSDF value in this way.
Due to camera have it is multiple, so there may be differences the case where the parameter Estimation of multiple cameras, therefore, using unified
Mode that the corresponding initial local TSDF value of different cameral is converted to global TSDF value is obviously inaccurate.So the present invention
Power solves the problems, such as to obtain accurately overall situation TSDF value.
In addition, by foregoing description, it will be understood that step S1 can be realized by method shown in Fig. 2, to obtain initial office
Portion's TSDF value.
As shown in Fig. 2, in the step s 21, estimating each multiple magazine camera corresponding with multiple depth maps
Camera parameter, camera parameter include but is not limited to the position and orientation of camera.It is used in Kinect Fusion for example, can be used
Depth map and threedimensional model between matching algorithm carry out camera parameter estimation.Of course, it is possible to using known in the art
All suitable camera parameter estimation methods realize step S21.
In step S22, based on estimated camera parameter, one or more initial local TSDF values of voxel are calculated.
Step S22 can be realized by method shown in Fig. 3.
As shown in figure 3, each voxel and multiple depth maps for initial local TSDF value to be calculated are corresponding each
A camera, in step S31, the estimated camera parameter based on camera calculates first depth of the voxel relative to the camera
Value.In step s 32, determine that the voxel is corresponding, the second depth value in the associated depth map of the camera.In step S33,
Will after the truncation of the difference of the first depth value and the second depth value as a result, initial local corresponding with the camera as the voxel
TSDF value.
It should be noted that the range of voxel is the scheduled estimation energy in three-dimensional space when calculating the initial local TSDF value of voxel
Enough it is fully contemplated by the region of object.Since the calculating of initial local TSDF is related to break-in operation, so having initial after being truncated
The range of the voxel of local T SDF value will further reduce.By the threshold value of control truncation, can there will be initial local TSDF
The voxel of value is limited near the surface of object.The threshold value of truncation for example can be positive and negative 5 centimetres namely initial local
The value range of TSDF value can be [- 5,5].
For convenience of calculation, also initial local TSDF value is normalized.Normalized threshold value is the threshold value being truncated.
Truncation ensure that voxel near the surface of object, it will also be seen that, voxel is transformed (rigid from subsequent explanation
Transformation or non-rigid transformation) voxel itself or another voxel can be corresponded to, in order to guarantee the voxel of transformation front and back all in right
Near the surface of elephant, the initial local TSDF value further to voxel is limited.
Specifically, the absolute value for limiting the initial local TSDF value of voxel is less than or equal to specific threshold α, the specific threshold
α belongs to (0,1).That is, the voxel of the initial local TSDF value of [- α, α] is transformed to correspond to its initial local TSDF
The voxel being worth between [- 1,1].
In addition, if being calculated in the associated coordinate system of each camera (such as using the camera as the coordinate system of origin) respectively
Initial local TSDF value, then the initial local TSDF value for the voxel being calculated correspond to multiple magazine camera associations
Coordinate system.The initial local TSDF value for the voxel being calculated can be transformed into the same coordinate system.For example, will be calculated
The initial local TSDF value of voxel be transformed into first associated coordinate system of camera.By by initial local TSDF primary system one
To a coordinate system, it can be convenient and acquire the overall situation to the transformation of final local T SDF value and in turn from initial local TSDF value
TSDF value can reduce calculation amount, and can obtain globally optimal solution rather than locally optimal solution.
It is of course also possible to calculate initial local TSDF value of the voxel relative to each camera directly in global coordinate system.
Benefit using such mode is can to convert to avoid by the initial local TSDF value of voxel from the associated coordinate system of each camera
The loss of significance generated when to the same coordinate system.
An important means of the invention is to be grouped initial local TSDF value, then independent for each grouping
Processing, then the result integrated treatment to each group.In this way, it can cope with and be counted with many initial local TSDF values
According to the very big situation of amount.
In step s 2, by corresponding depth map, initial local TSDF value is grouped, at least one set of initial local TSDF value
Corresponding to more than one depth map.
As previously mentioned, each initial local TSDF value corresponds to a depth map in multiple depth maps.Therefore, Ke Yigen
Initial local TSDF value is assigned in different groups according to which depth map is initial local TSDF value correspond to.Each group can be with
Initial local TSDF value including corresponding to one or more depth maps, but the initial local TSDF value of at least one group is corresponding
In more than one depth map.All initial local TSDF values corresponding to same depth map are only in a group.
In step s3, for every group of initial local TSDF value, the global TSDF value of the group is obtained.
Since step S3 to be accomplished that the transformation from initial local TSDF value to global TSDF value, so with the present invention
To be solved be slave initial local TSDF value to the transformation of global TSDF value it is identical, the range only handled is defined in respectively
A grouping.Therefore, step S3 both can use traditional method and realize, also can use and realizes according to the method for the present invention.
When using conventional methods realization step S3, for every group of initial local TSDF value, calculates and correspond to one
The arithmetic mean of instantaneous value of the initial local TSDF value of element, the global TSDF value as the group for corresponding to the voxel.For example, at the beginning of certain group
Beginning local T SDF value correspond to three depth maps, then, in the group corresponding to voxel A initial local TSDF value it is most there are three
A1, A2, A3 calculate the value of (A1+A2+A3)/3, the global TSDF value corresponding to voxel A as the group.
Note that since initial local TSDF value being grouped according to corresponding depth map, so the same voxel may be more
There is corresponding initial local TSDF value in a grouping, correspondingly, by step S3, every group of global TSDF value corresponds to multiple
Voxel, each voxel have been likely to corresponding overall situation TSDF value in multiple groups.Hereinafter will how will in introduction step S4
The PRELIMINARY RESULTS that step S3 is obtained is further processed, finally to obtain final overall situation TSDF value unique to voxel.
When realizing step S3 using preferred method according to the present invention, for every group of initial local TSDF value, by asking
Optimization problem is solved, the global TSDF value of the group is obtained.
When with comparing hereinafter it is noted that due to realize step S3 when, processing be packets inner initial local
TSDF value, it is possible to weight is not used in optimization problem.And below for after grouping as a result, in order to eliminate point
The distortion such as pseudo-side that group may cause, uses weight preferably in optimization problem.
It is introduced below how to be realized by designing and solving optimization problem from initial local TSDF value to global TSDF
The conversion of value.Relevant thought is equally applicable to step S3 and step S4.
As previously described, because initial local TSDF value is calculated based on the camera parameter of estimation, and different depth maps pair
Not necessarily the situation of entirely accurate and inaccuracy is not necessarily identical the parameter of the estimation for the different cameral answered, so uncomfortable
Preferably all initial local TSDF values of voxel are handled in a uniform manner to obtain the global TSDF value of voxel.It is true
On, it is believed that the initial local TSDF value transformation (inverse transformation of rigid transformation or non-rigid transformation) of the first voxel to the second body
At element, become the final local T SDF value of the second voxel.First voxel and the second voxel can identical (null transformation), can also not
With (non-zero transform).Specific threshold α hereinbefore defines the value range of the initial local TSDF of the second voxel, to guarantee
The final local T SDF value of second voxel belongs to [- 1,1].
It should be noted that rigid transformation is carried out for same camera.That is, same camera is corresponding to belong to multiple first bodies
Multiple initial local TSDF values of element correspond to multiple second by the inverse transformation of the same rigid transformation in these first voxels
As multiple final local T SDF values corresponding to the camera of these the second voxels while voxel.So-called rigidity refers to correspondence
The second voxel is reached from the first voxel through unified rotationally and/or translationally transformation in multiple initial local TSDF values of same camera
(TSDF value itself is constant), there is no become before and after transformation for the relative positional relationship between voxel associated by these values
Change.
It should be noted that non-rigid transformation is carried out for voxel.The main body of transformation is voxel, transformation the result is that establishing first
The final local T SDF value of the corresponding relationship and the second voxel of voxel and the second voxel is equal to the initial of corresponding first voxel
Local T SDF value.That is, an initial local TSDF value of first voxel passes through non-rigid change in first voxel
Final local T SDF value while the inverse transformation changed corresponds to the second voxel as second voxel.It is non-rigid to refer to each
One voxel reach the second voxel it is experienced transformation be it is independent of each other, it is unrelated with camera, corresponding to same camera it is multiple just
Beginning inverse transformation of the local T SDF value through respective non-rigid transformation reaches the second voxel (TSDF value itself is constant) from the first voxel,
Relative positional relationship between voxel associated by these values can change before and after transformation.Relative to rigid transformation, adopt
It can more meticulously consider that the relevant transformation of multiple initial local TSDF values for same camera may with non-rigid transformation
It is different from each other.Therefore, non-rigid transformation is to further improve the final TSDF of voxel relative to the benefit of rigid transformation
The accuracy rate of value, and then improve the accuracy of global TSDF and the effect of three-dimensional reconstruction object.
By the amendment of rigid transformation/non-rigid transformation, the error of camera parameter estimation is counteracted, so that TSDF value and body
The corresponding relationship of element is more accurate.Meanwhile global TSDF value is obtained from final local T SDF value, so that it may and then based on the overall situation
TSDF value rebuilds three dimensional object.
Preferred embodiment in accordance with the present invention is realized by the way of designing and solving optimization problem from initial local
Rigid transformation/non-rigid transformation that TSDF value is related to final local T SDF value and from final local T SDF value to global TSDF
The calculating of value.
The design key of optimization problem is the design of variable and cost function.
For rigid transformation, in optimization problem, the global TSDF value of a voxel is based on the final of the voxel
Local T SDF value obtains, and the final local T SDF value of a voxel is equal to initial office of the voxel through the corresponding voxel of rigid transformation
Portion's TSDF value, variable are the parameters of global the TSDF value and rigid transformation of voxel, and cost function is related to following factors: voxel
The quadratic sum of the difference of the initial local TSDF value of global TSDF value and the voxel through the corresponding voxel of rigid transformation.
Cost function for example can be designed as the global TSDF value of voxel and the voxel through the corresponding voxel of rigid transformation
The summation of the quadratic sum of the difference of initial local TSDF value.Be noted above step S1 obtain initial local TSDF value when, due into
It has gone truncation, normalization, and has been further limited in some embodiments using threshold alpha, only part voxel is caused to have just
Beginning local T SDF value, subsequent step will be carried out for these voxels.Therefore, cost function can be designed as these voxels
Each calculates cost item, and cost function is the summation of cost item, and each cost item is the global TSDF value and the voxel of voxel
The quadratic sum of the difference of initial local TSDF value through the corresponding voxel of rigid transformation namely the global TSDF value of voxel and the voxel
Final local T SDF value difference quadratic sum.
Here the global TSDF value for why being designed as a voxel is obtained based on the final local T SDF value of the voxel, and one
The final local T SDF value of a voxel is equal to initial local TSDF value of the voxel through the corresponding voxel of rigid transformation, rather than sets
The global TSDF value for being calculated as a voxel is obtained based on the initial local TSDF value of the voxel, is because if setting according to the latter
Meter, then the solution of optimization problem can be a voxel global TSDF value be equal to the voxel initial local TSDF value arithmetic/
Weighted average.By introducing the rigid transformation that only relates to translate and rotate whole for a camera, increase from initial
The conversion of local T SDF value to final local T SDF value leads to three-dimensional reconstruction so that the solution of optimization problem is more accurate
The result of object is more accurate.
If overall situation TSDF value is V, depth map number is n, then the corresponding initial local TSDF value of each depth map is V1、
V2、……、Vn, the corresponding final local T SDF value of each depth map is V1’、V2’、……、Vn', the serial number i, i=of depth map
1,2,……,n.For the voxel p in V, in Vi' in correspondence voxel be still p, and in ViIn correspondence voxel be Ti(p),
Wherein TiFor the rigid transformation for needing to solve in optimization problem.
As it was noted above, voxel has initial local TSDF value through truncation, normalization, that is, require Vi(p) value is located at
[- 1,1] section.It also needs to seek V hereini(Ti(p))∈[-1,1].Due to ViWith Vi' between not will do it violent transformation,
That is voxel p and voxel TiThe distance between (p) will not be far, so definition set Pi: Pi=p | Vi(p)∈[-α,
α] }, make it possible to guarantee Vi(Ti(p))∈[-1,1]。
That is, when calculating cost function, for belonging to set PiVoxel p calculate cost item summation, Mei Gedai
Valence item is global TSDF value of the voxel p in V with voxel p through the corresponding V of rigid transformationiIn voxel Ti(p) initial local
The quadratic sum of the difference of TSDF value, a square summation are because voxel p may have multiple final local T SDF values, to be corresponding with more
A voxel Ti(p) initial local TSDF value.
So cost function can indicate are as follows:
Optimization problem is solved, that is, solves the final overall situation TSDF value and rigid transformation for minimizing above-mentioned cost function
The optimal value of parameter.
An overall situation TSDF value of voxel is to emphasize that a voxel finally only has an overall situation TSDF value herein, and in this way
The range of voxel be set Pi。
Optimization problem can be solved by Gauss-Newton (Gauss-Newton) method of iteration or by being directed to body
The global TSDF value of element and the parameter of rigid transformation calculate separately to iteratively solve.
But no matter which kind of solves mode, iteration is directed to, it is necessary to set the initial value of iteration.
In optimization problem, the initial value of the global TSDF value of voxel is equal to whole initial local TSDF values of the voxel
Weighted average.On the one hand weighting herein can be arithmetic average, on the other hand can also according to camera and voxel away from
From calculating weight.
The initial value of the parameter of rigid transformation makes the voxel through the corresponding voxel of rigid transformation be the voxel itself.One
Voxel may have multiple initial local TSDF values, if the voxel corresponds to the voxel itself through rigid transformation, this individual
Element initialization final local T SDF value also have it is multiple, each correspond to a camera.
For example, rigid transformation Ti(p)=Ri*p+ti, wherein RiIt is spin matrix, unit matrix, t can be initialized asiFor translation
Vector can be initialized as null vector.
For non-rigid transformation, in optimization problem, the global TSDF value of a voxel based on the voxel most
End portion TSDF value obtains, and the final local T SDF value of a voxel is equal to the voxel through the first of the corresponding voxel of non-rigid transformation
Beginning local T SDF value, variable are the parameters of global the TSDF value and non-rigid transformation of voxel, and cost function is related to following factors:
Square of the difference of the initial local TSDF value of the global TSDF value of particular voxel and the voxel through the corresponding voxel of non-rigid transformation
With the summation of the offset measurement of the non-rigid transformation of particular voxel experience.
Previously mentioned voxel will meet specific threshold condition, i.e. the absolute value of the initial local TSDF value of restriction voxel is less than
Or it is equal to specific threshold α, specific threshold α belongs to (0,1).If such voxel be not it is too many, can based on all this
Sample voxel designs cost function, and optimization problem can solve.At this point, particular voxel is all voxels for meeting specific threshold condition.
If such voxel is more, based on when all voxel designs cost function in this way, optimization problem may cannot be solved.?
In this case, by sample to all voxels for meeting specific threshold condition and cost can be designed based on sampling voxel
Function comes so that optimization problem can solve.The parameter of the non-rigid transformation of non-sampled voxel can be according to the non-rigid of sampling voxel
The parameter of transformation determines.At this point, particular voxel is to meet obtained sampling voxel after all voxels sampling of specific threshold condition.
In a preferred embodiment, it is sampled as uniform sampling.
Cost function for example can be designed as related to following factors: the global TSDF value of particular voxel is with the voxel through non-
Quadratic sum (the first cost item), the particular voxel of the difference of the initial local TSDF value of the corresponding voxel of rigid transformation are undergone non-rigid
Property transformation offset measurement summation (the second cost item).
Cost function can be designed as the sum of the first cost item and the second cost item, wherein the first cost item and the second cost
Item numerical difference is not larger, sums again with the first cost item after the second cost item can be multiplied with balance factor.
First cost item is global TSDF value and the voxel of particular voxel through the initial of the corresponding voxel of non-rigid transformation
The difference of the final local T SDF value of the global TSDF value and voxel of the quadratic sum namely particular voxel of the difference of local T SDF value
Quadratic sum.Here the summation in quadratic sum includes the summation of two levels, and one side particular voxel is not unique, multiple particular volumes
The correlation of element needs to sum, and on the other hand, each particular voxel may have multiple final local T SDF values, so each
The correlation of particular voxel is also required to sum.
Second cost item is the summation of the offset measurement of the non-rigid transformation of particular voxel experience.In the second voxel through non-rigid
Property convert corresponding first voxel in the case where, above-mentioned offset measurement refers to the mould of offset of first voxel relative to the second voxel
Square.Since same second voxel may have multiple final local T SDF values, so second voxel may be via a variety of
Non-rigid transformation corresponds to multiple first voxels, and multiple final local T SDF values of second voxel are equal to multiple first at this time
The initial local TSDF value of voxel.Therefore, correspondingly, offset measurement, i.e., offset of multiple first voxels relative to the second voxel,
There is also multiple, so carry out the summation operation of offset measurement.Also, there are multiple second voxels, so also needing pair
The offset measurement of multiple second voxels is summed.As it can be seen that the summation of the offset measurement of the non-rigid transformation of particular voxel experience
It is the summation of the offset measurement of all non-rigid transformations of all particular voxel experience.As described above, particular voxel is to meet
All voxels or particular voxel of specific threshold condition be meet specific threshold condition all voxels sampling after obtain adopt
Sample voxel.Cost item will be hereinafter further described for sampling voxel.
In addition, the purpose of cost function be to understand optimization problem, can be with when cost function is based only upon sampling voxel
The offset of the non-rigid transformation of non-sampled voxel experience is obtained according to the offset of the non-rigid transformation of sampling voxel experience.In this way,
Cost function be based only upon sampling voxel to optimization problem can solve and solve calculation amount reduce solving speed it is fast, remain to simultaneously
Obtain the global TSDF value of non-sampled voxel.For example, the offset of the non-rigid transformation of non-sampled voxel experience can be according to sampling
The offset of the non-rigid transformation of voxel experience passes through three-dimensional interpolation and obtains.In one embodiment, non-sampled voxel experience is non-
The offset of rigid transformation passes through three according to the offset of the non-rigid transformation of the sampling voxel experience close with the non-sampled voxel distance
Interpolation is tieed up to obtain.In another embodiment, the offset of the non-rigid transformation of non-sampled voxel experience according to the non-sampled body
The offset that element corresponds to the non-rigid transformation of the sampling voxel experience of same camera passes through three-dimensional interpolation and obtains.In another implementation
In example, the offset of the non-rigid transformation of non-sampled voxel experience according to and the non-sampled voxel correspond to same camera and with this
The offset of the non-rigid transformation of the close sampling voxel experience of non-sampled voxel distance passes through three-dimensional interpolation and obtains.Note: in three-dimensional
Inserting the offset obtained after offset may not correspond to voxel location for particular voxel, and correspond between voxel location
Sub- voxel location.In such a case, it is possible to move to voxel position from sub- voxel location by rounding up three-dimensional coordinate
It sets, so that particular voxel is corresponded to the voxel at voxel location.
Here the global TSDF value for why being designed as a voxel is obtained based on the final local T SDF value of the voxel, and one
The final local T SDF value of a voxel is equal to initial local TSDF value of the voxel through the corresponding voxel of non-rigid transformation, rather than
The global TSDF value for being designed as a voxel is obtained based on the initial local TSDF value of the voxel, is because if setting according to the latter
Meter, then the solution of optimization problem can be a voxel global TSDF value be equal to the voxel initial local TSDF value arithmetic/
Weighted average.By introducing non-rigid transformation, the conversion from initial local TSDF value to final local T SDF value is increased, from
And make the solution of optimization problem more accurate, cause the result of three-dimensional reconstruction object more accurate.
The design of cost function is explained further below with reference to formula.
If overall situation TSDF value is V, depth map number is n, then the corresponding initial local TSDF value of each depth map is V1、
V2、……、Vn, the corresponding final local T SDF value of each depth map is V1’、V2’、……、Vn', the serial number i, i=of depth map
1,2,……,n.For the voxel p in V, in Vi' in correspondence voxel be still p, and in ViIn correspondence voxel be Ci(p),
Wherein CiFor the non-rigid transformation for needing to solve in optimization problem.
The set of definition sampling voxel, is denoted as Di={ di,j, wherein j=1,2 ..., mi, miFor ViIt is middle to sample the total of voxel
Number.
The non-rigid transformation of voxel experience will be sampled is defined as: Ci(di,j)=di,j+si,j, wherein si,jTo sample voxel
Change in location between mapping result (sampling voxel is through the corresponding voxel location of non-rigid transformation) and sampling voxel home position
Vector deviates.And the mapping result of non-sampled voxel carries out trilinear interpolation according to the offset of sampling voxel and obtains.With root
For offset according to the non-rigid transformation of the calculations of offset sampling voxel of the non-rigid transformation of all sampling voxels,Wherein wi,j(p) be linear interpolation coefficient, can according to non-sampled voxel p and sampling voxel it
Between positional relationship (such as distance) acquire.
As it was noted above, voxel has initial local TSDF value through truncation, normalization, that is, require Vi(p) value is located at
[- 1,1] section.It also needs to seek V hereini(Ci(p))∈[-1,1].Due to ViWith Vi' between not will do it violent transformation,
That is voxel p and voxel CiThe distance between (p) will not be far, so definition set Pi: Pi=p | Vi(p)∈[-α,
α] }, make it possible to guarantee Vi(Ci(p))∈[-1,1]。
That is, when calculating cost function, for belonging to set PiVoxel p calculate the first cost item and the second cost
Summation, each first cost item is global TSDF value of the voxel p in V with voxel p through the corresponding V of non-rigid transformationiIn
Voxel Ci(p) quadratic sum of the difference of initial local TSDF value, a square summation are because voxel p may have multiple most ends
Portion's TSDF value, to be corresponding with multiple voxel Ci(p) initial local TSDF value, and there is also multiple by voxel p itself.
So the summation of the first cost item of cost function can indicate are as follows:
The second cost item of cost function is introduced in the present invention, to guarantee ViWith Vi' between not will do it violent transformation,
All associated voxels are also avoided to be mapped to the extreme case of the same point simultaneously.
The summation of second cost item of cost function can indicate are as follows:
In summary the factor of two aspects, defines cost function to be optimized are as follows:
Wherein λ is the balance factor of front and back two.
Optimization problem is solved, that is, solves the final overall situation TSDF value and non-rigid change for minimizing above-mentioned cost function
Change the optimal value of parameter.
An overall situation TSDF value of voxel is to emphasize that a voxel finally only has an overall situation TSDF value herein, and in this way
The range of voxel be set Pi。
Optimization problem can be solved by Gauss-Newton (Gauss-Newton) method of iteration or by being directed to body
The global TSDF value of element and the parameter of rigid transformation calculate separately to iteratively solve.
But no matter which kind of solves mode, iteration is directed to, it is necessary to set the initial value of iteration.
In optimization problem, the initial value of the global TSDF value of voxel is equal to whole initial local TSDF values of the voxel
Weighted average.On the one hand weighting herein can be arithmetic average, on the other hand can also according to camera and voxel away from
From calculating weight.
The initial value of the parameter of the non-rigid transformation of voxel makes the voxel through the corresponding voxel of non-rigid transformation be the body
Element itself.One voxel may have multiple initial local TSDF values, if the voxel corresponds to the voxel through non-rigid transformation
Itself, then the final local T SDF value of the initialization of this voxel also has multiple, each corresponds to a camera.
It is described above in the case of rigid transformation and non-rigid transformation, design and the method for solving optimization problem.Pass through
The above method can realize step S3 in a manner of solving optimization problem.
In step s 4, it using the global TSDF value of obtained each group as initial local TSDF value, solves to optimize and ask
Topic, to obtain final overall situation TSDF value.
That is, being carried out again using the global TSDF value of obtained each group as initial local TSDF value from initial
Conversion of the local T SDF value to final overall situation TSDF value.However, in step s 4, it can only be using the side for solving optimization problem
Formula.Solve the specific method of optimization problem and the explanation one carried out above for two kinds of situations of rigid transformation and non-rigid transformation
It causes, repeats no more.
A kind of preferred embodiment of introduction step S4 herein.
Due to being grouped before, so this artificial segmentation may result in the subject surface of final three-dimensional reconstruction
There are pseudo-side, pseudo-side is as caused by grouping.Therefore, in the preferred embodiment, introduce weight overcoming due to grouping and
The pseudo-side problem of generation.
Specifically, in optimization problem, it is still: final part of the global TSDF value based on the voxel of a voxel
TSDF value obtains, and the final local T SDF value of a voxel is equal to the initial local TSDF of the transformed corresponding voxel of the voxel
Value, variable is the parameter of global the TSDF value and transformation of voxel.Unlike the optimization problem illustrated before, preferred real
It applies in mode, cost function and following factors are related: the global TSDF value of particular voxel and the transformed corresponding voxel of the voxel
Initial local TSDF value difference square weighted sum, the weight of weighted sum is equal to the transformed corresponding voxel of particular voxel
The weight of corresponding group.
It should be noted that the alphabetic flag hereinafter occurred, unless specifically stated otherwise, with the rigid transformation that illustrates before and it is non-just
Property change situation optimization problem in meaning it is identical, if any place is specialized, be subject to and here indicate that.
Cost function in the case of the rigid transformation of front can indicate are as follows:
In the preferred embodiment, the cost function in the case of rigid transformation can indicate are as follows:
Cost function in the case of the non-rigid transformation of front can indicate are as follows:
In the preferred embodiment, the cost function in the case of non-rigid transformation can indicate are as follows:
E=Ec+λEr,Rigidity becomes herein
It changes and the unified writing of non-rigid transformation: Ti(p).For rigid transformation, Er=0.
If overall situation TSDF value is V, initial local TSDF value is divided into n group, i.e. V according to corresponding depth map1,V2,…,
Vn, the weight of the corresponding voxel of each initial local TSDF value is W1,W2,…,Wn, final local T SDF value is V after transformation1′,
V2′,…,Vn′.Organize serial number i, i=1,2 ... ..., n.For the voxel p in V, in Vi' in correspondence voxel be still p, and
In ViIn correspondence voxel be Ti(p) (rigid transformation or non-rigid transformation), wherein TiFor the change for needing to solve in optimization problem
It changes.
As it can be seen that main modification is that introducing weight Wi(Ti(p)).Weight changes with iterative process.Particular voxel p
Weight be equal to the transformed corresponding voxel T of particular voxeli(p) weight of corresponding group, and the corresponding group of corresponding voxel
Weight is above-mentioned W1,W2,…,Wn, the weight (W of a voxel1,W2,…,Wn) group is depended on, same voxel corresponds to difference
Group has different weights, corresponding to same group of different voxels weight having the same.The weight of particular voxel is equal to the spy
Determine the number of the voxel initial local TSDF value of transformed corresponding voxel in this set.In the case where so calculating weight,
It is considered that weight is equal to the sum of the weight corresponding to depth map of the particular voxel transformed corresponding voxel in this set, this
The weight corresponding to depth map of particular voxel transformed corresponding voxel in this set is in the particular voxel in this set through becoming
It changes corresponding voxel and is equal to 1 when having initial local TSDF value corresponding to depth map, be otherwise equal to 0.Alternatively, particular voxel
Weight is equal to the sum of the weight corresponding to depth map of the particular voxel transformed corresponding voxel in this set, the particular voxel
The weight corresponding to depth map of transformed corresponding voxel is transformed corresponding in this set with the particular voxel in this set
Voxel is related relative to the depth value that the depth map corresponds to camera.For example, the particular voxel transformed corresponding body in this set
Element is bigger relative to the depth value that depth map corresponds to camera, and transformed corresponding voxel corresponds to the particular voxel in this set
The weight of the depth map is smaller.For another example, transformed corresponding voxel corresponds to depth map to the particular voxel in this set
Weight is inversely proportional to the particular voxel, and transformed corresponding voxel corresponds to the depth value of camera relative to the depth map in this set.
In a preferred embodiment, calculating can be simplified in the following way: in (k+1) wheel iterative process,
It usesInstead ofCost function to be optimized is simplified are as follows:
Cost function to be optimized at this time is the form of non-linear least square, can pass through Gauss-Newton (Gauss-
Newton) method solves.
In some cases, possible initial local TSDF value is excessive, so that being not enough to effectively by a division operation
Ground reduces data volume, can be grouped again in step s 4 at this time.
Using the global TSDF value of obtained each group as initial local TSDF value, it is grouped again by corresponding depth map,
At least one set of initial local TSDF value corresponds to more than one and is grouped corresponding depth map for the first time;For every group of initial local TSDF
Value, obtains the global TSDF value of the group;Using the global TSDF value of obtained each group as initial local TSDF value, solve optimal
Change problem, to obtain final overall situation TSDF value.
It obtains in step s 4 and belongs to set PiAll voxel p final overall situation TSDF value after, so that it may be based on this
Reconstructed object.
In step s 5, obtained final overall situation TSDF value, object described in three-dimensional reconstruction are based on.
For example, it is based on obtained final overall situation TSDF value using marching cube (marching cube) algorithm,
The surface of object described in three-dimensional reconstruction.Marching cubes algorithm is known in the art algorithm, and details are not described herein.
The equipment of the three-dimensional reconstruction object of embodiment according to the present invention is described next, with reference to Fig. 4.
Fig. 4 shows the structural block diagram of the equipment of the three-dimensional reconstruction object of embodiment according to the present invention.Such as Fig. 4 institute
Show, three-dimensional reconstruction object-based device 400 according to the present invention includes: to obtain device 41, is configured as: obtaining voxel in three-dimensional space
Initial local be truncated signed distance function TSDF value, each initial local TSDF value correspond to multiple depth maps in a depth
Degree figure;Apparatus for grouping 42, is configured as: by corresponding depth map, initial local TSDF value being grouped, at least one set of initial local
TSDF value corresponds to more than one depth map;Group overall situation TSDF value acquisition device 43, is configured as: for every group of initial local
TSDF value obtains the global TSDF value of the group;Solving device 44, is configured as: the global TSDF value of obtained each group is made
For initial local TSDF value, optimization problem is solved, to obtain final overall situation TSDF value;And reconstructing device 45, it is configured as:
Based on obtained final overall situation TSDF value, object described in three-dimensional reconstruction;Wherein, in the optimization problem, a voxel
Global TSDF value obtained based on the final local T SDF value of the voxel, the final local T SDF value of a voxel is equal to the voxel
The initial local TSDF value of transformed corresponding voxel, variable are the parameter of global the TSDF value and transformation of voxel, cost function
And following factors are related: the global TSDF value of particular voxel and the initial local TSDF value of the transformed corresponding voxel of the voxel
Difference square weighted sum, the weight of weighted sum is equal to the weight of the corresponding group of the transformed corresponding voxel of particular voxel.
In one embodiment, the transformation includes rigid transformation.
In one embodiment, the transformation includes non-rigid transformation, and cost function is also related to following factors: particular volume
The summation of the offset measurement of the non-rigid transformation of element experience.
In one embodiment, described group of overall situation TSDF value acquisition device 43 is further configured to: initial for every group
Local T SDF value, by solving optimization problem, to obtain the global TSDF value of the group.
In one embodiment, described group of overall situation TSDF value acquisition device 43 is further configured to: initial for every group
Local T SDF value calculates the arithmetic mean of instantaneous value for corresponding to the initial local TSDF value of same voxel, as corresponding to the voxel
The global TSDF value of the group.
In one embodiment, the solving device 44 is further configured to: by the global TSDF of obtained each group
Value is used as initial local TSDF value, is grouped again by corresponding depth map, and at least one set of initial local TSDF value corresponds to more than
One is grouped corresponding depth map for the first time;For every group of initial local TSDF value, the global TSDF value of the group is obtained;It will be acquired
Each group global TSDF value as initial local TSDF value, optimization problem is solved, to obtain final overall situation TSDF value.
In one embodiment, in the optimization problem, each particular voxel initially has the weight specific to group,
Weight is equal to the particular voxel, and the number of the initial local TSDF value of transformed corresponding voxel or weight are equal in this set
The sum of the weight corresponding to depth map of the particular voxel transformed corresponding voxel in this set, the particular voxel is in this set
Transformed corresponding voxel is opposite in this set with the particular voxel for the weight corresponding to depth map of transformed corresponding voxel
It is related in the depth value that the depth map corresponds to camera.
In one embodiment, the acquisition device 41 further include: normalization unit is configured as: by what is be calculated
The initial local TSDF value of voxel normalizes.
In one embodiment, the absolute value of the initial local TSDF value of voxel is less than or equal to specific threshold, this is specific
Threshold value belongs to (0,1).
In one embodiment, particular voxel is all voxels for meeting specific threshold condition.
In one embodiment, particular voxel is to meet obtained sampling body after all voxels sampling of specific threshold condition
Element.
Due to the processing in included each device in three-dimensional reconstruction object-based device 400 according to the present invention, unit
It is similar with the processing in each step included in three-dimensional reconstruction object method described above respectively, therefore in order to succinctly rise
See, omits the detailed description of these devices, unit herein.
In addition, it is still necessary to, it is noted that each component devices, unit can be by softwares, firmware, hard in above equipment here
The mode of part or combinations thereof is configured.It configures workable specific means or mode is well known to those skilled in the art,
This is repeated no more.In the case where being realized by software or firmware, from storage medium or network to specialized hardware structure
Computer (such as general purpose computer 500 shown in fig. 5) installation constitutes the program of the software, which is being equipped with various journeys
When sequence, it is able to carry out various functions etc..
Fig. 5 shows the schematic frame for the computer that can be used for implementing the method and apparatus of embodiment according to the present invention
Figure.
In Fig. 5, central processing unit (CPU) 501 is according to the program stored in read-only memory (ROM) 502 or from depositing
The program that storage part 508 is loaded into random access memory (RAM) 503 executes various processing.In RAM 503, also according to need
Store the data required when CPU 501 executes various processing etc..CPU 501, ROM 502 and RAM 503 are via bus
504 are connected to each other.Input/output interface 505 is also connected to bus 504.
Components described below is connected to input/output interface 505: importation 506 (including keyboard, mouse etc.), output section
Divide 507 (including display, such as cathode-ray tube (CRT), liquid crystal display (LCD) etc. and loudspeakers etc.), storage section
508 (including hard disks etc.), communications portion 509 (including network interface card such as LAN card, modem etc.).Communications portion 509
Communication process is executed via network such as internet.As needed, driver 510 can be connected to input/output interface 505.
Detachable media 511 such as disk, CD, magneto-optic disk, semiconductor memory etc., which can according to need, is installed in driver
On 510, so that the computer program read out is mounted to as needed in storage section 508.
It is such as removable from network such as internet or storage medium in the case where series of processes above-mentioned by software realization
Unload the program that the installation of medium 511 constitutes software.
It will be understood by those of skill in the art that this storage medium be not limited to it is shown in fig. 5 be wherein stored with program,
Separately distribute with equipment to provide a user the detachable media 511 of program.The example of detachable media 511 includes disk
(including floppy disk (registered trademark)), CD (including compact disc read-only memory (CD-ROM) and digital versatile disc (DVD)), magneto-optic disk
(including mini-disk (MD) (registered trademark)) and semiconductor memory.Alternatively, storage medium can be ROM 502, storage section
Hard disk for including in 508 etc., wherein computer program stored, and user is distributed to together with the equipment comprising them.
The present invention also proposes a kind of program product of instruction code for being stored with machine-readable.Described instruction code is by machine
When device reads and executes, method that above-mentioned embodiment according to the present invention can be performed.
Correspondingly, it is also wrapped for carrying the storage medium of the program product of the above-mentioned instruction code for being stored with machine-readable
It includes in disclosure of the invention.The storage medium includes but is not limited to floppy disk, CD, magneto-optic disk, storage card, memory stick etc.
Deng.
In the description above to the specific embodiment of the invention, for the feature a kind of embodiment description and/or shown
It can be used in one or more other embodiments in a manner of same or similar, with the feature in other embodiment
It is combined, or the feature in substitution other embodiment.
It should be emphasized that term "comprises/comprising" refers to the presence of feature, element, step or component when using herein, but simultaneously
It is not excluded for the presence or additional of one or more other features, element, step or component.
In addition, method of the invention be not limited to specifications described in time sequencing execute, can also according to it
His time sequencing, concurrently or independently execute.Therefore, the execution sequence of method described in this specification is not to this hair
Bright technical scope is construed as limiting.
Although being had been disclosed above by the description to specific embodiments of the present invention to the present invention, it answers
The understanding, above-mentioned all embodiments and example are exemplary, and not restrictive.Those skilled in the art can be in institute
Design is to various modifications of the invention, improvement or equivalent in attached spirit and scope of the claims.These modification, improve or
Person's equivalent should also be as being to be considered as included in protection scope of the present invention.
Note
1. a kind of three-dimensional rebuilding method of object, comprising:
Signed distance function TSDF value, each initial local TSDF value is truncated in the initial local for obtaining voxel in three-dimensional space
Corresponding to a depth map in multiple depth maps;
By corresponding depth map, initial local TSDF value is grouped, at least one set of initial local TSDF value corresponds to more than
One depth map;
For every group of initial local TSDF value, the global TSDF value of the group is obtained;
Using the global TSDF value of obtained each group as initial local TSDF value, optimization problem is solved, to obtain most
Overall situation TSDF value eventually;And
Based on obtained final overall situation TSDF value, object described in three-dimensional reconstruction;
Wherein, in the optimization problem, the global TSDF value of a voxel is based on the final local T SDF of the voxel
Value obtains, and the final local T SDF value of a voxel is equal to the initial local TSDF value of the transformed corresponding voxel of the voxel, becomes
Amount is the parameter of global the TSDF value and transformation of voxel, and cost function is related to following factors: the global TSDF value of particular voxel
With the difference of the initial local TSDF value of the transformed corresponding voxel of the voxel square weighted sum, the weight of weighted sum is equal to spy
Determine the weight of the corresponding group of the transformed corresponding voxel of voxel.
2. the method as described in note 1, wherein the transformation includes rigid transformation.
3. note 1 as described in method, wherein it is described transformation include non-rigid transformation, cost function also with following factors
It is related: the summation of the offset measurement of the non-rigid transformation of particular voxel experience.
4. the method as described in note 1, wherein be directed to every group of initial local TSDF value, obtain the global TSDF value of the group
It include:, by solving optimization problem, to obtain the global TSDF value of the group for every group of initial local TSDF value.
5. the method as described in note 1, wherein be directed to every group of initial local TSDF value, obtain the global TSDF value of the group
Include: to calculate the arithmetic mean of instantaneous value for corresponding to the initial local TSDF value of same voxel for every group of initial local TSDF value, makees
For the global TSDF value of the group corresponding to the voxel.
6. the method as described in note 1, wherein described using the global TSDF value of obtained each group as initial local
TSDF value solves optimization problem, includes: to obtain final overall situation TSDF value
Using the global TSDF value of obtained each group as initial local TSDF value, it is grouped again by corresponding depth map,
At least one set of initial local TSDF value corresponds to more than one and is grouped corresponding depth map for the first time;
For every group of initial local TSDF value, the global TSDF value of the group is obtained;
Using the global TSDF value of obtained each group as initial local TSDF value, optimization problem is solved, to obtain most
Overall situation TSDF value eventually.
7. the method as described in note 1, wherein in the optimization problem, each particular voxel initially has specific
In the weight of group, weight be equal to the particular voxel in this set the number of the initial local TSDF value of transformed corresponding voxel or
Person's weight is equal to the sum of the weight corresponding to depth map of the particular voxel transformed corresponding voxel in this set, the particular volume
The weight corresponding to depth map of element corresponding voxel transformed in this set is transformed corresponding in this set with the particular voxel
The voxel depth value that corresponds to camera relative to the depth map it is related.
8. the method as described in note 1, wherein signed distance function is truncated in the initial local for obtaining voxel in three-dimensional space
TSDF value includes:
The initial local TSDF value for the voxel being calculated is normalized.
9. such as method described in note 8, wherein the absolute value of the initial local TSDF value of voxel is less than or equal to certain threshold
Value, the specific threshold belong to (0,1).
10. such as method described in note 9, wherein particular voxel is after meeting all voxels sampling of specific threshold condition
Obtained sampling voxel.
11. a kind of three-dimensional reconstruction equipment of object, comprising:
Device is obtained, is configured as: obtaining the initial local truncation signed distance function TSDF value of voxel in three-dimensional space,
Each initial local TSDF value corresponds to a depth map in multiple depth maps;
Apparatus for grouping is configured as: by corresponding depth map, initial local TSDF value being grouped, at least one set of initial office
Portion's TSDF value corresponds to more than one depth map;
Group overall situation TSDF value acquisition device, is configured as: for every group of initial local TSDF value, obtaining the overall situation of the group
TSDF value;
Solving device is configured as: using the global TSDF value of obtained each group as initial local TSDF value, being solved most
Optimization problem, to obtain final overall situation TSDF value;And
Reconstructing device is configured as: being based on obtained final overall situation TSDF value, object described in three-dimensional reconstruction;
Wherein, in the optimization problem, the global TSDF value of a voxel is based on the final local T SDF of the voxel
Value obtains, and the final local T SDF value of a voxel is equal to the initial local TSDF value of the transformed corresponding voxel of the voxel, becomes
Amount is the parameter of global the TSDF value and transformation of voxel, and cost function is related to following factors: the global TSDF value of particular voxel
With the difference of the initial local TSDF value of the transformed corresponding voxel of the voxel square weighted sum, the weight of weighted sum is equal to spy
Determine the weight of the corresponding group of the transformed corresponding voxel of voxel.
12. the equipment as described in note 11, wherein the transformation includes rigid transformation.
13. note 11 as described in equipment, wherein it is described transformation include non-rigid transformation, cost function also with it is following because
It is plain related: the summation of the offset measurement of the non-rigid transformation of particular voxel experience.
14. the equipment as described in note 11, wherein described group of overall situation TSDF value acquisition device is further configured to: needle
The global TSDF value of the group is obtained by solving optimization problem to every group of initial local TSDF value.
15. the equipment as described in note 11, wherein described group of overall situation TSDF value acquisition device is further configured to: needle
To every group of initial local TSDF value, the arithmetic mean of instantaneous value for corresponding to the initial local TSDF value of same voxel is calculated, as correspondence
In the global TSDF value of the group of the voxel.
16. the equipment as described in note 11, wherein the solving device is further configured to:
Using the global TSDF value of obtained each group as initial local TSDF value, it is grouped again by corresponding depth map,
At least one set of initial local TSDF value corresponds to more than one and is grouped corresponding depth map for the first time;
For every group of initial local TSDF value, the global TSDF value of the group is obtained;
Using the global TSDF value of obtained each group as initial local TSDF value, optimization problem is solved, to obtain most
Overall situation TSDF value eventually.
17. the equipment as described in note 11, wherein in the optimization problem, each particular voxel initially has spy
Due to the weight of group, weight is equal to the number of the particular voxel initial local TSDF value of transformed corresponding voxel in this set
Or weight is equal to the sum of the weight corresponding to depth map of the particular voxel transformed corresponding voxel in this set, this is specific
The weight corresponding to depth map and the particular voxel of voxel transformed corresponding voxel in this set are transformed right in this set
The voxel answered is related relative to the depth value that the depth map corresponds to camera.
18. the equipment as described in note 11, wherein the acquisition device further include: normalization unit is configured as: will
The initial local TSDF value for the voxel being calculated normalizes.
19. the equipment as described in note 18, wherein the absolute value of the initial local TSDF value of voxel is less than or equal to specific
Threshold value, the specific threshold belong to (0,1).
20. the equipment as described in note 19, wherein particular voxel is after meeting all voxels sampling of specific threshold condition
Obtained sampling voxel.
Claims (10)
1. a kind of three-dimensional rebuilding method of object, comprising:
Signed distance function TSDF value is truncated in the initial local for obtaining voxel in three-dimensional space, and each initial local TSDF value is corresponding
A depth map in multiple depth maps;
By corresponding depth map, initial local TSDF value is grouped, at least one set of initial local TSDF value corresponds to more than one
Depth map;
For every group of initial local TSDF value, the global TSDF value of the group is obtained;
Using the global TSDF value of obtained each group as initial local TSDF value, optimization problem is solved, it is final complete to obtain
Office's TSDF value;And
Based on obtained final overall situation TSDF value, object described in three-dimensional reconstruction;
Wherein, in the optimization problem, the global TSDF value of a voxel is worth based on the final local T SDF of the voxel
It arrives, the final local T SDF value of a voxel is equal to the initial local TSDF value of the transformed corresponding voxel of the voxel, and variable is
The parameter of global the TSDF value and transformation of voxel, cost function include following cost item: the global TSDF value of particular voxel with should
The difference of the initial local TSDF value of the transformed corresponding voxel of voxel square weighted sum, the weight of weighted sum is equal to particular volume
The weight of the corresponding group of the transformed corresponding voxel of element.
2. the method for claim 1, wherein the transformation includes rigid transformation.
3. the method for claim 1, wherein the transformation includes non-rigid transformation, cost function further includes another generation
Valence item: the summation of the offset measurement of the non-rigid transformation of particular voxel experience.
4. being the method for claim 1, wherein directed to every group of initial local TSDF value, the global TSDF value of the group is obtained
It include:, by solving optimization problem, to obtain the global TSDF value of the group for every group of initial local TSDF value.
5. being the method for claim 1, wherein directed to every group of initial local TSDF value, the global TSDF value of the group is obtained
Include: to calculate the arithmetic mean of instantaneous value for corresponding to the initial local TSDF value of same voxel for every group of initial local TSDF value, makees
For the global TSDF value of the group corresponding to the voxel.
6. the method for claim 1, wherein described using the global TSDF value of obtained each group as initial local
TSDF value solves optimization problem, includes: to obtain final overall situation TSDF value
Using the global TSDF value of obtained each group as initial local TSDF value, it is grouped again by corresponding depth map, at least
One group of initial local TSDF value corresponds to more than one and is grouped corresponding depth map for the first time;
For every group of initial local TSDF value, the global TSDF value of the group is obtained;
Using the global TSDF value of obtained each group as initial local TSDF value, optimization problem is solved, it is final complete to obtain
Office's TSDF value.
7. each particular voxel initially has specific the method for claim 1, wherein in the optimization problem
In the weight of group, weight be equal to the particular voxel in this set the number of the initial local TSDF value of transformed corresponding voxel or
Person's weight is equal to the sum of the weight corresponding to depth map of the particular voxel transformed corresponding voxel in this set, the particular volume
The weight corresponding to depth map of element corresponding voxel transformed in this set is transformed corresponding in this set with the particular voxel
The voxel depth value that corresponds to camera relative to the depth map it is related.
8. signed distance function is truncated in the initial local for the method for claim 1, wherein obtaining voxel in three-dimensional space
TSDF value includes:
The initial local TSDF value for the voxel being calculated is normalized.
9. method according to claim 8, wherein the absolute value of the initial local TSDF value of voxel is less than or equal to certain threshold
Value, the specific threshold belong to (0,1).
10. a kind of three-dimensional reconstruction equipment of object, comprising:
Device is obtained, is configured as: obtaining the initial local truncation signed distance function TSDF value of voxel in three-dimensional space, each
Initial local TSDF value corresponds to a depth map in multiple depth maps;
Apparatus for grouping is configured as: by corresponding depth map, initial local TSDF value being grouped, at least one set of initial local
TSDF value corresponds to more than one depth map;
Group overall situation TSDF value acquisition device, is configured as: for every group of initial local TSDF value, obtaining the global TSDF of the group
Value;
Solving device is configured as: using the global TSDF value of obtained each group as initial local TSDF value, being solved and is optimized
Problem, to obtain final overall situation TSDF value;And
Reconstructing device is configured as: being based on obtained final overall situation TSDF value, object described in three-dimensional reconstruction;
Wherein, in the optimization problem, the global TSDF value of a voxel is worth based on the final local T SDF of the voxel
It arrives, the final local T SDF value of a voxel is equal to the initial local TSDF value of the transformed corresponding voxel of the voxel, and variable is
The parameter of global the TSDF value and transformation of voxel, cost function include following cost item: the global TSDF value of particular voxel with should
The difference of the initial local TSDF value of the transformed corresponding voxel of voxel square weighted sum, the weight of weighted sum is equal to particular volume
The weight of the corresponding group of the transformed corresponding voxel of element.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201510590009.7A CN106558076B (en) | 2015-09-16 | 2015-09-16 | The method and apparatus of three-dimensional reconstruction object |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201510590009.7A CN106558076B (en) | 2015-09-16 | 2015-09-16 | The method and apparatus of three-dimensional reconstruction object |
Publications (2)
Publication Number | Publication Date |
---|---|
CN106558076A CN106558076A (en) | 2017-04-05 |
CN106558076B true CN106558076B (en) | 2019-06-18 |
Family
ID=58413603
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN201510590009.7A Active CN106558076B (en) | 2015-09-16 | 2015-09-16 | The method and apparatus of three-dimensional reconstruction object |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN106558076B (en) |
Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN104504671A (en) * | 2014-12-12 | 2015-04-08 | 浙江大学 | Method for generating virtual-real fusion image for stereo display |
CN104616345A (en) * | 2014-12-12 | 2015-05-13 | 浙江大学 | Octree forest compression based three-dimensional voxel access method |
EP2886043A1 (en) * | 2013-12-23 | 2015-06-24 | a.tron3d GmbH | Method for continuing recordings to detect three-dimensional geometries of objects |
Family Cites Families (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
KR101839035B1 (en) * | 2012-11-26 | 2018-03-19 | 한국전자통신연구원 | Apparatus for 3d reconstruction based on multiple gpus and method thereof |
-
2015
- 2015-09-16 CN CN201510590009.7A patent/CN106558076B/en active Active
Patent Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
EP2886043A1 (en) * | 2013-12-23 | 2015-06-24 | a.tron3d GmbH | Method for continuing recordings to detect three-dimensional geometries of objects |
CN104504671A (en) * | 2014-12-12 | 2015-04-08 | 浙江大学 | Method for generating virtual-real fusion image for stereo display |
CN104616345A (en) * | 2014-12-12 | 2015-05-13 | 浙江大学 | Octree forest compression based three-dimensional voxel access method |
Non-Patent Citations (1)
Title |
---|
基于Kinect深度图像的三维重建研究;韦羽棉;《中国优秀硕士学位论文全文数据库 信息科技辑》;20150115(第1期);I138-1237 |
Also Published As
Publication number | Publication date |
---|---|
CN106558076A (en) | 2017-04-05 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
Yu et al. | Pu-net: Point cloud upsampling network | |
Wang et al. | Pixel2mesh: Generating 3d mesh models from single rgb images | |
Tagliasacchi et al. | 3d skeletons: A state‐of‐the‐art report | |
Govindu et al. | On averaging multiview relations for 3D scan registration | |
Pauly et al. | Efficient simplification of point-sampled surfaces | |
Li et al. | Detail-preserving and content-aware variational multi-view stereo reconstruction | |
Liu et al. | Cubic stylization | |
Le Guyader et al. | Self-repelling snakes for topology-preserving segmentation models | |
Tagliasacchi et al. | Vase: Volume‐aware surface evolution for surface reconstruction from incomplete point clouds | |
CN110619681A (en) | Human body geometric reconstruction method based on Euler field deformation constraint | |
CN112581593B (en) | Training method of neural network model and related equipment | |
CN110889893B (en) | Three-dimensional model representation method and system for expressing geometric details and complex topology | |
CN113643336B (en) | Three-dimensional image rigid matching method based on spherical polar coordinate system depth neural network | |
Campos et al. | Splat-based surface reconstruction from defect-laden point sets | |
CN106548466B (en) | The method and apparatus of three-dimensional reconstruction object | |
Fabbri et al. | Camera pose estimation using first-order curve differential geometry | |
Eigensatz et al. | Positional, metric, and curvature control for constraint‐based surface deformation | |
CN104183002A (en) | Three-dimensional model change method and device | |
CN106548507A (en) | The method and apparatus of three-dimensional reconstruction object | |
Roberts et al. | Deformation transfer survey | |
CN107680073A (en) | The method and apparatus of geometrical reconstruction object | |
Gois et al. | Generalized hermitian radial basis functions implicits from polygonal mesh constraints | |
Woodford et al. | Large scale photometric bundle adjustment | |
Zach et al. | Practical methods for convex multi-view reconstruction | |
CN111340950B (en) | Digital elevation model reconstruction method and device |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
GR01 | Patent grant | ||
GR01 | Patent grant |