CN106558076A - The method and apparatus of three-dimensional reconstruction object - Google Patents
The method and apparatus of three-dimensional reconstruction object Download PDFInfo
- Publication number
- CN106558076A CN106558076A CN201510590009.7A CN201510590009A CN106558076A CN 106558076 A CN106558076 A CN 106558076A CN 201510590009 A CN201510590009 A CN 201510590009A CN 106558076 A CN106558076 A CN 106558076A
- Authority
- CN
- China
- Prior art keywords
- voxel
- tsdf
- tsdf values
- values
- initial local
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2200/00—Indexing scheme for image data processing or generation, in general
- G06T2200/28—Indexing scheme for image data processing or generation, in general involving image processing hardware
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10028—Range image; Depth image; 3D point clouds
Landscapes
- Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Computer Graphics (AREA)
- Geometry (AREA)
- Software Systems (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Image Generation (AREA)
Abstract
The invention discloses a kind of method and apparatus of three-dimensional reconstruction object.The method includes:The initial local TSDF values of voxel in three dimensions are obtained, each initial local TSDF values are corresponding to a depth map in multiple depth maps;By corresponding depth map, initial local TSDF values are grouped, least one set initial local TSDF values are corresponding to more than one depth map;For every group of initial local TSDF value, the global TSDF values of the group are obtained;Using the global TSDF values of resulting each group as initial local TSDF values, optimization problem is solved, to obtain final overall situation TSDF values;And based on resulting final global TSDF values, object described in three-dimensional reconstruction;Wherein, in the optimization problem, variable is the parameter of the global TSDF values of voxel and conversion, and cost function is related to following factors:The difference of the initial local TSDF values of the global TSDF values corresponding voxel transformed with the voxel of particular voxel square weighted sum, weight be equal to the transformed corresponding voxel of particular voxel correspondence group weight.
Description
Technical field
This invention relates generally to three-dimensional imaging field.Specifically, the present invention relates to one kind can be accurate
The method and apparatus for really carrying out the three-dimensional reconstruction of object.
Background technology
In recent years, the application of many correlations with the development of 3 Dimension Image Technique, has been emerged in large numbers, has such as been strengthened
Reality, digital museum, 3 D-printing etc..The importance of 3 Dimension Image Technique is three-dimensional reconstruction
Technology.The three-dimensional rebuilding method of main flow rebuilds three dimensional object based on one group of depth image.
Traditional three-dimensional rebuilding method focuses mainly on how more accurately carrying out the parameter of camera
Estimate, for example, deformation information is introduced to estimate the distortion of camera.But, traditional three-dimensional reconstruction
Method, (blocks signed distance function, Truncated Signed Distance overall situation TSDF is calculated
When Function) characterizing, simply simply all of local T SDF is characterized carries out arithmetic average,
Fully excavate local T SDF to characterize and the relation between overall situation TSDF signs.Change due to only relying on
Enter the estimation to camera parameter to lift the effect of three-dimensional reconstruction, so traditional three-dimensional rebuilding method
Effect improved limited, the accuracy of three-dimensional reconstruction object needs further to be improved.
Therefore, it is desirable to a kind of method and apparatus of three-dimensional reconstruction object, which can carry out object exactly
Three-dimensional reconstruction.
The content of the invention
The brief overview with regard to the present invention is given below, to provide with regard to some of the invention
The basic comprehension of aspect.It should be appreciated that this general introduction is not the exhaustive general introduction with regard to the present invention.
It is not intended to the crucial or pith for determining the present invention, nor the model of the intended limitation present invention
Enclose.Its purpose only provides some concepts in simplified form, more detailed in this, as what is discussed after a while
The preamble of thin description.
The purpose of the present invention is the problems referred to above for prior art, it is proposed that one kind lays particular emphasis on excavation office
The method and apparatus of the three-dimensional reconstruction object of the relation between portion TSDF is characterized and overall situation TSDF is characterized.
To achieve these goals, according to an aspect of the invention, there is provided a kind of three-dimensional of object
Method for reconstructing, the method include:The initial local for obtaining voxel in three dimensions blocks directed distance letter
Number TSDF values, each initial local TSDF values are corresponding to a depth map in multiple depth maps;Press
Corresponding depth map, initial local TSDF values are grouped, least one set initial local TSDF values correspondence
In more than one depth map;For every group of initial local TSDF value, the global TSDF of the group is obtained
Value;Using the global TSDF values of resulting each group as initial local TSDF values, solve optimization and ask
Topic, to obtain final overall situation TSDF values;And based on resulting final global TSDF values, it is three-dimensional
Rebuild the object;Wherein, in the optimization problem, the global TSDF values base of a voxel
It is worth in final local T SDF of the voxel, the final local T SDF value of a voxel is equal to the body
The initial local TSDF values of the transformed corresponding voxel of element, variable are the global TSDF values of voxel and become
The parameter changed, cost function are related to following factors:The global TSDF values of particular voxel and the voxel
The difference of the initial local TSDF values of transformed corresponding voxel square weighted sum, the power of weighted sum
The weight of the correspondence group of the transformed corresponding voxel of particular voxel is equal to again.
According to another aspect of the present invention, there is provided a kind of three-dimensional reconstruction equipment of object, the equipment
Including:Device is obtained, is configured to:Obtain three dimensions in voxel initial local block it is oriented away from
From function TSDF values, each initial local TSDF values are corresponding to a depth map in multiple depth maps;
Apparatus for grouping, is configured to:By corresponding depth map, initial local TSDF values are grouped, at least
One group of initial local TSDF value is corresponding to more than one depth map;Group overall situation TSDF value acquisition device,
It is configured to:For every group of initial local TSDF value, the global TSDF values of the group are obtained;Solve dress
Put, be configured to:Using the global TSDF values of resulting each group as initial local TSDF values, ask
Solution optimization problem, to obtain final overall situation TSDF values;And reconstructing device, it is configured to:Base
In resulting final global TSDF values, object described in three-dimensional reconstruction;Wherein, in the optimization
In problem, the global TSDF values of a voxel are worth to based on final local T SDF of the voxel, and one
The final local T SDF value of individual voxel is equal to the initial local TSDF of the transformed corresponding voxel of the voxel
Value, variable are the global TSDF values of voxel and the parameter for converting, and cost function is related to following factors:
The initial local TSDF values of the global TSDF values corresponding voxel transformed with the voxel of particular voxel
Poor square of weighted sum, the weight of weighted sum are equal to the correspondence of the transformed corresponding voxel of particular voxel
The weight of group.
In addition, according to a further aspect in the invention, additionally provide a kind of storage medium.The storage is situated between
Matter includes machine-readable program code, when described program code is performed on message processing device,
Described program code causes described information processing equipment to perform said method of the invention.
Additionally, in accordance with a further aspect of the present invention, additionally provide a kind of program product.Described program is produced
Product include the executable instruction of machine, when the instruction is performed on message processing device, the finger
Order causes described information processing equipment to perform said method of the invention.
Description of the drawings
With reference to explanation below in conjunction with the accompanying drawings to embodiments of the invention, this can be more readily understood that
Bright above and other objects, features and advantages.Part in accompanying drawing is intended merely to illustrate the present invention's
Principle.In the accompanying drawings, same or similar technical characteristic or part will be using same or similar attached
Icon is remembered to represent.In accompanying drawing:
The flow chart that Fig. 1 shows three-dimensional reconstruction object method according to an embodiment of the invention;
Fig. 2 shows the flow chart for obtaining initial local TSDF methods according to an embodiment of the invention;
Fig. 3 shows the flow chart for calculating initial local TSDF methods according to an embodiment of the invention;
Fig. 4 shows the structure square frame of the equipment of three-dimensional reconstruction object according to an embodiment of the invention
Figure;And
Fig. 5 shows the computer that can be used to implementing method and apparatus according to an embodiment of the invention
Schematic block diagram.
Specific embodiment
The one exemplary embodiment of the present invention is described in detail hereinafter in connection with accompanying drawing.In order to clear
Chu and it is simple and clear for the sake of, all features of actual embodiment are not described in the description.However, should
The understanding, must make many specific to enforcement during any this actual embodiment is developed
The decision of mode, to realize the objectives of developer, for example, meets and system and business phase
Those restrictive conditions for closing, and these restrictive conditions may be with the different of embodiment
Change.Additionally, it also should be appreciated that, although development is likely to be extremely complex and time-consuming, but
For the those skilled in the art for having benefited from present disclosure, this development is only routine
Task.
Here, in addition it is also necessary to which explanation is a bit, in order to avoid having obscured this because of unnecessary details
It is bright, illustrate only in the accompanying drawings the apparatus structure closely related with scheme of the invention and/or
Process step, and eliminate the other details little with relation of the present invention.In addition, it is also stated that
It is that the element and feature described in an a kind of accompanying drawing or embodiment of the present invention can be with one
Or the element that illustrates in more other accompanying drawings or embodiment and feature combine.
The three-dimensional rebuilding method of object according to an embodiment of the invention is described below with reference to Fig. 1
Flow process.
The flow chart that Fig. 1 shows three-dimensional reconstruction object method according to an embodiment of the invention.As schemed
Shown in 1, three-dimensional reconstruction object method comprises the steps according to an embodiment of the invention:Obtain three
In dimension space, the initial local of voxel blocks signed distance function TSDF values, each initial local
TSDF values are corresponding to a depth map (step S1) in multiple depth maps;By corresponding depth map,
Initial local TSDF values are grouped, least one set initial local TSDF values are corresponding to more than one depth
Degree figure (step S2);For every group of initial local TSDF value, the global TSDF values of the group are obtained
(step S3);Using the global TSDF values of resulting each group as initial local TSDF values, ask
Solution optimization problem, to obtain final overall situation TSDF values (step S4);And based on resulting
Final overall situation TSDF values, object (step S5) described in three-dimensional reconstruction;Wherein, in the optimization
In problem, the global TSDF values of a voxel are worth to based on final local T SDF of the voxel,
The final local T SDF value of one voxel is equal to the initial local of the transformed corresponding voxel of the voxel
TSDF values, variable be the global TSDF values of voxel and conversion parameter, cost function with it is following because
It is plain related:The initial local of the global TSDF values corresponding voxel transformed with the voxel of particular voxel
The difference of TSDF values square weighted sum, it is transformed corresponding that the weight of weighted sum is equal to particular voxel
The weight of the correspondence group of voxel.
In step sl, the initial local for obtaining voxel in three dimensions blocks signed distance function
TSDF values, each initial local TSDF values are corresponding to a depth map in multiple depth maps.
Specifically, three dimensions are evenly divided into some voxels, it is believed that voxel is three-dimensional
The elementary cell in space.Three dimensional object is rebuild due to based on one group of depth map, so three-dimensional reconstruction pair
As the input of method is one group of depth map.This group of depth map is shot by multiple cameras and is obtained, each
Camera corresponds to a depth map.Again as input does not include the parameter of camera, so needing based on deep
Degree figure is estimating the parameter of corresponding camera, and and then obtains the initial local TSDF values of associated voxels.
In addition, it should be noted here that " the multiple cameras " mentioned herein is included in different positions
With the situation of multiple cameras physically of direction, also including same camera by adjustment position and/
Or towards and form the situation of multiple cameras in logic, also including the feelings of above-mentioned two situations mixing
Condition.
The initial local TSDF values of voxel are defined as what is calculated in three dimensions based on camera parameter
Voxel relative between the depth of the voxel correspondence position in the depth and depth map of camera it is oriented away from
From the result Jing after blocking.Such TSDF values are referred to as into initial local TSDF values.From defined above
Understand, a voxel can calculate a directed distance defined above relative to a camera, one
Individual voxel can calculate multiple directed distances defined above relative to multiple cameras, each it is oriented away from
From corresponding to a camera.Directed distance is due to will be through break-in operation, so a voxel may be right
Should in one or more initial locals TSDF values, the corresponding initial local TSDF values of voxel it is upper
Limit is the number of camera/depth map.Jing after blocking, some voxels can no initial local TSDF values,
Such voxel is no longer considered.That is, being calculated initial local TSDF values in step sl
Voxel just can be continued with follow-up step S2, S3.In addition, knowable to above-mentioned definition, often
Individual initial local TSDF values are corresponding to multiple magazine cameras, namely correspond to multiple depth
A depth map in figure.
Due to camera capture depth map when, acquisition be subject surface information, it is possible to
Understand based on camera parameter to be the voxel relative to the depth of camera in the voxel that three dimensions are calculated
With the projection in the direction of the optical axis of the distance between the imaging point of camera, and voxel correspondence in depth map
The depth of position is in the subject surface passed through by the light between the voxel and the imaging point of camera
Projection of the point with the distance between the imaging point of camera in the direction of the optical axis.As the voxel may be relative
The point in the subject surface is farther or closer to so distance projection in the direction of the optical axis apart from camera
Difference be directed distance.Distance in the direction of the optical axis be projected as zero, show the voxel and subject surface
On the point overlap, namely the voxel is the point in subject surface.
In theory, the null all voxels of initial local TSDF values constitute subject surface.But,
As the calculating of the value of initial local TSDF depends on the estimation of camera parameter, the estimation of camera parameter
Value might not be completely the same with the actual value of camera parameter, so initial local TSDF values can not
It is used directly to rebuild the surface of three dimensional object.A kind of possible method is that have initial local to each
All initial local TSDF values of the voxel of TSDF values seek arithmetic mean of instantaneous value, using arithmetic mean of instantaneous value as
The global TSDF values of the voxel, are then based on the global TSDF values and the voxel with the value to rebuild
The surface of three dimensional object.However, so generate the problem that have cured initial local TSDF values with it is complete
Relation between office's TSDF values.
As camera has multiple, so the situation of the parameter estimation of multiple cameras there may be difference, because
Different cameral corresponding initial local TSDF values are converted to global TSDF by the way of unified by this
Value is obviously not accurate enough.So the present invention puts forth effort to solve the problems, such as to obtain accurately overall situation TSDF values.
In addition, through foregoing description, it will be understood that step S1 can pass through the method reality shown in Fig. 2
It is existing, to obtain initial local TSDF values.
As shown in Fig. 2 in the step s 21, estimate corresponding with multiple depth maps multiple magazine
The camera parameter of each camera, camera parameter include but is not limited to the position and orientation of camera.For example,
Can be carried out using the matching algorithm between the depth map and threedimensional model used in Kinect Fusion
Camera parameter is estimated.Of course, it is possible to adopt all suitable camera parameter estimation sides known in the art
Method realizes step S21.
In step S22, based on estimated camera parameter, one or more of calculating voxel are initial
Local T SDF value.
Step S22 can pass through the method shown in Fig. 3 and realize.
As shown in figure 3, for each voxel and multiple depth of initial local TSDF values to be calculated
Corresponding each camera of figure, in step S31, based on the camera parameter estimated by camera, counts
Calculate first depth value of the voxel relative to the camera.In step s 32, determine the voxel it is corresponding,
The second depth value in the depth map of the camera association.In step S33, by the first depth value and
The difference of two depth values block after result, as voxel initial local corresponding with the camera
TSDF values.
It should be noted that during when calculating the initial local TSDF values of voxel, the scope of voxel is three dimensions
Predetermined estimation can be fully contemplated by the region of object.As the calculating of initial local TSDF is related to cut
Disconnected operation, so Jing after blocking, the scope of the voxel with initial local TSDF values further will contract
It is little.By the threshold value for controlling to block, it is right to be limited to the voxel with initial local TSDF values
The near surface of elephant.The threshold value blocked can for example be positive and negative 5 centimetres, namely initial local
The span of TSDF values can be [- 5,5].
For convenience of calculation, also initial local TSDF values are normalized.Normalized threshold value is
For the threshold value blocked.
Block the near surface that voxel is ensure that in object, will also be seen that, voxel from follow-up explanation
Transformed (rigid transformation or non-rigid transformation) is understood corresponding to the voxel itself or another voxel, in order to
Near surface of the voxel before and after ensureing to convert all in object, by initial local further to voxel
TSDF values are limited.
Specifically, the absolute value for limiting the initial local TSDF values of voxel is less than or equal to specific threshold
α, specific threshold α belong to (0,1).That is, the initial local TSDF values of [- α, α]
The transformed voxel that can be corresponding to its initial local TSDF values between [- 1,1] of voxel.
If additionally, respectively in coordinate system (such as seat with the camera as origin of each camera association
Mark system) middle calculating initial local TSDF values, then the initial local TSDF values of calculated voxel are right
Should be in the coordinate system of multiple magazine camera associations.Can be by the initial office of calculated voxel
Portion's TSDF values are transformed in the same coordinate system.For example, by the initial local TSDF of calculated voxel
Value is transformed in the coordinate system of first camera association.By by initial local TSDF primary systems one to
Individual coordinate system, can facilitate from initial local TSDF values to the conversion of final local T SDF value and enter
And global TSDF values are tried to achieve, amount of calculation can be reduced, and globally optimal solution rather than office can be obtained
Portion's optimal solution.
It is of course also possible to direct calculate initial office of the voxel relative to each camera in global coordinate system
Portion's TSDF values.Using the benefit of this kind of mode can be avoid by the initial local TSDF values of voxel from
The coordinate system of each camera association is transformed into the loss of significance produced during the same coordinate system.
One important means of the present invention is that initial local TSDF values are grouped, and is then directed to
Each packet individual processing, then the result integrated treatment to each group.By such mode, Ke Yiying
To having a case that many initial local TSDF values are that data volume is very big.
In step s 2, by corresponding depth map, initial local TSDF values are grouped, at least one
Group initial local TSDF values are corresponding to more than one depth map.
As it was previously stated, each initial local TSDF values are corresponding to a depth in multiple depth maps
Figure.Therefore, it can according to initial local TSDF values corresponding to which depth map come by initial local
TSDF values are assigned in different groups.Each group can be included corresponding to one or more depth maps
Initial local TSDF values, but at least one group of initial local TSDF values are corresponding to more than one
Depth map.Corresponding to same depth map all initial local TSDF values only in a group.
In step s3, for every group of initial local TSDF value, obtain the global TSDF of the group
Value.
As step S3 will be accomplished that the transformation from initial local TSDF values to global TSDF values,
So with the invention solves the problems that be complete phase from initial local TSDF values to the transformation of global TSDF values
With, the scope for simply processing is defined in each packet.Therefore, step S3 both can be using tradition
Method realize, it is also possible to using the method according to the invention realize.
When use conventional methods realize step S3 when, for every group of initial local TSDF value, meter
The arithmetic mean of instantaneous value of the initial local TSDF values corresponding to same voxel is calculated, as corresponding to the voxel
The group global TSDF values.For example, certain organizes initial local TSDF values corresponding to three depth maps,
So, in the group corresponding to voxel A initial local TSDF values be up to three A1, A2,
A3, calculates the value of (A1+A2+A3)/3, as the global TSDF corresponding to voxel A of the group
Value.
Note, due to being grouped initial local TSDF values according to corresponding depth map, so same
Voxel may have corresponding initial local TSDF values in multiple packets, correspondingly, through step
S3, corresponding to multiple voxels, each voxel is likely to per group of global TSDF values in multiple groups
There are corresponding global TSDF values.Hereinafter will introduce what is how step S3 obtained in step S4
PRELIMINARY RESULTS is further processed, and obtains final overall situation TSDF values unique to voxel with final.
When step S3 is realized using method for optimizing of the invention, for every group of initial local
TSDF values, by solving optimization problem, obtain the global TSDF values of the group.
It is noted that during due to realizing step S3, process is in packet when with contrasting hereinafter
The initial local TSDF values in portion, it is possible to weight is not used in optimization problem.And later
In for the result after packet, in order to eliminate the distortions such as the pseudo-side that may cause of packet, preferably in optimum
Weight used in change problem.
It is introduced below how by designing and solving optimization problem realizing from initial local TSDF values
To the conversion of global TSDF values.Related thought is equally applicable to step S3 and step S4.
As previously described, because initial local TSDF values are calculated based on the camera parameter estimated, and not
With the corresponding different cameral of depth map estimation parameter not necessarily entirely accurate and inaccurate
Situation is not necessarily identical, so being not suitable for all initial locals in a uniform manner to voxel
TSDF values are processed to obtain the global TSDF values of voxel.It is in fact possible to think the first voxel
Initial local TSDF values conversion (inverse transformation of rigid transformation or non-rigid transformation) to the second voxel
Place, becomes the final local T SDF value of the second voxel.First voxel can be with identical (zero with the second voxel
Conversion), it is also possible to different (non-zero transforms).Specific threshold α hereinbefore defines the second voxel
The span of initial local TSDF, so as to the final local T SDF value that ensure that the second voxel belongs to
[-1,1]。
It should be noted that rigid transformation is carried out for same camera.That is, the corresponding category of same camera
In multiple first voxels multiple initial local TSDF values in these first voxels through same rigidity
The inverse transformation of conversion become while corresponding to multiple second voxels these the second voxels corresponding to this
The multiple final local T SDF value of camera.So-called rigidity is referred to corresponding to the multiple initial of same camera
The rotationally and/or translationally conversion of local T SDF value Jing unification reaches the second voxel (TSDF from the first voxel
Value itself is constant), the relative position relation between voxel associated by these values before and after conversion not
Change.
It should be noted that non-rigid transformation is carried out for voxel.The main body of conversion is voxel, the result of conversion
It is the final local of the corresponding relation and the second voxel that establish the first voxel and the second voxel
TSDF values are equal to the initial local TSDF values of corresponding first voxel.That is, first body
One initial local TSDF value of element is corresponded to through the inverse transformation of non-rigid transformation in first voxel
Become the final local T SDF value of second voxel while second voxel.It is non-rigid refer to each
One voxel reach the conversion experienced of the second voxel be it is independent of each other, it is unrelated with camera, corresponding to same
The inverse transformation of the respective non-rigid transformation of multiple initial local TSDF values Jing of one camera is from the first voxel
Reach the second voxel (TSDF values itself are constant), the relative position between voxel associated by these values
Relation can change before and after conversion.Relative to rigid transformation, can be more using non-rigid transformation
The conversion for being meticulously considered for multiple initial local TSDF values correlations of same camera may be also that
This is different.Therefore, non-rigid transformation is to further increase voxel relative to the benefit of rigid transformation
Final TSDF values accuracy rate, and then improve the accuracy and three-dimensional reconstruction object of global TSDF
Effect.
Through the amendment of rigid transformation/non-rigid transformation, the error of camera parameter estimation is counteracted, is made
Obtain TSDF values more accurate with the corresponding relation of voxel.Meanwhile, it is worth to entirely from final local T SDF
Office's TSDF values, it is possible to and then three dimensional object is rebuild based on overall situation TSDF values.
According to a preferred embodiment of the invention, using designing and solve by the way of optimization problem realizing
Rigid transformation/non-rigid transformation for being related to final local T SDF value from initial local TSDF values and
From final local T SDF value to the calculating of global TSDF values.
The design key of optimization problem is the design of variable and cost function.
For rigid transformation, in optimization problem, the global TSDF values of a voxel are based on
Final local T SDF of the voxel is worth to, and the final local T SDF value of a voxel is equal to the voxel
The initial local TSDF values of the corresponding voxel of Jing rigid transformations, variable be voxel global TSDF values and
The parameter of rigid transformation, cost function are related to following factors:The global TSDF values of voxel and the body
The quadratic sum of the difference of the initial local TSDF values of the corresponding voxel of plain Jing rigid transformations.
Cost function for example can be designed as the global TSDF values and voxel Jing rigid transformations pair of voxel
The summation of the quadratic sum of the difference of the initial local TSDF values of the voxel answered.Step S1 is noted above
When obtaining initial local TSDF values, due to having carried out blocking, normalization, and in some embodiments
Middle utilization threshold alpha is further limited, and causes only part voxel to have initial local TSDF values, after
Continuous step will be carried out for these voxels.Therefore, cost function is can be designed as these voxels
Each calculation cost item, cost function are the summations of cost item, and each cost item is the overall situation of voxel
The difference of the initial local TSDF values of TSDF values voxel corresponding with voxel Jing rigid transformations square
With, namely the quadratic sum of the difference of the final local T SDF value of the global TSDF values of voxel and the voxel.
Here why it is designed as final local of the global TSDF values of a voxel based on the voxel
TSDF is worth to, and the final local T SDF value of a voxel is corresponding equal to voxel Jing rigid transformations
The initial local TSDF values of voxel, rather than the global TSDF values of a voxel are designed as based on the body
The initial local TSDF of element is worth to, if being because designing according to the latter, optimization problem
Solution can be a voxel global TSDF values be equal to the voxel initial local TSDF values arithmetic/add
Weight average value.By introduce for camera it is overall only relate to the rigid transformation for translating and rotating,
The conversion from initial local TSDF values to final local T SDF value is increased, so that optimization is asked
The solution of topic is more accurate, causes the result of three-dimensional reconstruction object more accurate.
If overall situation TSDF values are V, depth map number is n, then the corresponding initial local of each depth map
TSDF values are V1、V2、……、Vn, the corresponding final local T SDF value of each depth map is V1’、
V2’、……、Vn', serial number i of depth map, i=1,2 ... ..., n.For the voxel p in V,
Which is in Vi' in corresponding voxel be still p, and in ViIn corresponding voxel be Ti(p), wherein TiFor optimization
The rigid transformation for solving is needed in problem.
As it was noted above, voxel Jing block, normalization and there is initial local TSDF values, that is, require
ViP the value of () is located at [- 1,1] interval.Also need to herein seek Vi(Ti(p))∈[-1,1].Due to ViWith Vi' between
Violent conversion will not be carried out, that is to say, that voxel p and voxel TiP the distance between () will not be far,
So definition set Pi:Pi=p | Vi(p) ∈ [- α, α] }, enabling ensure Vi(Ti(p))∈[-1,1]。
That is, during calculation cost function, for belonging to set PiVoxel p calculation cost items
Summation, each cost item is global TSDF values of the voxel p in V and voxel p Jing rigid transformations pair
The V for answeringiIn voxel TiP the quadratic sum of the difference of the initial local TSDF values of (), a square summation are because
Voxel p may have multiple final local T SDF values, so as to there is multiple voxel Ti(p) it is initial
Local T SDF value.
So cost function can be expressed as:
Optimization problem is solved, that is, solves the final global TSDF for minimizing above-mentioned cost function
The optimal value of value and rigid transformation parameters.
An overall situation TSDF value of voxel is to emphasize that a voxel is finally only global with one herein
TSDF values, and the scope of such voxel is set Pi。
Optimization problem can be solved by Gauss-Newton (Gauss-Newton) method of iteration or
Calculated come iterative respectively by the parameter of global TSDF values and rigid transformation for voxel.
But no matter which kind of solves mode, is directed to iteration, it is necessary to set the initial value of iteration.
In optimization problem, the initial value of the global TSDF values of voxel is equal to all first of the voxel
The weighted mean of beginning local T SDF value.On the one hand weighting herein can be arithmetic average, another
Aspect can also calculate weight according to the distance of camera and voxel.
It is the body that the initial value of the parameter of rigid transformation causes the corresponding voxel of voxel Jing rigid transformations
Element itself.One voxel may have multiple initial local TSDF values, if voxel Jing rigidly becomes
Change and correspond to the voxel itself, then the initialized final local T SDF value of this voxel also has many
It is individual, each correspond to a camera.
For example, rigid transformation Ti(p)=Ri*p+ti, wherein RiIt is spin matrix, list can be initialized as
Position battle array, tiFor translation vector, null vector can be initialized as.
For non-rigid transformation, in optimization problem, the global TSDF values base of a voxel
It is worth in final local T SDF of the voxel, the final local T SDF value of a voxel is equal to the body
The initial local TSDF values of the corresponding voxel of plain Jing non-rigid transformations, variable is the global TSDF of voxel
The parameter of value and non-rigid transformation, cost function are related to following factors:The global TSDF of particular voxel
The quadratic sum of the difference of the initial local TSDF values of value voxel corresponding with voxel Jing non-rigid transformations,
The summation of the skew tolerance of the non-rigid transformation of particular voxel experience.
Previously mentioned voxel will meet specific threshold condition, that is, limit the initial local TSDF values of voxel
Absolute value be less than or equal to specific threshold α, specific threshold α belong to (0,1).If such
Voxel is not too many, then be able to can be solved based on all such voxel design cost functions, optimization problem.
Now, particular voxel is all voxels for meeting specific threshold condition.If such voxel is more,
Then based on when all so voxel designs cost function, optimization problem may be solved.This
In the case of, sampling body can be sampled and is based on by all voxels to meeting specific threshold condition
Element designs cost function optimization problem can be solved.The parameter of the non-rigid transformation of non-sampled voxel
Can be according to the parameter determination of the non-rigid transformation of sampling voxel.Now, particular voxel is to meet specific
The sampling voxel obtained after all voxel samplings of threshold condition.In a preferred embodiment, it is sampled as
Uniform sampling.
Cost function for example can be designed as related to following factors:The global TSDF values of particular voxel
The quadratic sum (of the difference of the initial local TSDF values of voxel corresponding with voxel Jing non-rigid transformations
One cost item), particular voxel experience non-rigid transformation skew tolerance summation (the second cost item).
Cost function can be designed as the first cost item and the second cost item sum, wherein the first cost item
It is not larger with the second cost item number value difference, again with first after can be multiplied the second cost item with balance factor
Cost item is sued for peace.
First cost item is that the global TSDF values of particular voxel are corresponding with voxel Jing non-rigid transformations
The quadratic sum of the difference of the initial local TSDF values of voxel, namely the global TSDF values of particular voxel with should
The quadratic sum of the difference of the final local T SDF value of voxel.Here the summation in quadratic sum includes two
The summation of aspect, one side particular voxel be not unique, and the correlation of multiple particular voxels needs summation,
On the other hand, each particular voxel may have multiple final local T SDF values, so each is special
The correlation for determining voxel is also required to summation.
Second cost item is the summation of the skew tolerance of the non-rigid transformation of particular voxel experience.Second
In the case of voxel Jing non-rigid transformations the first voxel of correspondence, above-mentioned skew tolerance refers to the first voxel
Relative to the skew of the second voxel mould square.As same second voxel may be with multiple final
Local T SDF value, so second voxel may correspond to multiple first via various non-rigid transformations
Voxel, now the multiple final local T SDF value of second voxel is first equal to the plurality of first voxel
Beginning local T SDF value.Therefore, correspondingly, skew tolerance, i.e., multiple first voxels are relative to second
The skew of voxel, there is also it is multiple, so need carry out offset tolerance summation operation.Also, deposit
In multiple second voxels, so also needing to sue for peace the skew tolerance of multiple second voxels.It can be seen that,
The summation of the skew tolerance of the non-rigid transformation of particular voxel experience is the institute of all particular voxel experience
There is the summation of the skew tolerance of non-rigid transformation.As described above, particular voxel is to meet specific threshold
All voxels of condition or particular voxel are met after all voxels sampling of specific threshold condition
The sampling voxel for arriving.Hereinafter cost item will be further described by taking voxel of sampling as an example.
In addition, the purpose of cost function is to understand optimization problem, when cost function is based only upon sampling
During voxel, non-sampled voxel experience can be obtained according to the skew of the non-rigid transformation of sampling voxel experience
Non-rigid transformation skew.So, cost function is based only upon sampling voxel so as to optimization problem can
Solve and solve amount of calculation and reduce solving speed soon, while remaining to obtain the overall situation of non-sampled voxel
TSDF values.For example, the skew of the non-rigid transformation of non-sampled voxel experience can be according to sampling voxel
The skew of the non-rigid transformation of experience is obtained by three-dimensional interpolation.In one embodiment, non-sampled body
Skew basis and non-sampled voxel of the non-rigid transformation of element experience is experienced apart near sampling voxel
The skew of non-rigid transformation obtained by three-dimensional interpolation.In another embodiment, non-sampled voxel
The skew of the non-rigid transformation of experience is according to the sampling body that same camera is corresponded to the non-sampled voxel
The skew of the non-rigid transformation of element experience is obtained by three-dimensional interpolation.In yet another embodiment, it is non-to adopt
The skew of the non-rigid transformation of sample voxel experience corresponds to same camera according to the non-sampled voxel
And the skew with the non-sampled voxel apart from the non-rigid transformation of near sampling voxel experience passes through three-dimensional
Interpolation is obtained.Note:The side-play amount obtained after three-dimensional interpolation offset may not by particular voxel correspondence
In voxel location, and correspond to the sub- voxel location between voxel location.In this case, may be used
Voxel location is moved to from sub- voxel location with by three-dimensional coordinate is rounded up, so as to will be specific
Voxel is corresponding to the voxel at voxel location.
Here why it is designed as final local of the global TSDF values of a voxel based on the voxel
TSDF is worth to, and the final local T SDF value of a voxel is equal to voxel Jing non-rigid transformations correspondence
Voxel initial local TSDF values, rather than be designed as the global TSDF values of a voxel based on should
The initial local TSDF of voxel is worth to, if being because designing according to the latter, optimization problem
Solution can be a voxel global TSDF values be equal to the voxel initial local TSDF values arithmetic/
Weighted mean.By introducing non-rigid transformation, increased from initial local TSDF values to most end
The conversion of portion's TSDF values, so that the solution of optimization problem is more accurate, causes three-dimensional reconstruction pair
The result of elephant is more accurate.
The design of cost function is explained further with reference to formula.
If overall situation TSDF values are V, depth map number is n, then the corresponding initial local of each depth map
TSDF values are V1、V2、……、Vn, the corresponding final local T SDF value of each depth map is V1’、
V2’、……、Vn', serial number i of depth map, i=1,2 ... ..., n.For the voxel p in V,
Which is in Vi' in corresponding voxel be still p, and in ViIn corresponding voxel be Ci(p), wherein CiFor optimization
The non-rigid transformation for solving is needed in problem.
The set of definition sampling voxel, is denoted as Di={ di,j, wherein j=1,2 ..., mi, miFor ViMiddle sampling
The sum of voxel.
The non-rigid transformation of sampling voxel experience is defined as:Ci(di,j)=di,j+si,j, wherein, si,jFor
Sampling voxel maps result (the corresponding voxel location of sampling voxel Jing non-rigid transformations) and sampling voxel
Change in location vector between home position, that is, offset.And the mapping result of non-sampled voxel is according to adopting
The skew of sample voxel carries out trilinear interpolation and obtains.With the non-rigid transformation according to all sampling voxels
Calculations of offset sampling voxel non-rigid transformation skew as a example by,Wherein
wi,jP () is linear interpolation coefficient, can be according to the position relationship between non-sampled voxel p and sampling voxel
(such as distance) is tried to achieve.
As it was noted above, voxel Jing block, normalization and there is initial local TSDF values, that is, require
ViP the value of () is located at [- 1,1] interval.Also need to herein seek Vi(Ci(p))∈[-1,1].Due to ViWith Vi' between
Violent conversion will not be carried out, that is to say, that voxel p and voxel CiP the distance between () will not be far,
So definition set Pi:Pi=p | Vi(p) ∈ [- α, α] }, enabling ensure Vi(Ci(p))∈[-1,1]。
That is, during calculation cost function, for belonging to set PiVoxel p calculate the first generation
The summation of valency item and the second cost item, each first cost item are global TSDF values of the voxel p in V
V corresponding with voxel p Jing non-rigid transformationsiIn voxel CiThe difference of the initial local TSDF values of (p)
Quadratic sum, a square summation are because that voxel p may have multiple final local T SDF values, so as to right
There should be multiple voxel CiThe initial local TSDF values of (p), and voxel p there is also in itself it is multiple.
So the summation of the first cost item of cost function can be expressed as:
The second cost item of cost function is introduced in the present invention, to ensure ViWith Vi' between will not carry out play
Strong conversion, while also avoiding all associated voxels from being mapped to the extreme case of same point.
The summation of the second cost item of cost function can be expressed as:
The factor of two aspects of summary, defining cost function to be optimized is:
Wherein λ is the balance factor of before and after two.
Optimization problem is solved, that is, solves the final global TSDF for minimizing above-mentioned cost function
The optimal value of value and non-rigid transformation parameter.
An overall situation TSDF value of voxel is to emphasize that a voxel is finally only global with one herein
TSDF values, and the scope of such voxel is set Pi。
Optimization problem can be solved by Gauss-Newton (Gauss-Newton) method of iteration or
Calculated come iterative respectively by the parameter of global TSDF values and rigid transformation for voxel.
But no matter which kind of solves mode, is directed to iteration, it is necessary to set the initial value of iteration.
In optimization problem, the initial value of the global TSDF values of voxel is equal to all first of the voxel
The weighted mean of beginning local T SDF value.On the one hand weighting herein can be arithmetic average, another
Aspect can also calculate weight according to the distance of camera and voxel.
The initial value of the parameter of the non-rigid transformation of voxel causes voxel Jing non-rigid transformations corresponding
Voxel is the voxel itself.One voxel may have multiple initial local TSDF values, if the body
Plain Jing non-rigid transformations correspond to the voxel itself, then the initialized final local of this voxel
TSDF values also with multiple, each correspond to a camera.
In the case of being described above rigid transformation and non-rigid transformation, design and solve optimization problem
Method.By said method, step S3 can be realized in the way of solving optimization problem.
In step s 4, using the global TSDF values of resulting each group as initial local TSDF
Value, solves optimization problem, to obtain final overall situation TSDF values.
That is, using the global TSDF values of resulting each group as initial local TSDF values,
The conversion from initial local TSDF values to final global TSDF values is carried out again.Only, in step
In rapid S4, can only be by the way of optimization problem be solved.Solve the concrete grammar of optimization problem
It is consistent with the explanation carried out above for two kinds of situations of rigid transformation and non-rigid transformation, repeat no more.
A kind of preferred implementation of step S4 is introduced herein.
Due to being grouped before, so this artificial segmentation may result in final three-dimensional reconstruction
Subject surface there is pseudo-side, pseudo-side is due to caused by packet.Therefore, in a preferred embodiment,
Introduce weight to overcome the pseudo-side problem produced due to packet.
Specifically, in optimization problem, remain:The global TSDF values of one voxel are based on should
Final local T SDF of voxel is worth to, and the final local T SDF value of a voxel is equal to the voxel
The initial local TSDF values of transformed corresponding voxel, variable are the global TSDF values of voxel and become
The parameter changed.From unlike the optimization problem for illustrating before, in a preferred embodiment, cost
Function is related to following factors:The global TSDF values corresponding body transformed with the voxel of particular voxel
The difference of the initial local TSDF values of element square weighted sum, the weight of weighted sum is equal to particular voxel
The weight of the correspondence group of transformed corresponding voxel.
It should be noted that the alphabetic flag for hereinafter occurring, unless specifically stated otherwise, firm with what is illustrated before
Property conversion it is identical with the implication in the optimization problem of non-rigid transformation situation, if any specializing it
Place, by being defined for here indicating that.
The cost function in the case of rigid transformation above can be expressed as:
In a preferred embodiment, the cost function in the case of rigid transformation can be expressed as:
The cost function in the case of non-rigid transformation above can be expressed as:
In a preferred embodiment, the cost function in the case of non-rigid transformation can be expressed as:
E=Ec+λEr,It is rigid herein
Conversion and non-rigid transformation unification writing:Ti(p).For rigid transformation, Er=0.
If overall situation TSDF values are V, initial local TSDF values are divided into n according to corresponding depth map
Group, i.e. V1,V2,…,Vn, the weight of the corresponding voxel of each initial local TSDF values is W1,W2,…,Wn, become
After changing, final local T SDF value is V1′,V2′,…,Vn′.Group serial number i, i=1,2 ... ..., n.For in V
Voxel p, which is in Vi' in corresponding voxel be still p, and in ViIn corresponding voxel be Ti(p) (rigidity
Conversion or non-rigid transformation), wherein TiFor the conversion for needing to solve in optimization problem.
It can be seen that, main modification is that and introduce weight Wi(Ti(p)).Weight changes with iterative process.
The weight of particular voxel p is equal to the transformed corresponding voxel T of particular voxeliThe weight of the correspondence group of (p),
And the weight of the correspondence group of corresponding voxel is above-mentioned W1,W2,…,Wn, the weight of a voxel
(W1,W2,…,Wn) depending on group, same voxel is corresponding to different groups with different weights, correspondence
There is identical weight in same group of different voxels.The weight of particular voxel exists equal to the particular voxel
The number of the initial local TSDF values of transformed corresponding voxel in the group.Weight is being calculated so
In the case of, it is believed that weight is equal to the correspondence of the particular voxel transformed corresponding voxel in this set
In the weight sum of depth map, the particular voxel in this set transformed corresponding voxel corresponding to depth
In the particular voxel, transformed corresponding voxel has just the weight of degree figure corresponding to depth map in this set
1 is equal to during beginning local T SDF value, otherwise equal to 0.Alternatively, the weight of particular voxel is equal to and is somebody's turn to do
The weight sum corresponding to depth map of particular voxel transformed corresponding voxel in this set, this is specific
The weight corresponding to depth map of transformed corresponding voxel is existed voxel with the particular voxel in this set
In the group, transformed corresponding voxel is related relative to the depth value of the depth map correspondence camera.For example,
The particular voxel in this set transformed corresponding voxel relative to depth map correspondence camera depth value
Bigger, the weight corresponding to the depth map of the particular voxel transformed corresponding voxel in this set is got over
It is little.For another example, the power corresponding to depth map of the particular voxel transformed corresponding voxel in this set
Transformed corresponding voxel corresponds to phase relative to the depth map in this set to be inversely proportional to the particular voxel again
The depth value of machine.
In a preferred embodiment, calculating can be simplified in the following way:Take turns at (k+1)
In iterative process, useReplaceCost function to be optimized is reduced to:
Cost function now to be optimized is the form of non-linear least square, can pass through Gauss-Newton
(Gauss-Newton) method is solved.
In some cases, possible initial local TSDF values are excessive, so that through once packet behaviour
It is not enough to be effectively reduced data volume, now can be grouped again in step s 4.
Using the global TSDF values of resulting each group as initial local TSDF values, by corresponding depth
Figure is grouped again, and least one set initial local TSDF values are grouped corresponding first corresponding to more than one
Depth map;For every group of initial local TSDF value, the global TSDF values of the group are obtained;By gained
The global TSDF values of each group for arriving solve optimization problem as initial local TSDF values, with
To final global TSDF values.
Obtain in step s 4 and belong to set PiAll voxel p final global TSDF values after,
This reconstructed object can be just based on.
In step s 5, based on resulting final global TSDF values, object described in three-dimensional reconstruction.
For example, using marching cube (marching cube) algorithm, based on it is resulting most
Whole overall situation TSDF values, the surface of object described in three-dimensional reconstruction.Marching cubes algorithm be this area
The algorithm known, will not be described here.
The equipment of three-dimensional reconstruction object according to an embodiment of the invention is described next, with reference to Fig. 4.
Fig. 4 shows the structure square frame of the equipment of three-dimensional reconstruction object according to an embodiment of the invention
Figure.As shown in figure 4, three-dimensional reconstruction object-based device of the invention 400 includes:Obtain device
41, it is configured to:The initial local for obtaining voxel in three dimensions blocks signed distance function TSDF
Value, each initial local TSDF values are corresponding to a depth map in multiple depth maps;Apparatus for grouping
42, it is configured to:By corresponding depth map, initial local TSDF values are grouped, at the beginning of least one set
Beginning, local T SDF value was corresponding to more than one depth map;Group overall situation TSDF values acquisition device 43, quilt
It is configured to:For every group of initial local TSDF value, the global TSDF values of the group are obtained;Solving device
44, it is configured to:Using the global TSDF values of resulting each group as initial local TSDF values, ask
Solution optimization problem, to obtain final overall situation TSDF values;And reconstructing device 45, it is configured to:
Based on resulting final global TSDF values, object described in three-dimensional reconstruction;Wherein, in the optimum
In change problem, the global TSDF values of a voxel are worth to based on final local T SDF of the voxel,
The final local T SDF value of one voxel is equal to the initial local of the transformed corresponding voxel of the voxel
TSDF values, variable are the parameter of the global TSDF values of voxel and conversion, cost function and following factors
It is related:The initial local of the global TSDF values corresponding voxel transformed with the voxel of particular voxel
The difference of TSDF values square weighted sum, it is transformed corresponding that the weight of weighted sum is equal to particular voxel
The weight of the correspondence group of voxel.
In one embodiment, the conversion includes rigid transformation.
In one embodiment, the conversion includes non-rigid transformation, cost function also with following factors
It is related:The summation of the skew tolerance of the non-rigid transformation of particular voxel experience.
In one embodiment, described group of overall situation TSDF values acquisition device 43 is further configured to:
For every group of initial local TSDF value, by solving optimization problem, the global TSDF of the group is obtained
Value.
In one embodiment, described group of overall situation TSDF values acquisition device 43 is further configured to:
For every group of initial local TSDF value, the calculation of the initial local TSDF values corresponding to same voxel is calculated
Art meansigma methodss, as the global TSDF values of the group corresponding to the voxel.
In one embodiment, the solving device 44 is further configured to:Will be resulting each
The global TSDF values of group are grouped by corresponding depth map, at least again as initial local TSDF values
One group of initial local TSDF value is grouped corresponding depth map first corresponding to more than one;For per group
Initial local TSDF values, obtain the global TSDF values of the group;By the global TSDF of resulting each group
Value solves optimization problem as initial local TSDF values, to obtain final overall situation TSDF values.
In one embodiment, in the optimization problem, each particular voxel initially has specific
In the weight of group, weight is equal to the initial local of the particular voxel transformed corresponding voxel in this set
The number or weight of TSDF values is equal to the right of the particular voxel transformed corresponding voxel in this set
Should be in the weight sum of depth map, transformed corresponding voxel is corresponded to the particular voxel in this set
The weight of depth map and the particular voxel in this set transformed corresponding voxel relative to the depth map
The depth value of correspondence camera is related.
In one embodiment, the acquisition device 41 also includes:Normalization unit, is configured to:
By the initial local TSDF value normalization of calculated voxel.
In one embodiment, the absolute value of the initial local TSDF values of voxel is less than or equal to specific
Threshold value, the specific threshold belong to (0,1).
In one embodiment, particular voxel is all voxels for meeting specific threshold condition.
In one embodiment, particular voxel is met after all voxels sampling of specific threshold condition
The sampling voxel for arriving.
Due to included each device in three-dimensional reconstruction object-based device of the invention 400, list
In unit process respectively with each step included in three-dimensional reconstruction object method described above in
Process be similar to, therefore for simplicity, here omits the detailed description of these devices, unit.
Additionally, being still needed, it is noted that each component devices, unit can pass through in the said equipment here
The mode of software, firmware, hardware or its combination is configured.Specific means or side that configuration can be used
Formula is well known to those skilled in the art, and will not be described here.In the feelings realized by software or firmware
Under condition, from storage medium or network to the computer with specialized hardware structure (such as shown in Fig. 5
General purpose computer 500) install constitute the software program, the computer when various programs are provided with,
It is able to carry out various functions etc..
Fig. 5 shows the computer that can be used to implementing method and apparatus according to an embodiment of the invention
Schematic block diagram.
In Figure 5, CPU (CPU) 501 is stored according in read only memory (ROM) 502
Program or from storage part 508 be loaded into random access memory (RAM) 503 program performing it is each
Plant and process.In RAM 503, store when CPU 501 performs various process etc. always according to needs
Required data.CPU 501, ROM 502 and RAM 503 are connected to each other via bus 504.It is defeated
Enter/output interface 505 is also connected to bus 504.
Components described below is connected to input/output interface 505:Importation 506 (includes keyboard, mouse
Etc.), output par, c 507 (include display, such as cathode ray tube (CRT), liquid crystal display
(LCD) etc., and speaker etc.), storage part 508 (including hard disk etc.), communications portion 509 (wrap
Include NIC such as LAN card, modem etc.).Communications portion 509 via network such as
The Internet performs communication process.As needed, driver 510 can be connected to input/output interface
505.Detachable media 511 such as disk, CD, magneto-optic disk, semiconductor memory etc. can be with
Be installed in driver 510 as needed so that the computer program for reading out as needed by
It is installed in storage part 508.
In the case where above-mentioned series of processes is realized by software, it is situated between from network such as the Internet or storage
Matter such as detachable media 511 installs the program for constituting software.
It will be understood by those of skill in the art that this storage medium is not limited to shown in Fig. 5 wherein
Have program stored therein, and equipment separately distribute to provide a user with the detachable media 511 of program.
The example of detachable media 511 (includes CD comprising disk (including floppy disk (registered trade mark)), CD
Read only memory (CD-ROM) and digital universal disc (DVD)), magneto-optic disk (comprising mini-disk (MD) (note
Volume trade mark)) and semiconductor memory.Or, storage medium can be ROM 502, storage part 508
In the hard disk that includes etc., wherein computer program stored, and be distributed to together with comprising their equipment
User.
The present invention also proposes a kind of program product of the instruction code of the machine-readable that is stored with.The finger
When making code be read and performed by machine, above-mentioned method according to an embodiment of the invention is can perform.
Correspondingly, for carrying depositing for the program product of the instruction code of the above-mentioned machine-readable that is stored with
Storage media is also included within disclosure of the invention.The storage medium include but is not limited to floppy disk, CD,
Magneto-optic disk, storage card, memory stick etc..
In description above to the specific embodiment of the invention, for a kind of embodiment describe and/or
The feature for illustrating can be made in one or more other embodiments in same or similar mode
With, it is combined with the feature in other embodiment, or substitute the feature in other embodiment.
It should be emphasized that term "comprises/comprising" refer to when using herein feature, key element, step or
The presence of component, but it is not precluded from depositing for one or more further features, key element, step or component
Or it is additional.
Additionally, the method for the present invention be not limited to specifications described in time sequencing performing,
Can according to other time sequencings ground, concurrently or independently perform.Therefore, retouch in this specification
The execution sequence of the method stated is not construed as limiting to the technical scope of the present invention.
Although being draped over one's shoulders to the present invention by the description of the specific embodiment to the present invention above
Dew, however, it is to be understood that above-mentioned all embodiments and example are exemplary, and it is unrestricted
Property.Those skilled in the art can be designed in the spirit and scope of the appended claims to the present invention
Various modifications, improvement or equivalent.These modifications, improvement or equivalent should also be as being considered as
Including within the scope of the present invention.
Note
1. a kind of three-dimensional rebuilding method of object, including:
In obtaining three dimensions, the initial local of voxel blocks signed distance function TSDF values, at the beginning of each
Beginning local T SDF value is corresponding to a depth map in multiple depth maps;
By corresponding depth map, initial local TSDF values are grouped, least one set initial local TSDF
Value is corresponding to more than one depth map;
For every group of initial local TSDF value, the global TSDF values of the group are obtained;
Using the global TSDF values of resulting each group as initial local TSDF values, optimization is solved
Problem, to obtain final overall situation TSDF values;And
Based on resulting final global TSDF values, object described in three-dimensional reconstruction;
Wherein, in the optimization problem, the global TSDF values of a voxel are based on the voxel
Final local T SDF is worth to, and the final local T SDF value of a voxel is transformed equal to the voxel
The initial local TSDF values of corresponding voxel, variable are the ginsengs of the global TSDF values of voxel and conversion
Number, cost function are related to following factors:The global TSDF values of particular voxel are transformed with the voxel
The difference of the initial local TSDF values of corresponding voxel square weighted sum, the weight of weighted sum is equal to
The weight of the correspondence group of the transformed corresponding voxel of particular voxel.
2. note 1 as described in method, wherein, the conversion includes rigid transformation.
3. note 1 as described in method, wherein, the conversion includes non-rigid transformation, cost letter
Number is also related to following factors:The summation of the skew tolerance of the non-rigid transformation of particular voxel experience.
4. note 1 as described in method, wherein, for every group of initial local TSDF value, obtain
The global TSDF values of the group include:For every group of initial local TSDF value, by solving optimization
Problem, obtains the global TSDF values of the group.
5. note 1 as described in method, wherein, for every group of initial local TSDF value, obtain
The global TSDF values of the group include:For every group of initial local TSDF value, calculate corresponding to same
The arithmetic mean of instantaneous value of the initial local TSDF values of voxel, as the overall situation of the group corresponding to the voxel
TSDF values.
6. note 1 as described in method, wherein, the global TSDF by resulting each group
Value solves optimization problem as initial local TSDF values, to obtain final overall situation TSDF values bag
Include:
Using the global TSDF values of resulting each group as initial local TSDF values, by corresponding depth
Degree figure is grouped again, and least one set initial local TSDF values are grouped correspondence first corresponding to more than one
Depth map;
For every group of initial local TSDF value, the global TSDF values of the group are obtained;
Using the global TSDF values of resulting each group as initial local TSDF values, optimization is solved
Problem, to obtain final overall situation TSDF values.
7. note 1 as described in method, wherein, in the optimization problem, each particular volume
Initially with the weight specific to group, it is transformed corresponding in this set that weight is equal to the particular voxel to element
The number or weight of the initial local TSDF values of voxel is equal to the particular voxel, and Jing becomes in this set
The weight sum corresponding to depth map of corresponding voxel is changed, the particular voxel is transformed right in this set
The weight corresponding to depth map of the voxel answered and the particular voxel transformed corresponding body in this set
Element is related relative to the depth value of the depth map correspondence camera.
8. note 1 as described in method, wherein, obtain three dimensions in voxel initial local cut
Disconnected signed distance function TSDF values include:
By the initial local TSDF value normalization of calculated voxel.
9. note 8 as described in method, wherein, the absolute value of the initial local TSDF values of voxel
Less than or equal to specific threshold, the specific threshold belong to (0,1).
10. note 9 as described in method, wherein, particular voxel is to meet the institute of specific threshold condition
The sampling voxel obtained after having voxel sampling.
A kind of three-dimensional reconstruction equipment of 11. objects, including:
Device is obtained, is configured to:The initial local for obtaining voxel in three dimensions blocks directed distance
Function TSDF values, each initial local TSDF values are corresponding to a depth map in multiple depth maps;
Apparatus for grouping, is configured to:By corresponding depth map, initial local TSDF values are grouped,
Least one set initial local TSDF values are corresponding to more than one depth map;
Group overall situation TSDF value acquisition device, is configured to:For every group of initial local TSDF value,
Obtain the global TSDF values of the group;
Solving device, is configured to:Using the global TSDF values of resulting each group as initial local
TSDF values, solve optimization problem, to obtain final overall situation TSDF values;And
Reconstructing device, is configured to:Based on resulting final global TSDF values, three-dimensional reconstruction institute
State object;
Wherein, in the optimization problem, the global TSDF values of a voxel are based on the voxel
Final local T SDF is worth to, and the final local T SDF value of a voxel is transformed equal to the voxel
The initial local TSDF values of corresponding voxel, variable are the ginsengs of the global TSDF values of voxel and conversion
Number, cost function are related to following factors:The global TSDF values of particular voxel are transformed with the voxel
The difference of the initial local TSDF values of corresponding voxel square weighted sum, the weight of weighted sum is equal to
The weight of the correspondence group of the transformed corresponding voxel of particular voxel.
12. equipment as described in note 11, wherein, the conversion includes rigid transformation.
13. equipment as described in note 11, wherein, the conversion includes non-rigid transformation, cost
Function is also related to following factors:The summation of the skew tolerance of the non-rigid transformation of particular voxel experience.
14. equipment as described in note 11, wherein, described group of overall situation TSDF value acquisition device quilt
It is further configured to:For every group of initial local TSDF value, by solving optimization problem, obtain
The global TSDF values of the group.
15. equipment as described in note 11, wherein, described group of overall situation TSDF value acquisition device quilt
It is further configured to:For every group of initial local TSDF value, calculate corresponding to the initial of same voxel
The arithmetic mean of instantaneous value of local T SDF value, as the global TSDF values of the group corresponding to the voxel.
16. equipment as described in note 11, wherein, the solving device is further configured to:
Using the global TSDF values of resulting each group as initial local TSDF values, by corresponding depth
Degree figure is grouped again, and least one set initial local TSDF values are grouped correspondence first corresponding to more than one
Depth map;
For every group of initial local TSDF value, the global TSDF values of the group are obtained;
Using the global TSDF values of resulting each group as initial local TSDF values, optimization is solved
Problem, to obtain final overall situation TSDF values.
17. equipment as described in note 11, wherein, in the optimization problem, each is specific
Initially with the weight specific to group, weight is equal to the particular voxel transformed correspondence in this set to voxel
Voxel initial local TSDF values number or weight be equal to particular voxel Jing in this set
The weight sum corresponding to depth map of corresponding voxel is converted, the particular voxel is transformed in this set
The weight corresponding to depth map of corresponding voxel is transformed corresponding in this set with the particular voxel
Voxel is related relative to the depth value of the depth map correspondence camera.
18. equipment as described in note 11, wherein, the acquisition device also includes:Normalization list
Unit, is configured to:By the initial local TSDF value normalization of calculated voxel.
19. note 18 as described in equipment, wherein, the initial local TSDF values of voxel it is absolute
Value less than or equal to specific threshold, the specific threshold belong to (0,1).
20. equipment as described in note 19, wherein, particular voxel meets specific threshold condition
The sampling voxel obtained after all voxel samplings.
Claims (10)
1. a kind of three-dimensional rebuilding method of object, including:
In obtaining three dimensions, the initial local of voxel blocks signed distance function TSDF values, at the beginning of each
Beginning local T SDF value is corresponding to a depth map in multiple depth maps;
By corresponding depth map, initial local TSDF values are grouped, least one set initial local TSDF
Value is corresponding to more than one depth map;
For every group of initial local TSDF value, the global TSDF values of the group are obtained;
Using the global TSDF values of resulting each group as initial local TSDF values, optimization is solved
Problem, to obtain final overall situation TSDF values;And
Based on resulting final global TSDF values, object described in three-dimensional reconstruction;
Wherein, in the optimization problem, the global TSDF values of a voxel are based on the voxel
Final local T SDF is worth to, and the final local T SDF value of a voxel is transformed equal to the voxel
The initial local TSDF values of corresponding voxel, variable are the ginsengs of the global TSDF values of voxel and conversion
Number, cost function are related to following factors:The global TSDF values of particular voxel are transformed with the voxel
The difference of the initial local TSDF values of corresponding voxel square weighted sum, the weight of weighted sum is equal to
The weight of the correspondence group of the transformed corresponding voxel of particular voxel.
2. the method for claim 1, wherein the conversion includes rigid transformation.
3. the method for claim 1, wherein the conversion includes non-rigid transformation, generation
Valency function is also related to following factors:It is total that the skew of the non-rigid transformation of particular voxel experience is measured
With.
4. the method for claim 1, wherein every group of initial local TSDF value is directed to,
The global TSDF values for obtaining the group include:For every group of initial local TSDF value, by solving most
Optimization problem, obtains the global TSDF values of the group.
5. the method for claim 1, wherein every group of initial local TSDF value is directed to,
The global TSDF values for obtaining the group include:For every group of initial local TSDF value, calculate corresponding to
The arithmetic mean of instantaneous value of the initial local TSDF values of same voxel, as the group corresponding to the voxel
Global TSDF values.
6. the method for claim 1, wherein overall situation by resulting each group
TSDF values solve optimization problem as initial local TSDF values, to obtain final overall situation TSDF
Value includes:
Using the global TSDF values of resulting each group as initial local TSDF values, by corresponding depth
Degree figure is grouped again, and least one set initial local TSDF values are grouped correspondence first corresponding to more than one
Depth map;
For every group of initial local TSDF value, the global TSDF values of the group are obtained;
Using the global TSDF values of resulting each group as initial local TSDF values, optimization is solved
Problem, to obtain final overall situation TSDF values.
7., the method for claim 1, wherein in the optimization problem, each is special
Voxel is determined initially with the weight specific to group, it is transformed right in this set that weight is equal to the particular voxel
The number or weight of the initial local TSDF values of the voxel answered is equal to the particular voxel in this set
The weight sum corresponding to depth map of transformed corresponding voxel, Jing becomes the particular voxel in this set
The weight corresponding to depth map for changing corresponding voxel is transformed corresponding in this set with the particular voxel
Voxel it is related relative to the depth value of depth map correspondence camera.
8. the method for claim 1, wherein the initial office of voxel in three dimensions is obtained
Signed distance function TSDF values are blocked in portion to be included:
By the initial local TSDF value normalization of calculated voxel.
9. method as claimed in claim 8, wherein, the initial local TSDF values of voxel it is exhausted
To value less than or equal to specific threshold, the specific threshold belong to (0,1).
10. the three-dimensional reconstruction equipment of a kind of object, including:
Device is obtained, is configured to:The initial local for obtaining voxel in three dimensions blocks directed distance
Function TSDF values, each initial local TSDF values are corresponding to a depth map in multiple depth maps;
Apparatus for grouping, is configured to:By corresponding depth map, initial local TSDF values are grouped,
Least one set initial local TSDF values are corresponding to more than one depth map;
Group overall situation TSDF value acquisition device, is configured to:For every group of initial local TSDF value,
Obtain the global TSDF values of the group;
Solving device, is configured to:Using the global TSDF values of resulting each group as initial local
TSDF values, solve optimization problem, to obtain final overall situation TSDF values;And
Reconstructing device, is configured to:Based on resulting final global TSDF values, three-dimensional reconstruction institute
State object;
Wherein, in the optimization problem, the global TSDF values of a voxel are based on the voxel
Final local T SDF is worth to, and the final local T SDF value of a voxel is transformed equal to the voxel
The initial local TSDF values of corresponding voxel, variable are the ginsengs of the global TSDF values of voxel and conversion
Number, cost function are related to following factors:The global TSDF values of particular voxel are transformed with the voxel
The difference of the initial local TSDF values of corresponding voxel square weighted sum, the weight of weighted sum is equal to
The weight of the correspondence group of the transformed corresponding voxel of particular voxel.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201510590009.7A CN106558076B (en) | 2015-09-16 | 2015-09-16 | The method and apparatus of three-dimensional reconstruction object |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201510590009.7A CN106558076B (en) | 2015-09-16 | 2015-09-16 | The method and apparatus of three-dimensional reconstruction object |
Publications (2)
Publication Number | Publication Date |
---|---|
CN106558076A true CN106558076A (en) | 2017-04-05 |
CN106558076B CN106558076B (en) | 2019-06-18 |
Family
ID=58413603
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN201510590009.7A Active CN106558076B (en) | 2015-09-16 | 2015-09-16 | The method and apparatus of three-dimensional reconstruction object |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN106558076B (en) |
Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20140146057A1 (en) * | 2012-11-26 | 2014-05-29 | Electronics And Telecommunications Research Institute | Apparatus for 3d reconstruction based on multiple gpus and method thereof |
CN104504671A (en) * | 2014-12-12 | 2015-04-08 | 浙江大学 | Method for generating virtual-real fusion image for stereo display |
CN104616345A (en) * | 2014-12-12 | 2015-05-13 | 浙江大学 | Octree forest compression based three-dimensional voxel access method |
EP2886043A1 (en) * | 2013-12-23 | 2015-06-24 | a.tron3d GmbH | Method for continuing recordings to detect three-dimensional geometries of objects |
-
2015
- 2015-09-16 CN CN201510590009.7A patent/CN106558076B/en active Active
Patent Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20140146057A1 (en) * | 2012-11-26 | 2014-05-29 | Electronics And Telecommunications Research Institute | Apparatus for 3d reconstruction based on multiple gpus and method thereof |
EP2886043A1 (en) * | 2013-12-23 | 2015-06-24 | a.tron3d GmbH | Method for continuing recordings to detect three-dimensional geometries of objects |
CN104504671A (en) * | 2014-12-12 | 2015-04-08 | 浙江大学 | Method for generating virtual-real fusion image for stereo display |
CN104616345A (en) * | 2014-12-12 | 2015-05-13 | 浙江大学 | Octree forest compression based three-dimensional voxel access method |
Non-Patent Citations (1)
Title |
---|
韦羽棉: "基于Kinect深度图像的三维重建研究", 《中国优秀硕士学位论文全文数据库 信息科技辑》 * |
Also Published As
Publication number | Publication date |
---|---|
CN106558076B (en) | 2019-06-18 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
Yu et al. | Pu-net: Point cloud upsampling network | |
Kedward et al. | Efficient and exact mesh deformation using multiscale RBF interpolation | |
CN109584357B (en) | Three-dimensional modeling method, device and system based on multiple contour lines and storage medium | |
US7724943B2 (en) | Rapid and robust 3D/3D registration technique | |
Tagliasacchi et al. | Vase: Volume‐aware surface evolution for surface reconstruction from incomplete point clouds | |
CN110517348B (en) | Target object three-dimensional point cloud reconstruction method based on image foreground segmentation | |
CN109685841B (en) | Registration method and system of three-dimensional model and point cloud | |
CN110706332B (en) | Scene reconstruction method based on noise point cloud | |
CN111028356A (en) | Optimization method based on non-convex non-smooth second-order regular term and sparse fidelity term | |
CN110910433A (en) | Point cloud matching method based on deep learning | |
CN106548507A (en) | The method and apparatus of three-dimensional reconstruction object | |
CN115797561A (en) | Three-dimensional reconstruction method, device and readable storage medium | |
CN113643336B (en) | Three-dimensional image rigid matching method based on spherical polar coordinate system depth neural network | |
CN107680073A (en) | The method and apparatus of geometrical reconstruction object | |
Yan et al. | Plenvdb: Memory efficient vdb-based radiance fields for fast training and rendering | |
Roberts et al. | Deformation transfer survey | |
CN106548466A (en) | The method and apparatus of three-dimensional reconstruction object | |
Badshah et al. | On two multigrid algorithms for modeling variational multiphase image segmentation | |
WO2024041027A1 (en) | Panoramic-image processing method, and computer device and storage medium | |
CN117350929A (en) | Electric power image enhancement optimization method based on improved meta heuristic algorithm | |
Zach et al. | Practical methods for convex multi-view reconstruction | |
CN116758219A (en) | Region-aware multi-view stereo matching three-dimensional reconstruction method based on neural network | |
CN106558076A (en) | The method and apparatus of three-dimensional reconstruction object | |
CN111046958A (en) | Image classification and recognition method based on data-dependent kernel learning and dictionary learning | |
CN116468632A (en) | Grid denoising method and device based on self-adaptive feature preservation |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
GR01 | Patent grant | ||
GR01 | Patent grant |