CN106558076B

CN106558076B - The method and apparatus of three-dimensional reconstruction object

Info

Publication number: CN106558076B
Application number: CN201510590009.7A
Authority: CN
Inventors: 李斐; 刘汝杰
Original assignee: Fujitsu Ltd
Current assignee: Fujitsu Ltd
Priority date: 2015-09-16
Filing date: 2015-09-16
Publication date: 2019-06-18
Anticipated expiration: 2035-09-16
Also published as: CN106558076A

Abstract

The invention discloses a kind of methods and apparatus of three-dimensional reconstruction object.This method comprises: obtaining the initial local TSDF value of voxel in three-dimensional space, each initial local TSDF value corresponds to a depth map in multiple depth maps；By corresponding depth map, initial local TSDF value is grouped, at least one set of initial local TSDF value corresponds to more than one depth map；For every group of initial local TSDF value, the global TSDF value of the group is obtained；Using the global TSDF value of obtained each group as initial local TSDF value, optimization problem is solved, to obtain final overall situation TSDF value；And it is based on obtained final overall situation TSDF value, object described in three-dimensional reconstruction；Wherein, in the optimization problem, variable is the parameter of global the TSDF value and transformation of voxel, cost function and following factors are related: the global TSDF value of particular voxel and the difference of the initial local TSDF value of the transformed corresponding voxel of the voxel square weighted sum, weight is equal to the weight of the corresponding group of the transformed corresponding voxel of particular voxel.

Description

The method and apparatus of three-dimensional reconstruction object

Technical field

This invention relates generally to three-dimensional imaging fields.Specifically, can accurately be carried out pair the present invention relates to one kind The method and apparatus of the three-dimensional reconstruction of elephant.

Background technique

In recent years, with the development of 3 dimension imaging technology, many relevant applications have been emerged in large numbers, as augmented reality, number are rich Object shop, 3 D-printing etc..The importance of 3 dimension imaging technology is three-dimensional reconstruction.The three-dimensional rebuilding method of mainstream is based on One group of depth image rebuilds three dimensional object.

Traditional three-dimensional rebuilding method focuses mainly on how more accurately estimating the parameter of camera, such as draws Distortion of the deformation information to estimate camera is entered.But traditional three-dimensional rebuilding method, calculating overall situation TSDF, (truncation has When being characterized to distance function, Truncated Signed Distance Function), only simply to all parts TSDF characterization carries out arithmetic average, without sufficiently excavating the relationship between local T SDF characterization and overall situation TSDF characterization.Due to only The effect that three-dimensional reconstruction is promoted by the estimation improved to camera parameter, so traditional three-dimensional rebuilding method is effect improved Limited, the accuracy of three-dimensional reconstruction object needs to be further increased.

Therefore, it is desirable to which a kind of method and apparatus of three-dimensional reconstruction object, can accurately carry out the three-dimensional reconstruction of object.

Summary of the invention

It has been given below about brief overview of the invention, in order to provide about the basic of certain aspects of the invention Understand.It should be appreciated that this summary is not an exhaustive overview of the invention.It is not intended to determine pass of the invention Key or pith, nor is it intended to limit the scope of the present invention.Its purpose only provides certain concepts in simplified form, Taking this as a prelude to a more detailed description discussed later.

The purpose of the present invention is in view of the above problems in the prior art, propose one kind to lay particular emphasis on excavation local T SDF characterization The method and apparatus of the three-dimensional reconstruction object of relationship between global TSDF characterization.

To achieve the goals above, according to an aspect of the invention, there is provided a kind of three-dimensional rebuilding method of object, is somebody's turn to do Method includes: to obtain the initial local truncation signed distance function TSDF value of voxel in three-dimensional space, each initial local TSDF Value corresponds to a depth map in multiple depth maps；By corresponding depth map, initial local TSDF value is grouped, it is at least one set of Initial local TSDF value corresponds to more than one depth map；For every group of initial local TSDF value, the global TSDF of the group is obtained Value；Using the global TSDF value of obtained each group as initial local TSDF value, optimization problem is solved, to obtain the final overall situation TSDF value；And it is based on obtained final overall situation TSDF value, object described in three-dimensional reconstruction；Wherein, in the optimization problem In, the global TSDF value of a voxel is obtained based on the final local T SDF value of the voxel, the final local T SDF of a voxel Value is equal to the initial local TSDF value of the transformed corresponding voxel of the voxel, and variable is the ginseng of global the TSDF value and transformation of voxel Number, cost function and following factors are related: the global TSDF value of particular voxel is initial with the transformed corresponding voxel of the voxel The difference of local T SDF value square weighted sum, the weight of weighted sum is equal to the correspondence group of the transformed corresponding voxel of particular voxel Weight.

According to another aspect of the present invention, a kind of three-dimensional reconstruction equipment of object is provided, which includes: to be filled It sets, is configured as: obtaining the initial local truncation signed distance function TSDF value of voxel in three-dimensional space, each initial local TSDF value corresponds to a depth map in multiple depth maps；Apparatus for grouping is configured as:, will be initial by corresponding depth map The grouping of local T SDF value, at least one set of initial local TSDF value correspond to more than one depth map；Group overall situation TSDF value obtains dress It sets, is configured as: for every group of initial local TSDF value, obtaining the global TSDF value of the group；Solving device is configured as: will The global TSDF value of obtained each group solves optimization problem as initial local TSDF value, to obtain final overall situation TSDF Value；And reconstructing device, it is configured as: being based on obtained final overall situation TSDF value, object described in three-dimensional reconstruction；Wherein, exist In the optimization problem, the global TSDF value of a voxel is obtained based on the final local T SDF value of the voxel, a voxel Final local T SDF value be equal to the transformed corresponding voxel of the voxel initial local TSDF value, variable is the overall situation of voxel The parameter of TSDF value and transformation, cost function are related to following factors: the global TSDF value of particular voxel and the voxel are transformed The difference of the initial local TSDF value of corresponding voxel square weighted sum, it is transformed right that the weight of weighted sum is equal to particular voxel The weight of the corresponding group for the voxel answered.

In addition, according to another aspect of the present invention, additionally providing a kind of storage medium.The storage medium includes that machine can The program code of reading, when executing said program code on information processing equipment, said program code makes at the information Equipment is managed to execute according to the above method of the present invention.

In addition, in accordance with a further aspect of the present invention, additionally providing a kind of program product.Described program product includes that machine can The instruction of execution, when executing described instruction on information processing equipment, described instruction executes the information processing equipment According to the above method of the present invention.

Detailed description of the invention

Referring to reference to the accompanying drawing to the explanation of the embodiment of the present invention, the invention will be more easily understood it is above and Other objects, features and advantages.Component in attached drawing is intended merely to show the principle of the present invention.In the accompanying drawings, identical or class As technical characteristic or component will be indicated using same or similar appended drawing reference.In attached drawing:

Fig. 1 shows the flow chart of the three-dimensional reconstruction object method of embodiment according to the present invention；

Fig. 2 shows the flow charts of the acquisition initial local TSDF method of embodiment according to the present invention；

Fig. 3 shows the flow chart of the calculating initial local TSDF method of embodiment according to the present invention；

Fig. 4 shows the structural block diagram of the equipment of the three-dimensional reconstruction object of embodiment according to the present invention；And

Fig. 5 shows the schematic frame for the computer that can be used for implementing the method and apparatus of embodiment according to the present invention Figure.

Specific embodiment

Exemplary embodiment of the invention is described in detail hereinafter in connection with attached drawing.It rises for clarity and conciseness See, does not describe all features of actual implementation mode in the description.It should be understood, however, that developing any this reality Much decisions specific to embodiment must be made during embodiment, to realize the objectives of developer, For example, meeting restrictive condition those of related to system and business, and these restrictive conditions may be with embodiment It is different and change.In addition, it will also be appreciated that although development is likely to be extremely complex and time-consuming, to benefit For those skilled in the art of present disclosure, this development is only routine task.

Here, and also it should be noted is that, in order to avoid having obscured the present invention because of unnecessary details, in the accompanying drawings Illustrate only with closely related apparatus structure and/or processing step according to the solution of the present invention, and be omitted and the present invention The little other details of relationship.In addition, it may also be noted that being described in an attached drawing of the invention or a kind of embodiment Elements and features can be combined with elements and features shown in one or more other attached drawings or embodiment.

The process of the three-dimensional rebuilding method of the object of embodiment according to the present invention is described below with reference to Fig. 1.

Fig. 1 shows the flow chart of the three-dimensional reconstruction object method of embodiment according to the present invention.As shown in Figure 1, according to The three-dimensional reconstruction object method of the embodiment of the present invention includes the following steps: the initial local truncation for obtaining voxel in three-dimensional space Signed distance function TSDF value, each initial local TSDF value correspond to a depth map (step S1) in multiple depth maps； By corresponding depth map, initial local TSDF value is grouped, at least one set of initial local TSDF value corresponds to more than one depth Scheme (step S2)；For every group of initial local TSDF value, the global TSDF value (step S3) of the group is obtained；By obtained each group Global TSDF value as initial local TSDF value, optimization problem is solved, to obtain final overall situation TSDF value (step S4)；With And it is based on obtained final overall situation TSDF value, object (step S5) described in three-dimensional reconstruction；Wherein, in the optimization problem In, the global TSDF value of a voxel is obtained based on the final local T SDF value of the voxel, the final local T SDF of a voxel Value is equal to the initial local TSDF value of the transformed corresponding voxel of the voxel, and variable is the ginseng of global the TSDF value and transformation of voxel Number, cost function and following factors are related: the global TSDF value of particular voxel is initial with the transformed corresponding voxel of the voxel The difference of local T SDF value square weighted sum, the weight of weighted sum is equal to the correspondence group of the transformed corresponding voxel of particular voxel Weight.

In step sl, signed distance function TSDF value, Mei Gechu is truncated in the initial local for obtaining voxel in three-dimensional space Beginning local T SDF value corresponds to a depth map in multiple depth maps.

Specifically, three-dimensional space is evenly divided into several voxels, it is believed that voxel is the basic of three-dimensional space Unit.Due to rebuilding three dimensional object based on one group of depth map, so the input of three-dimensional reconstruction object method is one group of depth map. This group of depth map is shot by multiple cameras, and each camera corresponds to a depth map.Again since input does not include phase The parameter of machine so needing to estimate based on depth map the parameter of corresponding camera, and obtains the initial office of associated voxels in turn Portion's TSDF value.

In addition, it should be noted here that " the multiple cameras " mentioned herein includes the physics in different position and orientation On multiple cameras the case where, also include the same camera multiple phases in logic are formed by adjusting position and/or direction The case where machine, include thes case where that above-mentioned two situations mix.

The initial local TSDF value of voxel be defined as based on camera parameter the calculated voxel of three-dimensional space relative to Result of the directed distance after being truncated in the depth and depth map of camera between the depth of the voxel corresponding position.It will be such TSDF value is known as initial local TSDF value.From defined above it is found that a voxel can calculate one relative to a camera Directed distance defined above, a voxel can calculate multiple directed distances defined above relative to multiple cameras, often A directed distance corresponds to a camera.Directed distance is due to break-in operation to be passed through, so a voxel likely corresponds to one A or multiple initial local TSDF values, the upper limit of the corresponding initial local TSDF value of a voxel are camera/depth map numbers. After being truncated, some voxels can not have initial local TSDF value, and such voxel is no longer considered.That is, counting in step sl The voxel that calculation obtains initial local TSDF value can just continue in subsequent step S2, S3.In addition, can from above-mentioned definition Know, each initial local TSDF value corresponds to multiple magazine cameras, namely corresponding to one in multiple depth maps Depth map.

Due to camera capture depth map when, acquisition be subject surface information, it is possible to understand be based on phase Machine parameter the calculated voxel of three-dimensional space relative to the depth of camera be between imaging point of the voxel with camera at a distance from Projection in the direction of the optical axis, and the depth of the voxel corresponding position is between the voxel and the imaging point of camera in depth map The projection of the distance between imaging point of point and camera in the subject surface that light is passed through in the direction of the optical axis.Due to the body Element may be farther or closer apart from camera relative to the point of this in subject surface, so the difference of the projection of distance in the direction of the optical axis For directed distance.Distance in the direction of the optical axis be projected as zero, show that the voxel is overlapped with the point in subject surface, namely this Voxel is the point in subject surface.

Theoretically, the null all voxels of initial local TSDF value constitute subject surface.But due to initial local The calculating of the value of TSDF depends on the estimation of camera parameter, and the estimated value of camera parameter and the actual value of camera parameter might not It is completely the same, so initial local TSDF value can not be used directly to rebuild the surface of three dimensional object.A kind of possible method is Arithmetic mean of instantaneous value is asked to all initial local TSDF values of each voxel with initial local TSDF value, arithmetic mean of instantaneous value is made For the global TSDF value of the voxel, overall situation TSDF value and voxel with the value are then based on to rebuild the table of three dimensional object Face.However, generating the problem that the relationship having cured between initial local TSDF value and overall situation TSDF value in this way.

Due to camera have it is multiple, so there may be differences the case where the parameter Estimation of multiple cameras, therefore, using unified Mode that the corresponding initial local TSDF value of different cameral is converted to global TSDF value is obviously inaccurate.So the present invention Power solves the problems, such as to obtain accurately overall situation TSDF value.

In addition, by foregoing description, it will be understood that step S1 can be realized by method shown in Fig. 2, to obtain initial office Portion's TSDF value.

As shown in Fig. 2, in the step s 21, estimating each multiple magazine camera corresponding with multiple depth maps Camera parameter, camera parameter include but is not limited to the position and orientation of camera.It is used in Kinect Fusion for example, can be used Depth map and threedimensional model between matching algorithm carry out camera parameter estimation.Of course, it is possible to using known in the art All suitable camera parameter estimation methods realize step S21.

In step S22, based on estimated camera parameter, one or more initial local TSDF values of voxel are calculated.

Step S22 can be realized by method shown in Fig. 3.

As shown in figure 3, each voxel and multiple depth maps for initial local TSDF value to be calculated are corresponding each A camera, in step S31, the estimated camera parameter based on camera calculates first depth of the voxel relative to the camera Value.In step s 32, determine that the voxel is corresponding, the second depth value in the associated depth map of the camera.In step S33, Will after the truncation of the difference of the first depth value and the second depth value as a result, initial local corresponding with the camera as the voxel TSDF value.

It should be noted that the range of voxel is the scheduled estimation energy in three-dimensional space when calculating the initial local TSDF value of voxel Enough it is fully contemplated by the region of object.Since the calculating of initial local TSDF is related to break-in operation, so having initial after being truncated The range of the voxel of local T SDF value will further reduce.By the threshold value of control truncation, can there will be initial local TSDF The voxel of value is limited near the surface of object.The threshold value of truncation for example can be positive and negative 5 centimetres namely initial local The value range of TSDF value can be [- 5,5].

For convenience of calculation, also initial local TSDF value is normalized.Normalized threshold value is the threshold value being truncated.

Truncation ensure that voxel near the surface of object, it will also be seen that, voxel is transformed (rigid from subsequent explanation Transformation or non-rigid transformation) voxel itself or another voxel can be corresponded to, in order to guarantee the voxel of transformation front and back all in right Near the surface of elephant, the initial local TSDF value further to voxel is limited.

Specifically, the absolute value for limiting the initial local TSDF value of voxel is less than or equal to specific threshold α, the specific threshold α belongs to (0,1).That is, the voxel of the initial local TSDF value of [- α, α] is transformed to correspond to its initial local TSDF The voxel being worth between [- 1,1].

In addition, if being calculated in the associated coordinate system of each camera (such as using the camera as the coordinate system of origin) respectively Initial local TSDF value, then the initial local TSDF value for the voxel being calculated correspond to multiple magazine camera associations Coordinate system.The initial local TSDF value for the voxel being calculated can be transformed into the same coordinate system.For example, will be calculated The initial local TSDF value of voxel be transformed into first associated coordinate system of camera.By by initial local TSDF primary system one To a coordinate system, it can be convenient and acquire the overall situation to the transformation of final local T SDF value and in turn from initial local TSDF value TSDF value can reduce calculation amount, and can obtain globally optimal solution rather than locally optimal solution.

It is of course also possible to calculate initial local TSDF value of the voxel relative to each camera directly in global coordinate system. Benefit using such mode is can to convert to avoid by the initial local TSDF value of voxel from the associated coordinate system of each camera The loss of significance generated when to the same coordinate system.

An important means of the invention is to be grouped initial local TSDF value, then independent for each grouping Processing, then the result integrated treatment to each group.In this way, it can cope with and be counted with many initial local TSDF values According to the very big situation of amount.

In step s 2, by corresponding depth map, initial local TSDF value is grouped, at least one set of initial local TSDF value Corresponding to more than one depth map.

As previously mentioned, each initial local TSDF value corresponds to a depth map in multiple depth maps.Therefore, Ke Yigen Initial local TSDF value is assigned in different groups according to which depth map is initial local TSDF value correspond to.Each group can be with Initial local TSDF value including corresponding to one or more depth maps, but the initial local TSDF value of at least one group is corresponding In more than one depth map.All initial local TSDF values corresponding to same depth map are only in a group.

In step s3, for every group of initial local TSDF value, the global TSDF value of the group is obtained.

Since step S3 to be accomplished that the transformation from initial local TSDF value to global TSDF value, so with the present invention To be solved be slave initial local TSDF value to the transformation of global TSDF value it is identical, the range only handled is defined in respectively A grouping.Therefore, step S3 both can use traditional method and realize, also can use and realizes according to the method for the present invention.

When using conventional methods realization step S3, for every group of initial local TSDF value, calculates and correspond to one The arithmetic mean of instantaneous value of the initial local TSDF value of element, the global TSDF value as the group for corresponding to the voxel.For example, at the beginning of certain group Beginning local T SDF value correspond to three depth maps, then, in the group corresponding to voxel A initial local TSDF value it is most there are three A1, A2, A3 calculate the value of (A1+A2+A3)/3, the global TSDF value corresponding to voxel A as the group.

Note that since initial local TSDF value being grouped according to corresponding depth map, so the same voxel may be more There is corresponding initial local TSDF value in a grouping, correspondingly, by step S3, every group of global TSDF value corresponds to multiple Voxel, each voxel have been likely to corresponding overall situation TSDF value in multiple groups.Hereinafter will how will in introduction step S4 The PRELIMINARY RESULTS that step S3 is obtained is further processed, finally to obtain final overall situation TSDF value unique to voxel.

When realizing step S3 using preferred method according to the present invention, for every group of initial local TSDF value, by asking Optimization problem is solved, the global TSDF value of the group is obtained.

When with comparing hereinafter it is noted that due to realize step S3 when, processing be packets inner initial local TSDF value, it is possible to weight is not used in optimization problem.And below for after grouping as a result, in order to eliminate point The distortion such as pseudo-side that group may cause, uses weight preferably in optimization problem.

It is introduced below how to be realized by designing and solving optimization problem from initial local TSDF value to global TSDF The conversion of value.Relevant thought is equally applicable to step S3 and step S4.

As previously described, because initial local TSDF value is calculated based on the camera parameter of estimation, and different depth maps pair Not necessarily the situation of entirely accurate and inaccuracy is not necessarily identical the parameter of the estimation for the different cameral answered, so uncomfortable Preferably all initial local TSDF values of voxel are handled in a uniform manner to obtain the global TSDF value of voxel.It is true On, it is believed that the initial local TSDF value transformation (inverse transformation of rigid transformation or non-rigid transformation) of the first voxel to the second body At element, become the final local T SDF value of the second voxel.First voxel and the second voxel can identical (null transformation), can also not With (non-zero transform).Specific threshold α hereinbefore defines the value range of the initial local TSDF of the second voxel, to guarantee The final local T SDF value of second voxel belongs to [- 1,1].

It should be noted that rigid transformation is carried out for same camera.That is, same camera is corresponding to belong to multiple first bodies Multiple initial local TSDF values of element correspond to multiple second by the inverse transformation of the same rigid transformation in these first voxels As multiple final local T SDF values corresponding to the camera of these the second voxels while voxel.So-called rigidity refers to correspondence The second voxel is reached from the first voxel through unified rotationally and/or translationally transformation in multiple initial local TSDF values of same camera (TSDF value itself is constant), there is no become before and after transformation for the relative positional relationship between voxel associated by these values Change.

It should be noted that non-rigid transformation is carried out for voxel.The main body of transformation is voxel, transformation the result is that establishing first The final local T SDF value of the corresponding relationship and the second voxel of voxel and the second voxel is equal to the initial of corresponding first voxel Local T SDF value.That is, an initial local TSDF value of first voxel passes through non-rigid change in first voxel Final local T SDF value while the inverse transformation changed corresponds to the second voxel as second voxel.It is non-rigid to refer to each One voxel reach the second voxel it is experienced transformation be it is independent of each other, it is unrelated with camera, corresponding to same camera it is multiple just Beginning inverse transformation of the local T SDF value through respective non-rigid transformation reaches the second voxel (TSDF value itself is constant) from the first voxel, Relative positional relationship between voxel associated by these values can change before and after transformation.Relative to rigid transformation, adopt It can more meticulously consider that the relevant transformation of multiple initial local TSDF values for same camera may with non-rigid transformation It is different from each other.Therefore, non-rigid transformation is to further improve the final TSDF of voxel relative to the benefit of rigid transformation The accuracy rate of value, and then improve the accuracy of global TSDF and the effect of three-dimensional reconstruction object.

By the amendment of rigid transformation/non-rigid transformation, the error of camera parameter estimation is counteracted, so that TSDF value and body The corresponding relationship of element is more accurate.Meanwhile global TSDF value is obtained from final local T SDF value, so that it may and then based on the overall situation TSDF value rebuilds three dimensional object.

Preferred embodiment in accordance with the present invention is realized by the way of designing and solving optimization problem from initial local Rigid transformation/non-rigid transformation that TSDF value is related to final local T SDF value and from final local T SDF value to global TSDF The calculating of value.

The design key of optimization problem is the design of variable and cost function.

For rigid transformation, in optimization problem, the global TSDF value of a voxel is based on the final of the voxel Local T SDF value obtains, and the final local T SDF value of a voxel is equal to initial office of the voxel through the corresponding voxel of rigid transformation Portion's TSDF value, variable are the parameters of global the TSDF value and rigid transformation of voxel, and cost function is related to following factors: voxel The quadratic sum of the difference of the initial local TSDF value of global TSDF value and the voxel through the corresponding voxel of rigid transformation.

Cost function for example can be designed as the global TSDF value of voxel and the voxel through the corresponding voxel of rigid transformation The summation of the quadratic sum of the difference of initial local TSDF value.Be noted above step S1 obtain initial local TSDF value when, due into It has gone truncation, normalization, and has been further limited in some embodiments using threshold alpha, only part voxel is caused to have just Beginning local T SDF value, subsequent step will be carried out for these voxels.Therefore, cost function can be designed as these voxels Each calculates cost item, and cost function is the summation of cost item, and each cost item is the global TSDF value and the voxel of voxel The quadratic sum of the difference of initial local TSDF value through the corresponding voxel of rigid transformation namely the global TSDF value of voxel and the voxel Final local T SDF value difference quadratic sum.

Here the global TSDF value for why being designed as a voxel is obtained based on the final local T SDF value of the voxel, and one The final local T SDF value of a voxel is equal to initial local TSDF value of the voxel through the corresponding voxel of rigid transformation, rather than sets The global TSDF value for being calculated as a voxel is obtained based on the initial local TSDF value of the voxel, is because if setting according to the latter Meter, then the solution of optimization problem can be a voxel global TSDF value be equal to the voxel initial local TSDF value arithmetic/ Weighted average.By introducing the rigid transformation that only relates to translate and rotate whole for a camera, increase from initial The conversion of local T SDF value to final local T SDF value leads to three-dimensional reconstruction so that the solution of optimization problem is more accurate The result of object is more accurate.

If overall situation TSDF value is V, depth map number is n, then the corresponding initial local TSDF value of each depth map is V₁、 V₂、……、V_n, the corresponding final local T SDF value of each depth map is V₁’、V₂’、……、V_n', the serial number i, i=of depth map 1,2,……,n.For the voxel p in V, in V_i' in correspondence voxel be still p, and in V_iIn correspondence voxel be T_i(p), Wherein T_iFor the rigid transformation for needing to solve in optimization problem.

As it was noted above, voxel has initial local TSDF value through truncation, normalization, that is, require V_i(p) value is located at [- 1,1] section.It also needs to seek V herein_i(T_i(p))∈[-1,1].Due to V_iWith V_i' between not will do it violent transformation, That is voxel p and voxel T_iThe distance between (p) will not be far, so definition set P_i: P_i=p | V_i(p)∈[-α, α] }, make it possible to guarantee V_i(T_i(p))∈[-1,1]。

That is, when calculating cost function, for belonging to set P_iVoxel p calculate cost item summation, Mei Gedai Valence item is global TSDF value of the voxel p in V with voxel p through the corresponding V of rigid transformation_iIn voxel T_i(p) initial local The quadratic sum of the difference of TSDF value, a square summation are because voxel p may have multiple final local T SDF values, to be corresponding with more A voxel T_i(p) initial local TSDF value.

So cost function can indicate are as follows:

Optimization problem is solved, that is, solves the final overall situation TSDF value and rigid transformation for minimizing above-mentioned cost function The optimal value of parameter.

An overall situation TSDF value of voxel is to emphasize that a voxel finally only has an overall situation TSDF value herein, and in this way The range of voxel be set P_i。

Optimization problem can be solved by Gauss-Newton (Gauss-Newton) method of iteration or by being directed to body The global TSDF value of element and the parameter of rigid transformation calculate separately to iteratively solve.

But no matter which kind of solves mode, iteration is directed to, it is necessary to set the initial value of iteration.

In optimization problem, the initial value of the global TSDF value of voxel is equal to whole initial local TSDF values of the voxel Weighted average.On the one hand weighting herein can be arithmetic average, on the other hand can also according to camera and voxel away from From calculating weight.

The initial value of the parameter of rigid transformation makes the voxel through the corresponding voxel of rigid transformation be the voxel itself.One Voxel may have multiple initial local TSDF values, if the voxel corresponds to the voxel itself through rigid transformation, this individual Element initialization final local T SDF value also have it is multiple, each correspond to a camera.

For example, rigid transformation T_i(p)=R_i*p+t_i, wherein R_iIt is spin matrix, unit matrix, t can be initialized as_iFor translation Vector can be initialized as null vector.

For non-rigid transformation, in optimization problem, the global TSDF value of a voxel based on the voxel most End portion TSDF value obtains, and the final local T SDF value of a voxel is equal to the voxel through the first of the corresponding voxel of non-rigid transformation Beginning local T SDF value, variable are the parameters of global the TSDF value and non-rigid transformation of voxel, and cost function is related to following factors: Square of the difference of the initial local TSDF value of the global TSDF value of particular voxel and the voxel through the corresponding voxel of non-rigid transformation With the summation of the offset measurement of the non-rigid transformation of particular voxel experience.

Previously mentioned voxel will meet specific threshold condition, i.e. the absolute value of the initial local TSDF value of restriction voxel is less than Or it is equal to specific threshold α, specific threshold α belongs to (0,1).If such voxel be not it is too many, can based on all this Sample voxel designs cost function, and optimization problem can solve.At this point, particular voxel is all voxels for meeting specific threshold condition. If such voxel is more, based on when all voxel designs cost function in this way, optimization problem may cannot be solved.? In this case, by sample to all voxels for meeting specific threshold condition and cost can be designed based on sampling voxel Function comes so that optimization problem can solve.The parameter of the non-rigid transformation of non-sampled voxel can be according to the non-rigid of sampling voxel The parameter of transformation determines.At this point, particular voxel is to meet obtained sampling voxel after all voxels sampling of specific threshold condition. In a preferred embodiment, it is sampled as uniform sampling.

Cost function for example can be designed as related to following factors: the global TSDF value of particular voxel is with the voxel through non- Quadratic sum (the first cost item), the particular voxel of the difference of the initial local TSDF value of the corresponding voxel of rigid transformation are undergone non-rigid Property transformation offset measurement summation (the second cost item).

Cost function can be designed as the sum of the first cost item and the second cost item, wherein the first cost item and the second cost Item numerical difference is not larger, sums again with the first cost item after the second cost item can be multiplied with balance factor.

First cost item is global TSDF value and the voxel of particular voxel through the initial of the corresponding voxel of non-rigid transformation The difference of the final local T SDF value of the global TSDF value and voxel of the quadratic sum namely particular voxel of the difference of local T SDF value Quadratic sum.Here the summation in quadratic sum includes the summation of two levels, and one side particular voxel is not unique, multiple particular volumes The correlation of element needs to sum, and on the other hand, each particular voxel may have multiple final local T SDF values, so each The correlation of particular voxel is also required to sum.

Second cost item is the summation of the offset measurement of the non-rigid transformation of particular voxel experience.In the second voxel through non-rigid Property convert corresponding first voxel in the case where, above-mentioned offset measurement refers to the mould of offset of first voxel relative to the second voxel Square.Since same second voxel may have multiple final local T SDF values, so second voxel may be via a variety of Non-rigid transformation corresponds to multiple first voxels, and multiple final local T SDF values of second voxel are equal to multiple first at this time The initial local TSDF value of voxel.Therefore, correspondingly, offset measurement, i.e., offset of multiple first voxels relative to the second voxel, There is also multiple, so carry out the summation operation of offset measurement.Also, there are multiple second voxels, so also needing pair The offset measurement of multiple second voxels is summed.As it can be seen that the summation of the offset measurement of the non-rigid transformation of particular voxel experience It is the summation of the offset measurement of all non-rigid transformations of all particular voxel experience.As described above, particular voxel is to meet All voxels or particular voxel of specific threshold condition be meet specific threshold condition all voxels sampling after obtain adopt Sample voxel.Cost item will be hereinafter further described for sampling voxel.

In addition, the purpose of cost function be to understand optimization problem, can be with when cost function is based only upon sampling voxel The offset of the non-rigid transformation of non-sampled voxel experience is obtained according to the offset of the non-rigid transformation of sampling voxel experience.In this way, Cost function be based only upon sampling voxel to optimization problem can solve and solve calculation amount reduce solving speed it is fast, remain to simultaneously Obtain the global TSDF value of non-sampled voxel.For example, the offset of the non-rigid transformation of non-sampled voxel experience can be according to sampling The offset of the non-rigid transformation of voxel experience passes through three-dimensional interpolation and obtains.In one embodiment, non-sampled voxel experience is non- The offset of rigid transformation passes through three according to the offset of the non-rigid transformation of the sampling voxel experience close with the non-sampled voxel distance Interpolation is tieed up to obtain.In another embodiment, the offset of the non-rigid transformation of non-sampled voxel experience according to the non-sampled body The offset that element corresponds to the non-rigid transformation of the sampling voxel experience of same camera passes through three-dimensional interpolation and obtains.In another implementation In example, the offset of the non-rigid transformation of non-sampled voxel experience according to and the non-sampled voxel correspond to same camera and with this The offset of the non-rigid transformation of the close sampling voxel experience of non-sampled voxel distance passes through three-dimensional interpolation and obtains.Note: in three-dimensional Inserting the offset obtained after offset may not correspond to voxel location for particular voxel, and correspond between voxel location Sub- voxel location.In such a case, it is possible to move to voxel position from sub- voxel location by rounding up three-dimensional coordinate It sets, so that particular voxel is corresponded to the voxel at voxel location.

Here the global TSDF value for why being designed as a voxel is obtained based on the final local T SDF value of the voxel, and one The final local T SDF value of a voxel is equal to initial local TSDF value of the voxel through the corresponding voxel of non-rigid transformation, rather than The global TSDF value for being designed as a voxel is obtained based on the initial local TSDF value of the voxel, is because if setting according to the latter Meter, then the solution of optimization problem can be a voxel global TSDF value be equal to the voxel initial local TSDF value arithmetic/ Weighted average.By introducing non-rigid transformation, the conversion from initial local TSDF value to final local T SDF value is increased, from And make the solution of optimization problem more accurate, cause the result of three-dimensional reconstruction object more accurate.

The design of cost function is explained further below with reference to formula.

If overall situation TSDF value is V, depth map number is n, then the corresponding initial local TSDF value of each depth map is V₁、 V₂、……、V_n, the corresponding final local T SDF value of each depth map is V₁’、V₂’、……、V_n', the serial number i, i=of depth map 1,2,……,n.For the voxel p in V, in V_i' in correspondence voxel be still p, and in V_iIn correspondence voxel be C_i(p), Wherein C_iFor the non-rigid transformation for needing to solve in optimization problem.

The set of definition sampling voxel, is denoted as D_i={ d_i,j, wherein j=1,2 ..., m_i, m_iFor V_iIt is middle to sample the total of voxel Number.

The non-rigid transformation of voxel experience will be sampled is defined as: C_i(d_i,j)=d_i,j+s_i,j, wherein s_i,jTo sample voxel Change in location between mapping result (sampling voxel is through the corresponding voxel location of non-rigid transformation) and sampling voxel home position Vector deviates.And the mapping result of non-sampled voxel carries out trilinear interpolation according to the offset of sampling voxel and obtains.With root For offset according to the non-rigid transformation of the calculations of offset sampling voxel of the non-rigid transformation of all sampling voxels,Wherein w_i,j(p) be linear interpolation coefficient, can according to non-sampled voxel p and sampling voxel it Between positional relationship (such as distance) acquire.

As it was noted above, voxel has initial local TSDF value through truncation, normalization, that is, require V_i(p) value is located at [- 1,1] section.It also needs to seek V herein_i(C_i(p))∈[-1,1].Due to V_iWith V_i' between not will do it violent transformation, That is voxel p and voxel C_iThe distance between (p) will not be far, so definition set P_i: P_i=p | V_i(p)∈[-α, α] }, make it possible to guarantee V_i(C_i(p))∈[-1,1]。

That is, when calculating cost function, for belonging to set P_iVoxel p calculate the first cost item and the second cost Summation, each first cost item is global TSDF value of the voxel p in V with voxel p through the corresponding V of non-rigid transformation_iIn Voxel C_i(p) quadratic sum of the difference of initial local TSDF value, a square summation are because voxel p may have multiple most ends Portion's TSDF value, to be corresponding with multiple voxel C_i(p) initial local TSDF value, and there is also multiple by voxel p itself.

So the summation of the first cost item of cost function can indicate are as follows:

The second cost item of cost function is introduced in the present invention, to guarantee V_iWith V_i' between not will do it violent transformation, All associated voxels are also avoided to be mapped to the extreme case of the same point simultaneously.

The summation of second cost item of cost function can indicate are as follows:

In summary the factor of two aspects, defines cost function to be optimized are as follows:

Wherein λ is the balance factor of front and back two.

Optimization problem is solved, that is, solves the final overall situation TSDF value and non-rigid change for minimizing above-mentioned cost function Change the optimal value of parameter.

The initial value of the parameter of the non-rigid transformation of voxel makes the voxel through the corresponding voxel of non-rigid transformation be the body Element itself.One voxel may have multiple initial local TSDF values, if the voxel corresponds to the voxel through non-rigid transformation Itself, then the final local T SDF value of the initialization of this voxel also has multiple, each corresponds to a camera.

It is described above in the case of rigid transformation and non-rigid transformation, design and the method for solving optimization problem.Pass through The above method can realize step S3 in a manner of solving optimization problem.

In step s 4, it using the global TSDF value of obtained each group as initial local TSDF value, solves to optimize and ask Topic, to obtain final overall situation TSDF value.

That is, being carried out again using the global TSDF value of obtained each group as initial local TSDF value from initial Conversion of the local T SDF value to final overall situation TSDF value.However, in step s 4, it can only be using the side for solving optimization problem Formula.Solve the specific method of optimization problem and the explanation one carried out above for two kinds of situations of rigid transformation and non-rigid transformation It causes, repeats no more.

A kind of preferred embodiment of introduction step S4 herein.

Due to being grouped before, so this artificial segmentation may result in the subject surface of final three-dimensional reconstruction There are pseudo-side, pseudo-side is as caused by grouping.Therefore, in the preferred embodiment, introduce weight overcoming due to grouping and The pseudo-side problem of generation.

Specifically, in optimization problem, it is still: final part of the global TSDF value based on the voxel of a voxel TSDF value obtains, and the final local T SDF value of a voxel is equal to the initial local TSDF of the transformed corresponding voxel of the voxel Value, variable is the parameter of global the TSDF value and transformation of voxel.Unlike the optimization problem illustrated before, preferred real It applies in mode, cost function and following factors are related: the global TSDF value of particular voxel and the transformed corresponding voxel of the voxel Initial local TSDF value difference square weighted sum, the weight of weighted sum is equal to the transformed corresponding voxel of particular voxel The weight of corresponding group.

It should be noted that the alphabetic flag hereinafter occurred, unless specifically stated otherwise, with the rigid transformation that illustrates before and it is non-just Property change situation optimization problem in meaning it is identical, if any place is specialized, be subject to and here indicate that.

Cost function in the case of the rigid transformation of front can indicate are as follows:

In the preferred embodiment, the cost function in the case of rigid transformation can indicate are as follows:

Cost function in the case of the non-rigid transformation of front can indicate are as follows:

In the preferred embodiment, the cost function in the case of non-rigid transformation can indicate are as follows:

E=E_c+λE_r,Rigidity becomes herein It changes and the unified writing of non-rigid transformation: T_i(p).For rigid transformation, E_r=0.

If overall situation TSDF value is V, initial local TSDF value is divided into n group, i.e. V according to corresponding depth map₁,V₂,…, V_n, the weight of the corresponding voxel of each initial local TSDF value is W₁,W₂,…,W_n, final local T SDF value is V after transformation₁′, V₂′,…,V_n′.Organize serial number i, i=1,2 ... ..., n.For the voxel p in V, in V_i' in correspondence voxel be still p, and In V_iIn correspondence voxel be T_i(p) (rigid transformation or non-rigid transformation), wherein T_iFor the change for needing to solve in optimization problem It changes.

As it can be seen that main modification is that introducing weight W_i(T_i(p)).Weight changes with iterative process.Particular voxel p Weight be equal to the transformed corresponding voxel T of particular voxel_i(p) weight of corresponding group, and the corresponding group of corresponding voxel Weight is above-mentioned W₁,W₂,…,W_n, the weight (W of a voxel₁,W₂,…,W_n) group is depended on, same voxel corresponds to difference Group has different weights, corresponding to same group of different voxels weight having the same.The weight of particular voxel is equal to the spy Determine the number of the voxel initial local TSDF value of transformed corresponding voxel in this set.In the case where so calculating weight, It is considered that weight is equal to the sum of the weight corresponding to depth map of the particular voxel transformed corresponding voxel in this set, this The weight corresponding to depth map of particular voxel transformed corresponding voxel in this set is in the particular voxel in this set through becoming It changes corresponding voxel and is equal to 1 when having initial local TSDF value corresponding to depth map, be otherwise equal to 0.Alternatively, particular voxel Weight is equal to the sum of the weight corresponding to depth map of the particular voxel transformed corresponding voxel in this set, the particular voxel The weight corresponding to depth map of transformed corresponding voxel is transformed corresponding in this set with the particular voxel in this set Voxel is related relative to the depth value that the depth map corresponds to camera.For example, the particular voxel transformed corresponding body in this set Element is bigger relative to the depth value that depth map corresponds to camera, and transformed corresponding voxel corresponds to the particular voxel in this set The weight of the depth map is smaller.For another example, transformed corresponding voxel corresponds to depth map to the particular voxel in this set Weight is inversely proportional to the particular voxel, and transformed corresponding voxel corresponds to the depth value of camera relative to the depth map in this set.

In a preferred embodiment, calculating can be simplified in the following way: in (k+1) wheel iterative process, It usesInstead ofCost function to be optimized is simplified are as follows:

Cost function to be optimized at this time is the form of non-linear least square, can pass through Gauss-Newton (Gauss- Newton) method solves.

In some cases, possible initial local TSDF value is excessive, so that being not enough to effectively by a division operation Ground reduces data volume, can be grouped again in step s 4 at this time.

Using the global TSDF value of obtained each group as initial local TSDF value, it is grouped again by corresponding depth map, At least one set of initial local TSDF value corresponds to more than one and is grouped corresponding depth map for the first time；For every group of initial local TSDF Value, obtains the global TSDF value of the group；Using the global TSDF value of obtained each group as initial local TSDF value, solve optimal Change problem, to obtain final overall situation TSDF value.

It obtains in step s 4 and belongs to set P_iAll voxel p final overall situation TSDF value after, so that it may be based on this Reconstructed object.

In step s 5, obtained final overall situation TSDF value, object described in three-dimensional reconstruction are based on.

For example, it is based on obtained final overall situation TSDF value using marching cube (marching cube) algorithm, The surface of object described in three-dimensional reconstruction.Marching cubes algorithm is known in the art algorithm, and details are not described herein.

The equipment of the three-dimensional reconstruction object of embodiment according to the present invention is described next, with reference to Fig. 4.

Fig. 4 shows the structural block diagram of the equipment of the three-dimensional reconstruction object of embodiment according to the present invention.Such as Fig. 4 institute Show, three-dimensional reconstruction object-based device 400 according to the present invention includes: to obtain device 41, is configured as: obtaining voxel in three-dimensional space Initial local be truncated signed distance function TSDF value, each initial local TSDF value correspond to multiple depth maps in a depth Degree figure；Apparatus for grouping 42, is configured as: by corresponding depth map, initial local TSDF value being grouped, at least one set of initial local TSDF value corresponds to more than one depth map；Group overall situation TSDF value acquisition device 43, is configured as: for every group of initial local TSDF value obtains the global TSDF value of the group；Solving device 44, is configured as: the global TSDF value of obtained each group is made For initial local TSDF value, optimization problem is solved, to obtain final overall situation TSDF value；And reconstructing device 45, it is configured as: Based on obtained final overall situation TSDF value, object described in three-dimensional reconstruction；Wherein, in the optimization problem, a voxel Global TSDF value obtained based on the final local T SDF value of the voxel, the final local T SDF value of a voxel is equal to the voxel The initial local TSDF value of transformed corresponding voxel, variable are the parameter of global the TSDF value and transformation of voxel, cost function And following factors are related: the global TSDF value of particular voxel and the initial local TSDF value of the transformed corresponding voxel of the voxel Difference square weighted sum, the weight of weighted sum is equal to the weight of the corresponding group of the transformed corresponding voxel of particular voxel.

In one embodiment, the transformation includes rigid transformation.

In one embodiment, the transformation includes non-rigid transformation, and cost function is also related to following factors: particular volume The summation of the offset measurement of the non-rigid transformation of element experience.

In one embodiment, described group of overall situation TSDF value acquisition device 43 is further configured to: initial for every group Local T SDF value, by solving optimization problem, to obtain the global TSDF value of the group.

In one embodiment, described group of overall situation TSDF value acquisition device 43 is further configured to: initial for every group Local T SDF value calculates the arithmetic mean of instantaneous value for corresponding to the initial local TSDF value of same voxel, as corresponding to the voxel The global TSDF value of the group.

In one embodiment, the solving device 44 is further configured to: by the global TSDF of obtained each group Value is used as initial local TSDF value, is grouped again by corresponding depth map, and at least one set of initial local TSDF value corresponds to more than One is grouped corresponding depth map for the first time；For every group of initial local TSDF value, the global TSDF value of the group is obtained；It will be acquired Each group global TSDF value as initial local TSDF value, optimization problem is solved, to obtain final overall situation TSDF value.

In one embodiment, in the optimization problem, each particular voxel initially has the weight specific to group, Weight is equal to the particular voxel, and the number of the initial local TSDF value of transformed corresponding voxel or weight are equal in this set The sum of the weight corresponding to depth map of the particular voxel transformed corresponding voxel in this set, the particular voxel is in this set Transformed corresponding voxel is opposite in this set with the particular voxel for the weight corresponding to depth map of transformed corresponding voxel It is related in the depth value that the depth map corresponds to camera.

In one embodiment, the acquisition device 41 further include: normalization unit is configured as: by what is be calculated The initial local TSDF value of voxel normalizes.

In one embodiment, the absolute value of the initial local TSDF value of voxel is less than or equal to specific threshold, this is specific Threshold value belongs to (0,1).

In one embodiment, particular voxel is all voxels for meeting specific threshold condition.

In one embodiment, particular voxel is to meet obtained sampling body after all voxels sampling of specific threshold condition Element.

Due to the processing in included each device in three-dimensional reconstruction object-based device 400 according to the present invention, unit It is similar with the processing in each step included in three-dimensional reconstruction object method described above respectively, therefore in order to succinctly rise See, omits the detailed description of these devices, unit herein.

In addition, it is still necessary to, it is noted that each component devices, unit can be by softwares, firmware, hard in above equipment here The mode of part or combinations thereof is configured.It configures workable specific means or mode is well known to those skilled in the art, This is repeated no more.In the case where being realized by software or firmware, from storage medium or network to specialized hardware structure Computer (such as general purpose computer 500 shown in fig. 5) installation constitutes the program of the software, which is being equipped with various journeys When sequence, it is able to carry out various functions etc..

In Fig. 5, central processing unit (CPU) 501 is according to the program stored in read-only memory (ROM) 502 or from depositing The program that storage part 508 is loaded into random access memory (RAM) 503 executes various processing.In RAM 503, also according to need Store the data required when CPU 501 executes various processing etc..CPU 501, ROM 502 and RAM 503 are via bus 504 are connected to each other.Input/output interface 505 is also connected to bus 504.

Components described below is connected to input/output interface 505: importation 506 (including keyboard, mouse etc.), output section Divide 507 (including display, such as cathode-ray tube (CRT), liquid crystal display (LCD) etc. and loudspeakers etc.), storage section 508 (including hard disks etc.), communications portion 509 (including network interface card such as LAN card, modem etc.).Communications portion 509 Communication process is executed via network such as internet.As needed, driver 510 can be connected to input/output interface 505. Detachable media 511 such as disk, CD, magneto-optic disk, semiconductor memory etc., which can according to need, is installed in driver On 510, so that the computer program read out is mounted to as needed in storage section 508.

It is such as removable from network such as internet or storage medium in the case where series of processes above-mentioned by software realization Unload the program that the installation of medium 511 constitutes software.

It will be understood by those of skill in the art that this storage medium be not limited to it is shown in fig. 5 be wherein stored with program, Separately distribute with equipment to provide a user the detachable media 511 of program.The example of detachable media 511 includes disk (including floppy disk (registered trademark)), CD (including compact disc read-only memory (CD-ROM) and digital versatile disc (DVD)), magneto-optic disk (including mini-disk (MD) (registered trademark)) and semiconductor memory.Alternatively, storage medium can be ROM 502, storage section Hard disk for including in 508 etc., wherein computer program stored, and user is distributed to together with the equipment comprising them.

The present invention also proposes a kind of program product of instruction code for being stored with machine-readable.Described instruction code is by machine When device reads and executes, method that above-mentioned embodiment according to the present invention can be performed.

Correspondingly, it is also wrapped for carrying the storage medium of the program product of the above-mentioned instruction code for being stored with machine-readable It includes in disclosure of the invention.The storage medium includes but is not limited to floppy disk, CD, magneto-optic disk, storage card, memory stick etc. Deng.

In the description above to the specific embodiment of the invention, for the feature a kind of embodiment description and/or shown It can be used in one or more other embodiments in a manner of same or similar, with the feature in other embodiment It is combined, or the feature in substitution other embodiment.

It should be emphasized that term "comprises/comprising" refers to the presence of feature, element, step or component when using herein, but simultaneously It is not excluded for the presence or additional of one or more other features, element, step or component.

In addition, method of the invention be not limited to specifications described in time sequencing execute, can also according to it His time sequencing, concurrently or independently execute.Therefore, the execution sequence of method described in this specification is not to this hair Bright technical scope is construed as limiting.

Although being had been disclosed above by the description to specific embodiments of the present invention to the present invention, it answers The understanding, above-mentioned all embodiments and example are exemplary, and not restrictive.Those skilled in the art can be in institute Design is to various modifications of the invention, improvement or equivalent in attached spirit and scope of the claims.These modification, improve or Person's equivalent should also be as being to be considered as included in protection scope of the present invention.

Note

1. a kind of three-dimensional rebuilding method of object, comprising:

Signed distance function TSDF value, each initial local TSDF value is truncated in the initial local for obtaining voxel in three-dimensional space Corresponding to a depth map in multiple depth maps；

By corresponding depth map, initial local TSDF value is grouped, at least one set of initial local TSDF value corresponds to more than One depth map；

For every group of initial local TSDF value, the global TSDF value of the group is obtained；

Using the global TSDF value of obtained each group as initial local TSDF value, optimization problem is solved, to obtain most Overall situation TSDF value eventually；And

Based on obtained final overall situation TSDF value, object described in three-dimensional reconstruction；

Wherein, in the optimization problem, the global TSDF value of a voxel is based on the final local T SDF of the voxel Value obtains, and the final local T SDF value of a voxel is equal to the initial local TSDF value of the transformed corresponding voxel of the voxel, becomes Amount is the parameter of global the TSDF value and transformation of voxel, and cost function is related to following factors: the global TSDF value of particular voxel With the difference of the initial local TSDF value of the transformed corresponding voxel of the voxel square weighted sum, the weight of weighted sum is equal to spy Determine the weight of the corresponding group of the transformed corresponding voxel of voxel.

2. the method as described in note 1, wherein the transformation includes rigid transformation.

3. note 1 as described in method, wherein it is described transformation include non-rigid transformation, cost function also with following factors It is related: the summation of the offset measurement of the non-rigid transformation of particular voxel experience.

4. the method as described in note 1, wherein be directed to every group of initial local TSDF value, obtain the global TSDF value of the group It include:, by solving optimization problem, to obtain the global TSDF value of the group for every group of initial local TSDF value.

5. the method as described in note 1, wherein be directed to every group of initial local TSDF value, obtain the global TSDF value of the group Include: to calculate the arithmetic mean of instantaneous value for corresponding to the initial local TSDF value of same voxel for every group of initial local TSDF value, makees For the global TSDF value of the group corresponding to the voxel.

6. the method as described in note 1, wherein described using the global TSDF value of obtained each group as initial local TSDF value solves optimization problem, includes: to obtain final overall situation TSDF value

Using the global TSDF value of obtained each group as initial local TSDF value, it is grouped again by corresponding depth map, At least one set of initial local TSDF value corresponds to more than one and is grouped corresponding depth map for the first time；

Using the global TSDF value of obtained each group as initial local TSDF value, optimization problem is solved, to obtain most Overall situation TSDF value eventually.

7. the method as described in note 1, wherein in the optimization problem, each particular voxel initially has specific In the weight of group, weight be equal to the particular voxel in this set the number of the initial local TSDF value of transformed corresponding voxel or Person's weight is equal to the sum of the weight corresponding to depth map of the particular voxel transformed corresponding voxel in this set, the particular volume The weight corresponding to depth map of element corresponding voxel transformed in this set is transformed corresponding in this set with the particular voxel The voxel depth value that corresponds to camera relative to the depth map it is related.

8. the method as described in note 1, wherein signed distance function is truncated in the initial local for obtaining voxel in three-dimensional space TSDF value includes:

The initial local TSDF value for the voxel being calculated is normalized.

9. such as method described in note 8, wherein the absolute value of the initial local TSDF value of voxel is less than or equal to certain threshold Value, the specific threshold belong to (0,1).

10. such as method described in note 9, wherein particular voxel is after meeting all voxels sampling of specific threshold condition Obtained sampling voxel.

11. a kind of three-dimensional reconstruction equipment of object, comprising:

Device is obtained, is configured as: obtaining the initial local truncation signed distance function TSDF value of voxel in three-dimensional space, Each initial local TSDF value corresponds to a depth map in multiple depth maps；

Apparatus for grouping is configured as: by corresponding depth map, initial local TSDF value being grouped, at least one set of initial office Portion's TSDF value corresponds to more than one depth map；

Group overall situation TSDF value acquisition device, is configured as: for every group of initial local TSDF value, obtaining the overall situation of the group TSDF value；

Solving device is configured as: using the global TSDF value of obtained each group as initial local TSDF value, being solved most Optimization problem, to obtain final overall situation TSDF value；And

Reconstructing device is configured as: being based on obtained final overall situation TSDF value, object described in three-dimensional reconstruction；

12. the equipment as described in note 11, wherein the transformation includes rigid transformation.

13. note 11 as described in equipment, wherein it is described transformation include non-rigid transformation, cost function also with it is following because It is plain related: the summation of the offset measurement of the non-rigid transformation of particular voxel experience.

14. the equipment as described in note 11, wherein described group of overall situation TSDF value acquisition device is further configured to: needle The global TSDF value of the group is obtained by solving optimization problem to every group of initial local TSDF value.

15. the equipment as described in note 11, wherein described group of overall situation TSDF value acquisition device is further configured to: needle To every group of initial local TSDF value, the arithmetic mean of instantaneous value for corresponding to the initial local TSDF value of same voxel is calculated, as correspondence In the global TSDF value of the group of the voxel.

16. the equipment as described in note 11, wherein the solving device is further configured to:

17. the equipment as described in note 11, wherein in the optimization problem, each particular voxel initially has spy Due to the weight of group, weight is equal to the number of the particular voxel initial local TSDF value of transformed corresponding voxel in this set Or weight is equal to the sum of the weight corresponding to depth map of the particular voxel transformed corresponding voxel in this set, this is specific The weight corresponding to depth map and the particular voxel of voxel transformed corresponding voxel in this set are transformed right in this set The voxel answered is related relative to the depth value that the depth map corresponds to camera.

18. the equipment as described in note 11, wherein the acquisition device further include: normalization unit is configured as: will The initial local TSDF value for the voxel being calculated normalizes.

19. the equipment as described in note 18, wherein the absolute value of the initial local TSDF value of voxel is less than or equal to specific Threshold value, the specific threshold belong to (0,1).

20. the equipment as described in note 19, wherein particular voxel is after meeting all voxels sampling of specific threshold condition Obtained sampling voxel.

Claims

1. a kind of three-dimensional rebuilding method of object, comprising:

Signed distance function TSDF value is truncated in the initial local for obtaining voxel in three-dimensional space, and each initial local TSDF value is corresponding A depth map in multiple depth maps；

Using the global TSDF value of obtained each group as initial local TSDF value, optimization problem is solved, it is final complete to obtain Office's TSDF value；And

Wherein, in the optimization problem, the global TSDF value of a voxel is worth based on the final local T SDF of the voxel It arrives, the final local T SDF value of a voxel is equal to the initial local TSDF value of the transformed corresponding voxel of the voxel, and variable is The parameter of global the TSDF value and transformation of voxel, cost function include following cost item: the global TSDF value of particular voxel with should The difference of the initial local TSDF value of the transformed corresponding voxel of voxel square weighted sum, the weight of weighted sum is equal to particular volume The weight of the corresponding group of the transformed corresponding voxel of element.

2. the method for claim 1, wherein the transformation includes rigid transformation.

3. the method for claim 1, wherein the transformation includes non-rigid transformation, cost function further includes another generation Valence item: the summation of the offset measurement of the non-rigid transformation of particular voxel experience.

4. being the method for claim 1, wherein directed to every group of initial local TSDF value, the global TSDF value of the group is obtained It include:, by solving optimization problem, to obtain the global TSDF value of the group for every group of initial local TSDF value.

5. being the method for claim 1, wherein directed to every group of initial local TSDF value, the global TSDF value of the group is obtained Include: to calculate the arithmetic mean of instantaneous value for corresponding to the initial local TSDF value of same voxel for every group of initial local TSDF value, makees For the global TSDF value of the group corresponding to the voxel.

6. the method for claim 1, wherein described using the global TSDF value of obtained each group as initial local TSDF value solves optimization problem, includes: to obtain final overall situation TSDF value

Using the global TSDF value of obtained each group as initial local TSDF value, it is grouped again by corresponding depth map, at least One group of initial local TSDF value corresponds to more than one and is grouped corresponding depth map for the first time；

Using the global TSDF value of obtained each group as initial local TSDF value, optimization problem is solved, it is final complete to obtain Office's TSDF value.

7. each particular voxel initially has specific the method for claim 1, wherein in the optimization problem In the weight of group, weight be equal to the particular voxel in this set the number of the initial local TSDF value of transformed corresponding voxel or Person's weight is equal to the sum of the weight corresponding to depth map of the particular voxel transformed corresponding voxel in this set, the particular volume The weight corresponding to depth map of element corresponding voxel transformed in this set is transformed corresponding in this set with the particular voxel The voxel depth value that corresponds to camera relative to the depth map it is related.

8. signed distance function is truncated in the initial local for the method for claim 1, wherein obtaining voxel in three-dimensional space TSDF value includes:

The initial local TSDF value for the voxel being calculated is normalized.

9. method according to claim 8, wherein the absolute value of the initial local TSDF value of voxel is less than or equal to certain threshold Value, the specific threshold belong to (0,1).

10. a kind of three-dimensional reconstruction equipment of object, comprising:

Apparatus for grouping is configured as: by corresponding depth map, initial local TSDF value being grouped, at least one set of initial local TSDF value corresponds to more than one depth map；

Group overall situation TSDF value acquisition device, is configured as: for every group of initial local TSDF value, obtaining the global TSDF of the group Value；

Solving device is configured as: using the global TSDF value of obtained each group as initial local TSDF value, being solved and is optimized Problem, to obtain final overall situation TSDF value；And