CN109978986B - Three-dimensional model reconstruction method and device, storage medium and terminal equipment - Google Patents
Three-dimensional model reconstruction method and device, storage medium and terminal equipment Download PDFInfo
- Publication number
- CN109978986B CN109978986B CN201711460297.XA CN201711460297A CN109978986B CN 109978986 B CN109978986 B CN 109978986B CN 201711460297 A CN201711460297 A CN 201711460297A CN 109978986 B CN109978986 B CN 109978986B
- Authority
- CN
- China
- Prior art keywords
- model
- average shape
- dimensional model
- shape model
- sample
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 67
- 238000012545 processing Methods 0.000 claims abstract description 17
- 238000005457 optimization Methods 0.000 claims abstract description 13
- 230000006870 function Effects 0.000 claims description 19
- 238000004590 computer program Methods 0.000 claims description 18
- 230000008569 process Effects 0.000 claims description 10
- 238000002372 labelling Methods 0.000 claims description 6
- 238000004364 calculation method Methods 0.000 claims description 4
- 238000010276 construction Methods 0.000 claims description 3
- 238000012821 model calculation Methods 0.000 claims description 3
- 230000009467 reduction Effects 0.000 abstract description 5
- 230000002159 abnormal effect Effects 0.000 description 4
- 238000010586 diagram Methods 0.000 description 4
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 230000007547 defect Effects 0.000 description 3
- 238000004891 communication Methods 0.000 description 2
- 230000009466 transformation Effects 0.000 description 2
- 238000004422 calculation algorithm Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000012805 post-processing Methods 0.000 description 1
- 238000011084 recovery Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
Landscapes
- Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Computer Graphics (AREA)
- Geometry (AREA)
- Software Systems (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Image Processing (AREA)
Abstract
The invention discloses a method and a device for reconstructing a three-dimensional model, a storage medium and a terminal device, which are used for solving the problems that the reconstructed model cannot display the smooth surface of an artificially manufactured commodity and is low in reduction degree. The method provided by the invention comprises the following steps: calculating an SFM model of a target object of a model to be reconstructed by adopting an SFM method; obtaining an average shape model under the classification of the target object, wherein anchor points are marked on the average shape model in advance; with the assistance of an anchor point, performing deformation processing on the average shape model so that the average shape model is aligned with the SFM model; constructing a three-dimensional model of the target object by adopting a multi-view stereoscopic vision method; performing optimization compensation on the deformed average shape model according to the three-dimensional model, so that the error between the deformed average shape model and the three-dimensional model is minimum under a preset objective function after compensation is added; and outputting the compensated average shape model as a reconstruction model of the target object.
Description
Technical Field
The invention relates to the technical field of three-dimensional models, in particular to a method and a device for reconstructing a three-dimensional model, a storage medium and terminal equipment.
Background
Motion restoration Structures (SFM) and multi-view stereo vision (MVS) are conventional procedures for reconstructing three-dimensional models. The SFM is a process of obtaining three-dimensional structure information by analyzing the motion of an object, and a three-dimensional model of the object can be obtained by calculation through an SFM method, and is called an SFM model. In the pure vision-based three-dimensional reconstruction, the output result of the multi-view stereo-vision-based three-dimensional reconstruction often has some problems such as abnormal points or holes. The result of the smooth surface reconstruction tends to show some unevenness. For example, some e-commerce (Taobao, etc.) platforms desire to display a three-dimensional model of their merchandise on a web page for viewing by a user, and if conventional multi-view stereo vision is used to reconstruct the three-dimensional model of the merchandise, the reconstructed model often does not show the smooth surface of the artificially manufactured merchandise and may have flaws such as holes.
Disclosure of Invention
The embodiment of the invention provides a method and a device for reconstructing a three-dimensional model, a storage medium and a terminal device, which can reconstruct the three-dimensional model of an object, improve the reduction degree of the smooth surface of the object and reduce defects such as abnormal points, cavities and the like.
In a first aspect, a method for reconstructing a three-dimensional model is provided, including:
calculating an SFM model of a target object of a model to be reconstructed by adopting an SFM method;
obtaining an average shape model under the classification of the target object, wherein the average shape model is obtained by taking an average shape of each three-dimensional model sample which is constructed by adopting scanning equipment in advance for each different object sample under the classification of the target object and is used for representing the average shape of the object under the classification, and anchor points are labeled on the average shape model in advance;
with the aid of an anchor point, performing deformation processing on the average shape model so that the average shape model is aligned with the SFM model;
constructing a three-dimensional model of the target object by adopting a multi-view stereoscopic vision method;
performing optimization compensation on the deformed average shape model according to the three-dimensional model, so that the error between the deformed average shape model and the three-dimensional model is minimum under a preset objective function after compensation is added;
and outputting the compensated average shape model as a reconstruction model of the target object.
Optionally, the average shape model under the classification to which the target object belongs is obtained in advance through the following steps:
collecting a real object picture of each different object sample under the classification of the target object;
adopting scanning equipment to construct three-dimensional model samples corresponding to the different object samples according to the object picture;
marking an anchor point on the three-dimensional model sample, wherein the anchor point comprises position information of the anchor point on the three-dimensional model sample;
aligning each three-dimensional model sample according to an anchor point on each three-dimensional model sample;
and calculating an average shape model under the classification of the target object according to the aligned three-dimensional model samples.
Optionally, the marking of the anchor point on the three-dimensional model sample specifically includes: marking anchor points on the three-dimensional model sample by adopting a preset feature point set, wherein the feature point set represents the real object contour of the object sample corresponding to the three-dimensional model sample;
the feature point set is obtained in advance through the following steps:
constructing a labeled three-dimensional model corresponding to each different object sample according to the collected object pictures of each different object sample;
marking characteristic points for representing the outline of the real object on the marked three-dimensional model;
and collecting all the characteristic points on the same marked three-dimensional model to obtain a characteristic point set of the object sample corresponding to the marked three-dimensional model.
Optionally, the collecting the physical pictures of the different object samples under the classification to which the target object belongs includes:
placing a physical object of each of the different object samples on the ground;
aiming at a single real object in each real object, controlling the robot to walk around the single real object at least for one circle by taking the single real object as a circle center, and photographing the single real object in the walking process to obtain pictures of the single real object at different angles.
Optionally, the deforming the average shape model with the assistance of an anchor point such that the average shape model aligns with the SFM model comprises:
searching each target point corresponding to each anchor point of the average shape model on the SFM model;
calculating deformation parameters of the average shape model according to the corresponding relation between each anchor point and each target point;
and carrying out deformation processing on the average shape model according to the deformation parameters so that the average shape model is aligned with the SFM model.
In a second aspect, there is provided an apparatus for reconstructing a three-dimensional model, comprising:
the SFM model calculation module is used for calculating an SFM model of a target object of a model to be reconstructed by adopting an SFM method;
an average shape model obtaining module, configured to obtain an average shape model under the classification to which the target object belongs, where the average shape model is obtained by taking an average shape of each three-dimensional model sample, which is constructed by using scanning equipment in advance, of each different object sample under the classification to which the target object belongs, and is used to represent an average shape of the classified object, and anchor points are labeled in advance on the average shape model;
a model alignment module, configured to perform deformation processing on the average shape model with the aid of an anchor point, so that the average shape model is aligned with the SFM model;
the multi-view stereo model building module is used for building a three-dimensional model of the target object by adopting a multi-view stereo vision method;
the optimization compensation module is used for carrying out optimization compensation on the deformed average shape model according to the three-dimensional model, so that the error between the deformed average shape model and the three-dimensional model is the minimum under a preset objective function after compensation is added;
and the reconstruction model output module is used for outputting the compensated average shape model as the reconstruction model of the target object.
Optionally, the average shape model under the classification to which the target object belongs is obtained in advance through the following modules:
the object picture collecting module is used for collecting object pictures of different object samples under the classification of the target object;
the model sample construction module is used for constructing three-dimensional model samples corresponding to the different object samples according to the object picture by adopting scanning equipment;
the anchor point marking module is used for marking an anchor point on the three-dimensional model sample, and the anchor point comprises position information of the anchor point on the three-dimensional model sample;
the model sample alignment module is used for aligning the three-dimensional model samples according to anchor points on the three-dimensional model samples;
and the average shape calculation module is used for calculating an average shape model under the classification of the target object according to the aligned three-dimensional model samples.
Optionally, the model alignment module comprises:
the target point searching unit is used for searching each target point corresponding to each anchor point of the average shape model on the SFM model;
calculating deformation parameters of the average shape model according to the corresponding relation between each anchor point and each target point;
and carrying out deformation processing on the average shape model according to the deformation parameters so that the average shape model is aligned with the SFM model.
In a third aspect, a terminal device is provided, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the steps of the above-mentioned three-dimensional model reconstruction method when executing the computer program.
In a fourth aspect, a computer-readable storage medium is provided, which stores a computer program that, when executed by a processor, implements the steps of the above-described method for reconstructing a three-dimensional model.
According to the technical scheme, the embodiment of the invention has the following advantages:
in the embodiment of the invention, firstly, an SFM method is adopted to calculate the SFM model of the target object of the model to be reconstructed; then, obtaining an average shape model under the classification of the target object, wherein the average shape model is obtained by taking an average shape of each three-dimensional model sample which is constructed by adopting scanning equipment in advance from each different object sample under the classification of the target object, and is used for representing the average shape of the classified object, and anchor points are marked on the average shape model in advance; then, with the assistance of an anchor point, performing deformation processing on the average shape model to make the average shape model aligned with the SFM model; thirdly, constructing a three-dimensional model of the target object by adopting a multi-eye stereo vision method; performing optimization compensation on the deformed average shape model according to the three-dimensional model, so that the error between the deformed average shape model and the three-dimensional model is minimum under a preset objective function after compensation is added; and finally, outputting the compensated average shape model as a reconstruction model of the target object. Therefore, the method and the device align the prior average shape model with the model of the target object, optimize and compensate the model by adopting the three-dimensional model constructed by the multi-view stereo vision method, minimize the error between the prior average shape model and the model of the target object, output the reconstructed model more accurately, improve the reduction degree of the smooth surface of the object and reduce the defects of abnormal points, cavities and the like.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings needed to be used in the embodiments or the prior art descriptions will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without inventive exercise.
Fig. 1 is a flowchart of an embodiment of a method for reconstructing a three-dimensional model according to an embodiment of the present invention;
FIG. 2 is a schematic flow chart of a method for reconstructing a three-dimensional model according to an embodiment of the present invention, in which an average shape model under classification is pre-calculated in an application scenario;
FIG. 3 is a schematic flow chart of a method for reconstructing a three-dimensional model according to an embodiment of the present invention, in which a feature point set is collected in advance in an application scenario;
FIG. 4 is a schematic flowchart of step 103 of a method for reconstructing a three-dimensional model in an application scenario according to an embodiment of the present invention;
FIG. 5 is a diagram illustrating an embodiment of a device for reconstructing a three-dimensional model according to an embodiment of the present invention;
fig. 6 is a schematic diagram of a terminal device according to an embodiment of the present invention.
Detailed Description
The embodiment of the invention provides a method and a device for reconstructing a three-dimensional model, a storage medium and terminal equipment, which are used for solving the problems that the reconstructed model cannot show the smooth surface of an artificially manufactured commodity and has low reduction degree.
In order to make the objects, features and advantages of the present invention more obvious and understandable, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention, and it is obvious that the embodiments described below are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Referring to fig. 1, an embodiment of a method for reconstructing a three-dimensional model according to the embodiment of the present invention includes:
101. calculating an SFM model of a target object of a model to be reconstructed by adopting an SFM method;
first, in this embodiment, a motion recovery Structure (SFM) method may be adopted to calculate an SFM model of a target object to be modeled as model _ SFM.
102. Obtaining an average shape model under the classification of the target object, wherein the average shape model is obtained by taking an average shape of each three-dimensional model sample which is constructed by adopting scanning equipment in advance for each different object sample under the classification of the target object and is used for representing the average shape of the object under the classification, and anchor points are labeled on the average shape model in advance;
it should be noted that, in this embodiment, the average shape models under different object classifications are sorted in advance, and the classification of these object classifications may adopt a tree-like structure, for example, one large classification includes multiple small classifications, and each small classification includes multiple different objects. For example, the major category "car" includes small categories such as "truck", "car", "bus", etc., wherein the small category "car" includes cars of various models, such as bme x5, gallop c200, etc. It is understood that the object category may be set according to actual conditions, and is not limited specifically herein.
After determining the different classifications, the average shape model under each classification is calculated in advance, specifically, as shown in fig. 2, the average shape model under the classification to which the target object belongs can be obtained in advance through the following steps:
201. collecting real object pictures of different object samples under the classification of the target object;
202. adopting scanning equipment to construct three-dimensional model samples corresponding to the different object samples according to the object picture;
203. marking an anchor point on the three-dimensional model sample, wherein the anchor point comprises position information of the anchor point on the three-dimensional model sample;
204. aligning each three-dimensional model sample according to an anchor point on each three-dimensional model sample;
205. and calculating an average shape model under the classification of the target object according to the aligned three-dimensional model samples.
The step 201 may specifically include: placing a physical object of each of the different object samples on the ground; then, aiming at a single real object in each real object, controlling the robot to walk around the single real object at least for one circle by taking the single real object as a circle center, and photographing the single real object in the walking process to obtain pictures of the single real object at different angles. In an application scene, the robot can be programmed first, so that the robot can automatically photograph the real object in a preset mode. Firstly, the real object to be photographed is placed on the vacant ground, interference objects are avoided as much as possible around the real object to be photographed, and the work of post-processing of pictures is reduced. Then, the robot walks in a circular route around the real object under the control of programming, the circle center of the circle is the position of the real object, the robot walks and takes pictures at the same time, and the taken pictures can be uniformly distributed on the circular route, so that the real object pictures of the real object at different angles can be obtained. Preferably, the robot can be controlled to travel three circles along the circular route through programming in order to take the details of each angle of the real object, and the camera shooting angle of each circle of the robot is different from that of the other two circles, so that more details of the real object can be taken.
With respect to step 202, it can be understood that, the three-dimensional model sample constructed by using the scanning device is more accurate than the model constructed by the SFM method, and the present embodiment adopts the scanning device to construct the model of the sample, and the cost can be controlled within a reasonable range due to the limited number of samples. After the three-dimensional model sample is constructed, the three-dimensional model sample is recorded as model _ scan _ i.
For step 203, the anchor point may include, in addition to the position information of the anchor point on the three-dimensional model sample, a SIFT feature of the anchor point in the real image and a preset weight value, where the weight value represents the importance of the anchor point, and the larger the number of occurrences of the anchor point, the smaller the variation amplitude of the corresponding position and appearance, and thus the higher the importance of the anchor point.
Furthermore, the marking of the anchor point on the three-dimensional model sample may specifically be: and marking anchor points on the three-dimensional model sample by adopting a preset feature point set, wherein the feature point set represents the real object contour of the object sample corresponding to the three-dimensional model sample. As shown in fig. 3, the feature point set may be obtained in advance through the following steps:
301. constructing a labeled three-dimensional model corresponding to each different object sample according to the collected object pictures of each different object sample;
302. marking feature points for representing the real object outline on the marked three-dimensional model;
303. and collecting all the characteristic points on the same marked three-dimensional model to obtain a characteristic point set of the object sample corresponding to the marked three-dimensional model.
For the above steps 301 to 303, specifically, the robot is used to obtain the object pictures, and a three-dimensional model corresponding to the object is constructed based on the object pictures, that is, the labeled three-dimensional model is labeled as a model _ sfm _ i, some feature points point _ sfm _ i in the model _ sfm _ i may be manually labeled, and the feature points should be able to relatively well show the outline of the model. Therefore, the set of all feature points in the same model _ sfm _ i model is the feature motor of the object sample corresponding to the model _ sfm _ i model.
With respect to step 204, it can be understood that the anchor points on the three-dimensional model samples can represent the contours of the object samples corresponding to the three-dimensional model samples, and therefore, for the three-dimensional model samples under the same classification, the alignment can be completed according to the anchor points on the three-dimensional model samples.
For step 205, after the three-dimensional model samples in the same classification are aligned, an average shape model in the classification can be calculated, that is, an average shape model in the classification to which the target object belongs can be obtained, and the average shape model is referred to as model _ mean.
103. With the aid of an anchor point, performing deformation processing on the average shape model so that the average shape model is aligned with the SFM model;
it will be appreciated that the SFM model is from the target object and the average shape model corresponds to the class to which the target object belongs, and thus the SFM model may correspond to the average shape model in general. For example, although cars of different models may differ in shape, three-dimensional models of different cars are connected in configuration, and thus the average shape model can be aligned with the SFM model through a deformation process.
Further, as shown in fig. 4, the step 103 may include:
401. searching each target point corresponding to each anchor point of the average shape model on the SFM model;
402. calculating deformation parameters of the average shape model according to the corresponding relation between each anchor point and each target point;
403. and carrying out deformation processing on the average shape model according to the deformation parameters so that the average shape model is aligned with the SFM model.
With respect to the above steps 401 to 403, it can be understood that a local search method can be used to find the corresponding point of each anchor point above the average shape model _ mean, which is above the model _ sfm, i.e. the target point. Then, deformation parameters of the average shape model _ mean are calculated by using the correspondence relationship between the anchor points and the target points, so that the deformed model _ estimate = T (model _ mean, parameters), wherein parameters are deformation parameters and T is a deformation function (such as an algorithm function of the TPS method). The deformed model _ estimate is more fit with the SFM model.
104. Constructing a three-dimensional model of the target object by adopting a multi-view stereoscopic vision method;
105. performing optimization compensation on the deformed average shape model according to the three-dimensional model, so that the error between the deformed average shape model and the three-dimensional model is minimum under a preset objective function after compensation is added;
for the above steps 104 to 105, after the average shape model _ mean and the SFM model are aligned, the deformed average shape model _ estimate is obtained. Since the mean shape model _ estimate was previously derived via a deformation function, it represents a smoother transformation induced from the model _ mean to model _ sfm anchor alignment. Such a transformation may sometimes cause model _ estimate to lose some of the specific details of model _ sfm. For example, a specific running automobile (model _ estimate) is reconstructed from a template (model _ mean) of a general automobile, but since the running automobile to be reconstructed is an unmanned automobile and a radar camera and other positioning loads which are not available in the general automobile (model _ mean) are loaded on the automobile, an error function needs to be made between the model _ estimate and the model _ sfm (information of the radar camera positioning devices) to compensate for the improved model _ estimate.
Therefore, the model _ estimate model needs to be improved to refine the characteristic features of the target object thereon. Firstly, a three-dimensional model _ mvs can be constructed by using a multi-view stereo vision method, the grid structure of the model _ estimate model is optimized, and the compensation is calculated, so that the error between the model _ estimate model and the three-dimensional model _ mvs is minimum under a certain objective function after the compensation is added.
In practical application, any function for measuring the similarity of two mesh structures can be selected, and the specific function form is not specifically limited in this embodiment.
106. And outputting the compensated average shape model as a reconstruction model of the target object.
Finally, the deformed average shape model _ estimate is added with the compensation amount calculated in the step 105, so that the specific characteristics of the target object can be refined on the average shape model _ estimate, and therefore the compensated average shape model _ estimate is considered to be the reconstruction model of the target object to be reconstructed, the reconstruction model is output, and the model reconstruction work of the target object is completed.
In this embodiment, first, an SFM method is used to calculate an SFM model of a target object to be modeled; then, obtaining an average shape model under the classification of the target object, wherein the average shape model is obtained by taking an average shape of each three-dimensional model sample which is constructed by adopting scanning equipment in advance from each different object sample under the classification of the target object, and is used for representing the average shape of the classified object, and anchor points are marked on the average shape model in advance; then, with the assistance of an anchor point, performing deformation processing on the average shape model to make the average shape model aligned with the SFM model; thirdly, constructing a three-dimensional model of the target object by adopting a multi-view stereo vision method; performing optimization compensation on the deformed average shape model according to the three-dimensional model, so that the error between the deformed average shape model and the three-dimensional model is minimum under a preset objective function after compensation is added; and finally, outputting the compensated average shape model as a reconstruction model of the target object. Therefore, in the embodiment, the priori average shape model is aligned with the model of the target object, and the three-dimensional model constructed by the multi-view stereo vision method is adopted to perform optimization compensation, so that the error between the model and the model is minimum, the finally output reconstructed model is more accurate, the reduction degree of the smooth surface of the object is improved, and defects such as abnormal points, cavities and the like are reduced.
It should be understood that, the sequence numbers of the steps in the foregoing embodiments do not imply an execution sequence, and the execution sequence of each process should be determined by its function and inherent logic, and should not constitute any limitation to the implementation process of the embodiments of the present invention.
The above mainly describes a method for reconstructing a three-dimensional model, and a detailed description will be given below of a three-dimensional model reconstruction apparatus.
Fig. 5 is a block diagram illustrating an embodiment of a three-dimensional model reconstruction apparatus according to an embodiment of the present invention.
In this embodiment, a three-dimensional model reconstruction apparatus includes:
an SFM model calculation module 501, configured to calculate an SFM model of a target object to be modeled by using an SFM method;
an average shape model obtaining module 502, configured to obtain an average shape model under the classification to which the target object belongs, where the average shape model is obtained by taking an average shape of each three-dimensional model sample, which is constructed by using scanning equipment in advance, of each different object sample under the classification to which the target object belongs, and is used to represent an average shape of the object under the classification, and an anchor point is labeled on the average shape model in advance;
a model alignment module 503, configured to perform deformation processing on the average shape model with the aid of an anchor point, so that the average shape model is aligned with the SFM model;
a multi-view stereo model building module 504, configured to build a three-dimensional model of the target object by using a multi-view stereo vision method;
an optimization compensation module 505, configured to perform optimization compensation on the deformed average shape model according to the three-dimensional model, so that an error between the deformed average shape model and the three-dimensional model is minimum under a preset objective function after a compensation amount is added to the deformed average shape model;
a reconstruction model output module 506, configured to output the compensated average shape model as a reconstruction model of the target object.
Further, the average shape model under the classification to which the target object belongs may be obtained in advance through the following modules:
the object picture collecting module is used for collecting object pictures of different object samples under the classification of the target object;
the model sample construction module is used for constructing three-dimensional model samples corresponding to the different object samples according to the object pictures by adopting scanning equipment;
the anchor point marking module is used for marking an anchor point on the three-dimensional model sample, and the anchor point comprises position information of the anchor point on the three-dimensional model sample;
the model sample alignment module is used for aligning the three-dimensional model samples according to anchor points on the three-dimensional model samples;
and the average shape calculation module is used for calculating an average shape model under the classification of the target object according to the aligned three-dimensional model samples.
Further, the anchor point labeling module is specifically configured to: marking an anchor point on the three-dimensional model sample by adopting a preset feature point set, wherein the feature point set represents the real object contour of the object sample corresponding to the three-dimensional model sample;
the feature point set can be obtained in advance through the following modules:
the labeling model building module is used for building a labeling three-dimensional model corresponding to each different object sample according to the collected object pictures of each different object sample;
the characteristic point marking module is used for marking characteristic points for representing the real object outline on the marked three-dimensional model;
and the characteristic point collection module is used for collecting all characteristic points on the same marked three-dimensional model to obtain a characteristic point set of the object sample corresponding to the marked three-dimensional model.
Further, the real object picture collecting module comprises:
the object placing unit is used for placing the objects of different object samples on the ground;
and the robot photographing unit is used for controlling the robot to walk around the single real object for at least one circle by taking the single real object as a circle center according to the single real object in each real object, photographing the single real object in the walking process and acquiring pictures of the real objects of the single real object at different angles.
Further, the model alignment module may include:
the target point searching unit is used for searching target points corresponding to anchor points of the average shape model on the SFM model;
calculating deformation parameters of the average shape model according to the corresponding relation between each anchor point and each target point;
and carrying out deformation processing on the average shape model according to the deformation parameters so that the average shape model is aligned with the SFM model.
Fig. 6 is a schematic diagram of a terminal device according to an embodiment of the present invention. As shown in fig. 6, the terminal device 6 of this embodiment includes: a processor 60, a memory 61 and a computer program 62 stored in said memory 61 and executable on said processor 60, such as a program for performing the above-mentioned reconstruction method of a three-dimensional model. The processor 60, when executing the computer program 62, implements the steps in the above-described embodiments of the method of reconstructing a three-dimensional model, such as the steps 101 to 106 shown in fig. 1. Alternatively, the processor 60, when executing the computer program 62, implements the functions of the modules/units in the above-mentioned device embodiments, such as the functions of the modules 501 to 506 shown in fig. 5.
Illustratively, the computer program 62 may be partitioned into one or more modules/units that are stored in the memory 61 and executed by the processor 60 to implement the present invention. The one or more modules/units may be a series of computer program instruction segments capable of performing specific functions, which are used to describe the execution of the computer program 62 in the terminal device 6.
The terminal device 6 may be a computing device such as a desktop computer, a notebook, a palm computer, and a cloud server. The terminal device may include, but is not limited to, a processor 60, a memory 61. Those skilled in the art will appreciate that fig. 6 is merely an example of a terminal device 6 and does not constitute a limitation of terminal device 6 and may include more or less components than those shown, or some components in combination, or different components, for example, the terminal device may also include input output devices, network access devices, buses, etc.
The Processor 60 may be a Central Processing Unit (CPU), other general purpose Processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), an off-the-shelf Programmable Gate Array (FPGA) or other Programmable logic device, discrete Gate or transistor logic device, discrete hardware component, etc. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
The memory 61 may be an internal storage unit of the terminal device 6, such as a hard disk or a memory of the terminal device 6. The memory 61 may also be an external storage device of the terminal device 6, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), and the like, which are provided on the terminal device 6. Further, the memory 61 may also include both an internal storage unit and an external storage device of the terminal device 6. The memory 61 is used for storing the computer program and other programs and data required by the terminal device. The memory 61 may also be used to temporarily store data that has been output or is to be output.
It is clear to those skilled in the art that, for convenience and brevity of description, the specific working processes of the above-described systems, apparatuses and units may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.
In the above embodiments, the descriptions of the respective embodiments have respective emphasis, and reference may be made to the related descriptions of other embodiments for parts that are not described or illustrated in a certain embodiment.
Those of ordinary skill in the art would appreciate that the modules, elements, and/or method steps of the various embodiments described in connection with the embodiments disclosed herein may be implemented as electronic hardware, or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the technical solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.
In the several embodiments provided in the present application, it should be understood that the disclosed system, apparatus and method may be implemented in other manners. For example, the above-described apparatus embodiments are merely illustrative, and for example, the division of the units is only one logical division, and other divisions may be realized in practice, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.
The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, all or part of the flow of the method according to the embodiments of the present invention may also be implemented by a computer program, which may be stored in a computer-readable storage medium, and when the computer program is executed by a processor, the steps of the method embodiments may be implemented. Wherein the computer program comprises computer program code, which may be in the form of source code, object code, an executable file or some intermediate form, etc. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, usb disk, removable hard disk, magnetic disk, optical disk, computer Memory, read-Only Memory (ROM), random Access Memory (RAM), electrical carrier wave signals, telecommunications signals, software distribution medium, and the like. It should be noted that the computer readable medium may contain content that is subject to appropriate increase or decrease as required by legislation and patent practice in jurisdictions, for example, in some jurisdictions, computer readable media does not include electrical carrier signals and telecommunications signals as is required by legislation and patent practice.
The above-mentioned embodiments are only used for illustrating the technical solutions of the present invention, and not for limiting the same; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.
Claims (8)
1. A method of reconstructing a three-dimensional model, comprising:
calculating an SFM model of a target object of a model to be reconstructed by adopting an SFM method;
obtaining an average shape model under the classification of the target object, wherein the average shape model is obtained by taking an average shape of each three-dimensional model sample which is constructed by adopting scanning equipment in advance for each different object sample under the classification of the target object and is used for representing the average shape of the object under the classification, and anchor points are labeled on the average shape model in advance;
with the aid of an anchor point, performing deformation processing on the average shape model so that the average shape model is aligned with the SFM model;
constructing a three-dimensional model of the target object by adopting a multi-view stereoscopic vision method;
performing optimization compensation on the deformed average shape model according to the three-dimensional model, so that the error between the deformed average shape model and the three-dimensional model is minimum under a preset objective function after compensation is added;
outputting the compensated average shape model as a reconstruction model of the target object;
the average shape model under the classification to which the target object belongs is obtained in advance through the following steps:
collecting real object pictures of different object samples under the classification of the target object;
adopting scanning equipment to construct three-dimensional model samples corresponding to the different object samples according to the object picture;
marking an anchor point on the three-dimensional model sample, wherein the anchor point comprises position information of the anchor point on the three-dimensional model sample, and further comprises SIFT characteristics of the anchor point in a real picture and a preset weight value, and the weight value represents the importance of the anchor point;
aligning each three-dimensional model sample according to an anchor point on each three-dimensional model sample;
and calculating an average shape model under the classification of the target object according to the aligned three-dimensional model samples.
2. The method for reconstructing a three-dimensional model according to claim 1, wherein the step of labeling anchor points on the three-dimensional model sample specifically comprises: marking anchor points on the three-dimensional model sample by adopting a preset feature point set, wherein the feature point set represents the real object contour of the object sample corresponding to the three-dimensional model sample;
the feature point set is obtained in advance through the following steps:
constructing a labeled three-dimensional model corresponding to each different object sample according to the collected object pictures of each different object sample;
marking feature points for representing the real object outline on the marked three-dimensional model;
and collecting all the characteristic points on the same marked three-dimensional model to obtain a characteristic point set of the object sample corresponding to the marked three-dimensional model.
3. The method according to claim 1, wherein the collecting the physical pictures of the samples of the different objects under the classification of the target object comprises:
placing a real object of each different object sample on the ground;
aiming at a single real object in each real object, the robot is controlled to walk around the single real object for at least one circle by taking the single real object as a circle center, and the single real object is photographed in the walking process to obtain real object pictures of the single real object at different angles.
4. The method of reconstructing a three-dimensional model according to any of claims 1 to 3, wherein said deforming said mean shape model with the aid of an anchor point such that said mean shape model aligns with said SFM model comprises:
searching each target point corresponding to each anchor point of the average shape model on the SFM model;
calculating deformation parameters of the average shape model according to the corresponding relation between each anchor point and each target point;
and carrying out deformation processing on the average shape model according to the deformation parameters so that the average shape model is aligned with the SFM model.
5. An apparatus for reconstructing a three-dimensional model, comprising:
the SFM model calculation module is used for calculating an SFM model of a target object of a model to be reconstructed by adopting an SFM method;
an average shape model obtaining module, configured to obtain an average shape model under the category to which the target object belongs, where the average shape model is obtained by taking an average shape of three-dimensional model samples, which are pre-constructed by using scanning equipment, of different object samples under the category to which the target object belongs, and is used to represent an average shape of the object under the category, and an anchor point is pre-marked on the average shape model;
a model alignment module, configured to perform deformation processing on the average shape model with the aid of an anchor point, so that the average shape model is aligned with the SFM model;
the multi-view stereo model building module is used for building a three-dimensional model of the target object by adopting a multi-view stereo vision method;
the optimization compensation module is used for carrying out optimization compensation on the deformed average shape model according to the three-dimensional model, so that the error between the deformed average shape model and the three-dimensional model is the minimum under a preset objective function after compensation is added;
a reconstructed model output module, configured to output the compensated average shape model as a reconstructed model of the target object;
the average shape model under the classification to which the target object belongs is obtained in advance through the following modules:
the object picture collecting module is used for collecting object pictures of different object samples under the classification of the target object;
the model sample construction module is used for constructing three-dimensional model samples corresponding to the different object samples according to the object pictures by adopting scanning equipment;
the anchor point labeling module is used for labeling an anchor point on the three-dimensional model sample, wherein the anchor point comprises position information of the anchor point on the three-dimensional model sample, SIFT characteristics of the anchor point in a real picture and a preset weight value, and the weight value represents the importance of the anchor point;
a model sample alignment module for aligning each of the three-dimensional model samples according to an anchor point on each of the three-dimensional model samples;
and the average shape calculation module is used for calculating an average shape model under the classification of the target object according to the aligned three-dimensional model samples.
6. The apparatus for reconstructing a three-dimensional model according to claim 5, wherein the model alignment module comprises:
the target point searching unit is used for searching target points corresponding to anchor points of the average shape model on the SFM model;
calculating deformation parameters of the average shape model according to the corresponding relation between each anchor point and each target point;
and carrying out deformation processing on the average shape model according to the deformation parameters so that the average shape model is aligned with the SFM model.
7. A terminal device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that the processor implements the steps of the method for reconstructing a three-dimensional model according to any one of claims 1 to 4 when executing the computer program.
8. A computer-readable storage medium, in which a computer program is stored which, when being executed by a processor, carries out the steps of a method for reconstructing a three-dimensional model as claimed in any one of claims 1 to 4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711460297.XA CN109978986B (en) | 2017-12-28 | 2017-12-28 | Three-dimensional model reconstruction method and device, storage medium and terminal equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711460297.XA CN109978986B (en) | 2017-12-28 | 2017-12-28 | Three-dimensional model reconstruction method and device, storage medium and terminal equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109978986A CN109978986A (en) | 2019-07-05 |
| CN109978986B true CN109978986B (en) | 2023-03-07 |
Family
ID=67074741
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201711460297.XA Active CN109978986B (en) | 2017-12-28 | 2017-12-28 | Three-dimensional model reconstruction method and device, storage medium and terminal equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109978986B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111540045B (en) * | 2020-07-07 | 2020-11-24 | 深圳市优必选科技股份有限公司 | Mechanical arm and three-dimensional reconstruction method and device thereof |
Family Cites Families (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9489768B2 (en) * | 2012-11-14 | 2016-11-08 | Nec Corporation | Semantic dense 3D reconstruction |
| US9269187B2 (en) * | 2013-03-20 | 2016-02-23 | Siemens Product Lifecycle Management Software Inc. | Image-based 3D panorama |
| CN104200517A (en) * | 2014-08-06 | 2014-12-10 | 北京工业大学 | Three-dimensional reconstruction method |
| CN105427385B (en) * | 2015-12-07 | 2018-03-27 | 华中科技大学 | A kind of high-fidelity face three-dimensional rebuilding method based on multilayer deformation model |
| CN106293092A (en) * | 2016-08-15 | 2017-01-04 | 成都通甲优博科技有限责任公司 | The method realizing virtual wearing based on multi-view stereo vision 3-D technology |
| CN107358631B (en) * | 2017-06-27 | 2020-05-19 | 大连理工大学 | Binocular vision reconstruction method considering three-dimensional distortion |
-
2017
- 2017-12-28 CN CN201711460297.XA patent/CN109978986B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN109978986A (en) | 2019-07-05 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109523597B (en) | Method and device for calibrating external parameters of camera | |
| CN108364253B (en) | Vehicle damage assessment method and system and electronic equipment | |
| CN105938559B (en) | Use the Digital Image Processing of convolutional neural networks | |
| CN108279670B (en) | Method, apparatus and computer readable medium for adjusting point cloud data acquisition trajectory | |
| CN108230235B (en) | Disparity map generation system, method and storage medium | |
| CN110070610B (en) | Feature point matching method, and feature point matching method and device in three-dimensional reconstruction process | |
| CN111402413B (en) | Three-dimensional visual positioning method and device, computing equipment and storage medium | |
| CN114219855A (en) | Point cloud normal vector estimation method and device, computer equipment and storage medium | |
| CN111696196A (en) | Three-dimensional face model reconstruction method and device | |
| CN109784171A (en) | Car damage identification method for screening images, device, readable storage medium storing program for executing and server | |
| CN111964606A (en) | Three-dimensional information processing method and device | |
| CN112613107A (en) | Method and device for determining construction progress of tower project, storage medium and equipment | |
| US11120308B2 (en) | Vehicle damage detection method based on image analysis, electronic device and storage medium | |
| CN110673607B (en) | Feature point extraction method and device under dynamic scene and terminal equipment | |
| CN114663598A (en) | Three-dimensional modeling method, device and storage medium | |
| CN109978986B (en) | Three-dimensional model reconstruction method and device, storage medium and terminal equipment | |
| CN112270748B (en) | Three-dimensional reconstruction method and device based on image | |
| CN112215887A (en) | Pose determination method and device, storage medium and mobile robot | |
| CN112258647A (en) | Map reconstruction method and device, computer readable medium and electronic device | |
| CN112212851B (en) | Pose determination method and device, storage medium and mobile robot | |
| CN115587943B (en) | Denoising method and device for point cloud data, electronic equipment and storage medium | |
| CN111104965A (en) | Vehicle target identification method and device | |
| CN116012609A (en) | Multi-target tracking method, device, electronic equipment and medium for looking around fish eyes | |
| CN112288817B (en) | Three-dimensional reconstruction processing method and device based on image | |
| CN112329596B (en) | Target damage assessment method and device, electronic equipment and computer-readable storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CP01 | Change in the name or title of a patent holder |
Address after: 518000 16th and 22nd Floors, C1 Building, Nanshan Zhiyuan, 1001 Xueyuan Avenue, Nanshan District, Shenzhen City, Guangdong Province Patentee after: Shenzhen UBTECH Technology Co.,Ltd. Address before: 518000 16th and 22nd Floors, C1 Building, Nanshan Zhiyuan, 1001 Xueyuan Avenue, Nanshan District, Shenzhen City, Guangdong Province Patentee before: Shenzhen UBTECH Technology Co.,Ltd. |
|
| CP01 | Change in the name or title of a patent holder | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20231204 Address after: Room 601, 6th Floor, Building 13, No. 3 Jinghai Fifth Road, Beijing Economic and Technological Development Zone (Tongzhou), Tongzhou District, Beijing, 100176 Patentee after: Beijing Youbixuan Intelligent Robot Co.,Ltd. Address before: 518000 16th and 22nd Floors, C1 Building, Nanshan Zhiyuan, 1001 Xueyuan Avenue, Nanshan District, Shenzhen City, Guangdong Province Patentee before: Shenzhen UBTECH Technology Co.,Ltd. |
|
| TR01 | Transfer of patent right |