CN110309554A - Video human three-dimensional rebuilding method and device based on clothes modeling and simulation - Google Patents
Video human three-dimensional rebuilding method and device based on clothes modeling and simulation Download PDFInfo
- Publication number
- CN110309554A CN110309554A CN201910507845.2A CN201910507845A CN110309554A CN 110309554 A CN110309554 A CN 110309554A CN 201910507845 A CN201910507845 A CN 201910507845A CN 110309554 A CN110309554 A CN 110309554A
- Authority
- CN
- China
- Prior art keywords
- clothing
- human body
- video
- simulation
- parameter
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000004088 simulation Methods 0.000 title claims abstract description 75
- 238000000034 method Methods 0.000 title claims abstract description 70
- 230000011218 segmentation Effects 0.000 claims abstract description 31
- 230000008569 process Effects 0.000 claims description 16
- 239000004744 fabric Substances 0.000 claims description 14
- 238000005457 optimization Methods 0.000 claims description 14
- 238000009877 rendering Methods 0.000 claims description 13
- 230000008859 change Effects 0.000 claims description 7
- 238000005286 illumination Methods 0.000 claims description 7
- 230000007704 transition Effects 0.000 claims description 7
- 238000006243 chemical reaction Methods 0.000 claims description 6
- 238000004364 calculation method Methods 0.000 claims description 5
- 239000000203 mixture Substances 0.000 claims description 5
- 230000001131 transforming effect Effects 0.000 claims description 2
- 230000000386 athletic effect Effects 0.000 abstract description 2
- 239000002245 particle Substances 0.000 abstract description 2
- 239000000463 material Substances 0.000 description 4
- 238000013135 deep learning Methods 0.000 description 3
- 230000008901 benefit Effects 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 230000005484 gravity Effects 0.000 description 2
- 241000208340 Araliaceae Species 0.000 description 1
- 244000086443 Craterellus fallax Species 0.000 description 1
- 235000007926 Craterellus fallax Nutrition 0.000 description 1
- 235000005035 Panax pseudoginseng ssp. pseudoginseng Nutrition 0.000 description 1
- 235000003140 Panax quinquefolius Nutrition 0.000 description 1
- 230000009471 action Effects 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 235000008434 ginseng Nutrition 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 238000000465 moulding Methods 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F30/00—Computer-aided design [CAD]
- G06F30/20—Design optimisation, verification or simulation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Geometry (AREA)
- General Physics & Mathematics (AREA)
- Computer Graphics (AREA)
- Software Systems (AREA)
- Computer Hardware Design (AREA)
- Evolutionary Computation (AREA)
- General Engineering & Computer Science (AREA)
- Processing Or Creating Images (AREA)
Abstract
The invention discloses a kind of video human three-dimensional rebuilding methods and device based on clothes modeling and simulation, wherein, method includes: to acquire human motion using video camera, pass through the posture and posture information of image zooming-out human body, and extract the segmentation information of personage's difference clothing, to model to human body and surface laundry, and physical analogy and textures simulation are carried out for surface laundry according to the athletic posture of human body.Manikin method for reconstructing is based on the matched method of body templates, carries out human body attitude according to the existing method based on single RGB image estimation manikin and figure solves；Clothing simulation mainly utilizes particle simulation method, carries out physical analogy by the way that external force and interior force constraint is added and models.This method can enable the Union Movement of human body and clothing to obtain relatively good reconstruction by physical simulating method, and be suitable for human body and the clothing reconstruction of list RGB acquisition human motion.
Description
Technical field
The present invention relates to technical field of computer vision, in particular to a kind of video human based on clothes modeling and simulation
Three-dimensional rebuilding method and device.
Background technique
Three-dimensional reconstruction is the previous scientific research of mesh and the skill that industry is all paid close attention in this field of computer vision
Art.The model obtained by three-dimensional reconstruction the fields such as video games, architecture, basic industries have it is higher research and it is practical
Value.
However, about human body three-dimensional and clothes method for reconstructing, since human action and clothing material are compared with horn of plenty, it is
One biggish problem in three-dimensional reconstruction field.Existing human body reconstruction technology is largely that human body and clothing is complete as one
Integral mould is rebuild, therefore can not carry out true physical simulation and modeling to clothing.
Summary of the invention
The present invention is directed to solve at least some of the technical problems in related technologies.
For this purpose, an object of the present invention is to provide a kind of video human three-dimensional reconstructions based on clothes modeling and simulation
Method, this method can enable the Union Movement of human body and clothing to obtain relatively good reconstruction by physical simulating method,
And it can be adapted for human body and the clothing reconstruction of single RGB acquisition human motion.
It is another object of the present invention to propose a kind of video human three-dimensional reconstruction dress based on clothes modeling and simulation
It sets.
In order to achieve the above objectives, one aspect of the present invention embodiment proposes a kind of video people based on clothes modeling and simulation
Body three-dimensional rebuilding method, comprising: human body movement data is acquired by single RGB, and preceding background is carried out to the human body movement data
Segmentation, and estimate that the method multi-frame joint of body templates estimates human body attitude and human somatotype by list RGB；According to the human body
Figure models human body initial attitude using body templates, and clothing original two-dimensional cloth simulation is emulated to suture and wear
To with the human body of initial attitude；Clothing parameter is adjusted according to the collision information under external force between clothing and human body, and makes three
Clothing under dimension simulation meets fit condition；Personage's posture transition is transformed in video under the posture of the 1st frame, while to three-dimensional
Clothing carry out joint physical analogy；The segmentation of different clothings, and the clothing based on parameter model are carried out to the present frame of video
The segmentation figure of fitting clothing makes the marginal portion of clothing and present frame clothing in the video under three-dimensional simulation reach identical
Condition, and it is crucial for k frame in the video, clothing parameter is solved by multiframe Advance data quality；According to the clothing parameter
Human body attitude is carried out to frame each in the video with the human body attitude and clothing combined simulation models；Pass through camera projection relation
With the RGB information in original video, texture calculating and textures are carried out to the clothing of modeling, and entire fortune is rendered by illumination again again
Dynamic sequence, obtains body three-dimensional reconstruction result.
The video human three-dimensional rebuilding method based on clothes modeling and simulation of the embodiment of the present invention, can use clothes
Two-dimentional cloth parameter information carries out personage's wearing to clothing and movement is simulated, and is fitted the true picture acquired in video, thus
To the personage and clothing in single rgb video while modeling and simulation can be carried out, and then by physical simulating method, can made
The Union Movement of human body and clothing can obtain relatively good reconstruction, and can be adapted for the human body of single RGB acquisition human motion
It is rebuild with clothing.
In addition, the video human three-dimensional rebuilding method according to the above embodiment of the present invention based on clothes modeling and simulation is also
It can have following additional technical characteristic:
Further, in one embodiment of the invention, wherein the segmentation figure of every clothing is and original video is differentiated
The bianry image of rate same size.
Further, in one embodiment of the invention, parameter adjustment and optimization process in, clothing always with human body into
Row combined simulation.
Further, in one embodiment of the invention, described that clothing parameter, packet are solved by multiframe Advance data quality
It includes: range conversion operation is done to bianry image, find out two norm distances of each pixel to the Edge tracking of binary image, and structure
At image；Threshold process is made to the described image of composition.
Further, in one embodiment of the invention, optimization item is the deviation for rendering image and segmented image:
Wherein,For in parameter xiUnder, the rendering image of i-th clothing,For segmented image；
The calculation formula of Optimal Parameters are as follows:
Wherein, Δ xiFor the independent change amount of each parameter.
In order to achieve the above objectives, another aspect of the present invention embodiment proposes a kind of video based on clothes modeling and simulation
Human body three-dimensional reconstructing device, comprising: acquisition module, for acquiring human body movement data by list RGB, and to the human motion
Background segment before data carry out, and estimate that the method multi-frame joint of body templates estimates human body attitude and body by list RGB
Type；First modeling module, for being modeled using body templates to human body initial attitude according to the human somatotype, and by clothing
The emulation of object original two-dimensional cloth simulation is to suture and through with the human body of initial attitude；Module is adjusted, for according under external force
Collision information between clothing and human body adjusts clothing parameter, and the clothing under three-dimensional simulation is made to meet fit condition；Transformation
Module carries out joint physics for transforming to personage's posture transition in video under the posture of the 1st frame, while to three-dimensional clothing
Simulation；Module is solved, the segmentation of different clothings is carried out for the present frame to video, and the clothing based on parameter model is fitted clothing
The segmentation figure of object makes the marginal portion of present frame clothing in clothing and the video under three-dimensional simulation reach identical condition, and
And it is crucial for k frame in the video, clothing parameter is solved by multiframe Advance data quality；Second modeling module, for according to institute
It states clothing parameter and the human body attitude and human body attitude and the modeling of clothing combined simulation is carried out to frame each in the video；Rebuild mould
Block, for carrying out texture calculating and textures to the clothing of modeling by the RGB information in camera projection relation and original video, and
Entire motion sequence is rendered again by illumination again, obtains body three-dimensional reconstruction result.
The video human three-dimensional reconstruction apparatus based on clothes modeling and simulation of the embodiment of the present invention, can use clothes
Two-dimentional cloth parameter information carries out personage's wearing to clothing and movement is simulated, and is fitted the true picture acquired in video, thus
To the personage and clothing in single rgb video while modeling and simulation can be carried out, and then by physical simulating method, can made
The Union Movement of human body and clothing can obtain relatively good reconstruction, and can be adapted for the human body of single RGB acquisition human motion
It is rebuild with clothing.
In addition, the video human three-dimensional reconstruction apparatus according to the above embodiment of the present invention based on clothes modeling and simulation is also
It can have following additional technical characteristic:
Further, in one embodiment of the invention, wherein the segmentation figure of every clothing is and original video is differentiated
The bianry image of rate same size.
Further, in one embodiment of the invention, parameter adjustment and optimization process in, clothing always with human body into
Row combined simulation.
Further, in one embodiment of the invention, the solution module be further used for doing bianry image away from
From map function, each pixel is found out to two norm distances of the Edge tracking of binary image, and constitutes image, and to composition
Described image makees threshold process.
Further, in one embodiment of the invention, optimization item is the deviation for rendering image and segmented image:
Wherein,For in parameter xiUnder, the rendering image of i-th clothing,For segmented image；
The calculation formula of Optimal Parameters are as follows:
Wherein, Δ xiFor the independent change amount of each parameter.
The additional aspect of the present invention and advantage will be set forth in part in the description, and will partially become from the following description
Obviously, or practice through the invention is recognized.
Detailed description of the invention
Above-mentioned and/or additional aspect and advantage of the invention will become from the following description of the accompanying drawings of embodiments
Obviously and it is readily appreciated that, in which:
Fig. 1 is the process according to the video human three-dimensional rebuilding method based on clothes modeling and simulation of the embodiment of the present invention
Figure；
Fig. 2 is the structure according to the video human three-dimensional reconstruction apparatus based on clothes modeling and simulation of the embodiment of the present invention
Schematic diagram.
Specific embodiment
The embodiment of the present invention is described below in detail, examples of the embodiments are shown in the accompanying drawings, wherein from beginning to end
Same or similar label indicates same or similar element or element with the same or similar functions.Below with reference to attached
The embodiment of figure description is exemplary, it is intended to is used to explain the present invention, and is not considered as limiting the invention.
The video human three based on clothes modeling and simulation proposed according to embodiments of the present invention is described with reference to the accompanying drawings
Tie up method for reconstructing and device, describe to propose according to embodiments of the present invention first with reference to the accompanying drawings based on clothes modeling and simulation
Video human three-dimensional rebuilding method.
Fig. 1 is the process of the video human three-dimensional rebuilding method based on clothes modeling and simulation of one embodiment of the invention
Figure.
As shown in Figure 1, should video human three-dimensional rebuilding method based on clothes modeling and simulation the following steps are included:
In step s101, human body movement data is acquired by single RGB, and preceding background point is carried out to human body movement data
It cuts, and estimates that the method multi-frame joint of body templates estimates human body attitude and human somatotype by list RGB.
It is understood that the embodiment of the present invention can by single RGB acquire human body movement data, and to acquisition data into
Background segment before row, and estimate that the method multi-frame joint of body templates estimates the posture and figure of human body by existing list RGB.
In step s 102, human body initial attitude is modeled using body templates according to human somatotype, and by clothing
The emulation of original two-dimensional cloth simulation is to suture and through with the human body of initial attitude.
It is understood that the embodiment of the present invention can be according to the human somatotype estimated, using body templates to human body
Initial attitude is modeled, and clothing original two-dimensional cloth simulation is emulated to suture and through with the human body of initial attitude.
In step s 103, clothing parameter is adjusted according to the collision information under external force between clothing and human body, and makes three
Clothing under dimension simulation meets fit condition.
It is understood that the embodiment of the present invention can be according to the collision information under external force between clothing and human body, rationally
Clothing parameter is adjusted, and makes the clothing under three-dimensional simulation more fit.
In step S104, personage's posture transition is transformed in video under the posture of the 1st frame, while to three-dimensional clothing
Carry out joint physical analogy.
It is understood that the embodiment of the present invention can transform to personage's posture transition in video under the posture of the 1st frame,
So as to carry out joint physical analogy to three-dimensional clothing in the process.
In step s105, the segmentation of different clothings is carried out to the present frame of video, and the clothing based on parameter model is quasi-
The segmentation figure for closing clothing makes the marginal portion of present frame clothing in clothing and video under three-dimensional simulation reach identical condition, and
And it is crucial for k frame in video, clothing parameter is solved by multiframe Advance data quality.
Wherein, identical condition be understood that be present frame clothing in the clothing and video under three-dimensional simulation marginal portion
It coincide as far as possible, certainly, specifically identical condition can be arranged in those skilled in the art according to the actual situation, do not do herein specific
It limits.
Further, in one embodiment of the invention, wherein the segmentation figure of every clothing is and original video is differentiated
The bianry image of rate same size.
Specifically, (1) carries out different clothing to the video frame, by existing method (such as Look into person etc.)
The segmentation of object, the segmentation figure of every clothing are the bianry image with original video resolution ratio same size.
(2) segmentation figure of the clothing is fitted with the clothing based on parameter model so that the clothing under three-dimensional simulation with
The marginal portion of the frame clothing coincide as far as possible in video.In parameter adjustment and optimization process, clothing is joined with human body always
Molding is quasi-.
(3) crucial for k frame in video, it repeats (1) and (2), clothing parameter is solved by multiframe Advance data quality.
In step s 106, human body attitude and clothing joint are carried out to frame each in video according to clothing parameter and human body attitude
Simulation modelling.
It is understood that each video frame in clothing parameter and step S101 that the embodiment of the present invention passes through solution
In personage's posture for extracting, human body attitude is carried out to frame each in video and clothing combined simulation models.
In step s 107, by the RGB information in camera projection relation and original video, texture is carried out to the clothing of modeling
Calculating and textures, and entire motion sequence is rendered by illumination again again, obtain body three-dimensional reconstruction result.
To sum up, video human three-dimensional reconstruction of the embodiment of the present invention based on clothes modeling and simulation, and RGB and figure is utilized
As segmentation information matches personage and clothing, Three-dimensional Gravity then is carried out to human body and clothing with calculated model parameter
It builds.
Below by by specific embodiment to the video human three-dimensional rebuilding method based on clothes modeling and simulation carry out into
One step illustrates, the specific steps are as follows:
Step S1 acquires the motion sequence of single human body using a RGB camera, it is ensured that each part of personage is (as carried on the back
Portion) it is all collected in video.
Step S2, by each frame picture in the RGB sequenceIt is carried out using the method based on deep learning semantic segmentation
Preceding background segment, to obtain the segmentation figure M of personagei。
Step S3, by personage's segmentation figure MiAnd original imageAs input, the existing list based on deep learning is used
The method of RGB image estimation human body attitude and figure obtains the posture P of the body templates based on skeleton coveringiWith the letter of figure S
Breath.
Step S4, clothes worn to personage carry out rough classification (such as trousers/shorts/skirt), and use corresponding ginseng
Numberization cloth information, models clothing.
Step S5, employment object type S and model initial attitude, model personage and render.Fixing human posture, will
Clothing after modeling is divided into former and later two parts, under physical simulation system, by applying attraction to front and rear part suture,
Clothing after enabling modeling is completely through with human body.
Step S6, to clothing apply gravity and human body collision and internal constraint power, enable emulation after clothing
Enough closer reality.
Step S7, the collision information after detection emulation between clothing and human body, for human body different piece, if detected
The part of article receives to obtain tension, too strong with personage's impact simulation, then relaxes corresponding cloth parameter.
Personage's posture is transitioned into personage's posture P that first frame is estimated in video by step S81On, in this process and under connecing
In the process come, clothing emulation can continue to carry out.
Step S9, to the frame F in video1, different clothing are carried out by the personage position dividing method based on deep learning
The segmentation of object, the segmentation figure of every clothing are the bianry image with original video resolution ratio same size.Assuming that the figure after segmentation
As beingN is worn the clothes the quantity of object by personage.
Step S10 carries out cloth parametric solution: assuming that in parameter xiUnder, (two-value turns to the rendering image of i-th clothing
Preceding background) beAnd segmented image isTarget is the image boundary of both fittings.Concrete form is as follows: concrete form
As follows: being defined as follows operation to bianry image C first isThe operation does range conversion operation to bianry image C first, finds out
Each pixel and constitutes image D to two norm distances of Edge tracking of binary image, does threshold process to image D later:
Therefore, required optimization item is that rendering image and segmented image existDeviation under operation:
Due toWhen there is no explicit expression, therefore doing gradient estimation, x is enabledi←xi+Δxi, obtained by analogy method
It arrives:
By setting Δ xiFor the independent change amount of each parameter, recycle Gauss-Newton alternative manner, can iteration ask
Solve Optimal Parameters xi。
Step S11, to key frame F each in videoj, step S9 and step S10 operation is repeated, according to the clothing of frame each in video
Object segmentation information successive optimization clothing parameter.
Step S12 passes through the personage's posture estimated in the parameter of solving optimization and each frame by step S1, mould
It is quasi- to reconstruct every frame personage and clothes.
Step S13 calculates vertex correspondence on every clothing by the RGB information in camera projection relation and original video
Projected position, and the texture information of the vertex and dough sheet is acquired by RGB information, and optimize according to each frame information progressive updating
Texture.
Step S14 carries out texture mapping to clothing, carries out illumination again by rendering mode, render entire sequence again, complete
At the reconstruction of personage and clothes.
To sum up, the method for the embodiment of the present invention can use video camera acquisition human motion, pass through image zooming-out human body
Posture and posture information, and the segmentation information of personage's difference clothing is extracted, so that human body and surface laundry are modeled, and
Physical analogy and textures simulation are carried out for surface laundry according to the athletic posture of human body.Manikin method for reconstructing is based on human body
The method of template matching carries out human body attitude according to the existing method based on single RGB image estimation manikin and figure is asked
Solution；Clothing simulation mainly utilizes particle simulation method, carries out physical analogy by the way that external force and interior force constraint is added and models.
The video human three-dimensional rebuilding method based on clothes modeling and simulation proposed according to embodiments of the present invention, Ke Yili
With the two-dimentional cloth parameter information of clothes, personage's wearing is carried out to clothing and movement is simulated, and be fitted acquired in video it is true
Image, so as to in single rgb video personage and clothing carry out modeling and simulation simultaneously, and then pass through physical analogy side
Method can enable the Union Movement of human body and clothing to obtain relatively good reconstruction, and can be adapted for single RGB acquisition human body
The human body and clothing of movement are rebuild.
The video human three based on clothes modeling and simulation proposed according to embodiments of the present invention is described referring next to attached drawing
Tie up reconstructing device.
Fig. 2 is the structure of the video human three-dimensional reconstruction apparatus based on clothes modeling and simulation of one embodiment of the invention
Schematic diagram.
As shown in Fig. 2, being somebody's turn to do the video human three-dimensional reconstruction apparatus 10 based on clothes modeling and simulation includes: acquisition module
100, the first modeling module 200, adjustment module 300, conversion module 400, solution module 500, the second modeling module 600 and reconstruction
Module 700.
Wherein, acquisition module 100 be used for by list RGB acquire human body movement data, and to human body movement data progress before
Background segment, and estimate that the method multi-frame joint of body templates estimates human body attitude and human somatotype by list RGB.First modeling
Module 200 is for modeling human body initial attitude using body templates according to human somatotype, and by clothing original two-dimensional cloth
Material analog simulation is to suture and through with the human body of initial attitude.Module 300 is adjusted to be used for according to clothing under external force and human body
Between collision information adjust clothing parameter, and the clothing under three-dimensional simulation is made to meet fit condition.Conversion module 400 is used for
Personage's posture transition is transformed in video under the posture of the 1st frame, while joint physical analogy is carried out to three-dimensional clothing.It solves
Module 500 is used to carry out the present frame of video the segmentation of different clothings, and point of the fitting clothing of the clothing based on parameter model
Cutting figure makes the marginal portion of present frame clothing in clothing and video under three-dimensional simulation reach identical condition, and for video
Middle k frame is crucial, solves clothing parameter by multiframe Advance data quality.Second modeling module 600 is used for according to clothing parameter and human body
Posture carries out human body attitude to frame each in video and clothing combined simulation models.Module 700 is rebuild to be used to project pass by camera
RGB information in system and original video carries out texture calculating and textures to the clothing of modeling, and is rendered again entirely by illumination again
Motion sequence obtains body three-dimensional reconstruction result.The device 10 of the embodiment of the present invention can make human body by physical simulating method
Union Movement with clothing can obtain relatively good reconstruction, and can be adapted for the human body and clothing of single RGB acquisition human motion
Object is rebuild.
Further, in one embodiment of the invention, wherein the segmentation figure of every clothing is and original video is differentiated
The bianry image of rate same size.
Further, in one embodiment of the invention, parameter adjustment and optimization process in, clothing always with human body into
Row combined simulation.
Further, in one embodiment of the invention, module is solved to be further used for doing bianry image distance change
Operation is changed, each pixel is found out to two norm distances of Edge tracking of binary image, and constitutes image, and threshold is made to the image of composition
Value processing.
Further, in one embodiment of the invention, optimization item is the deviation for rendering image and segmented image:
Wherein,For in parameter xiUnder, the rendering image of i-th clothing,For segmented image；
The calculation formula of Optimal Parameters are as follows:
Wherein, Δ xiFor the independent change amount of each parameter.
It should be noted that the aforementioned solution to the video human three-dimensional rebuilding method embodiment based on clothes modeling and simulation
The video human three-dimensional reconstruction apparatus based on clothes modeling and simulation that explanation is also applied for the embodiment is released, it is no longer superfluous herein
It states.
The video human three-dimensional reconstruction apparatus based on clothes modeling and simulation proposed according to embodiments of the present invention, Ke Yili
With the two-dimentional cloth parameter information of clothes, personage's wearing is carried out to clothing and movement is simulated, and be fitted acquired in video it is true
Image, so as to in single rgb video personage and clothing carry out modeling and simulation simultaneously, and then pass through physical analogy side
Method can enable the Union Movement of human body and clothing to obtain relatively good reconstruction, and can be adapted for single RGB acquisition human body
The human body and clothing of movement are rebuild.
In addition, term " first ", " second " are used for descriptive purposes only and cannot be understood as indicating or suggesting relative importance
Or implicitly indicate the quantity of indicated technical characteristic.Define " first " as a result, the feature of " second " can be expressed or
Implicitly include at least one this feature.In the description of the present invention, the meaning of " plurality " is at least two, such as two, three
It is a etc., unless otherwise specifically defined.
In the description of this specification, reference term " one embodiment ", " some embodiments ", " example ", " specifically show
The description of example " or " some examples " etc. means specific features, structure, material or spy described in conjunction with this embodiment or example
Point is included at least one embodiment or example of the invention.In the present specification, schematic expression of the above terms are not
It must be directed to identical embodiment or example.Moreover, particular features, structures, materials, or characteristics described can be in office
It can be combined in any suitable manner in one or more embodiment or examples.In addition, without conflicting with each other, the skill of this field
Art personnel can tie the feature of different embodiments or examples described in this specification and different embodiments or examples
It closes and combines.
Although the embodiments of the present invention has been shown and described above, it is to be understood that above-described embodiment is example
Property, it is not considered as limiting the invention, those skilled in the art within the scope of the invention can be to above-mentioned
Embodiment is changed, modifies, replacement and variant.
Claims (10)
1. a kind of video human three-dimensional rebuilding method based on clothes modeling and simulation characterized by comprising
Human body movement data is acquired by single RGB, and preceding background segment is carried out to the human body movement data, and pass through list RGB
Estimate the method multi-frame joint estimation human body attitude and human somatotype of body templates；
Human body initial attitude is modeled using body templates according to the human somatotype, and by clothing original two-dimensional cloth mould
Quasi- emulation is to suture and through with the human body of initial attitude；
Clothing parameter is adjusted according to the collision information under external force between clothing and human body, and the clothing under three-dimensional simulation is met
Fit condition；
Personage's posture transition is transformed in video under the posture of the 1st frame, while joint physical analogy is carried out to three-dimensional clothing；
The segmentation of different clothings is carried out to the present frame of video, and the segmentation figure of the fitting clothing of the clothing based on parameter model makes
The marginal portion of present frame clothing reaches identical condition in clothing and the video under three-dimensional simulation, and for the video
Middle k frame is crucial, solves clothing parameter by multiframe Advance data quality；
Human body attitude and clothing combined simulation are carried out to frame each in the video according to the clothing parameter and the human body attitude
Modeling；And
By the RGB information in camera projection relation and original video, texture calculating and textures are carried out to the clothing of modeling, and pass through
Illumination renders entire motion sequence again again, obtains body three-dimensional reconstruction result.
2. the method according to claim 1, wherein wherein, the segmentation figure of every clothing is and original video divides
The bianry image of resolution same size.
3. the method according to claim 1, wherein clothing is always and human body in parameter adjustment and optimization process
Carry out combined simulation.
4. the method according to claim 1, wherein described solve clothing parameter, packet by multiframe Advance data quality
It includes:
Range conversion operation is done to bianry image, find out each pixel to the Edge tracking of binary image two norm distances, and
Constitute image；
Threshold process is made to the described image of composition.
5. according to the method described in claim 4, it is characterized in that, optimization item is the deviation for rendering image and segmented image:
Wherein,For in parameter xiUnder, the rendering image of i-th clothing,For segmented image；
The calculation formula of Optimal Parameters are as follows:
Wherein, Δ xiFor the independent change amount of each parameter.
6. a kind of video human three-dimensional reconstruction apparatus based on clothes modeling and simulation characterized by comprising
Acquisition module for acquiring human body movement data by list RGB, and carries out preceding background point to the human body movement data
It cuts, and estimates that the method multi-frame joint of body templates estimates human body attitude and human somatotype by list RGB；
First modeling module, for being modeled using body templates to human body initial attitude according to the human somatotype, and will
The emulation of clothing original two-dimensional cloth simulation is to suture and through with the human body of initial attitude；
Module is adjusted, for adjusting clothing parameter according to the collision information under external force between clothing and human body, and makes three-dimensional mould
Clothing under quasi- meets fit condition；
Conversion module is carried out for transforming to personage's posture transition in video under the posture of the 1st frame, while to three-dimensional clothing
Joint physical analogy；
Module is solved, the segmentation of different clothings is carried out for the present frame to video, and the clothing based on parameter model is fitted clothing
The segmentation figure of object makes the marginal portion of present frame clothing in clothing and the video under three-dimensional simulation reach identical condition, and
And it is crucial for k frame in the video, clothing parameter is solved by multiframe Advance data quality；
Second modeling module, for carrying out human body appearance to frame each in the video according to the clothing parameter and the human body attitude
State and the modeling of clothing combined simulation；And
Module is rebuild, for carrying out texture calculating to the clothing of modeling by the RGB information in camera projection relation and original video
And textures, and entire motion sequence is rendered by illumination again again, obtain body three-dimensional reconstruction result.
7. device according to claim 6, which is characterized in that wherein, the segmentation figure of every clothing is and original video divides
The bianry image of resolution same size.
8. device according to claim 6, which is characterized in that in parameter adjustment and optimization process, clothing is always and human body
Carry out combined simulation.
9. device according to claim 6, which is characterized in that the solution module be further used for doing bianry image away from
From map function, each pixel is found out to two norm distances of the Edge tracking of binary image, and constitutes image, and to composition
Described image makees threshold process.
10. device according to claim 9, which is characterized in that optimization item is the deviation for rendering image and segmented image:
Wherein,For in parameter xiUnder, the rendering image of i-th clothing,For segmented image；
The calculation formula of Optimal Parameters are as follows:
Wherein, Δ xiFor the independent change amount of each parameter.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910507845.2A CN110309554B (en) | 2019-06-12 | 2019-06-12 | Video human body three-dimensional reconstruction method and device based on garment modeling and simulation |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910507845.2A CN110309554B (en) | 2019-06-12 | 2019-06-12 | Video human body three-dimensional reconstruction method and device based on garment modeling and simulation |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110309554A true CN110309554A (en) | 2019-10-08 |
| CN110309554B CN110309554B (en) | 2021-01-15 |
Family
ID=68076411
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910507845.2A Expired - Fee Related CN110309554B (en) | 2019-06-12 | 2019-06-12 | Video human body three-dimensional reconstruction method and device based on garment modeling and simulation |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110309554B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115375823A (en) * | 2022-10-21 | 2022-11-22 | 北京百度网讯科技有限公司 | Three-dimensional virtual clothing generation method, device, equipment and storage medium |
Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR100967064B1 (en) * | 2007-10-24 | 2010-06-29 | 아주대학교산학협력단 | A Method for Distinction of Color Similarity for Clothes in Varying Illumination and Security System of Public Entrance Area based on Clothes Similarity |
| CN103955963A (en) * | 2014-04-30 | 2014-07-30 | 崔岩 | Digital human body three-dimensional reconstruction method and system based on Kinect device |
| CN104778736A (en) * | 2015-04-03 | 2015-07-15 | 北京航空航天大学 | Three-dimensional garment animation generation method driven by single video content |
| CN107204025A (en) * | 2017-04-18 | 2017-09-26 | 华北电力大学 | The adaptive clothing cartoon modeling method that view-based access control model is perceived |
| CN107924532A (en) * | 2015-08-10 | 2018-04-17 | 立体丈量公司 | Method and apparatus for the description for providing dress form |
| CN108053469A (en) * | 2017-12-26 | 2018-05-18 | 清华大学 | Complicated dynamic scene human body three-dimensional method for reconstructing and device under various visual angles camera |
| CN109102561A (en) * | 2018-07-13 | 2018-12-28 | 浙江百先得服饰有限公司 | A kind of 3D hybrid rending method online based on clothes |
| CN109377564A (en) * | 2018-09-30 | 2019-02-22 | 清华大学 | Virtual fit method and device based on monocular depth camera |
| EP3479296A1 (en) * | 2016-08-10 | 2019-05-08 | Zeekit Online Shopping Ltd. | System, device, and method of virtual dressing utilizing image processing, machine learning, and computer vision |
| CN109767487A (en) * | 2019-01-04 | 2019-05-17 | 北京达佳互联信息技术有限公司 | Face three-dimensional rebuilding method, device, electronic equipment and storage medium |
-
2019
- 2019-06-12 CN CN201910507845.2A patent/CN110309554B/en not_active Expired - Fee Related
Patent Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR100967064B1 (en) * | 2007-10-24 | 2010-06-29 | 아주대학교산학협력단 | A Method for Distinction of Color Similarity for Clothes in Varying Illumination and Security System of Public Entrance Area based on Clothes Similarity |
| CN103955963A (en) * | 2014-04-30 | 2014-07-30 | 崔岩 | Digital human body three-dimensional reconstruction method and system based on Kinect device |
| CN104778736A (en) * | 2015-04-03 | 2015-07-15 | 北京航空航天大学 | Three-dimensional garment animation generation method driven by single video content |
| CN107924532A (en) * | 2015-08-10 | 2018-04-17 | 立体丈量公司 | Method and apparatus for the description for providing dress form |
| EP3479296A1 (en) * | 2016-08-10 | 2019-05-08 | Zeekit Online Shopping Ltd. | System, device, and method of virtual dressing utilizing image processing, machine learning, and computer vision |
| CN107204025A (en) * | 2017-04-18 | 2017-09-26 | 华北电力大学 | The adaptive clothing cartoon modeling method that view-based access control model is perceived |
| CN108053469A (en) * | 2017-12-26 | 2018-05-18 | 清华大学 | Complicated dynamic scene human body three-dimensional method for reconstructing and device under various visual angles camera |
| CN109102561A (en) * | 2018-07-13 | 2018-12-28 | 浙江百先得服饰有限公司 | A kind of 3D hybrid rending method online based on clothes |
| CN109377564A (en) * | 2018-09-30 | 2019-02-22 | 清华大学 | Virtual fit method and device based on monocular depth camera |
| CN109767487A (en) * | 2019-01-04 | 2019-05-17 | 北京达佳互联信息技术有限公司 | Face three-dimensional rebuilding method, device, electronic equipment and storage medium |
Non-Patent Citations (3)
| Title |
|---|
| TAO YU等: "DoubleFusion: Real-time Capture of Human Performances with Inner Body Shapes from a Single Depth Sensor", 《IEEE XPLORE》 * |
| TAO YU等: "Video-Based Outdoor Human Reconstruction", 《IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY》 * |
| 胡鹏鹏: "基于RGBD深度相机的三维扫描***开发与多重着装仿真算法研究", 《中国博士学位论文全文数据库 信息科技辑》 * |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115375823A (en) * | 2022-10-21 | 2022-11-22 | 北京百度网讯科技有限公司 | Three-dimensional virtual clothing generation method, device, equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110309554B (en) | 2021-01-15 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Yang et al. | Physics-inspired garment recovery from a single-view image | |
| Cheng et al. | Parametric modeling of 3D human body shape—A survey | |
| Hasler et al. | Multilinear pose and body shape estimation of dressed subjects from image sets | |
| Vlasic et al. | Articulated mesh animation from multi-view silhouettes | |
| Ersotelos et al. | Building highly realistic facial modeling and animation: a survey | |
| CN106600626B (en) | Three-dimensional human motion capture method and system | |
| CN110310319B (en) | Illumination-separated single-view human body clothing geometric detail reconstruction method and device | |
| US20140333614A1 (en) | System and method for simulating realistic clothing | |
| Huang et al. | Human-centric design personalization of 3D glasses frame in markerless augmented reality | |
| CN107578323B (en) | Three-dimensional real human body online virtual fitting method | |
| Neophytou et al. | A layered model of human body and garment deformation | |
| CN100386778C (en) | Human face image age changing method based on average face and senile proportional image | |
| CN103268623A (en) | Static human face expression synthesizing method based on frequency domain analysis | |
| WO2014081394A1 (en) | Method, apparatus and system for virtual clothes modelling | |
| US20210375045A1 (en) | System and method for reconstructing a 3d human body under clothing | |
| Jiang et al. | Transferring and fitting fixed-sized garments onto bodies of various dimensions and postures | |
| CN108363973A (en) | A kind of unconfined 3D expressions moving method | |
| CN105869217A (en) | Virtual method for trying on clothes by real person | |
| CN111950430A (en) | Color texture based multi-scale makeup style difference measurement and migration method and system | |
| WO2020104990A1 (en) | Virtually trying cloths & accessories on body model | |
| CN114693570A (en) | Human body model image fusion processing method, device and storage medium | |
| De Luigi et al. | Drapenet: Garment generation and self-supervised draping | |
| CN110309554A (en) | Video human three-dimensional rebuilding method and device based on clothes modeling and simulation | |
| JP6818219B1 (en) | 3D avatar generator, 3D avatar generation method and 3D avatar generation program | |
| Straka et al. | Rapid skin: estimating the 3D human pose and shape in real-time |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20210115 |