US11087517B2 - Sketch-based abstraction for character posing and synthesis - Google Patents
Sketch-based abstraction for character posing and synthesis Download PDFInfo
- Publication number
- US11087517B2 US11087517B2 US15/172,078 US201615172078A US11087517B2 US 11087517 B2 US11087517 B2 US 11087517B2 US 201615172078 A US201615172078 A US 201615172078A US 11087517 B2 US11087517 B2 US 11087517B2
- Authority
- US
- United States
- Prior art keywords
- representation
- character
- sketch
- deformation
- sketch input
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T13/00—Animation
- G06T13/20—3D [Three Dimensional] animation
- G06T13/40—3D [Three Dimensional] animation of characters, e.g. humans, animals or virtual beings
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T11/00—2D [Two Dimensional] image generation
- G06T11/20—Drawing from basic elements, e.g. lines or circles
- G06T11/203—Drawing of straight lines or curves
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T11/00—2D [Two Dimensional] image generation
- G06T11/60—Editing figures and text; Combining figures or text
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T13/00—Animation
- G06T13/80—2D [Two Dimensional] animation, e.g. using sprites
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T19/00—Manipulating 3D models or images for computer graphics
- G06T19/20—Editing of 3D images, e.g. changing shapes or colours, aligning objects or positioning parts
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2219/00—Indexing scheme for manipulating 3D models or images for computer graphics
- G06T2219/20—Indexing scheme for editing of 3D models
- G06T2219/2021—Shape modification
Definitions
- This disclosure generally relates to animation tools.
- posing a character may be a more involved endeavor, since it may entails the coordinated movement of thousands or more vertices.
- rigging artists may build rigs that may define the space of meaningful deformations for a character in terms of abstract rigging parameters (e.g., a full character rig or a rig for a portion of a character).
- animators may determine a character's pose indirectly by choosing values for these parameters.
- Rigging is the process of taking a static 3D mesh and preparing it for animation. This process involves creating a hierarchical set of interconnected “bones” (referred to as the skeleton or character rig) which is then used to animate the mesh. Additionally, rigging a character may involve placement of the character rig (or the skeleton), creation of a joint hierarchy, forward kinematics, inverse kinematics, defining degrees of freedom and constraints, and/or facial rigging. In reference to the joint hierarchy, in order for a character rig to function properly the joints must follow a logical hierarchy. For example, the first joint created in the character rig is called the joint root and thereafter every subsequent joint may be connected directly or indirectly to the joint root.
- Forward kinematics refers to one of the basic ways to calculate the joint movement of a fully rigged character. For example, moving a character's hand may involve the movement of not only the character's hand, but also the character's shoulder, elbow, etc. With inverse kinematics (IK), degrees of freedom and constraints are used to ensure that a characters rig stays within the bounds of nature, that is, not allowing a character head to rotate a full 360 degrees. Lastly, facial rigging may include an entirely separate rig defined solely for the face of a character.
- a sketch-based abstraction method may enable sketch-based posing of 3D characters or the design and customization of 3D characters.
- a method augments a 3D character with a 2D representation and provides a general-purpose energy formulation that measures the distance between the character's 2D representation and a user-drawn 2D sketched curve.
- the system may provide a bridge between 2D sketches and the 3D model deformed by the same rig parameters.
- the method may be applied to different posing tasks and may also be used to create customized character designs.
- the method may be applied to mechanical assemblies.
- the character rig's subspace may include the entire character rig as defined by the initial rigging of the character.
- the 2D representation may be created on the fly by sketching on top of the character's mesh. In other embodiments, the 2D representation may be built directly into the character rig as a set of rigged curves during character design.
- the character rig describes all of the rigging parameters and features of a particular character.
- the algorithm may minimize the energy using the character's rigging controls as unknowns. Since the rigging may influence both the 3D shape and the 2D representation, the algorithm provides a bridge between the 2D world of sketching and the character's 3D rigging controls. By minimizing the energy metric, the described embodiment may deform the 3D character. Therefore, the character rig may be treated as a black-box system, that is, in particular embodiments, the method may enable arbitrary rigging controls, including but not limited to, skeletal deformations, blend shapes, and arbitrary nonlinear procedural functions.
- Applications of the method may include draw-over posing for an entire character or for components thereof, creation of characters and character components, and manipulation of non-intuitive subspaces defined by the constraints of mechanical assemblies.
- Embodiments disclosed above are only examples, and the scope of this disclosure is not limited to them. Particular embodiments may include all, some, or none of the components, elements, features, functions, operations, or steps of the embodiments disclosed above.
- Embodiments according to the invention are in particular disclosed in the attached claims directed to a method, a storage medium, a system and a computer program product, wherein any feature mentioned in one claim category, e.g. method, can be claimed in another claim category, e.g. system, as well.
- the dependencies or references back in the attached claims are chosen for formal reasons only.
- any subject matter resulting from a deliberate reference back to any previous claims can be claimed as well, so that any combination of claims and the features thereof are disclosed and can be claimed regardless of the dependencies chosen in the attached claims.
- the subject-matter which can be claimed comprises not only the combinations of features as set out in the attached claims but also any other combination of features in the claims, wherein each feature mentioned in the claims can be combined with any other feature or combination of other features in the claims.
- any of the embodiments and features described or depicted herein can be claimed in a separate claim and/or in any combination with any embodiment or feature described or depicted herein or with any of the features of the attached claims.
- FIG. 1 illustrates an example method of a sketch-based abstraction on top of arbitrary rigs.
- FIG. 2A illustrates an example of the draw-over posing embodiment using a production quality character rig based on a skeletal structure focusing on the region of the head.
- FIG. 2B illustrates an example of the draw-over posing embodiment using a production quality character rig based on a skeletal structure focusing on the right leg of the elephant.
- FIG. 2C illustrates an example of the draw-over posing embodiment using a production quality character rig based on a skeletal structure focusing on the left leg of the elephant.
- FIG. 2D illustrates an example of the draw-over posing embodiment using a production quality character rig based on a skeletal structure focusing on the tail of the elephant.
- FIG. 3 illustrates an example of face-posing using the draw-over posing embodiment using a facial rig based on a large set of blend shapes.
- FIG. 4 illustrates an example of redraw posing by embedding the 2D representation.
- FIG. 5 illustrates an animation created using redraw posing.
- FIG. 6 illustrates an example of sketch-based character individualization.
- FIG. 7 illustrates an example of the different results achieved through sketch-based character individualization.
- FIG. 8 illustrates a mechanical leg and its corresponding 2D representation with accompanying motion curve.
- FIG. 9 illustrates an example method of creating mechanical assemblies.
- FIG. 10 illustrates an example of making fine-scale adjustments to a character rig.
- FIG. 11A illustrates a flowchart for an example method of sketch-based posing of 3D characters.
- FIG. 11B illustrates a flowchart for an example method of design and customization of 3D characters.
- FIG. 12 illustrates an example computer system.
- Particular embodiments provide sketch-based abstraction method that works on top of arbitrary rigs, allowing sketch-based posing and/or the design and customization of novel creatures.
- the embodiment described herein relates to a sketch-based abstraction on top of artist-designed subspaces.
- embodiments described herein may focus on 3D character models created and rigged using standard animation software.
- sketch-based abstraction method For purposes of illustration, multiple examples of the sketch-based abstraction method are disclosed. These examples include, but are not limited to, draw-over posing, redraw posing, sketch-based character individualization, and sketch-based design of mechanical characters.
- the term “sketch” may refer to a stick-figure-style drawing of a character or portions thereof; the sketch may be received as user input by way of any appropriate input device, such as, for example, an electronic, mechanical, or capacitive stylus used with a touch pad/touch screen.
- a user may draw a sketch on a character (either the entire character or a portion thereof) and then redraw the sketch in the desired posed position.
- the user's first drawn sketch shall be referred to as the source sketch and the user's redrawn sketch shall be referred to as the target sketch.
- the character's rig may be automatically updated to optimally match the two sketches.
- Draw-over posing may be used for, by way of example but not limitation, skeletal posing, curve deformers, facial blend shapes, and other more elaborate deforms that are not typically accommodated by inverse kinematics systems.
- a user may construct the 2D character representation by embedding curves in the character's structure and rigging them alongside the character's mesh. Therefore, at runtime a user need only draw the target sketch.
- the sketch-based abstraction may be applied to individual components of a character, wherein the individual components may be authored separately.
- a user may create a variety of custom characters by sketching individual components such as body segments, legs, heads, wings, and tails. Because the user may have the ability to define the appropriate 2D abstraction for each component, the 2D representation may be much simpler than the actual 3D shape. Therefore, a simple pictorial sketch may result in a complex character design.
- particular embodiments may provide methods of manipulating non-intuitive subspaces defined by the constraints of mechanical assemblies.
- FIG. 1 illustrates an example pipeline for a method for creating and utilizing a sketch-based abstraction on top of a rig.
- the rig may have a broad range of expressive deformations and may employ hundreds or thousands of rigging controls, which may vary in complexity from blend shapes to skeletal deformations to complex procedural functions.
- a 3D model's vertices may define the space of all possible deformations, while the rig defines the subspace of meaningful ones.
- the corresponding rig may employ a range of controls, including, but not limited to, skeletal kinematics, blend shapes, or arbitrary nonlinear procedural deformations.
- the system may treat the rig as a black box, taking the character mesh and a list of rig parameters as input.
- Input phase 100 may comprise creation of an asset, in which a rig (parameterized 3D model 120 ) forms the basis for creating a design subspace 130 (2D representation of the rig) by projection into the camera's viewing plane.
- a rig parametrimeterized 3D model 120
- the system may enhance the classical rig parameterization with a 2D representation of the model that may be deformed by the same controls as the surface mesh. This extended mapping may be referred to herein as a “design subspace.”
- a matching energy optimization may be performed (step 150 ), in which a correspondence between the 2D sketch 140 and the 2D representation of the rig is determined.
- the optimized parameters for the 2D representation are then applied to the rig, resulting in posed 3D model 160 .
- the rig may define an abstract mapping from a set of rig parameters p to a corresponding surface deformation x(p).
- Enhancing the rig with a 2D representation of the model 120 consisting of k points z (z 2 ; z 2 ; . . . ; z k )
- the system may generate the extended mapping: p ⁇ x(p), z(p) ⁇ .
- the specific choice of mapping from the parameters p to the 2D points z may depend on the application.
- the user may then sketch input to achieve a new pose.
- a user may first draw the source sketch, indicating the region of the character to move, and then draw the target sketch, indicating where the user would like the target sketch to move to in order to achieve a new pose.
- a user may not need to first sketch on the character, the user may build a 2D representation directly into the rig. This procedure allows the user to define a “2D language” for posing.
- the system may automatically determine the optimal pose in response to the user-specified 2D representation.
- the 2D representation may be created on the fly by drawing a curve onto the surface of the character which is then carried along as the character deforms.
- the user may embed a set of curves into the character itself and may rig them alongside the character's mesh.
- a projection into the camera's viewing plane may yield a 2D representation of the character's current pose.
- the system may connect these two representations via the rig, allowing the system to control the character's 3D shape by minimizing a 2D energy based on sketch matching.
- the system may define an optimization problem in the form of a nonlinear iterative closest point (hereinafter “ICP”) objective that may attempt to align the character's 2D representation to match the user defined sketch.
- ICP nonlinear iterative closest point
- optimization may be defined over the rig parameters, thereby minimizing the ICP energy that compares 2D objects. Minimizing the ICP energy that compares 2D objects may also deform the 3D shape to match the sketch.
- An application-dependent regularization term may be used to resolve ambiguities in a way appropriate for each particular application.
- ⁇ i,j denotes the correspondence weight between points y i and z j that associates them with each other.
- the system may then alternate between fixing p (and thus z) to compute correspondence weights ⁇ , and fixing ⁇ to optimize for p.
- a selection of one of two correspondence procedures may be predefined during the asset creation phase.
- the first correspondence procedure may perform an arc-length parameterized resampling using the same number of sample points for both user sketch 140 and the model's 2D curve representation.
- One-to-one correspondences between y and z may be established for ⁇ i,j by considering the drawing direction of both lines.
- the second correspondence procedure may be selected for more complicated gestures, and it may be based on equidistant sampling of both user sketch 140 and the model part's 2D representation. This may result in sets of points of different sizes. Therefore, to handle the multiple sets of different points, computing many-to-many correspondences between y and z to minimize W match with respect to ⁇ may be required using a greedy two-step approach.
- the first step may require computing one-to-many correspondences from y to z that may be called forward correspondences a i,j ⁇ 0, 1 ⁇ by assigning the closest match z to every point y i .
- the system may minimize the matching energy 150 W match by modifying the parameters p that control the 2D representation points z of design subspace 130 .
- the optimization problem given that the correspondence weights ⁇ are fixed, the optimization problem:
- performing the matching energy optimization 150 in terms of a 2D representation may lead to an under constrained problem, since potentially many subspace parameter configurations p—originally deforming the surface points x—may map to the same 2D point set z. In numerical terms, this may manifest as a large number of local minima in W match , which may lead to unpredictable behavior and high sensitivity of the solution to changes in the initial parameter vector p 0 . In addition, many of these local minima may express themselves as solutions where the surface representation x in 3D is highly distorted even though the 2D points z match well. To prevent such artifacts, a regularization energy component ⁇ W reg may be added to the optimization problem in Eq. 4.
- a parameter-based L2 regularizer ⁇ p ⁇ p 0 ⁇ 2 may be used to favor solutions that may require the least amount of change from the initial pose.
- employing a vertex-based regularization may be used depending on x.
- Parameter-based regularization may be inexpensive to include, whereas vertex-based regularization comes at a higher cost.
- the derivatives may need to be adjusted by adding
- J x ⁇ x ⁇ p additionally needs to be estimated using finite differences.
- FIGS. 2A-2D and FIG. 3 illustrate an example of the draw-over posing embodiment.
- FIGS. 2A-2D illustrate an example of an embodiment of the method applied to a rig for a character (Elephant), which is a production quality character rig based on a skeletal structure.
- FIG. 3 illustrates the facial rig for a character (Mike), which is based on a large set of blend shapes that have been sculpted individually.
- draw-over posing targets on-the-fly sketch-based posing of an arbitrarily rigged character.
- the user may create a custom 2D representation on the fly by placing a stroke on the character.
- the system may store the corresponding UV values and may use them to reconstruct the curve after deformation by the rig.
- projecting the drawn curve into the camera plane yields its 2D representation.
- the system may optimize the rig's parameters to make the two curves match.
- FIG. 2A illustrates an example of the draw-over posing embodiment for initial pose 210 , focusing on Elephant's head 210 A region.
- the solid line represents source sketch 210 B, which is the first sketch a user may draw.
- Source sketch 210 B indicates to the system the region of the character rig that the user wishes to move to a new position.
- the dashed line represents target sketch 210 B, which is the second sketch a user may draw.
- Target sketch 210 C indicates to the system the position the region selected by source sketch 210 B is to be moved toward.
- Elephant's head 210 A has been moved back a small amount to final pose 220 as a result of source sketch 210 B and target sketch 210 C.
- final pose 220 may alter the entire representation of the elephant, since tilting back Elephant's head 210 A affects multiple portions of the character rig.
- FIG. 2B illustrates an example of the draw-over posing embodiment for initial pose 220 , focusing on Elephant's right leg 230 A region.
- the solid line represents source sketch 230 B, which is the first sketch a user may draw.
- Source sketch 230 B indicates to the system the region of the character rig that the user wishes to move to a new position.
- the dashed line represents target sketch 230 B, which is the second sketch a user may draw.
- Target sketch 230 C indicates to the system the position the region selected by source sketch 230 B is to be moved toward.
- Elephant's right leg 230 A has been lifted up and toward the right to final pose 240 as a result of source sketch 230 B and target sketch 230 C.
- final pose 240 may alter the entire representation of the elephant as lifting Elephant's right leg 230 A affects multiple portions of the character rig.
- FIG. 2C illustrates an example of the draw-over posing embodiment for initial pose 250 , focusing on Elephant's left leg 250 A region.
- the solid line represents source sketch 250 B, which is the first sketch a user may draw.
- Source sketch 250 B indicates to the system the region of the character rig that the user wishes to move to a new position.
- the dashed line represents target sketch 230 B, which is the second sketch a user may draw.
- Target sketch 250 C indicates to the system the position the region selected by source sketch 250 B is to be moved toward.
- Elephant's left leg 250 A has been lifted up to final pose 260 as a result of source sketch 250 B and target sketch 250 C. Additionally, final pose 260 may alter the entire representation of the elephant as lifting Elephant's left leg 250 A affects multiple portions of the character rig.
- FIG. 2D illustrates an example of the draw-over posing embodiment for initial pose 270 , focusing on Elephant's tail 270 A region.
- the solid line represents source sketch 270 B, which is the first sketch a user may draw.
- Source sketch 270 B indicates to the system the region of the character rig that the user wishes to move to a new position.
- the dashed line represents target sketch 230 B, which is the second sketch a user may draw.
- Target sketch 270 C indicates to the system the position the region selected by source sketch 270 B is to be moved toward.
- Elephant's tail 270 A has been raised to final pose 280 as a result of source sketch 270 B and target sketch 270 C.
- final pose 280 may alter the entire representation of the elephant as lifting Elephant's tail 270 A affects multiple portions of the character rig as illustrated FIG. 2D . For example, by lifting Elephant's tail, the back is now arched and the belly is protruding.
- FIG. 3 illustrates an example of face-posing using the draw-over posing embodiment using a facial rig for a character (Mike) based on a large set of blend shapes.
- initial facial expression 300 A is manipulated as shown in sketch compilation 300 B, thereby resulting in final facial expression 300 C.
- the initial poses for right eyebrow 305 and left eyebrow 310 are represented by solid sketch lines 305 A and 310 A, which are subsequently drawn over by dashed sketch lines 305 B (raising the right eyebrow to final pose 305 C) and 310 B (dropping the left eyebrow to final pose 310 C).
- the initial poses for right eyelid 315 and left eyelid 320 are represented by solid sketch lines 315 A and 320 A, which are subsequently drawn over by dashed sketch lines 315 B (dropping the right eyelid to final pose 315 C) and 320 B (dropping the left eyelid to final pose 320 C).
- the initial pose for mouth and mustache 325 is represented by solid sketch line 325 A, which is subsequently drawn over by dashed sketch line 325 B (tweaking the mouth and mustache into final pose 325 C).
- small local deformations are preferred over global deformations.
- local deformations may be favored by penalizing deformations in regions far away from the curve that is used to generate the 2D representation.
- the system may optionally further improve the quality of the posing by adding a surface-based physical shell energy as a regularizer.
- static energy components of the discrete shells integrate directly into the system and may prevent surface distortions.
- a user may construct the 2D character representation by embedding curves in the character structure and rigging them alongside the character's mesh; in particular embodiments, this may eliminate the need to draw source sketch lines. Projecting these curves into the camera plane yields the 2D representation.
- the user may be required to draw the different curves for the target sketch in a prescribed order in order to facilitate the correspondence computation.
- z i 0 denotes the vertex position i prior to optimization.
- the physical shell regularization energy described above may be used to optionally improve the posing quality.
- FIG. 4 illustrates several expressive poses that may be derived from stick-figure sketches; as shown in FIG. 4 , rest pose 400 A and embedded 2D representation 400 B may be transformed into five different poses using sketch input. Because the 2D representation has previously been embedded into the character structure and rigged alongside the character's mesh, a user need only to draw the target sketch. For example, a user may want to display the figure in a seated position—to move the figure from rest pose 400 A to seated pose 410 A the user may draw sketch 410 B. If the user wants to display the figure in running pose 420 A, the user may draw sketch 420 B. Similarly, the user may pose the figure in pre-dive pose 430 A, striding pose 440 A, or lunging pose 450 A by drawing the corresponding sketch ( 430 B, 440 B, or 450 B, respectively).
- FIG. 5 to validate the redraw posing operation, a Cartoon Man character is used which includes embedded line representations for all body parts.
- FIG. 5 depicts an animation 500 of the Cartoon Man falling backward.
- the Cartoon Man starts in rest pose 510 and after eight sketch inputs transforming 2D representation 515 , the Cartoon Man has fallen backward and landed on his back in final pose 520 .
- a user may make fine-scale adjustments to a rig as shown in FIG. 6 .
- a user may first select a specific location on or region of the character rig to manipulate (e.g., the character's neck).
- the system may display a control element (e.g., control wheel 610 ), enabling the user to manipulate the rig into the desired position (e.g., by turning control wheel 610 to cause the head and neck to tilt).
- a control element e.g., control wheel 610
- the system may enable sketch-based character individualization by allowing users to design a virtual character based on simple sketches and predefined adaptive model parts that may expose rig controls that affect the shape in addition to the pose.
- Sketch-based character individualization enables the user to design individual character parts offline, which can then be easily combined and posed interactively using sketch input as illustrated in FIG. 6 .
- FIG. 7 illustrates an example application of sketch-based character individualization for a character comprising parts of a dragon model that have been rigged: torso 710 A, wings 705 A, head 715 A, tail 720 A, front legs 740 A and hind legs 730 A.
- each part of the dragon model also contains an embedded stroke gesture that serves as the 2D representation to be deformed by the rig and as an identifier to help determine classification of the drawn part.
- a user must sketch oval 710 B.
- a user in order to instantiate wing 705 A, head 715 A, tail 720 A, front legs 740 A, or hind legs 730 A, a user must sketch triangle 705 B, head 715 B, tail 720 B, front legs 740 B, or hind legs 730 B, respectively, as shown in FIG. 7 .
- the rigs may expose the local scaling parameters of the underlying bones used to skin the models in addition to typical posing controls such as rigid transformation and joint rotations.
- typical posing controls such as rigid transformation and joint rotations.
- the system may use a database of 2D sketches containing several example instances of each body part together with a category recognition approach to classify, and thereby detect, each drawn stroke.
- the system then may instantiate the detected part and optimize it for both its shape and pose parameters (in some embodiments, simultaneously) to place it into the scene.
- FIG. 8 illustrates results using a character individualization method as described herein.
- a user in order to create a torso part, a user may be able to sketch a large oval ( 810 A), small oval ( 810 B), or an elongated and saggy oval ( 810 C), which are detected, instantiated, and then optimized as a large torso ( 820 A), a small torso ( 820 B), or an elongated and saggy torso ( 820 C). Also, as shown in FIG.
- a user may be able to sketch a variable number of foreleg sketches ( 830 A- 830 C) or hindleg sketches ( 850 A- 850 C), which are detected, instantiated, and then optimized in the appropriate number, shape, and pose as foreleg parts 840 A- 840 C and hindleg parts 860 A- 860 C; as can be seen in the middle character in FIG. 8 , in particular embodiments, sketches of a single foreleg 830 B and a single hindleg 850 B may be instantiated as a pair of foreleg parts 840 B and a pair of hindleg parts 860 B. As further shown in FIG. 8 , sketches 870 A- 870 C may be similarly detected, instantiated, and then optimized in the appropriate number, shape, and pose as wings 880 A- 880 C.
- the user may either redraw any stroke and repeat the fitting of the corresponding part, or continue to add new parts to the character.
- the design spaces explored by the method may either be explicitly provided by artists through parametrized rigs, or they can be implicitly defined.
- both the mechanical structure and the motions of a complex mechanical character may be specified using techniques as described herein.
- Such mechanical assemblies may be modeled using a collection of components and constraints.
- the vector s aggregates the states of all components in a mechanical assembly. Constraints C (s) are introduced to restrict the relative motion between pairs of components, and they are used to model virtual motors and different types of mechanical joints. To solve for the motion of the assembly, the system computes the state vector s that minimizes 1 ⁇ 2C(s) T C(s) using Newton's method.
- the system may receive as input a library of parametrized mechanisms that represent, for example, different types of limbs that a mechanical character might have.
- skeletal curves may be rigidly attached to the main mechanical structure of the mechanisms. These skeletal curves may be used to differentiate between the different classes of mechanism that are available, and to define the energy term introduced above.
- FIG. 9 illustrates mechanical leg 910 B and its corresponding 2D representation 910 A. Additionally, mechanical leg 910 B may be assigned a motion curve 920 which determines the range of motion for mechanical leg 910 B.
- the parameters p may be optimized to allow the system to simultaneously change a mechanisms' kinematic properties (e.g., the length of any rigid segment) and motion; in particular embodiments, they may consist of the local coordinates that define the location of each joint.
- a design system to generate two types of legs and combine them with a rigid body shape parameterized in translation, rotation and scale along the x and y axes is presented.
- a user may first sketch mechanical body 1010 A of any size, which results in displaying its corresponding display body 1010 B. Additionally, a user may then sketch mechanical leg 1020 A and motion curves 1030 resulting in a complete mechanical assembly with a range of motion defined in the mechanical legs 1020 B.
- the user may create a customized mechanically simulated beast.
- the user may adapt the shape of the legs, their motion paths, or the relative phase between them may be done quickly and intuitively.
- FIG. 11A illustrates a flowchart for an example method 1100 A of sketch-based posing of 3D characters.
- the method may begin at step 1110 A, where particular embodiments may provide a 2D visual representation of an object to be animated.
- step 1120 A particular embodiments may receive sketch input with respect to the object, wherein the sketch input identifies a target position for a specified portion of the object.
- step 1130 A particular embodiments may minimize an energy metric by incorporating aspects of the character rig specification as unknown values.
- particular embodiments may calculate the closest point between the original position of the specified portion and the updated position of the specified portion.
- particular embodiments may compute a deformation for the object.
- Particular embodiments may repeat one or more steps of the method of FIG. 11A , where appropriate.
- this disclosure describes and illustrates particular steps of the method of FIG. 11A as occurring in a particular order, this disclosure contemplates any suitable steps of the method of FIG. 11A occurring in any suitable order.
- this disclosure describes and illustrates an example method for sketch-based posing of 3D characters, including the particular steps of the method of FIG. 11A
- this disclosure contemplates any suitable method for sketch-based posing of 3D characters, including any suitable steps, which may include all, some, or none of the steps of the method of FIG. 11A , where appropriate.
- this disclosure describes and illustrates particular components, devices, or systems carrying out particular steps of the method of FIG. 11A
- this disclosure contemplates any suitable combination of any suitable components, devices, or systems carrying out any suitable steps of the method of FIG. 11A .
- FIG. 11B illustrates a flowchart for an example method 1100 B of design and customization of 3D characters using sketch-based techniques.
- the method may begin at step 1110 B, where particular embodiments may provide a plurality of objects, wherein each of the objects comprises a 2D representation and a 3D model.
- step 1120 B particular embodiments may detect sketch input.
- step 1130 B particular embodiments may classify the sketch input as corresponding to a detected one of the objects.
- particular embodiments may instantiate the detected object.
- particular embodiments may display a 3D visual representation of the instantiated object. Particular embodiments may repeat one or more steps of the method of FIG. 11B , where appropriate.
- this disclosure describes and illustrates particular steps of the method of FIG. 11B as occurring in a particular order, this disclosure contemplates any suitable steps of the method of FIG. 11B occurring in any suitable order.
- this disclosure describes and illustrates an example method for design and customization of 3D characters using sketch-based techniques, including the particular steps of the method of FIG. 11B
- this disclosure contemplates any suitable method for design and customization of 3D characters using sketch-based techniques, including any suitable steps, which may include all, some, or none of the steps of the method of FIG. 11B , where appropriate.
- this disclosure describes and illustrates particular components, devices, or systems carrying out particular steps of the method of FIG. 11B
- this disclosure contemplates any suitable combination of any suitable components, devices, or systems carrying out any suitable steps of the method of FIG. 11B .
- FIG. 12 illustrates an example computer system 1200 .
- one or more computer systems 1200 perform one or more steps of one or more methods described or illustrated herein.
- one or more computer systems 1200 provide functionality described or illustrated herein.
- software running on one or more computer systems 1200 performs one or more steps of one or more methods described or illustrated herein or provides functionality described or illustrated herein.
- Particular embodiments include one or more portions of one or more computer systems 1200 .
- reference to a computer system may encompass a computing device, and vice versa, where appropriate.
- reference to a computer system may encompass one or more computer systems, where appropriate.
- computer system 1200 may be an embedded computer system, a system-on-chip (SOC), a single-board computer system (SBC) (such as, for example, a computer-on-module (COM) or system-on-module (SOM)), a desktop computer system, a laptop or notebook computer system, an interactive kiosk, a mainframe, a mesh of computer systems, a mobile telephone, a personal digital assistant (PDA), a server, a tablet computer system, or a combination of two or more of these.
- SOC system-on-chip
- SBC single-board computer system
- COM computer-on-module
- SOM system-on-module
- computer system 1200 may include one or more computer systems 1200 ; be unitary or distributed; span multiple locations; span multiple machines; span multiple data centers; or reside in a cloud, which may include one or more cloud components in one or more networks.
- one or more computer systems 1200 may perform without substantial spatial or temporal limitation one or more steps of one or more methods described or illustrated herein.
- one or more computer systems 1200 may perform in real time or in batch mode one or more steps of one or more methods described or illustrated herein.
- One or more computer systems 1200 may perform at different times or at different locations one or more steps of one or more methods described or illustrated herein, where appropriate.
- computer system 1200 includes a processor 1202 , memory 1204 , storage 1206 , an input/output (I/O) interface 1208 , a communication interface 1210 , and a bus 1212 .
- I/O input/output
- this disclosure describes and illustrates a particular computer system having a particular number of particular components in a particular arrangement, this disclosure contemplates any suitable computer system having any suitable number of any suitable components in any suitable arrangement.
- processor 1202 includes hardware for executing instructions, such as those making up a computer program.
- processor 1202 may retrieve (or fetch) the instructions from an internal register, an internal cache, memory 1204 , or storage 1206 ; decode and execute them; and then write one or more results to an internal register, an internal cache, memory 1204 , or storage 1206 .
- processor 1202 may include one or more internal caches for data, instructions, or addresses. This disclosure contemplates processor 1202 including any suitable number of any suitable internal caches, where appropriate.
- processor 1202 may include one or more instruction caches, one or more data caches, and one or more translation lookaside buffers (TLBs). Instructions in the instruction caches may be copies of instructions in memory 1204 or storage 1206 , and the instruction caches may speed up retrieval of those instructions by processor 1202 . Data in the data caches may be copies of data in memory 1204 or storage 1206 for instructions executing at processor 1202 to operate on; the results of previous instructions executed at processor 1202 for access by subsequent instructions executing at processor 1202 or for writing to memory 1204 or storage 1206 ; or other suitable data. The data caches may speed up read or write operations by processor 1202 . The TLBs may speed up virtual-address translation for processor 1202 .
- TLBs translation lookaside buffers
- processor 1202 may include one or more internal registers for data, instructions, or addresses. This disclosure contemplates processor 1202 including any suitable number of any suitable internal registers, where appropriate. Where appropriate, processor 1202 may include one or more arithmetic logic units (ALUs); be a multi-core processor; or include one or more processors 1202 . Although this disclosure describes and illustrates a particular processor, this disclosure contemplates any suitable processor.
- ALUs arithmetic logic units
- memory 1204 includes main memory for storing instructions for processor 1202 to execute or data for processor 1202 to operate on.
- computer system 1200 may load instructions from storage 1206 or another source (such as, for example, another computer system 1200 ) to memory 1204 .
- Processor 1202 may then load the instructions from memory 1204 to an internal register or internal cache.
- processor 1202 may retrieve the instructions from the internal register or internal cache and decode them.
- processor 1202 may write one or more results (which may be intermediate or final results) to the internal register or internal cache.
- Processor 1202 may then write one or more of those results to memory 1204 .
- processor 1202 executes only instructions in one or more internal registers or internal caches or in memory 1204 (as opposed to storage 1206 or elsewhere) and operates only on data in one or more internal registers or internal caches or in memory 1204 (as opposed to storage 1206 or elsewhere).
- One or more memory buses (which may each include an address bus and a data bus) may couple processor 1202 to memory 1204 .
- Bus 1212 may include one or more memory buses, as described below.
- one or more memory management units reside between processor 1202 and memory 1204 and facilitate accesses to memory 1204 requested by processor 1202 .
- memory 1204 includes random access memory (RAM).
- This RAM may be volatile memory, where appropriate Where appropriate, this RAM may be dynamic RAM (DRAM) or static RAM (SRAM). Moreover, where appropriate, this RAM may be single-ported or multi-ported RAM. This disclosure contemplates any suitable RAM.
- Memory 1204 may include one or more memories 1204 , where appropriate. Although this disclosure describes and illustrates particular memory, this disclosure contemplates any suitable memory.
- storage 1206 includes mass storage for data or instructions.
- storage 1206 may include a hard disk drive (HDD), a floppy disk drive, flash memory, an optical disc, a magneto-optical disc, magnetic tape, or a Universal Serial Bus (USB) drive or a combination of two or more of these.
- Storage 1206 may include removable or non-removable (or fixed) media, where appropriate.
- Storage 1206 may be internal or external to computer system 1200 , where appropriate.
- storage 1206 is non-volatile, solid-state memory.
- storage 1206 includes read-only memory (ROM).
- this ROM may be mask-programmed ROM, programmable ROM (PROM), erasable PROM (EPROM), electrically erasable PROM (EEPROM), electrically alterable ROM (EAROM), or flash memory or a combination of two or more of these.
- This disclosure contemplates mass storage 1206 taking any suitable physical form.
- Storage 1206 may include one or more storage control units facilitating communication between processor 1202 and storage 1206 , where appropriate.
- storage 1206 may include one or more storages 1206 .
- this disclosure describes and illustrates particular storage, this disclosure contemplates any suitable storage.
- I/O interface 1208 includes hardware, software, or both, providing one or more interfaces for communication between computer system 1200 and one or more I/O devices.
- Computer system 1200 may include one or more of these I/O devices, where appropriate.
- One or more of these I/O devices may enable communication between a person and computer system 1200 .
- an I/O device may include a keyboard, keypad, microphone, monitor, mouse, printer, scanner, speaker, still camera, stylus, tablet, touch screen, trackball, video camera, another suitable I/O device or a combination of two or more of these.
- An I/O device may include one or more sensors. This disclosure contemplates any suitable I/O devices and any suitable I/O interfaces 1208 for them.
- I/O interface 1208 may include one or more device or software drivers enabling processor 1202 to drive one or more of these I/O devices.
- I/O interface 1208 may include one or more I/O interfaces 1208 , where appropriate. Although this disclosure describes and illustrates a particular I/O interface, this disclosure contemplates any suitable I/O interface.
- communication interface 1210 includes hardware, software, or both providing one or more interfaces for communication (such as, for example, packet-based communication) between computer system 1200 and one or more other computer systems 1200 or one or more networks.
- communication interface 1210 may include a network interface controller (NIC) or network adapter for communicating with an Ethernet or other wire-based network or a wireless NIC (WNIC) or wireless adapter for communicating with a wireless network, such as a WI-FI network.
- NIC network interface controller
- WNIC wireless NIC
- WI-FI network wireless network
- computer system 1200 may communicate with an ad hoc network, a personal area network (PAN), a local area network (LAN), a wide area network (WAN), a metropolitan area network (MAN), or one or more portions of the Internet or a combination of two or more of these.
- PAN personal area network
- LAN local area network
- WAN wide area network
- MAN metropolitan area network
- computer system 1200 may communicate with a wireless PAN (WPAN) (such as, for example, a BLUETOOTH WPAN), a WI-FI network, a WI-MAX network, a cellular telephone network (such as, for example, a Global System for Mobile Communications (GSM) network), or other suitable wireless network or a combination of two or more of these.
- Computer system 1200 may include any suitable communication interface 1210 for any of these networks, where appropriate.
- Communication interface 1210 may include one or more communication interfaces 1210 , where appropriate.
- bus 1212 includes hardware, software, or both coupling components of computer system 1200 to each other.
- bus 1212 may include an Accelerated Graphics Port (AGP) or other graphics bus, an Enhanced Industry Standard Architecture (EISA) bus, a front-side bus (FSB), a HYPERTRANSPORT (HT) interconnect, an Industry Standard Architecture (ISA) bus, an INFINIBAND interconnect, a low-pin-count (LPC) bus, a memory bus, a Micro Channel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCIe) bus, a serial advanced technology attachment (SATA) bus, a Video Electronics Standards Association local (VLB) bus, or another suitable bus or a combination of two or more of these.
- Bus 1212 may include one or more buses 1212 , where appropriate.
- a computer-readable non-transitory storage medium or media may include one or more semiconductor-based or other integrated circuits (ICs) (such, as for example, field-programmable gate arrays (FPGAs) or application-specific ICs (ASICs)), hard disk drives (HDDs), hybrid hard drives (HHDs), optical discs, optical disc drives (ODDs), magneto-optical discs, magneto-optical drives, floppy diskettes, floppy disk drives (FDDs), magnetic tapes, solid-state drives (SSDs), RAM-drives, SECURE DIGITAL cards or drives, any other suitable computer-readable non-transitory storage media, or any suitable combination of two or more of these, where appropriate.
- ICs such, as for example, field-programmable gate arrays (FPGAs) or application-specific ICs (ASICs)
- HDDs hard disk drives
- HHDs hybrid hard drives
- ODDs optical disc drives
- magneto-optical discs magneto-optical drives
- an apparatus or system or a component of an apparatus or system being adapted to, arranged to, capable of, configured to, enabled to, operable to, or operative to perform a particular function encompasses that apparatus, system, component, whether or not it or that particular function is activated, turned on, or unlocked, as long as that apparatus, system, or component is so adapted, arranged, capable, configured, enabled, operable, or operative.
Abstract
Description
W match(ωz(p))=Σi=1 mΣi,j=1 kωi,j ·∥y i −z j(p)∥2 2 (Eq. 1)
ωi,j =a i,j +b i,j. (Eq. 2)
where ∀∈Σi=1 m a i,j=1 and ∀∈Σi=1 k b i,j=1. (Eq. 3)
z(p)≈z(p 0)+J(p 0)·(p−p 0), (Eq. 5)
using finite differences around the initial parameter vector p0. Given that Wmatch is a quadratic function in terms of z, its derivatives with respect to z are trivially obtained, and the gradient and the Hessian with respect to the parameters p may be derived using the chain rule as
respectively, where the vertex Jacobian
additionally needs to be estimated using finite differences.
W reg=ƒ(d(x i 0))Σi=1 n(x i(p)−x i 0)2. (Eq. 7)
Claims (20)
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US15/172,078 US11087517B2 (en) | 2015-06-03 | 2016-06-02 | Sketch-based abstraction for character posing and synthesis |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US201562170458P | 2015-06-03 | 2015-06-03 | |
US15/172,078 US11087517B2 (en) | 2015-06-03 | 2016-06-02 | Sketch-based abstraction for character posing and synthesis |
Publications (2)
Publication Number | Publication Date |
---|---|
US20160358365A1 US20160358365A1 (en) | 2016-12-08 |
US11087517B2 true US11087517B2 (en) | 2021-08-10 |
Family
ID=57451311
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US15/172,078 Active US11087517B2 (en) | 2015-06-03 | 2016-06-02 | Sketch-based abstraction for character posing and synthesis |
Country Status (1)
Country | Link |
---|---|
US (1) | US11087517B2 (en) |
Families Citing this family (8)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US10546406B2 (en) * | 2016-05-09 | 2020-01-28 | Activision Publishing, Inc. | User generated character animation |
KR101867991B1 (en) * | 2016-12-13 | 2018-06-20 | 한국과학기술원 | Motion edit method and apparatus for articulated object |
US20180225858A1 (en) * | 2017-02-03 | 2018-08-09 | Sony Corporation | Apparatus and method to generate realistic rigged three dimensional (3d) model animation for view-point transform |
CN107657651B (en) * | 2017-08-28 | 2019-06-07 | 腾讯科技(上海)有限公司 | Expression animation generation method and device, storage medium and electronic device |
US10417802B2 (en) | 2017-12-20 | 2019-09-17 | Binary Bubbles, Inc. | System and method for creating customized characters and selectively displaying them in an augmented or virtual reality display |
US10949756B2 (en) | 2018-02-01 | 2021-03-16 | Binary Bubbles, Inc. | System and method for creating and selectively modifying characters and conditionally presenting customized characters via electronic channels |
WO2019177870A1 (en) * | 2018-03-15 | 2019-09-19 | Magic Leap, Inc. | Animating virtual avatar facial movements |
US11763507B2 (en) * | 2018-12-05 | 2023-09-19 | Sony Group Corporation | Emulating hand-drawn lines in CG animation |
Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5680531A (en) * | 1993-07-02 | 1997-10-21 | Apple Computer, Inc. | Animation system which employs scattered data interpolation and discontinuities for limiting interpolation ranges |
US20160093084A1 (en) * | 2014-09-30 | 2016-03-31 | Pixar | Subspace clothing simulation using adaptive bases |
US20160110595A1 (en) * | 2014-10-17 | 2016-04-21 | Qiaosong Wang | Fast 3d model fitting and anthropometrics using synthetic data |
US20170186165A1 (en) * | 2015-12-29 | 2017-06-29 | Microsoft Technology Licensing, Llc | Tracking rigged smooth-surface models of articulated objects |
-
2016
- 2016-06-02 US US15/172,078 patent/US11087517B2/en active Active
Patent Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5680531A (en) * | 1993-07-02 | 1997-10-21 | Apple Computer, Inc. | Animation system which employs scattered data interpolation and discontinuities for limiting interpolation ranges |
US20160093084A1 (en) * | 2014-09-30 | 2016-03-31 | Pixar | Subspace clothing simulation using adaptive bases |
US20160110595A1 (en) * | 2014-10-17 | 2016-04-21 | Qiaosong Wang | Fast 3d model fitting and anthropometrics using synthetic data |
US20170186165A1 (en) * | 2015-12-29 | 2017-06-29 | Microsoft Technology Licensing, Llc | Tracking rigged smooth-surface models of articulated objects |
Non-Patent Citations (28)
Title |
---|
Baskin, et al., Mike and Tina character rig 2.8.0 [Online; accessed Jan. 8, 2019] http://www.creativecrash.com/maya/downloads/character-rigs/c/mike-and-tina-character-rig, 3 pages. |
Choi, et al., Retrieval and Visulization of Human Motion Data via Stick Figures, 2012 Comput. Graph. Forum 31,7-1, pp. 2057-2065. |
Cingold, et al., Structured annotation for 2d-to-3d modeling, ACM Trans. Graph. vol. 28 Issue 5, (Dec. 2009) 148:1-148:9. |
Cocos, et al., Computational design of mechanical characters, ACM Trans. on Graphics 32,4 (Jul. 2013), 83:1-83:12. |
Davis, et al., A sketching interface fa articulated figure animation. In Proceedings of the 2003 ACM Sig-Graph/Eurographics Symposium on Computer Animation, Eurographics Association, Aire-la-Ville, Switzerland, SCA 03, pp. 320-328. |
Eitz, et al., How do humans sketch objects? ACM Trans. on Graphics, 2012, (Proc. SIGGRAPII) 31, 4,44:1-44:10. |
Funkhouser, et al., Modeling by example. ACM Tian an Graphics (Proc. SIGGRAPH) (Aug. 2004). |
Grinspun, et al., Discrete shells, In Proceedings of The 2003 ACM SIG-GRAPH/Eurograpltics Symposium Computer Animation, Eurographics Association, Aiea-Ville, Switzerland, Switzerland, SCA '03, pp. 62-67. |
Guay, et al., The line of action: An intuitive interface for expressive character posing, ACM Trans. on Graphics (Nov. 2013), 32(6):205:1-205:8. |
Gunnarsson, et al., Sketch-based posing of 3d faces for facial animation, In TPCG, Eurographics Association, John Collomosse, Ian Grimstead, 2010, pp. 1-8. |
Hahn, et al., Efficient simulation of secondary motion in rig-space. In Proceedings of the ACM SIGGRAPH/Eurographics Symposium on Computer Animation, 2013, SCA '13 |
Hecker, et al., Real-time motion retargeting to highly varied user-created morphologies. In ACM SIGGRAPH 2008 Papers, ACM, New York, NY, USA, SIG-GRAPH '08, 27:1-27:11. |
Igarashi, et al., Teddy: A sketching interface for 3d freeform design. In Proceedings of he 26th Annual Conference on Computer Graphics and Interactive Techniques, ACM Press/Addison-Wesley Publishing Co., New York, NY, USA, 1999, SIGGRAPH '99, pp. 409-416. |
Karpenko, et al., Smoothsketch:3d free-form shapes from complex sketches. ACM Trans. Graph. 25, 3 (Jul. 2006), pp. 589-598. |
Lee, et al., Sketch-based search and composition of 3D models, In EUROGRAPHICS Workshop on Sketch-Based Interfaces and Modeling, 2008. |
Lin, et al., A sketching interface for sitting-pose design. In Proceedings of the Seventh Sketch-Based Interfaces and Modeling Symposium, Eurographics Association, Aire-la-Ville, Switzerland, Switzerland, 800 SBIM '10, 2010, pp. 111-118. |
Miranda, et al., Sketch express: A sketching interface for facial animation. Computers & Graphics, (Oct. 2012),36(6):585-595. |
Nealen, et al., A sketch-based interface for detail-preserving mesh editing. ACM Trans. on Graphics (Proc. SIGGRAPH) 2005, 24, 3, pp. 1142-1147. |
Nealen, et al., FiberMesh: Designing freeform surfaces with 3D curves, ACM Trans. on Graphics (Proc. SIGGRAPH) 2007, 26, 3, article No. 41. |
Noris, et al., Smart scribbles for sketch segmentation. Computer Graphics Forum 31, 2012, 8, pp. 2516-2527. |
Olsen, et al., Sketch-based modeling: A survey, Computers & Graphics 33, (2009), pp. 85-103. |
Oztireli, et al., Differential blending for expressive sketch-based posing. In Proceedings of the 12th ACM SIGGRAPH/Eurographics Symposium on Computer Animation, ACM, New York, NY, USA, 2013, SCA '13, pp. 155-164. |
Rusinkiewicz, et al., Efficient variants of the ICP algorithm. In 3-D Digital Imaging and Modeling, IEEE Computer Society, 2001, pp. 145-152. |
Shin, et al., Magic canvas: Interactive design of a 3-D scene prototype from freehand sketches. In Proceedings of Graphics Interface 2007, ACM, New York, NY, USA, GI '07, pp. 63-70. |
Terzopoulos, "The Computation of Visible-Surface Representations" IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 10, No. 4, Jul. 1988, p. 417-438. * |
Wei, et al., Intuitive interactive Human-character posing with millions of example poses, Computer Graphics and Applications, IEEE 31, 4 (Jul. 2011), pp. 78-88. |
Wyvill, et al., Sketch based construction and rendering of implicit models. In Proceedings of the First Euro-graphics Conference on Computational Aesthetics in Graphics, Visualization and Imaging, Eurographics Association, Aire-la Ville, Switzerland, Switzerland, Computational Aesthetics'05, 2005, pp. 67-74. |
Zimmermann, et al., SilSketch: Automated sketch-based editing of surface meshes. In Proceedings of the 4th Eurographics Workshop on Sketch-based Interfaces and Modeling, ACM, New York, NY, USA, SBIM '07, 2007, pp. 23-30. |
Also Published As
Publication number | Publication date |
---|---|
US20160358365A1 (en) | 2016-12-08 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US11087517B2 (en) | Sketch-based abstraction for character posing and synthesis | |
US11928592B2 (en) | Visual sign language translation training device and method | |
US11769259B2 (en) | Region-based stabilized face tracking | |
KR102616010B1 (en) | System and method for photorealistic real-time human animation | |
US11158121B1 (en) | Systems and methods for generating accurate and realistic clothing models with wrinkles | |
CN113361705A (en) | Unsupervised learning of scene structures for synthetic data generation | |
JP2023549821A (en) | Deformable neural radiance field | |
US11816404B2 (en) | Neural network control variates | |
CN115151915A (en) | Neural rendering for reverse graphics generation | |
US11335056B1 (en) | Real-time rendering with implicit shapes | |
US11062502B2 (en) | Three-dimensional modeling volume for rendering images | |
US20230298243A1 (en) | 3d digital avatar generation from a single or few portrait images | |
US20230274492A1 (en) | Texture transfer and synthesis using aligned maps in image generation systems and applications | |
US11922558B2 (en) | Hybrid differentiable rendering for light transport simulation systems and applications | |
CN116664807A (en) | Texture transfer and synthesis using alignment maps in image generation systems and applications | |
US20130311969A1 (en) | Systems and Methods for Runtime Control of Hierarchical Objects | |
US20240013497A1 (en) | Learning Articulated Shape Reconstruction from Imagery | |
WO2022251619A1 (en) | Hybrid differentiable rendering for light transport simulation systems and applications | |
CN116686006A (en) | Three-dimensional scan registration based on deformable model | |
US8890889B1 (en) | System and method for generating a pose of an object | |
US11640695B2 (en) | Digital garment generation | |
US20230360327A1 (en) | Generating three-dimensional representations for digital objects utilizing mesh-based thin volumes | |
CN109002163A (en) | Three-dimension interaction gesture sample method, apparatus, computer equipment and storage medium | |
US20240112401A1 (en) | 3d garment generation from 2d scribble images | |
US20230368451A1 (en) | System and method for ai assisted character pose authoring |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: EIDGENOESSISCHE TECHNISCHE HOCHSCHULE ZUERICH, SWI Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:SUMNER, ROBERT WALKER;NITTI, MAURIZIO;COROS, STELIAN;AND OTHERS;SIGNING DATES FROM 20160630 TO 20160706;REEL/FRAME:039915/0675 Owner name: THE WALT DISNEY COMPANY, SWITZERLAND Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:SUMNER, ROBERT WALKER;NITTI, MAURIZIO;COROS, STELIAN;AND OTHERS;SIGNING DATES FROM 20160630 TO 20160706;REEL/FRAME:039915/0675 Owner name: DISNEY ENTERPRISES, INC., CALIFORNIA Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:THE WALT DISNEY COMPANY;REEL/FRAME:039915/0703 Effective date: 20160630 Owner name: EIDGENOESSISCHE TECHNISCHE HOCHSCHULE ZUERICH, SWITZERLAND Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:SUMNER, ROBERT WALKER;NITTI, MAURIZIO;COROS, STELIAN;AND OTHERS;SIGNING DATES FROM 20160630 TO 20160706;REEL/FRAME:039915/0675 |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: RESPONSE TO NON-FINAL OFFICE ACTION ENTERED AND FORWARDED TO EXAMINER |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: FINAL REJECTION MAILED |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: DOCKETED NEW CASE - READY FOR EXAMINATION |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: NON FINAL ACTION MAILED |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: RESPONSE TO NON-FINAL OFFICE ACTION ENTERED AND FORWARDED TO EXAMINER |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: DOCKETED NEW CASE - READY FOR EXAMINATION |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: NON FINAL ACTION MAILED |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: RESPONSE TO NON-FINAL OFFICE ACTION ENTERED AND FORWARDED TO EXAMINER |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: NOTICE OF ALLOWANCE MAILED -- APPLICATION RECEIVED IN OFFICE OF PUBLICATIONS |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: PUBLICATIONS -- ISSUE FEE PAYMENT VERIFIED |
|
STCF | Information on status: patent grant |
Free format text: PATENTED CASE |