WO2022111525A1

WO2022111525A1 - Posture capturing method and apparatus, electronic device, and storage medium

Info

Publication number: WO2022111525A1
Application number: PCT/CN2021/132799
Authority: WO
Inventors: 柴金祥; 赵文平; 刘博�
Original assignee: 魔珐(上海)信息科技有限公司; 上海墨舞科技有限公司
Priority date: 2020-11-30
Filing date: 2021-11-24
Publication date: 2022-06-02
Also published as: CN112515661B; CN112515661A

Abstract

A posture capturing method and apparatus, an electronic device, and a storage medium. The posture capturing method comprises: determining a spatial position of a first marker object attached to a subject and tag information of the part where the first marker object is located on the subject (S110); and adjusting an initial posture model by using the spatial position of the first marker object and the tag information to generate a reference posture model corresponding to the subject (S120). With said method, the spatial position of the first marker object and the tag information can be used to generate the reference posture model of the subject, facilitating subsequent motion capturing.

Description

姿态捕捉方法、装置、电子设备及存储介质Attitude capture method, device, electronic device and storage medium

本申请要求于2020年11月30日提交中国专利局、申请号为202011376569.X、申请名称为“姿态捕捉方法、装置、电子设备及存储介质”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。This application claims the priority of the Chinese patent application filed on November 30, 2020 with the application number 202011376569.X and the application name "Posture Capture Method, Device, Electronic Device and Storage Medium", the entire content of which is approved by Reference is incorporated in this application.

技术领域technical field

本申请涉及计算机技术领域，尤其涉及一种姿态捕捉方法、装置、电子设备及存储介质。The present application relates to the field of computer technology, and in particular, to a gesture capture method, device, electronic device, and storage medium.

背景技术Background technique

随着计算机技术、传感器技术以及虚拟现实产业的高速发展，运动捕捉技术的发展异常迅速并且应用范围日益广泛，尤其是在体育行业、游戏制作、动画制作以及影视特效制作等诸多领域中起着举足轻重的作用，其形成了一种新型的艺术与技术的相互渗透和融合的方式，并将成为未来的一种发展趋势。With the rapid development of computer technology, sensor technology and virtual reality industry, the development of motion capture technology is extremely rapid and its application range is increasingly extensive, especially in many fields such as sports industry, game production, animation production and film and television special effects production. It has formed a new way of mutual penetration and integration of art and technology, and will become a development trend in the future.

目前的光学捕捉方式中，在演员身体上各个关节放置标记点，利用一些高速红外摄像头获取标记点的三维坐标，通过标记点的三维坐标，推断出演员的表演。该方案中假定人体各个骨骼是个刚体，因此标记点要放置在演员身上的骨节点处的位置，且放置点位置必须精确。并且通过活动各个关节来重建出演员的骨骼模型，耗时长且繁琐。现有的方式无法重建出与演员匹配的三维人体模型，由于人体的每根骨骼有肌肉附着，并不是严格意义上的刚体，这与现有光学捕捉技术的假设有偏差，因此，现有的光学捕捉技术对于演员表演的捕捉存在一些精度上的损失。此外，由于现有的光学捕捉技术无法重建出与演员匹配的三维人体模型，因此无法通过增加贴在演员身上标记点的数目来提升捕捉精度，增强鲁棒性。In the current optical capture method, markers are placed on each joint of the actor's body, and some high-speed infrared cameras are used to obtain the three-dimensional coordinates of the marked points, and the actor's performance can be inferred from the three-dimensional coordinates of the marked points. In this scheme, it is assumed that each bone of the human body is a rigid body, so the marker point should be placed at the position of the bone node on the actor, and the position of the placement point must be accurate. And reconstructing the actor's skeletal model by activating each joint is time-consuming and cumbersome. The existing methods cannot reconstruct a 3D human model that matches the actor. Since each bone of the human body has muscles attached, it is not a rigid body in the strict sense, which deviates from the assumptions of the existing optical capture technology. Therefore, the existing There is some loss of precision in the capture of an actor's performance by optical capture technology. In addition, since the existing optical capture technology cannot reconstruct a 3D human model that matches the actor, it cannot improve the capture accuracy and enhance the robustness by increasing the number of marked points attached to the actor.

发明内容SUMMARY OF THE INVENTION

目前针对于演员运动捕捉的方式中，精度比较高的方式主要有两种，分别是光学运动捕捉与惯性运行捕捉，下面一一进行说明。光学捕捉是在捕捉场地四周布置一些高速红外摄像头，在演员身体上各个关节放置标记点，获取标记点的三维坐标，通过标记点的三维坐标，推断出演员的表演。光学捕捉有以下问题，首先，现有光学捕捉假设演员的每根骨骼都是一个刚体，因此标记点必须放置在演员身上的骨节点处(骨节点指身体上骨头突出的位置，如果放在其他位置，标记点可能随着肌肉的变化产生滑动)，演员活动各个关节，得出演员各个关节骨骼的长度，该过程要求标记点的放置点位置必须精确，并且需要活动各个关节，耗时长且繁琐。由于人体的每根骨骼有肌肉附着，并不是严格意义上的刚体，这与现有光学捕捉技术的假设有偏差，因此，现有的光学捕捉技术对于演员表演的捕捉存在一些精度上的损失。此外，由于现有的光学捕捉无法重建出与演员匹配的三维人体模型，因此无法通过增加贴在演员身上标记点的数目来提升捕捉精度，增强鲁棒性。其次，在捕捉比较复杂的单人运动(比如下跪、地上打滚)或者多人运动捕捉(比如多人拥抱)时，由于演员身体自遮挡原因，光捕相机看不到演员身上标记点，这种情况下捕捉***无法获取到演员身上附着的标记点，导致捕捉精度下降或者出错。At present, among the methods of motion capture for actors, there are mainly two methods with relatively high precision, namely optical motion capture and inertial motion capture, which will be explained below. Optical capture is to arrange some high-speed infrared cameras around the capture site, place markers on each joint on the actor's body, obtain the three-dimensional coordinates of the marked points, and infer the actor's performance through the three-dimensional coordinates of the marked points. Optical capture has the following problems. First, the existing optical capture assumes that each bone of the actor is a rigid body, so the marker point must be placed at the bone node on the actor's body (the bone node refers to the protruding position of the bone on the body, if placed on other position, the marker point may slide with the change of the muscle), the actor moves each joint, and the length of each joint bone of the actor is obtained. This process requires that the position of the marker point must be accurate, and each joint needs to be moved, which is time-consuming and cumbersome . Since each bone of the human body has muscles attached, it is not a rigid body in the strict sense, which deviates from the assumption of the existing optical capture technology. Therefore, the existing optical capture technology has some loss of accuracy in capturing the performance of the actors. In addition, since the existing optical capture cannot reconstruct a 3D human model that matches the actor, it cannot improve the capture accuracy and enhance the robustness by increasing the number of marked points attached to the actor. Secondly, when capturing more complex single-person movements (such as kneeling, rolling on the ground) or multi-person motion capture (such as multiple people hugging), due to the self-occlusion of the actor's body, the light capture camera cannot see the marked points on the actor's body. In this case, the capture system cannot obtain the marker points attached to the actor, resulting in decreased capture accuracy or errors.

另一种捕捉的方式是惯性捕捉，通过在各个关节处放置惯性传感器(角速度计、线性加速度计等)，在演员运动时，获取各个关节的角速度、线性加速度等信息，由于惯性捕捉无法直接获取到演员各个关节的绝对位置和方向信息，所以需要对关节的角速度、加速度进行积分，得到各个关节的绝对姿势。相对于光学捕捉，惯性捕捉无法直接获取关节姿势，只能通过积分角速度和加速度获取关节姿势，另外惯性传感器的观测数据一般都有一定量的噪声，随着捕捉时间的增加，积分造成的误差累积会不断增大，因此，惯性捕捉无法进行长时间的精准人体运动捕捉，并且捕捉到三维人体姿态与真实的人体姿态会有一定的偏差，且偏差会随着捕捉时间的增加而增大；惯性捕捉需要佩戴各种传感器，穿戴并不方便；传感器需要电池进行驱动，录制的时长受到电池的限制。Another way to capture is inertial capture. By placing inertial sensors (angular velocity meters, linear accelerometers, etc.) at each joint, when the actor moves, the angular velocity, linear acceleration and other information of each joint can be obtained. Due to inertial capture, it cannot be directly obtained. To obtain the absolute position and direction information of each joint of the actor, it is necessary to integrate the angular velocity and acceleration of the joint to obtain the absolute posture of each joint. Compared with optical capture, inertial capture cannot directly obtain joint poses, but can only obtain joint poses by integrating angular velocity and acceleration. In addition, the observation data of inertial sensors generally have a certain amount of noise. With the increase of capture time, the error caused by integration will accumulate. Constantly increasing, therefore, inertial capture cannot perform accurate human motion capture for a long time, and there will be a certain deviation between the captured three-dimensional human posture and the real human posture, and the deviation will increase with the increase of capture time; inertial capture Various sensors need to be worn, which is inconvenient to wear; the sensor needs to be driven by a battery, and the recording time is limited by the battery.

针对上述问题，本申请实施例提供一种姿态捕捉方法、装置、电子设备及存储介质，用于至少解决以上提及的问题。In view of the above problems, embodiments of the present application provide a gesture capturing method, apparatus, electronic device, and storage medium, which are used to at least solve the above-mentioned problems.

本申请实施例还提供一种姿态捕捉方法，所述方法包括：确定附着在被摄体上的第一标记对象的空间位置以及第一标记对象在所述被摄体的部位的标签信息；利用第一标记对象的所述空间位置以及所述标签信息对初始姿态模型进行调整，生成与所述被摄体对应的基准姿态模型。The embodiment of the present application further provides a gesture capture method, the method includes: determining the spatial position of the first marked object attached to the subject and the label information of the first marked object on the part of the subject; using The spatial position of the first marked object and the label information adjust the initial posture model to generate a reference posture model corresponding to the subject.

本方法实施例存在以下有益效果，标记点无需放置在演员身上的骨节点处，也无需必须精确到某个固定位置，节省时间。并且能够重建出与演员匹配的三维人体模型。The embodiments of this method have the following beneficial effects, and the marking point does not need to be placed at the bone node on the actor, and does not need to be accurate to a certain fixed position, which saves time. And it can reconstruct a three-dimensional human model that matches the actor.

可选地，所述初始姿态模型包括用于描述形体的形体参数以及用于描述动作的动作参数。Optionally, the initial posture model includes a body parameter for describing a body and an action parameter for describing an action.

可选地，所述利用第一标记对象的所述空间位置以及所述标签信息对初始姿态模型进行调整，生成与所述被摄体对应的基准姿态模型包括：通过使所述被摄体做出特定动作，设置所述初始姿态模型中的动作参数；在所述动作参数确定的情况下，获取第一标记对象在所述被摄体做出所述特定动作的情况下的所述空间位置以及所述标签信息；利用所述动作参数、所述空间位置以及所述标签信息，对所述初始姿态模型的形体参数进行调整，生成与所述被摄体对应的所述基准姿态模型。Optionally, the adjusting the initial posture model by using the spatial position and the label information of the first marked object, and generating the reference posture model corresponding to the subject includes: making the subject do select a specific action, and set the action parameters in the initial posture model; if the action parameters are determined, obtain the spatial position of the first marker object when the subject performs the specific action and the label information; using the motion parameters, the spatial position and the label information, the body parameters of the initial posture model are adjusted to generate the reference posture model corresponding to the subject.

本方法实施例可以通过获取动作参数和第一标记对象的空间位置和标签信息，重建出与演员匹配的三维人体模型。In this embodiment of the method, a three-dimensional human body model matching the actor can be reconstructed by acquiring the action parameters and the spatial position and label information of the first marked object.

可选地，在生成与所述被摄体对应的所述基准姿态模型后还包括：响应于被摄体的动作，获取第一标记对象的当前空间位置以及当前标签信息；利用所述当前空间位置以及所述当前标签信息，对所述基准姿态模型进行调整，获取被摄体的当前姿态模型，以用于捕捉所述被摄体的当前姿态。Optionally, after generating the reference pose model corresponding to the subject, the method further includes: in response to the action of the subject, acquiring the current spatial position and current label information of the first marked object; using the current space The position and the current label information are adjusted, the reference attitude model is adjusted, and the current attitude model of the subject is obtained, so as to be used to capture the current attitude of the subject.

本方法实施例可以利用第一标记对象的当前空间位置以及当前标签信息，精准地捕捉人体运动的动作，捕捉的姿态与真实的人体姿态匹配度较高。This embodiment of the method can use the current spatial position and current label information of the first marked object to accurately capture the motion of the human body, and the captured posture has a high degree of matching with the real human body posture.

可选地，获取第一标记对象的当前标签信息包括：获取所述当前标签信息的预测空间位置和所述第一标记对象的当前空间位置；在确定所述第一标记对象的当前空间位置处于所述当前标签信息的预测空间位置的预设范围内的情况下，将所述第一标记对象与所述当前标签信息进行匹配，获得匹配关系，其中，所述预设范围是根据对所述被摄体的动作轨迹预测所设定的范围；根据所述匹配关系，确定所述第一标记对象对应的所述当前标签信息。Optionally, acquiring the current label information of the first marked object includes: acquiring the predicted spatial position of the current label information and the current spatial position of the first marked object; after determining that the current spatial position of the first marked object is in the In the case that the predicted spatial position of the current label information is within a preset range, the first marked object is matched with the current label information to obtain a matching relationship, wherein the preset range is based on the The range set by the motion trajectory prediction of the subject; according to the matching relationship, the current label information corresponding to the first marked object is determined.

可选地，所述利用所述当前空间位置以及所述当前标签信息对所述被摄体对应的所述基准姿态模型进行调整获取被摄体的当前姿态模型以用于捕捉所述被摄体的当前姿态包括：在所述基准姿态模型的形体参数确定的情况下，通过不断调整所述动作参数使所有所述标签信息的虚拟空间位置与所有所述标签信息对应的所述第一标记对象的空间位置距离之和最小，获取所述当前姿态模型，以用于捕捉所述被摄体的姿态。Optionally, adjusting the reference attitude model corresponding to the subject by using the current spatial position and the current label information to obtain the current attitude model of the subject for capturing the subject. The current posture includes: when the physical parameters of the reference posture model are determined, by continuously adjusting the action parameters, the virtual space positions of all the label information are made to correspond to the first marked object of the label information. The sum of the distances between the spatial positions is the smallest, and the current posture model is obtained to capture the posture of the subject.

可选地，所述方法，还包括：利用预先设置的动作库生成的先验动作模型，对所述当前姿态模型进行约束，获取约束后的当前姿态模型。Optionally, the method further includes: using a priori action model generated by a preset action library, constraining the current posture model, and obtaining a constrained current posture model.

本方法实施例可以利用先验动作模型，对动作进行约束，避免捕捉表演者的表演是出现不合理或者不连续的动作。可以解决由于遮挡原因导致标记点缺失时，捕捉到不合理不连续的动作的问题。This embodiment of the method can use a priori action model to constrain actions to avoid capturing unreasonable or discontinuous actions in a performer's performance. It can solve the problem of capturing unreasonable and discontinuous movements when markers are missing due to occlusion.

此外，在演员表演过程中，可能由于表演需求，会出现一些自遮挡情况，如双手抱胸，手臂、胸部的点不可避免造成了遮挡，蹲在地上，腿部、肚子上的反光点也造成了遮挡。此时，公式中的标记点会有缺失，抱胸时无法找到胸部标签点对应的三维坐标，可以通过加入先验信息，来，来捕捉表演者的表演，避免捕捉出不合理的动作。In addition, during the performance of the actors, there may be some self-blocking situations due to the performance requirements, such as holding the chest with both hands, the points of the arms and chest inevitably cause blockage, and squatting on the ground, the reflection points on the legs and stomach also cause blocked. At this time, the marker points in the formula will be missing, and the three-dimensional coordinates corresponding to the chest label points cannot be found when holding the chest. You can capture the performer's performance by adding prior information, so as to avoid capturing unreasonable actions.

此外，还可以加入先验动作模型对捕捉的动作进行限定，以免生成被摄体无法做出的动作。利用预先设置的动作库生成的先验动作模型，对所述当前姿态模型进行约束，获取约束后的当前姿态模型。In addition, a prior action model can also be added to limit the captured actions, so as to avoid generating actions that the subject cannot perform. Using a priori action model generated by a preset action library, the current posture model is constrained, and the constrained current posture model is obtained.

例如，在被摄体为人类的情况下，由于人体骨骼的自由度很高，如果不加以约束，会生成一些人体做不到的动作，此外，人体在作出连续性动作时，这些连续性动作要连贯且合理。For example, when the subject is a human being, due to the high degree of freedom of the human skeleton, if it is not constrained, some actions that the human body cannot do will be generated. In addition, when the human body performs continuous actions, these continuous actions Be consistent and reasonable.

可选地，所述方法，还包括：确定与所述被摄体执行互动的互动道具；通过在所述互动道具上附着的道具标记对象，获取所述互动道具的道具空间位置及道具标签信息；利用所述道具空间位置以及道具标签信息，对与所述互动道具对应的基本道具姿态模型进行调整，生成当前道具姿态模型，以用于捕捉所述互动道具的运动。Optionally, the method further includes: determining an interactive prop that interacts with the subject; obtaining the prop space position and prop label information of the interactive prop through a prop tag object attached to the interactive prop ; Using the prop space position and prop label information, adjust the basic prop pose model corresponding to the interactive prop to generate a current prop pose model for capturing the movement of the interactive prop.

可选地，确定附着在被摄体上的第一标记对象的空间位置包括：利用至少两个相机对第一标记对象拍摄的图像，确定第一标记的对象的至少两个二维位置；利用所述至少两个二维位置以及所述至少两个相机的标定信息，确定第一标记对象的空间位置。Optionally, determining the spatial position of the first marked object attached to the subject includes: using images captured by at least two cameras of the first marked object to determine at least two two-dimensional positions of the first marked object; The at least two two-dimensional positions and the calibration information of the at least two cameras determine the spatial position of the first marked object.

可选地，利用所述至少两个二维位置以及所述至少两个相机的标定信息确定第一标记对象的空间位置包括：利用所述至少两个二维位置以及所述至少两个相机的标定信息，确定所述至少两个相机对应于第一标记对象的至少两条射线；通过第一标记对象在所述空间位置上距所述至少两条射线的距离最小的方式，确定第一标记对象的空间位置。Optionally, using the at least two two-dimensional positions and the calibration information of the at least two cameras to determine the spatial position of the first marked object includes: using the at least two two-dimensional positions and the calibration information of the at least two cameras. Calibration information, determining that the at least two cameras correspond to at least two rays of the first marking object; determining the first marking in a way that the distance between the first marking object and the at least two rays at the spatial position is the smallest The spatial location of the object.

可选地，所述空间位置包括第一标记对象在用于捕捉所述被摄体的捕捉空间对应的空间坐标系内的坐标数据。Optionally, the spatial position includes coordinate data of the first marked object in a spatial coordinate system corresponding to a capture space for capturing the subject.

可选地，所述方法，还包括：通过对用于捕捉所述被摄体的所有相机执行标定，获取所有相机的标定信息；根据所述所有相机的标定信息和具有第二标记对象的标记装置设置比例关系；利用所述所有相机的标定信息对所述捕捉空间的地面进行标定，确定地面信息以及利用所述地面信息确定的所述空间坐标系。Optionally, the method further includes: obtaining calibration information of all cameras by performing calibration on all cameras used to capture the subject; according to the calibration information of all cameras and a marker having a second marker object The device sets a proportional relationship; uses the calibration information of all the cameras to calibrate the ground of the captured space, and determines the ground information and the space coordinate system determined by using the ground information.

可选地，所述确定附着在被摄体上的第一标记对象的空间位置以及第一标记对象在所述被摄体的部位的标签信息还包括：对所述第一标记对象和所述标签信息进行匹配，获得所述第一标记对象和所述标签信息的对应关系。Optionally, the determining the spatial position of the first marked object attached to the subject and the label information of the position of the first marked object on the subject further includes: comparing the first marked object and the The tag information is matched to obtain the corresponding relationship between the first tag object and the tag information.

本申请实施例还提供一种姿态捕捉方法，所述方法包括：响应于被摄体的动作，确定附着在所述被摄体上的第一标记对象的当前空间位置以及用于描述第一标记对象在所述被摄体的部位的标签信息；利用第一标记对象的当前空间位置和标签信息，对被摄体的基准姿态模型进行调整，获取被摄体的当前姿态模型，从而捕捉到所述被摄体的当前姿态。The embodiment of the present application further provides a gesture capturing method, the method includes: in response to the action of the subject, determining the current spatial position of the first marker object attached to the subject and describing the first marker Label information of the object on the part of the subject; use the current spatial position and label information of the first marked object to adjust the reference attitude model of the subject, and obtain the current attitude model of the subject, so as to capture the the current pose of the subject.

本方法实施例可以通过响应演员的动作，并且根据第一标记对象的空间位置和标签信息，精确地捕捉出与演员动作匹配的动作。This embodiment of the method can accurately capture the action matching the action of the actor by responding to the action of the actor and according to the spatial position and label information of the first marked object.

本申请实施例还提供一种姿态捕捉装置，所述装置包括：标签信息确定单元，用于确定附着在被摄体上的第一标记对象的空间位置以及用于描述第一标记对象在所述被摄体的部位的标签信息；基准姿态模型生成单元，用于利用第一标记对象的所述空间位置以及所述标签信息对初始姿态模型进行调整，生成与所述被摄体对应的基准姿态模型。An embodiment of the present application further provides a gesture capture device, the device includes: a label information determination unit for determining the spatial position of the first marked object attached to the subject and for describing the location of the first marked object in the Label information of the part of the subject; a reference pose model generating unit, used to adjust the initial pose model by using the spatial position of the first marked object and the label information, and generate a reference pose corresponding to the subject Model.

本申请实施例还提供一种电子设备，包括：一个或多个处理器；存储器；以及一个或多个程序，其中所述一个或多个程序存储在所述存储器中并被配置为由所述一个或多个处理器执行，所述一个或多个程序包括用于执行以上方法。Embodiments of the present application further provide an electronic device, including: one or more processors; a memory; and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the One or more processors execute, the one or more programs included for performing the above method.

本申请实施例还提供一种存储一个或多个程序的计算机可读存储介质，所述一个或多个程序包括指令，所述指令当由计算设备执行时，使得所述计算设备执行所述的方法。Embodiments of the present application further provide a computer-readable storage medium storing one or more programs, where the one or more programs include instructions, and when executed by a computing device, the instructions cause the computing device to execute the described method.

本申请实施例采用的上述至少一个技术方案能够达到以下有益效果：The above-mentioned at least one technical solution adopted in the embodiments of the present application can achieve the following beneficial effects:

综上可述，根据本申请的示例性实施例的姿态捕捉方法能够在不对第一标记对象进行约束的情况下仅利用第一标记对象的空间位置以及标签信息即可确定与被摄体对应的基准姿态模型，无需将第一标记对象限制在骨骼点上，使用起来更加灵活，能够更加灵活且方便地获取与被摄体对应的基准姿态模型。To sum up, the gesture capture method according to the exemplary embodiment of the present application can only use the spatial position and label information of the first marked object to determine the object corresponding to the subject without constraining the first marked object. The reference pose model does not need to limit the first marked object to the skeleton point, is more flexible to use, and can acquire the reference pose model corresponding to the subject more flexibly and conveniently.

标记点不需要放置在演员的骨节点处，可以放置在演员身上的任意位置；增加了演员身上布点的自由度。获取到与演员匹配的模型，使得捕捉演员表演的精度进一步得以提升。通过姿态数据库建立了人工智能人体姿态先验模型，并且在捕捉中使用该先验模型，使得在演员表演时，由于自遮挡造成了重建的反光点大量缺失时，仍然能够捕捉到演员的表演。The marker point does not need to be placed at the bone node of the actor, and can be placed at any position on the actor's body; it increases the degree of freedom of point placement on the actor's body. Obtaining a model that matches the actor further improves the accuracy of capturing the actor's performance. The artificial intelligence human pose prior model is established through the pose database, and the prior model is used in the capture, so that the actor's performance can still be captured when a large number of reconstructed reflective points are missing due to self-occlusion.

附图说明Description of drawings

此处所说明的附图用来提供对本申请的进一步理解，构成本申请的一部分，本申请的示意性实施例及其说明用于解释本申请，并不构成对本申请的不当限定。在附图中：The drawings described herein are used to provide further understanding of the present application and constitute a part of the present application. The schematic embodiments and descriptions of the present application are used to explain the present application and do not constitute an improper limitation of the present application. In the attached image:

图1是根据本申请的示例性实施例的姿态捕捉方法的步骤流程图；1 is a flowchart of steps of a gesture capturing method according to an exemplary embodiment of the present application;

图2是根据本申请的示例性实施例的利用第二标记装置对多个相机执行标定操作的示图；2 is a diagram of performing a calibration operation on a plurality of cameras using a second marking device according to an exemplary embodiment of the present application;

图3是根据本申请的示例性实施例的对多个相机执行标定操作的框图；3 is a block diagram of performing a calibration operation on a plurality of cameras according to an exemplary embodiment of the present application;

图4是根据本申请的示例性实施例的获取第一标记对象的空间位置的框图；4 is a block diagram of obtaining a spatial position of a first marked object according to an exemplary embodiment of the present application;

图5是根据本申请的示例性实施例的空间坐标匹配的示图；5 is a diagram of spatial coordinate matching according to an exemplary embodiment of the present application;

图6是根据本申请的示例性实施例的生成被摄体的基准姿态模型的框图；6 is a block diagram of generating a reference pose model of a subject according to an exemplary embodiment of the present application;

图7是根据本申请的示例性实施例的获取当前姿态模型的框图；7 is a block diagram of obtaining a current pose model according to an exemplary embodiment of the present application;

图8是根据本申请的示例性实施例的确定当前标签信息的框图；8 is a block diagram of determining current tag information according to an exemplary embodiment of the present application;

图9是根据本申请的示例性实施例的姿态捕捉装置的框图；9 is a block diagram of a gesture capture device according to an exemplary embodiment of the present application;

图10是根据本申请的示例性实施例的姿态捕捉时第一标记对象在人***置的示意图。FIG. 10 is a schematic diagram of the position of the first marker object on the human body during gesture capture according to an exemplary embodiment of the present application.

具体实施方式Detailed ways

为使本申请的目的、技术方案和优点更加清楚，下面将结合本申请具体实施例及相应的附图对本申请技术方案进行清楚、完整地描述。显然，所描述的实施例仅是本申请一部分实施例，而不是全部的实施例。基于本申请中的实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例，都属于本申请保护的范围。In order to make the purpose, technical solutions and advantages of the present application clearer, the technical solutions of the present application will be described clearly and completely below in conjunction with the specific embodiments of the present application and the corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of the present application, but not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of this application.

以下结合附图，详细说明本申请各实施例提供的技术方案。为了能够解决以上技术问题，本申请的示例性实施例的姿态捕捉方法可利用至少两个相机，确定对附着在被摄体上的第一标记对象的空间位置，并利用该空间位置生成被摄体的基准姿态模型。并且在被摄体作出动作后，利用第一标记对象的空间位置以及标签信息，可确定基准姿态模型中的动作参数，以生成与该动作对应的当前姿态模型，达到捕捉被摄体的姿态的目的。The technical solutions provided by the embodiments of the present application will be described in detail below with reference to the accompanying drawings. In order to solve the above technical problems, the gesture capture method of the exemplary embodiment of the present application may use at least two cameras to determine the spatial position of the first marker object attached to the subject, and use the spatial position to generate the subject The baseline pose model of the body. And after the subject makes an action, the spatial position and label information of the first marked object can be used to determine the action parameters in the reference posture model, so as to generate the current posture model corresponding to the action, so as to achieve the goal of capturing the posture of the subject. Purpose.

本申请的示例性实施例的姿态捕捉方法可应用于各个领域，包括且不限于动画领域、体育领域、游戏制作、动作制作以及影视制作。The gesture capturing method of the exemplary embodiment of the present application can be applied to various fields, including but not limited to the animation field, sports field, game production, action production, and film and television production.

图1是根据本申请的示例性实施例的姿态捕捉方法的步骤流程图。FIG. 1 is a flowchart of steps of a gesture capturing method according to an exemplary embodiment of the present application.

如图1所示，在步骤S110，确定附着在被摄体上的第一标记对象的空间位置以及用于描述第一标记对象在所述被摄体的部位的标签信息。As shown in FIG. 1 , in step S110 , the spatial position of the first marking object attached to the subject and the label information for describing the position of the first marking object on the subject are determined.

在实施中，所述被摄体是指被相机拍摄的对象，所述对象可以是能够自行作出各种动作的生命体，包括不限于人类(包括男性或者女性)或动物(例如，熊猫、马等)，还可以是接收指令后执行各种动作的机械体，例如，自动行走设备(诸如，机器人)，还可以是非生命体，例如，配合被摄体作出各种动作的互动道具，诸如，足球、篮球或者捧花等。In implementation, the subject refers to an object photographed by a camera, and the object may be a living body capable of performing various actions on its own, including but not limited to humans (including men or women) or animals (for example, pandas, horses, etc.) etc.), it can also be a mechanical body that performs various actions after receiving instructions, for example, an automatic walking device (such as a robot), or a non-living body, for example, an interactive prop that cooperates with the subject to perform various actions, such as, Football, basketball or bouquets etc.

此外，根据本申请的示例性实施例的被摄体可以是单个拍摄对象，即，对单个对象进行姿态捕捉，也可以是多个拍摄对象，在多个拍摄对象的情况下，这些拍摄对象可以各自执行各自的动作，也可以发生各种互动。例如，两个被摄体可拥抱在一起。In addition, the subject according to the exemplary embodiment of the present application may be a single subject, that is, a single subject is gesture-captured, or may be multiple subjects, and in the case of multiple subjects, these subjects may be Each performs its own actions, and various interactions can also occur. For example, two subjects can hug each other.

标记对象是指表面覆盖有特殊反光材料的标记物(marker点)，例如球形的标记物。在本申请中的第一标记对象与第二标记对象仅为了在命名上进行区分。在实施中，可利用相机发射红外光，该红外光经过标记物后反射并获取到该标记物的平面坐标(即，二维坐标)。此外，本申请中提及的第一标记对象和第二标记对象均未对数量进行限制，也就是说，可能存在多个第一标记对象以及多个第二标记对象。针对每个第一标记对象和第二次标记对象均可按照以下方式执行处理。The marking object refers to a marker (marker point) whose surface is covered with a special reflective material, such as a spherical marker. The first marked object and the second marked object in this application are only distinguished by naming. In an implementation, a camera can be used to emit infrared light, which is reflected after passing through the marker and obtains the plane coordinates (ie, two-dimensional coordinates) of the marker. In addition, neither the first marking object nor the second marking object mentioned in the present application is limited in quantity, that is, there may be multiple first marking objects and multiple second marking objects. Processing may be performed in the following manner for each of the first marked object and the second marked object.

此外，所述空间位置包括第一标记对象在用于捕捉所述被摄体的捕捉空间内的坐标数据。具体来说，为了能够捕捉所述被摄体的姿态，首先需要为被摄体搭建一个标定场地，其中，标定场地由多个相机构成，然后在搭建好标定场地后，利用这些相机以及第二标记对象，确定与该标定场地对应的虚拟的捕捉空间，然后确定该捕捉空间对应的空间坐标系。以下将结合图2和图3对相机标定操作进行描述。Furthermore, the spatial position includes coordinate data of the first marker object within a capture space for capturing the subject. Specifically, in order to be able to capture the posture of the subject, it is first necessary to build a calibration site for the subject, wherein the calibration site is composed of multiple cameras, and after the calibration site is built, use these cameras and the second Mark the object, determine the virtual capture space corresponding to the calibration site, and then determine the space coordinate system corresponding to the capture space. The camera calibration operation will be described below with reference to FIGS. 2 and 3 .

图2是根据本申请的示例性实施例的利用标记装置对多个相机执行标定操作的示图。图3是根据本申请的示例性实施例的对多个相机执行标定操作的框图。FIG. 2 is a diagram of performing a calibration operation on a plurality of cameras using a marking device according to an exemplary embodiment of the present application. 3 is a block diagram of performing a calibration operation on a plurality of cameras according to an exemplary embodiment of the present application.

如图2所示，这些相机构成了标定空间。然后，可利用如图2中的标定装置(例如，标定杆)对其进行标定，其中，所述标定装置上设置标记对象(即，第二标记对象)，优选地，可在标定装置上设置有三个标记对象。As shown in Figure 2, these cameras constitute the calibration space. Then, it can be calibrated using a calibration device (eg, a calibration rod) as shown in FIG. 2 , wherein a marking object (ie, a second marking object) is provided on the calibration device, preferably, the calibration device can be set on There are three marker objects.

随后使用标定装置扫场，具体地，具有第二标记对象的标记装置可以是具有标记点的标定杆。在实施中，用户(例如，技术人员)可在标定场地中挥动具有标记点的标定杆，每个相机获取标记点的二维坐标，根据二维坐标对所有相机进行标定，获得所有相机之间的标定信息，标定信息包括相机之间相对位置关系以及相机的内参。其中，标定场地是真实空间。The field is then scanned using a marking device, in particular, the marking device with the second marking object may be a marking rod with marking points. In implementation, a user (eg, a technician) can swing a calibration rod with a marker in the calibration field, each camera obtains the two-dimensional coordinates of the marker, and all cameras are calibrated according to the two-dimensional coordinates, and the distance between all cameras is obtained. The calibration information includes the relative positional relationship between the cameras and the internal parameters of the cameras. Among them, the calibration site is the real space.

如图2所示，图2中的每个相机可拍摄到包括有标记对象的标定杆的图像，并计算出标定信息。在实施中，图2中相机可至少包括两个相机。As shown in FIG. 2 , each camera in FIG. 2 can capture an image of a calibration rod including a marked object, and calculate calibration information. In an implementation, the cameras in FIG. 2 may include at least two cameras.

具体来说，如块301所示，可排除捕捉空间内的各个反光点。由于在场地中，不可避免有一些反光点被相机捕捉，因此需要对相机进行测试，将影响到相机捕捉的反光点进行排除，即确保相机捕获到就是标记对象。Specifically, as indicated by block 301, individual reflective spots within the capture space may be excluded. Since some reflective points are inevitably captured by the camera in the field, it is necessary to test the camera to exclude the reflective points that affect the camera's capture, that is, to ensure that the camera captures the marked object.

随后，如块302所示，使用标定装置进行扫场。该标定装置上可安装有三个共线的第二标记对象，这三个标记对象之间的距离是确定的。在该标定空间中，挥动该标定装置，这些相机可捕获到这三个标记对象点的平面位置，最后在每个相机获取到以上平面位置后扫场完毕。Then, as indicated by block 302, a field sweep is performed using the calibration device. Three collinear second marking objects can be installed on the calibration device, and the distance between the three marking objects is determined. In the calibration space, by waving the calibration device, these cameras can capture the plane positions of the three marked object points, and finally scan the field after each camera obtains the above plane positions.

随后，如块303所示，确定所有相机的标定信息，其中，所述标定信息包括相机的参数信息、相对位置与尺度信息。在实施中，所述参数信息包括相机的内参以及外参，内参是指相机的内部参数，包括焦距、畸变参数等，外参是指相机的位置、朝向。Then, as shown in block 303, the calibration information of all cameras is determined, wherein the calibration information includes parameter information, relative position and scale information of the cameras. In implementation, the parameter information includes internal parameters and external parameters of the camera, the internal parameters refer to the internal parameters of the camera, including focal length, distortion parameters, etc., and the external parameters refer to the position and orientation of the camera.

在实施中，可通过所述相机拍摄在所述标定空间内的具有第二标记对象的标记装置，获取所述标记装置的图像；通过所述相机中的所有相机在对应图像的同一位置的位置，确定所述相机的参数信息和相对位置，最后，还可通过标定装置中第二标记对象之间的距离进行确定比例关系。In implementation, a marking device with a second marking object in the calibration space can be photographed by the camera to obtain an image of the marking device; the positions of all cameras in the cameras at the same position corresponding to the image can be used , to determine the parameter information and relative position of the camera, and finally, the proportional relationship can also be determined by the distance between the second marked objects in the calibration device.

具体来说，在对相机标定的同时，利用上述标定好的所有相机捕捉标定杆上的标记点，在捕捉空间中重建出捕捉到的标记点的三维坐标，将重建出来的标记点的三维坐标的距离与实际标定杆上标记点之间的距离进行比较，获得比例关系，比例关系用于后续计算。Specifically, while calibrating the camera, use all the above calibrated cameras to capture the marked points on the calibration rod, reconstruct the three-dimensional coordinates of the captured marked points in the capture space, and use the reconstructed three-dimensional coordinates of the marked points. Compare the distance between the marked points on the actual calibration rod to obtain a proportional relationship, which is used for subsequent calculations.

同时，如块304所示，可在捕捉空间内放置三角板(三角板上3个顶点分别有标记对象)对地面进行标定，从而确定地面信息。具体来说，在标定场地上放置每个角上具有标记点的L型三角杆。在捕捉空间中重建出L型三角杆上三个标记点的三维坐标，在捕捉空间形成虚拟L型三角杆，虚拟L型三角杆的直角点是原点，短边是Z轴，长边是X轴，通过X轴和Z轴可以建立出Y轴，通过X轴和Z轴可以建立捕捉空间的地面信息，其中，原点以及X轴、Y轴和Z轴是捕捉空间中的空间坐标系。其中，捕捉空间是虚拟空间。At the same time, as shown in block 304, a triangular board (marked objects are respectively provided on the three vertices of the triangular board) can be placed in the capture space to calibrate the ground, so as to determine the ground information. Specifically, an L-shaped triangular pole with marked points on each corner is placed on the calibration field. The three-dimensional coordinates of the three marked points on the L-shaped triangular rod are reconstructed in the capture space, and a virtual L-shaped triangular rod is formed in the capture space. The right-angle point of the virtual L-shaped triangular rod is the origin, the short side is the Z axis, and the long side is the X axis The Y axis can be established through the X axis and the Z axis, and the ground information of the capture space can be established through the X axis and the Z axis, wherein the origin and the X axis, the Y axis and the Z axis are the spatial coordinate systems in the capture space. Among them, the capture space is a virtual space.

最后，如块305所示，利用所述标定信息以及在块304中确定的地面信息，确定所述空间坐标系。也就是说，在确定了捕捉空间的地面信息后，可确定以地面信息为基础的捕捉空间的空间坐标系。Finally, as shown in block 305, using the calibration information and the ground information determined in block 304, the spatial coordinate system is determined. That is, after the ground information of the capture space is determined, the space coordinate system of the capture space based on the ground information can be determined.

确定捕捉空间的空间坐标系后，可确定第一标记对象的空间位置。以下将参照图4进行详细描述。图4是根据本申请的示例性实施例的获取第一标记对象的空间位置的框图。After the spatial coordinate system of the capture space is determined, the spatial position of the first marker object can be determined. A detailed description will be made below with reference to FIG. 4 . FIG. 4 is a block diagram of obtaining a spatial position of a first marked object according to an exemplary embodiment of the present application.

如块401，可利用若干相机获取到若干第一标记对象的二维位置。在实施中，利用至少两个相机对每个第一标记对象进行拍摄，获取上述至少两个相机拍摄的同一第一标记对象的至少两个图像，然后，利用该至少两个图像获取针对同一标记对象的至少两个二维位置。在块402，获取上述至少两个相机的标定信息。随后，在块403，可利用上述至少两个相机的标定信息以及与其对应的至少两个二维位置生成对应于同一第一标记对象的至少两条射线。As in block 401, the two-dimensional positions of several first marked objects may be acquired using several cameras. In the implementation, each first marked object is photographed with at least two cameras, at least two images of the same first marked object photographed by the at least two cameras are acquired, and then, the at least two images are used to obtain images for the same mark At least two two-dimensional positions of the object. At block 402, the calibration information of the at least two cameras is obtained. Then, at block 403, at least two rays corresponding to the same first marker object may be generated using the calibration information of the at least two cameras and at least two two-dimensional positions corresponding thereto.

随后，如块404，可根据各种约束条件，获取到不同相机对于同一标记对象的对应关系。并且利用相机的参数信息为每个二维位置生成对应的一条射线。Then, as in block 404, the correspondence between different cameras for the same marked object may be acquired according to various constraints. And use the parameter information of the camera to generate a corresponding ray for each two-dimensional position.

最后，在块406，在获取以上对应关系后，可以通过使不同相机针对同一第一标记对象生成的射线相交，确定该第一标记对象的三维位置。也就是说，找出与所有射线的距离最小的点作为该标记对象的三维坐标点。Finally, at block 406, after obtaining the above correspondence, the three-dimensional position of the first marked object may be determined by intersecting rays generated by different cameras for the same first marked object. That is to say, find the point with the smallest distance from all rays as the three-dimensional coordinate point of the marked object.

在实施中，这些射线可能无法相交于一点，可采用如块405所示的优化处理，使得重建的三维位置更稳定。简言之，所述优化处理可根据生成的三维坐标点与不同射线的距离不同，通过迭代性调节不同射线的权重，使得生成的三维坐标点与大部分射线的距离最近。In an implementation, these rays may not intersect at a point, and an optimization process as shown in block 405 may be employed to make the reconstructed three-dimensional position more stable. In short, the optimization process may adjust the weights of different rays iteratively according to the different distances between the generated three-dimensional coordinate points and different rays, so that the distances between the generated three-dimensional coordinate points and most of the rays are the closest.

以上给出了针对单个第一标记对象确定空间位置的处理过程，在实施中，在多个第一标记对象的情况下，可针对每个第一标记对象均采用以上处理过程获取对应的空间坐标。The processing procedure for determining the spatial position of a single first marked object is given above. In the implementation, in the case of multiple first marked objects, the above processing procedure can be used for each first marked object to obtain the corresponding spatial coordinates .

为了更好地解释，以下将结合图5进行描述。图5是根据本申请的示例性实施例的空间坐标匹配的示图。如图5所示，第一标记对象可通过不同的相机生成不同的图像510和图像520。第一标记对象在图像510中的二维位置为P _L, 第一标记对象在图像520中的二维位置为P _R。图像510对应的相机的光学中心为O _L，图像520对应的相机的光学中心为O _R。这样形成的射线P _LO _L以及P _RO _R可相交于点P，则点P为第一标记对象重建后的空间位置。图5可称为为第一标记对象的空间位置的三维重建过程。 For better explanation, the following description will be made in conjunction with FIG. 5 . FIG. 5 is a diagram of spatial coordinate matching according to an exemplary embodiment of the present application. As shown in FIG. 5 , the first marked object may generate

different images

510 and 520 by different cameras. The two-dimensional position of the first marked object in the image 510 is P _L , and the two-dimensional position of the first marked object in the image 520 is P _R . The optical center of the camera corresponding to the image 510 is O _L , and the optical center of the camera corresponding to the image 520 is _OR . The rays P _L O _L and P _R O _R formed in this way can intersect at the point P, and then the point P is the reconstructed spatial position of the first marking object. Figure 5 may be referred to as a three-dimensional reconstruction process for the spatial position of the first marked object.

随后，可确定第一标记对象在被摄体上的部位。优选地，预先设定标记对象集(markerset)，即，预先设定好在被摄体的哪些部位的哪些位置贴第一标记对象。例如，在被摄体的某个部分的可以贴多个标记对象，且多个标记对象在该部位的不同位置。将定义的贴在某个部位的某个位置称之为标签信息。Subsequently, the location of the first marking object on the subject may be determined. Preferably, a marker object set (marker set) is preset, that is, it is preset in which parts and positions of the object the first marker objects are to be affixed. For example, multiple marking objects can be attached to a certain part of the subject, and the multiple marking objects are at different positions of the part. Sticking the definition to a certain position in a certain part is called label information.

在实施中，可让被摄体摆出特定姿势(例如，人型姿势或者T型姿势)，然后获取放置在被摄体的动捕服上的各个第一标记对象上的空间位置，根据预先设定markerset确定每个第一标记对象的标签信息，例如，可将位于最上部且中间位置的第一标记对象的标签信息确定为头部的上部位置。In the implementation, the subject can be made to assume a specific posture (for example, a human-shaped posture or a T-shaped posture), and then the spatial position of each first marked object placed on the subject's motion capture suit can be obtained. The markerset is set to determine the label information of each first marker object, for example, the label information of the first marker object located at the uppermost and middle position may be determined as the upper position of the head.

随后，可执行步骤S120，利用第一标记对象的所述空间位置以及所述标签信息对初始姿态模型进行调整，生成与所述被摄体对应的基准姿态模型。为了更好地描述该步骤，以下将参照图6进行描述。Then, step S120 may be performed to adjust the initial posture model by using the spatial position of the first marked object and the label information, and generate a reference posture model corresponding to the object. In order to better describe this step, it will be described below with reference to FIG. 6 .

图6是根据本申请的示例性实施例的生成被摄体的基准姿态模型的框图。FIG. 6 is a block diagram of generating a reference pose model of a subject according to an exemplary embodiment of the present application.

在块610，技术人员可通过三维扫描获取到大量的模型数据，以被摄体为人类为例，所述姿态数据库可包括各种体态和/或动作的姿态数据，例如，高矮胖瘦、男女等各类人体。At block 610, the technician can obtain a large amount of model data through 3D scanning. Taking the subject as a human as an example, the posture database can include posture data of various postures and/or actions, such as height, shortness, fatness, thinness, male and female and other types of human body.

在块620，可利用块610中的姿态数据库生成低维人体分布。可以对该分布进行取样，生成不同形态的人体。At block 620 , a low-dimensional body distribution may be generated using the pose database in block 610 . This distribution can be sampled to generate human bodies in different shapes.

在块630，建立初始姿态模型，其中，所述初始姿态模型包括用于描述形体的形体参数以及用于描述动作的动作参数。如以下公式1所示，FK用于指示初始姿态模型，

可分别代表身高和胖瘦，pose代表被摄体的姿态，由于

和pose均是未知的，因此，需要利用第一标记对象的空间位置对其进行求解。 At block 630, an initial pose model is established, wherein the initial pose model includes body parameters for describing a body and motion parameters for describing an action. As shown in Equation 1 below, FK is used to indicate the initial pose model,

It can represent height and fat and thin respectively, and pose represents the posture of the subject.

and pose are unknown, so it needs to be solved using the spatial position of the first marked object.

在块640，获取第一标记对象的空间位置和标签信息，在块650，利用第一标记对象的所述空间位置以及所述标签信息对初始姿态模型进行调整，生成与所述被摄体对应的基准姿态模型。At block 640, the spatial position and label information of the first marked object are obtained, and at block 650, the initial pose model is adjusted by using the spatial position and the label information of the first marked object to generate a corresponding image of the object. the baseline pose model.

可选地，在步骤S110，确定附着在被摄体上的第一标记对象的空间位置以及用于描述第一标记对象在所述被摄体的部位的标签信息中，还可以包括：对所述第一标记对象和所述标签信息进行匹配，获得所述第一标记对象和所述标签信息的对应关系。Optionally, in step S110, determining the spatial position of the first marked object attached to the subject and the label information used to describe the position of the first marked object on the subject may further include: The first marking object and the label information are matched to obtain the corresponding relationship between the first marking object and the label information.

在实施中，由于α,ρ和pose均是未知的，因此可获取第一标记对象在不同时间点和/或被摄体的不同动作下的各个空间位置和各个标签信息，然后利用这些空间位置和各个标签信息进行求解，确定FK中的参数，并且FK中的参数

要满足低维人体分布。应注意，第一标记对象在空间位置发生变化的情况下如何确定其标签信息，将在以下结合图7进行详细解释，在此将不再赘述。 In implementation, since α, ρ and pose are all unknown, each spatial position and each label information of the first marked object at different time points and/or different actions of the subject can be obtained, and then these spatial positions can be used. Solve with each label information, determine the parameters in FK, and the parameters in FK

To meet the low-dimensional human distribution. It should be noted that how to determine the label information of the first marked object when the spatial position changes will be explained in detail below with reference to FIG. 7 , and will not be repeated here.

优选地，可通过使所述被摄体做出特定动作(例如，摆出T型)，设置所述初始姿态模型中的动作参数。为了使结果更准确，设定的动作参数为标准的特定动作。在所述动作参数确定的情况下，获取第一标记对象在所述被摄体做出所述特定动作的情况下的所述空间位置以及所述标签信息。Preferably, the action parameters in the initial posture model can be set by causing the subject to perform a specific action (eg, posing in a T-shape). In order to make the results more accurate, the set action parameters are standard specific actions. When the action parameter is determined, the spatial position and the label information of the first marked object when the subject performs the specific action are acquired.

最后，利用所述动作参数、所述空间位置以及所述标签信息，对所述初始姿态模型的形体参数进行调整，生成与所述被摄体对应的所述基准姿态模型。也就是说，利用以下公式1的方式不断调整FK模型中的形体参数直至公式1收敛。Finally, using the motion parameters, the spatial position and the label information, the body parameters of the initial posture model are adjusted to generate the reference posture model corresponding to the subject. That is to say, the shape parameters in the FK model are continuously adjusted using the following formula 1 until formula 1 converges.

在实施中，可利用如下公式1，对初始姿态模型进行调整：In implementation, the following formula 1 can be used to adjust the initial attitude model:

公式中，α,ρ可分别代表被摄体的形体参数(身高和胖瘦)，pose代表被摄体的动作参数(姿态)；FK代表姿态模型，利用α,ρ,pose以及FK姿态模型，可重建出与被摄体对应的带有动作的虚拟人体模型。In the formula, α and ρ can represent the body parameters (height, fat and thin) of the subject, respectively, and pose represents the action parameters (posture) of the subject; FK represents the posture model, using α, ρ, pose and the FK posture model, A virtual human body model with actions corresponding to the subject can be reconstructed.

Corr表示标签信息与第一标记对象的匹配关系，即标签信息i对应哪个第一标记对象(或第一标记对象m属于哪个标签信息)。i代表该第一标记对象的标签信息，公式中bodyMarker _i表示第i个标签信息在与被摄体对应的虚拟人体模型上的位置，通过bodyMarker _i(FK(α，ρ，pose)可以得出第i个标签信息的虚拟三维坐标，Marker _m表示第m个第一标记对象的三维坐标。 Corr represents the matching relationship between the tag information and the first tag object, that is, which first tag object the tag information i corresponds to (or which tag information the first tag object m belongs to). i represents the label information of the first marked object. In the formula, bodyMarker _i represents the position of the i-th label information on the virtual human body model corresponding to the subject. It can be obtained by bodyMarker _i (FK(α, ρ, pose) The virtual three-dimensional coordinates of the i-th label information, and Marker _m represents the three-dimensional coordinates of the m-th first marker object.

标签信息与第一标记对象通过匹配关系Corr进行匹配，即标签信息i对应第m个第一标记对象。Dis表示bodyMarker _i(FK(α，ρ，pose)与Marker _m的距离。 The tag information and the first tag object are matched through the matching relationship Corr, that is, tag information i corresponds to the m-th first tag object. Dis represents the distance between bodyMarker _i (FK(α, ρ, pose) and Marker _m .

公式(1)表示获取到第一标记对象的三维坐标后，确定每个第一标记对象的标签信息(在公式中使用Corr表示第一标记对象与标签信息的匹配关系)，然后优化公式(1)中的变量，使虚拟人体模型上所有标签信息的虚拟三维坐标与所有标签信息对应的第一标记对象的三维坐标距离之和最小，来获取与被摄体对应的虚拟人体模型。The formula (1) indicates that after obtaining the three-dimensional coordinates of the first marked object, the label information of each first marked object is determined (Corr is used in the formula to represent the matching relationship between the first marked object and the label information), and then the formula (1) is optimized. ) to minimize the sum of the virtual three-dimensional coordinates of all label information on the virtual human body model and the three-dimensional coordinate distance of the first marked object corresponding to all label information to obtain the virtual human body model corresponding to the subject.

公式(1)的优化过程如下所述：The optimization process of formula (1) is as follows:

1.设定变量初始值。用定义markerset的初始值设定bodyMarker _i，即定义markerset时，bodyMarker _i表示标签信息i设定在虚拟人体模型上的位置；通过被摄体身高(第一标记对象的三维坐标高度差)，在姿态数据库的低维人体分布中获取所述身高约束下的人体分布，取约束下人体分布的均值作为α,ρ的初始值；被摄体的pose与预设的姿势接近，初始值使用预设的特定动作(如T-pose)。 1. Set the initial value of the variable. Set bodyMarker _i with the initial value of defining markerset, that is, when defining markerset, bodyMarker _i indicates the position of label information i set on the virtual human body model; Obtain the human body distribution under the height constraint from the low-dimensional human body distribution of the pose database, and take the mean value of the human body distribution under the constraint as the initial value of α, ρ; the pose of the subject is close to the preset pose, and the initial value uses the preset specific actions (such as T-pose).

2.获取标签信息与第一标记对象的匹配关系(公式(1)中的Corr)。通过bodyMarker _i(FK(α，ρ，pose)可得出标签信息的虚拟三维坐标，所有标签信息的虚拟三维坐标为集合A，所有第一标记对象的三维坐标为集合B。将集合A与集合B进行匹配，由于集合A中每个虚拟三维坐标都有标签信息，匹配后集合B中匹配成功的三维坐标具有与集合A中虚拟三维坐标一致的标签信息。 2. Obtain the matching relationship between the tag information and the first tag object (Corr in formula (1)). The virtual three-dimensional coordinates of the label information can be obtained through bodyMarker _i (FK(α, ρ, pose), the virtual three-dimensional coordinates of all label information are set A, and the three-dimensional coordinates of all the first marked objects are set B. Set A and set B performs matching. Since each virtual 3D coordinate in set A has label information, the successfully matched 3D coordinates in set B after matching have the same label information as the virtual 3D coordinates in set A.

例如，匹配方法可采用最近邻匹配方法。最近邻匹配方法指从集合A某点a出发，寻找另一个集合B中与点a距离最近的点，形成匹配的方法。在实际使用中，本发明不限制匹配方法(即，可以使用其他匹配方法)。For example, the matching method may adopt the nearest neighbor matching method. The nearest neighbor matching method refers to the method of starting from a certain point a in set A and finding the closest point to point a in another set B to form a matching method. In actual use, the present invention does not limit the matching method (ie, other matching methods can be used).

3.优化变量α,ρ，pose，bodyMarker _i，使虚拟人体模型上所有标签信息的虚拟三维坐标与所有标签信息对应的第一标记对象的三维坐标距离之和最小。 3. Optimize the variables α, ρ, pose, bodyMarker _i to minimize the sum of the virtual three-dimensional coordinates of all label information on the virtual human body model and the three-dimensional coordinate distance of the first marked object corresponding to all label information.

4.回到步骤3进行迭代优化，直到公式(1)收敛或者达到最大迭代次数。4. Go back to step 3 for iterative optimization until formula (1) converges or the maximum number of iterations is reached.

由于每个被摄体的形体各不相同，因此为了能够更准确地表征各个被摄体，因此每个被摄体在进行动作捕捉之前均需要执行以上操作，确定该被摄体的基准姿态模型，以上操作称为标定过程。并且为了更准确地生成被摄体的基准姿态模型，可将被摄体的启动动作设置为某个特定动作。举例来说，每个演员在执行动作捕捉前均作出T型动作，然后生成该演员的基准姿态模型。Since each subject has a different shape, in order to more accurately characterize each subject, each subject needs to perform the above operations before motion capture to determine the subject's baseline pose model , the above operation is called the calibration process. And in order to generate the reference pose model of the subject more accurately, the activation action of the subject can be set as a specific action. For example, each actor makes a T-movement before performing motion capture, and then generates a baseline pose model for the actor.

所述方法在已确定所述被摄体对应的基准姿态模型的情况下，可利用该基准姿态模型生成与被摄体的当前姿态对应的当前姿态模型，从而能够捕捉被摄体的当前姿态。为了更好地描述该过程，以下将结合图7进行具体描述。In the method, when the reference pose model corresponding to the object has been determined, the reference pose model can be used to generate a current pose model corresponding to the current pose of the object, so that the current pose of the object can be captured. In order to better describe the process, a detailed description will be made below with reference to FIG. 7 .

按照上述所示的操作完成对被摄体的标定操作后，可对被摄体执行捕捉操作。捕捉时，先摆出特定动作(例如，T-pose)。获取第一标记对象的三维坐标以及标签信息与第一标记对象的匹配关系。After completing the calibration operation on the subject according to the operations shown above, the capture operation can be performed on the subject. When capturing, start with a specific action (eg, T-pose). Acquire the three-dimensional coordinates of the first marked object and the matching relationship between the label information and the first marked object.

具体地，第一标记对象的当前空间位置通过三维重建获取。匹配关系的获取与标定过程类似，此时优化pose与匹配关系Corr即可，因为公式(1)中α,ρ，bodyMarker _i在标定过程中已经获取，保持不变。 Specifically, the current spatial position of the first marked object is acquired through three-dimensional reconstruction. The acquisition of the matching relationship is similar to the calibration process. At this time, the pose and the matching relationship Corr can be optimized, because α, ρ, and bodyMarker _i in formula (1) have been obtained during the calibration process and remain unchanged.

所述被摄体可根据实际需求执行各种动作，例如，演员可按照剧本作出各种表演。在这种情况下，响应于被摄体的动作，在块710，在该动作下，获取第一标记对象的当前空间位置以及当前标签信息。The subject can perform various actions according to actual needs, for example, the actor can perform various performances according to the script. In this case, in response to the action of the subject, at block 710, under the action, the current spatial position of the first marked object and the current label information are obtained.

第一标记对象的当前空间位置通过三维重建获取，第一标记对象的当前标签信息通过标签信息与第一标记对象的匹配关系获取。The current spatial position of the first marked object is obtained through three-dimensional reconstruction, and the current label information of the first marked object is obtained through the matching relationship between the label information and the first marked object.

第一标记对象的当前空间位置是指被摄体执行动作后导致的在其上的第一标记对象位置发生移动后的空间位置。在实施中，可按照如上如图4所示的方法确定第一标记对象的当前空间位置。The current spatial position of the first marked object refers to the spatial position after the position of the first marked object on the subject moves after the action is performed. In implementation, the current spatial position of the first marked object may be determined according to the method shown in FIG. 4 above.

在已经确定第一标记对象的当前空间位置的情况下，可确定第一标记对象的当前标签信息。以下将结合图8描述确定当前标签信息的处理过程。Having determined the current spatial position of the first marked object, current label information of the first marked object may be determined. The process of determining the current tag information will be described below with reference to FIG. 8 .

图8是示出根据本申请的示例性实施例的确定当前标签信息的框图。FIG. 8 is a block diagram illustrating determining current tag information according to an exemplary embodiment of the present application.

在块810，获取当前标签信息i的预测空间位置。在实施中，可根据标签信息i对应的第一标记对象的在先空间位置预测当前标签信息i在后空间位置，并将在后空间位置确定为预测空间位置，其中，在先空间位置是指标签信息i对应的第一标记对象在上一时刻(即，上一帧)的空间位置。在上一时刻中，标签信息i与第一标记对象的对应关系是确定的，即标签信息i与第一标记对象的空间位置一致。At block 810, the predicted spatial location of the current label information i is obtained. In implementation, the following spatial position of the current label information i may be predicted according to the previous spatial position of the first marked object corresponding to the label information i, and the latter spatial position is determined as the predicted spatial position, wherein the previous spatial position refers to The spatial position of the first marked object corresponding to the tag information i at the previous moment (ie, the previous frame). In the last moment, the corresponding relationship between the label information i and the first marked object is determined, that is, the spatial position of the label information i and the first marked object is consistent.

预测空间位置指当前标签信息i当前时刻(即，当前帧)预测的空间位置。在实施中，可利用针对被摄体的运动轨迹的预测方法确定所述预测空间位置，在此，将不限制所述预测方法。The predicted spatial position refers to the predicted spatial position of the current label information i at the current moment (ie, the current frame). In implementation, the predicted spatial position may be determined using a prediction method for the motion trajectory of the subject, and the prediction method will not be limited here.

在块820，获取第一标记对象P的当前空间位置。在实施中，可利用以上已经描述的方法确定当前空间位置。At block 820, the current spatial position of the first marked object P is obtained. In an implementation, the current spatial position may be determined using the methods already described above.

执行块830，判断第一标记对象P的当前空间位置是否在当前标签信息i的预测空间位置的预设范围内。如果是，即在确定第一标记对象P的当前空间位置在当前标签信息i的预测空间位置的预设范围内时，使用最近邻方法对第一标记对象P和当前标签信息i进行匹配并判断匹配是否正确(最近邻方法在上述标定时已介绍，不再赘述)，若匹配正确，则该第一标记对象P与标签信息i是有效匹配关系；如果不是，则不需要进行匹配。其中，所述预设范围是根据对被摄体的动作轨迹预测所设定的范围，通过上述过程，获取第一标记对象与标签信息的匹配关系。Go to block 830 to determine whether the current spatial position of the first marked object P is within the preset range of the predicted spatial position of the current tag information i. If yes, that is, when it is determined that the current spatial position of the first marked object P is within the preset range of the predicted spatial position of the current label information i, the nearest neighbor method is used to match the first marked object P and the current label information i and determine Whether the matching is correct (the nearest neighbor method has been introduced in the above calibration and will not be repeated here), if the matching is correct, the first marked object P and the tag information i are valid matching relationships; The preset range is a range set according to the prediction of the motion trajectory of the subject, and through the above process, the matching relationship between the first marked object and the label information is acquired.

在块840，确定当前时刻下标签信息i与第一标记对象P的匹配关系，根据所述匹配关系确定匹配成功后的第一标记对象P对应的标签信息i。In block 840, the matching relationship between the tag information i and the first tag object P at the current moment is determined, and the tag information i corresponding to the successfully matched first tag object P is determined according to the matching relationship.

在已经确定当前标签信息的情况下，在块730，利用所述当前空间位置以及所述当前标签信息，对块720中的基准姿态模型进行调整，获取被摄体的当前姿态模型，以用于捕捉所述被摄体的当前姿态。In the case where the current label information has been determined, at block 730, the reference attitude model in block 720 is adjusted using the current spatial position and the current label information to obtain the current attitude model of the subject for use in Capture the current pose of the subject.

具体来说，所述基准姿态模型是在按照以上处理已获取的模型，该模型是由形体参数以及动作参数构成的模型，在已经确定形体参数的情况下，可利用第一标记对象的当前空间位置以及当前标签信息确定该模型中的动作参数。Specifically, the reference pose model is a model that has been obtained according to the above process. The model is a model composed of body parameters and action parameters. When the body parameters have been determined, the current space of the first marked object can be used. The location and current label information determine the action parameters in this model.

在实施中，可通过不断调整动作参数使虚拟人体模型上所有标签信息的虚拟三维坐标与所有标签信息对应的第一标记对象的三维坐标距离之和最小。In implementation, the sum of the virtual three-dimensional coordinates of all label information on the virtual human body model and the three-dimensional coordinate distance of the first marked object corresponding to all label information can be minimized by continuously adjusting the action parameters.

举例来说，针对标签信息为头部、左臂和右腿的第一标记对象，可通过使头部标签对应的第一标记对象的空间位置(三维坐标)与头部标签在基准姿态模型中对应的虚拟空间位置(虚拟三维坐标)之差、左臂标签对应的第一标记对象的空间位置与与左臂标签在基准姿态模型中对应的虚拟空间位置之差以及右腿标签对应的第一标记对象的空间位置与与右腿标签在基准姿态模型中对应的虚拟空间位置之差求和最小的方式获取基准姿态模型中的动作参数，其中，各个标签在基准姿态模型中对应的虚拟空间位置指示的是对应的第一标记对象附着在基准姿态模型(也就是在虚拟人体)接触点的位置。For example, for the first marked object whose label information is the head, the left arm and the right leg, the spatial position (three-dimensional coordinates) of the first marked object corresponding to the head label can be compared with the head label in the reference pose model. The difference between the corresponding virtual space positions (virtual three-dimensional coordinates), the difference between the space position of the first marked object corresponding to the left arm label and the virtual space position corresponding to the left arm label in the reference pose model, and the first mark corresponding to the right leg label. The action parameters in the reference pose model are obtained by summing the difference between the space position of the marked object and the virtual space position corresponding to the right leg label in the reference pose model, and the virtual space position corresponding to each label in the reference pose model is obtained. What is indicated is the position where the corresponding first marked object is attached to the contact point of the reference pose model (that is, on the virtual human body).

在确定动作参数后，可利用形体参数和动作参数确定当前姿态模型，从而捕捉到被摄体的当前姿态。After the action parameters are determined, the body parameters and the action parameters can be used to determine the current pose model, thereby capturing the current pose of the subject.

此外，在演员表演过程中，可能由于表演需求，会出现一些自遮挡情况，如双手抱胸，手臂、胸部的点不可避免造成了遮挡，蹲在地上，腿部、肚子上的反光点也造成了遮挡。此时，公式中的标记点会有缺失，抱胸时无法找到胸部标签点对应的三维坐标，可以通过加入先验信息，来对pose进行约束，来捕捉表演者的表演，避免捕捉出不合理的动作。In addition, during the performance of the actors, there may be some self-blocking situations due to the performance requirements, such as holding the chest with both hands, the points of the arms and chest inevitably cause blockage, and squatting on the ground, the reflection points on the legs and stomach also cause blocked. At this time, the marker points in the formula will be missing, and the 3D coordinates corresponding to the chest label points cannot be found when holding the chest. The pose can be constrained by adding prior information to capture the performer's performance and avoid unreasonable capture. Actions.

基于以上考虑，可利用预先设置的动作库生成先验动作模型，其中，所述动作库包括符合人体骨骼以及动作连贯性的动作。随后，在确定当前姿态模型的过程中，可利用先验动作模型对其进行约束。在实施中，可按照以下公式2执行处理：Based on the above considerations, a prior motion model can be generated by using a preset motion library, wherein the motion library includes motions conforming to human skeleton and motion coherence. Then, in the process of determining the current pose model, the prior motion model can be used to constrain it. In an implementation, processing may be performed according to Equation 2 below:

公式中，i代表第i个标签，j代表当前帧，pose ^j代表当前帧pose，pose ^j-1代表前一帧pose，(pose ^j-1，...，pose ^j-k)代表当前帧前k帧pose。 In the formula, i represents the ith label, j represents the current frame, pose ^j represents the current frame pose, pose ^j-1 represents the previous frame pose, (pose ^j-1 , ..., pose ^jk ) represents the previous k of the current frame frame pose.

其中，FK代表基准姿态模型，α,ρ代表基准姿态模型的形体参数，pose表示被摄体的动作参数；Marker _m表示第m个第一标记对象的三维坐标；i代表标签信息，标签信息与第一标记对象通过匹配关系Corr进行匹配。如公式2所示，Prior2(pose ^j)表示获取的pose满足动作预设的先验动作模型，Prior1(pose ^j|(pose ^j-1，...，pose ^j-k))表示获取的pose需要满足时序信号，不会出现突变。bodyMarker _i表示第i个标签在与被摄体对应的虚拟人体模型上的位置，通过bodyMarker _i(FK(α，ρ，pose)可以得出第i个标签的虚拟三维坐标。 Among them, FK represents the reference pose model, α, ρ represent the body parameters of the reference pose model, pose represents the action parameters of the subject; Marker _m represents the three-dimensional coordinates of the m-th first marked object; i represents the label information, the label information and The first marked object is matched by the matching relation Corr. As shown in Equation 2, Prior2(pose ^j ) indicates that the acquired pose satisfies the a priori action model of the action preset, and Prior1(pose ^j |(pose ^j-1 , ..., pose ^jk )) indicates that the acquired pose needs to satisfy Timing signals, no sudden changes will occur. bodyMarker _i represents the position of the ith label on the virtual human body model corresponding to the subject, and the virtual three-dimensional coordinates of the ith label can be obtained through bodyMarker _i (FK(α, ρ, pose).

公式(2)中的α,ρ，bodyMarker _i在标定过程中获取，在公式(2)为固定变量。标签信息与第一标记对象的对应关系(Corr)已知。通过优化pose，使虚拟人体模型上所有标签信息的虚拟三维坐标与所有标签信息对应的第一标记对象的三维坐标距离之和最小，来获取被摄体的动作参数。 α, ρ, bodyMarker _i in formula (2) are obtained during the calibration process, and are fixed variables in formula (2). The correspondence (Corr) between the label information and the first label object is known. By optimizing the pose, the sum of the virtual three-dimensional coordinates of all the label information on the virtual human body model and the three-dimensional coordinate distance of the first marked object corresponding to all the label information is minimized, so as to obtain the action parameters of the subject.

此外，在所述被摄体还可包括互动道具，举例来说，可利用多个相机同时拍摄演员以及与演员互动的篮球。应注意，所述互动道具不限定数量和类别，也就是说，该互动道具可以是单个或者多个，可以是同一类别的也可以是不同类别的。In addition, interactive props may also be included in the subject, for example, multiple cameras may be used to simultaneously photograph an actor and a basketball interacting with the actor. It should be noted that the number and type of the interactive props are not limited, that is, the interactive props may be single or multiple, and may be of the same type or of different types.

在此情况下，可预先在互动道具上放置道具标记对象，其中，所述道具标记对象与如上所述的第一标记对象是相同的标记对象，并且不限定数量。随后，通过在所述互动道具上附着的道具标记对象，获取所述互动道具的道具空间位置及道具标签信息，其中，所述道具空间位置和道具标签信息可以按照如上方式获取。In this case, prop mark objects can be placed on the interactive props in advance, wherein the prop mark objects are the same mark objects as the first mark objects described above, and the number is not limited. Subsequently, the prop space position and prop label information of the interactive prop are obtained through the prop tag object attached to the interactive prop, wherein the prop space position and prop label information can be obtained in the above manner.

最后，利用道具标记点的空间位置和标签信息对基本道具姿态模型进行调整，获得当前道具姿态模型，捕捉当前道具的动作。对于人体而言，虚拟人体模型是通过算法生成的，对于道具而言，虚拟道具模型是人工做出来的，在标定时，根据道具的标记点信息，人工制作虚拟道具模型，在捕捉时，根据道具上的标记点的空间位置和标签信息进行道具动作捕捉，从而使虚拟道具模型可以做出与真实道具相同的动作或者姿态。Finally, the basic prop pose model is adjusted by using the spatial position and label information of the prop marker points to obtain the current prop pose model and capture the action of the current prop. For the human body, the virtual human body model is generated by an algorithm. For the props, the virtual prop model is made manually. When calibrating, the virtual prop model is manually made according to the marking point information of the prop. The spatial position and label information of the marked points on the props are used for prop motion capture, so that the virtual prop model can make the same actions or gestures as the real props.

此外，所述方法还可执行重定向操作，也就是说，根据预先设置的对应关系，将所述当前姿态模型重定向处理至虚拟对象上。此外，在所述被摄体包括互动道具的情况下，可根据预先设置的对应关系，将所述当前道具姿态模型重定向处理至与虚拟物体上。In addition, the method can also perform a redirection operation, that is, according to a preset corresponding relationship, the current posture model is redirected to the virtual object. In addition, when the subject includes interactive props, the current prop pose model can be redirected to a virtual object according to a preset corresponding relationship.

本发明提出一种新的光学身体动作捕捉方式，该光学捕捉方式通过建立人体模型数据库，得出人体模型的低维分布，在光学捕捉时，可以直接获取与演员对应的身体模型(包括骨骼、演员身材)。可以获得以下效果，标记点不需要放置在演员的骨节点处，可以放置在演员身上的任意位置；增加了演员身上布点的自由度。获取到与演员匹配的模型，使得捕捉演员表演的精度进一步得以提升。通过姿态数据库建立了姿态先验模型，并且在捕捉中使用先验模型，使得在演员表演时，由于自遮挡造成了重建的反光点大量缺失时，仍然能够捕捉到演员的表演。The present invention proposes a new optical body motion capture method. The optical capture method obtains the low-dimensional distribution of the human body model by establishing a human body model database. During optical capture, the body model (including bones, actor body). The following effects can be obtained. The marker point does not need to be placed at the bone node of the actor, but can be placed at any position on the actor; the degree of freedom of the placement of points on the actor is increased. Obtaining a model that matches the actor further improves the accuracy of capturing the actor's performance. The pose prior model is established through the pose database, and the prior model is used in the capture, so that the actor's performance can still be captured when a large number of reconstructed reflective points are missing due to self-occlusion.

综上可述，根据本申请的示例性实施例的姿态捕捉方法能够在不对第一标记对象进行约束的情况下仅利用第一标记对象的空间位置以及标签信息即可确定与被摄体对应的基准姿态模型，无需将第一标记对象限制在骨骼点上，使用起来更加灵活，能够更加灵活且方便地获取与被摄体对应的基准姿态模型。更进一步地，可通过使被摄体作出特定动作的情况下，确定基准姿态模型的形体参数，从而能够使基准姿态模型更符合被摄体的形体。更进一步地，可利用第一标记对象的当前空间位置、当前标签信息以及基准姿态模型，可获取到被摄体的当前姿态模型，从而捕捉到被摄体的当前姿态，实现对被摄体动作的捕捉，降低了动作捕捉的难度。To sum up, the gesture capture method according to the exemplary embodiment of the present application can only use the spatial position and label information of the first marked object to determine the object corresponding to the subject without constraining the first marked object. The reference pose model does not need to limit the first marked object to the skeleton point, is more flexible to use, and can acquire the reference pose model corresponding to the subject more flexibly and conveniently. Furthermore, the body parameters of the reference posture model can be determined when the subject performs a specific action, so that the reference posture model can be made more in line with the body of the subject. Further, the current spatial position of the first marked object, the current label information and the reference attitude model can be used to obtain the current attitude model of the subject, thereby capturing the current attitude of the subject and realizing the action of the subject. capture, reducing the difficulty of motion capture.

上述姿态捕捉方法可用在表演动画制作领域以及虚拟直播领域，尤其是高质量的三维动画生成，可以通过捕捉真实被摄体的动作和/或姿态从而生成虚拟角色的动作和/或姿态。上述提及的姿态捕捉方法可以实现单人的捕捉，也可以实现多人的捕捉，即在同一画面中可以实现单个虚拟角色的输出也可以实现多个虚拟角色的输出。在多人捕捉的情况下，可以捕捉演员之间的互动，例如，拥抱，握手等，根据多个演员之间的互动输出虚拟角色的互动。姿态捕捉方法可以支持离线动画生成的模式和实时在线动画生成的模式。The above gesture capture method can be used in the field of performance animation production and virtual live broadcast, especially in the production of high-quality 3D animation, and the movements and/or gestures of virtual characters can be generated by capturing the movements and/or gestures of real subjects. The gesture capture method mentioned above can realize the capture of a single person or a plurality of people, that is, the output of a single virtual character or the output of multiple virtual characters can be realized in the same picture. In the case of multi-person capture, interactions between actors, such as hugs, handshakes, etc., can be captured, and the interactions of virtual characters are output according to the interactions between multiple actors. The pose capture method can support offline animation generation mode and real-time online animation generation mode.

本申请实施例可以根据上述方法示例对上述终端等进行功能模块的划分，例如，可以对应各个功能划分各个功能模块，也可以将两个或两个以上的功能集成在一个处理模块中。上述集成的模块既可以采用硬件的形式实现，也可以采用软件功能模块的形式实现。需要说明的是，本申请实施例中对模块的划分是示意性的，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式。In this embodiment of the present application, functional modules may be divided into the above terminal and the like according to the above method examples. For example, each functional module may be divided corresponding to each function, or two or more functions may be integrated into one processing module. The above-mentioned integrated modules can be implemented in the form of hardware, and can also be implemented in the form of software function modules. It should be noted that, the division of modules in the embodiments of the present application is schematic, and is only a logical function division, and there may be other division manners in actual implementation.

在采用对应各个功能划分各个功能模块的情况下，图9是根据本申请的示例性实施例的姿态捕捉装置的框图。In the case where each functional module is divided corresponding to each function, FIG. 9 is a block diagram of a gesture capturing apparatus according to an exemplary embodiment of the present application.

如图9所示，姿态捕捉装置900可包括标签信息确定单元910和基准姿态模型生成单元920，其中，标签信息确定单元910可确定附着在被摄体上的第一标记对象的空间位置以及第一标记对象在所述被摄体的部位的标签信息；基准姿态模型生成单元920可利用第一标记对象的所述空间位置以及所述标签信息对初始姿态模型进行调整，生成与所述被摄体对应的基准姿态模型。As shown in FIG. 9 , the posture capturing device 900 may include a label information determining unit 910 and a reference posture model generating unit 920, wherein the label information determining unit 910 may determine the spatial position and the first labeling object attached to the subject. Label information of the part of the marked object on the subject; the reference pose model generation unit 920 can use the spatial position and the label information of the first marked object to adjust the initial pose model, and generate a The reference pose model corresponding to the body.

基准姿态模型生成单元920包括动作参数设置模块、第一标记对象信息获取模块以及基准姿态模型生成模块，其中，动作参数设置模块用于通过使所述被摄体做出特定动作，设置所述初始姿态模型中的动作参数；第一标记对象信息获取模块用于在所述动作参数确定的情况下，获取第一标记对象在所述被摄体做出所述特定动作的情况下的所述空间位置以及所述标签信息；基准姿态模型生成模块用于利用所述动作参数、所述空间位置以及所述标签信息，对所述初始姿态模型的形体参数进行调整，生成与所述被摄体对应的所述基准姿态模型。The reference posture model generation unit 920 includes an action parameter setting module, a first marked object information acquisition module, and a reference posture model generation module, wherein the action parameter setting module is used to set the initial Action parameters in the posture model; the first marked object information acquisition module is configured to acquire the space of the first marked object when the subject performs the specific action when the action parameter is determined The position and the label information; the reference attitude model generation module is used to adjust the physical parameters of the initial attitude model by using the motion parameters, the spatial position and the label information, and generate a corresponding model corresponding to the subject. of the reference pose model.

可选地，第一标记对象信息获取模块还可响应于被摄体的动作获取第一标记对象的当前空间位置以及当前标签信息，并且基准姿态模型生成模块还可利用所述当前空间位置以及所述当前标签信息，对所述基准姿态模型进行调整，获取被摄体的当前姿态模型，以用于捕捉所述被摄体的当前姿态。Optionally, the first marked object information acquisition module can also acquire the current spatial position and current label information of the first marked object in response to the action of the subject, and the reference pose model generation module can also use the current spatial position and all The current label information is adjusted, the reference attitude model is adjusted, and the current attitude model of the subject is acquired, so as to be used to capture the current attitude of the subject.

可选地，第一标记对象信息获取模块具体用于获取所述当前标签信息的预测空间位置和所述第一标记对象的当前空间位置；在确定所述第一标记对象的当前空间位置处于所述当前标签信息的预测空间位置的预设范围内的情况下，将所述第一标记对象与所述当前标签信息进行匹配，获得匹配关系，其中，所述预设范围是根据对所述被摄体的动作轨迹预测所设定的范围；根据所述匹配关系，确定所述第一标记对象对应的所述当前标签信息。Optionally, the first marked object information acquisition module is specifically configured to acquire the predicted spatial position of the current tag information and the current spatial position of the first marked object; In the case of being within the preset range of the predicted spatial position of the current tag information, the first tag object is matched with the current tag information to obtain a matching relationship, wherein the preset range is based on the The range set by the motion trajectory prediction of the subject; and the current label information corresponding to the first marked object is determined according to the matching relationship.

可选地，基准姿态模型生成模块具体用于在所述基准姿态模型的形体参数确定的情况下，通过不断调整所述动作参数使所有所述标签信息的虚拟空间位置与所有所述标签信息对应的所述第一标记对象的空间位置距离之和最小，，获取所述当前姿态模型，以用于捕捉所述被摄体的姿态。Optionally, the reference attitude model generation module is specifically configured to make the virtual space positions of all the label information correspond to all the label information by continuously adjusting the action parameters when the body parameters of the reference attitude model are determined. The sum of the spatial position distances of the first marked object is the smallest, and the current posture model is obtained to capture the posture of the subject.

可选地，所述姿态捕捉装置900还包括约束单元，所述约束单元用于利用预先设置的动作库生成的先验动作模型，对所述当前姿态模型进行约束，获取约束后的当前姿态模型。Optionally, the posture capturing device 900 further includes a constraining unit, and the constraining unit is configured to constrain the current posture model using a priori action model generated by a preset action library, and obtain a constrained current posture model. .

可选地，所述姿态捕捉装置900还包括互动道具确定单元、道具信息确定单元以及当前道具模型确定单元，其中，所述互动道具确定单元用于确定与所述被摄体执行互动的互动道具；道具信息确定单元用于通过在所述互动道具上附着的道具标记对象，获取所述互动道具的道具空间位置及道具标签信息；当前道具模型确当单元用于利用所述道具空间位置以及道具标签信息，对与所述互动道具对应的基本道具姿态模型进行调整，生成当前道具姿态模型，以用于捕捉所述互动道具的运动。Optionally, the gesture capturing apparatus 900 further includes an interactive prop determining unit, a prop information determining unit, and a current prop model determining unit, wherein the interactive prop determining unit is used to determine an interactive prop that interacts with the subject The prop information determination unit is used to obtain the prop space position and prop label information of the interactive prop through the prop tag object attached to the interactive prop; the current prop model confirmation unit is used to use the prop space position and prop label information information, adjust the basic prop pose model corresponding to the interactive prop, and generate a current prop pose model for capturing the movement of the interactive prop.

可选地，第一标记对象信息获取模块包括二维位置确定子模块以及空间位置确定子模块，其中，二维位置确定子模块用于利用至少两个相机对第一标记对象拍摄的图像，确定第一标记对象的至少两个二维位置，空间位置确定子模块用于利用所述至少两个二维位置以及所述至少两个相机的标定信息，确定第一标记对象的空间位置。Optionally, the first marked object information acquisition module includes a two-dimensional position determination sub-module and a spatial position determination sub-module, wherein the two-dimensional position determination sub-module is used to determine the image captured by at least two cameras of the first marked object. At least two two-dimensional positions of the first marked object, and the spatial position determination sub-module is configured to use the at least two two-dimensional positions and the calibration information of the at least two cameras to determine the spatial position of the first marked object.

可选地，空间位置确定子模块具体用于利用所述至少两个二维位置以及所述至少两个相机的标定信息，确定所述至少两个相机对应于第一标记对象的至少两条射线；通过第一标记对象在所述空间位置上距所述至少两条射线的距离最小的方式，确定第一标记对象的空间位置。Optionally, the spatial position determination sub-module is specifically configured to use the at least two two-dimensional positions and the calibration information of the at least two cameras to determine that the at least two cameras correspond to at least two rays of the first marked object. ; Determine the spatial position of the first marking object by means of the smallest distance between the first marking object and the at least two rays at the spatial position.

可选地，所述姿态捕捉装置900还包括相机标定信息获取单元、比例关系设置单元以及空间坐标系确定单元，其中，相机标定信息获取单元用于通过对用于捕捉所述被摄体的所有相机执行标定，获取所有相机的标定信息；比例关系设置单元用于根据所述所有相机的标定信息和具有第二标记对象的标记装置设置比例关系；空间坐标系确定单元用于利用所述所有相机的标定信息对所述捕捉空间的地面进行标定，确定地面信息以及利用所述地面信息确定的所述空间坐标系。Optionally, the posture capturing device 900 further includes a camera calibration information acquisition unit, a scale relationship setting unit, and a space coordinate system determination unit, wherein the camera calibration information acquisition unit is configured to The camera performs calibration, and obtains the calibration information of all cameras; the scale relationship setting unit is used for setting the scale relationship according to the calibration information of all the cameras and the marking device with the second marking object; the spatial coordinate system determining unit is used for using all the cameras. The calibration information is used to calibrate the ground of the captured space, and the ground information and the space coordinate system determined by using the ground information are determined.

可选地，所述姿态捕捉装置900还包括标签信息匹配单元，其中，标签信息匹配单元对所述第一标记对象和所述标签信息进行匹配，获得所述第一标记对象和所述标签信息的对应关系。Optionally, the gesture capturing device 900 further includes a tag information matching unit, wherein the tag information matching unit matches the first tag object and the tag information to obtain the first tag object and the tag information. corresponding relationship.

如图10所示，在实际姿态捕捉时，可以在人体不同的位置放置第一标记对象。被摄体需要穿上特制的姿态捕捉服装，包括姿态捕捉手套和姿态捕捉鞋，在姿态捕捉服装上贴上标记点。图10仅仅是一个示例图，没有表示出服装，所以有些第一标记对象并没有与皮肤完全接触，实际上，第一标记对象是与服装相接触的。As shown in FIG. 10 , in the actual pose capture, the first marker objects can be placed at different positions of the human body. The subject needs to wear special pose capture clothing, including pose capture gloves and pose capture shoes, and put markers on the pose capture garment. FIG. 10 is only an example diagram, and does not show clothing, so some of the first marked objects are not in complete contact with the skin. In fact, the first marked objects are in contact with the clothing.

综上可述，根据本申请的示例性实施例的姿态捕捉装置能够在不对第一标记对象进行约束的情况下仅利用第一标记对象的空间位置以及标签信息即可确定与被摄体对应的基准姿态模型，无需将第一标记对象限制在骨骼点上，使用起来更加灵活，能够更加灵活且方便地获取与被摄体对应的基准姿态模型。更进一步地，可通过使被摄体作出特定动作的情况下，确定基准姿态模型的形体参数，从而能够使基准姿态模型更符合被摄体的形体。更进一步地，可利用第一标记对象的当前空间位置、当前标签信息以及基准姿态模型，可获取到被摄体的当前姿态模型，从而捕捉到被摄体的当前姿态，实现对被摄体动作的捕捉，降低了动作捕捉的难度。To sum up, the gesture capture device according to the exemplary embodiment of the present application can determine the object corresponding to the subject by only using the spatial position and label information of the first marked object without constraining the first marked object. The reference pose model does not need to limit the first marked object to the skeleton point, is more flexible to use, and can acquire the reference pose model corresponding to the subject more flexibly and conveniently. Furthermore, the body parameters of the reference posture model can be determined when the subject performs a specific action, so that the reference posture model can be made more in line with the body of the subject. Further, the current spatial position of the first marked object, the current label information and the reference attitude model can be used to obtain the current attitude model of the subject, thereby capturing the current attitude of the subject and realizing the action of the subject. capture, reducing the difficulty of motion capture.

需要说明的是，实施例1所提供方法的各步骤的执行主体均可以是同一设备，或者，该方法也由不同设备作为执行主体。比如，步骤21和步骤22的执行主体可以为设备1，步骤23的执行主体可以为设备2；又比如，步骤21的执行主体可以为设备1，步骤22和步骤23的执行主体可以为设备2；等等。It should be noted that, each step of the method provided in Embodiment 1 may be executed by the same device, or the method may also be executed by different devices. For example, the execution body of step 21 and step 22 may be device 1, and the execution body of step 23 may be device 2; for another example, the execution body of step 21 may be device 1, and the execution body of step 22 and step 23 may be device 2 ;and many more.

本领域内的技术人员应明白，本发明的实施例可提供为方法、***、或计算机程序产品。因此，本发明可采用完全硬件实施例、完全软件实施例、或结合软件和硬件方面的实施例的形式。而且，本发明可采用在一个或多个其中包含有计算机可用程序代码的计算机可用存储介质(包括但不限于磁盘存储器、CD-ROM、光学存储器等)上实施的计算机程序产品的形式。As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) having computer-usable program code embodied therein.

本发明是参照根据本发明实施例的方法、设备(***)、和计算机程序产品的流程图和/或方框图来描述的。应理解可由计算机程序指令实现流程图和 /或方框图中的每一流程和/或方框、以及流程图和/或方框图中的流程和/或方框的结合。可提供这些计算机程序指令到通用计算机、专用计算机、嵌入式处理机或其他可编程姿态捕捉设备的处理器以产生一个机器，使得通过计算机或其他可编程姿态捕捉设备的处理器执行的指令产生用于实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能的装置。The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each process and/or block in the flowchart illustrations and/or block diagrams, and combinations of processes and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to the processor of a general purpose computer, special purpose computer, embedded processor or other programmable gesture capture device to produce a machine such that the instructions executed by the processor of the computer or other programmable gesture capture device produce Means for implementing the functions specified in a flow or flow of a flowchart and/or a block or blocks of a block diagram.

这些计算机程序指令也可存储在能引导计算机或其他可编程姿态捕捉设备以特定方式工作的计算机可读存储器中，使得存储在该计算机可读存储器中的指令产生包括指令装置的制造品，该指令装置实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能。These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable gesture capture device to function in a particular manner, such that the instructions stored in the computer-readable memory result in an article of manufacture comprising instruction means, the instructions The apparatus implements the functions specified in the flow or flow of the flowcharts and/or the block or blocks of the block diagrams.

这些计算机程序指令也可装载到计算机或其他可编程姿态捕捉设备上，使得在计算机或其他可编程设备上执行一系列操作步骤以产生计算机实现的处理，从而在计算机或其他可编程设备上执行的指令提供用于实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能的步骤。These computer program instructions can also be loaded onto a computer or other programmable gesture capture device to cause a series of operational steps to be performed on the computer or other programmable device to produce a computer-implemented process such that The instructions provide steps for implementing the functions specified in the flow or blocks of the flowcharts and/or the block or blocks of the block diagrams.

在一个典型的配置中，计算设备包括一个或多个处理器(CPU)、输入/输出接口、网络接口和内存。In a typical configuration, a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.

内存可能包括计算机可读介质中的非永久性存储器，随机存取存储器(RAM)和/或非易失性内存等形式，如只读存储器(ROM)或闪存(flash RAM)。内存是计算机可读介质的示例。Memory may include forms of non-persistent memory, random access memory (RAM) and/or non-volatile memory in computer readable media, such as read only memory (ROM) or flash memory (flash RAM). Memory is an example of a computer-readable medium.

计算机可读介质包括永久性和非永久性、可移动和非可移动媒体可以由任何方法或技术来实现信息存储。信息可以是计算机可读指令、数据结构、程序的模块或其他数据。计算机的存储介质的例子包括，但不限于相变内存(PRAM)、静态随机存取存储器(SRAM)、动态随机存取存储器(DRAM)、其他类型的随机存取存储器(RAM)、只读存储器(ROM)、电可擦除可编程只读存储器(EEPROM)、快闪记忆体或其他内存技术、只读光盘只读存储器(CD-ROM)、数字多功能光盘(DVD)或其他光学存储、磁盒式磁带，磁带磁磁盘存储或其他磁性存储设备或任何其他非传输介质，可用于存储可以被计算设备访问的信息。按照本文中的界定，计算机可读介质不包括暂存电脑可读媒体(transitory media)，如调制的数据信号和载波。Computer-readable media includes both persistent and non-permanent, removable and non-removable media, and storage of information may be implemented by any method or technology. Information may be computer readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read only memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), Flash Memory or other memory technology, Compact Disc Read Only Memory (CD-ROM), Digital Versatile Disc (DVD) or other optical storage, Magnetic tape cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission medium that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory computer-readable media, such as modulated data signals and carrier waves.

还需要说明的是，术语“包括”、“包含”或者其任何其他变体意在涵盖非排他性的包含，从而使得包括一系列要素的过程、方法、商品或者设备不仅包括那些要素，而且还包括没有明确列出的其他要素，或者是还包括为这种过程、方法、商品或者设备所固有的要素。在没有更多限制的情况下，由语句“包括一个……”限定的要素，并不排除在包括所述要素的过程、方法、商品或者设备中还存在另外的相同要素。It should also be noted that the terms "comprising", "comprising" or any other variation thereof are intended to encompass a non-exclusive inclusion such that a process, method, article or device comprising a series of elements includes not only those elements, but also Other elements not expressly listed, or which are inherent to such a process, method, article of manufacture, or apparatus are also included. Without further limitation, an element qualified by the phrase "comprising a..." does not preclude the presence of additional identical elements in the process, method, article of manufacture, or device that includes the element.

本领域技术人员应明白，本申请的实施例可提供为方法、***或计算机程序产品。因此，本申请可采用完全硬件实施例、完全软件实施例或结合软件和硬件方面的实施例的形式。而且，本申请可采用在一个或多个其中包含有计算机可用程序代码的计算机可用存储介质(包括但不限于磁盘存储器、CD-ROM、光学存储器等)上实施的计算机程序产品的形式。It will be appreciated by those skilled in the art that the embodiments of the present application may be provided as a method, a system or a computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) having computer-usable program code embodied therein.

以上所述仅为本申请的实施例而已，并不用于限制本申请。对于本领域技术人员来说，本申请可以有各种更改和变化。凡在本申请的精神和原理之内所作的任何修改、等同替换、改进等，均应包含在本申请的权利要求范围之内。The above descriptions are merely examples of the present application, and are not intended to limit the present application. Various modifications and variations of this application are possible for those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of this application shall be included within the scope of the claims of this application.

Claims

一种姿态捕捉方法，其特征在于，包括：A gesture capturing method, comprising:

确定附着在被摄体上的第一标记对象的空间位置以及第一标记对象在所述被摄体的部位的标签信息；determining the spatial position of the first marked object attached to the subject and the label information of the first marked object on the part of the subject;

利用第一标记对象的所述空间位置以及所述标签信息对初始姿态模型进行调整，生成与所述被摄体对应的基准姿态模型。The initial posture model is adjusted by using the spatial position of the first marked object and the label information to generate a reference posture model corresponding to the subject.
如权利要求1所述的方法，其特征在于，所述初始姿态模型包括用于描述形体的形体参数以及用于描述动作的动作参数。The method of claim 1, wherein the initial pose model includes a body parameter for describing a body and an action parameter for describing an action.
如权利要求2所述的方法，其特征在于，所述利用第一标记对象的所述空间位置以及所述标签信息对初始姿态模型进行调整，生成与所述被摄体对应的基准姿态模型包括：The method according to claim 2, wherein the adjusting the initial posture model by using the spatial position of the first marked object and the label information, and generating the reference posture model corresponding to the object comprises: :

通过使所述被摄体做出特定动作，设置所述初始姿态模型中的动作参数；Setting the action parameters in the initial posture model by causing the subject to perform a specific action;

在所述动作参数确定的情况下，获取第一标记对象在所述被摄体做出所述特定动作的情况下的所述空间位置以及所述标签信息；If the action parameter is determined, obtain the spatial position and the label information of the first marked object when the subject performs the specific action;

利用所述动作参数、所述空间位置以及所述标签信息，对所述初始姿态模型的形体参数进行调整，生成与所述被摄体对应的所述基准姿态模型。Using the motion parameters, the spatial position and the label information, the body parameters of the initial posture model are adjusted to generate the reference posture model corresponding to the subject.
如权利要求1至3中的任一权利要求所述的方法，其特征在于，在生成与所述被摄体对应的所述基准姿态模型后还包括：The method according to any one of claims 1 to 3, wherein after generating the reference pose model corresponding to the subject, the method further comprises:

响应于被摄体的动作，获取第一标记对象的当前空间位置以及当前标签信息；In response to the action of the subject, obtain the current spatial position and current label information of the first marked object;

利用所述当前空间位置以及所述当前标签信息，对所述基准姿态模型进行调整，获取被摄体的当前姿态模型，以用于捕捉所述被摄体的当前姿态。Using the current spatial position and the current label information, the reference attitude model is adjusted to obtain the current attitude model of the subject, so as to capture the current attitude of the subject.
如权利要求4所述的方法，其特征在于，获取第一标记对象的当前标签信息包括：The method of claim 4, wherein obtaining the current label information of the first labelled object comprises:

获取所述当前标签信息的预测空间位置和所述第一标记对象的当前空间位置；Obtain the predicted spatial position of the current label information and the current spatial position of the first marked object;

在确定所述第一标记对象的当前空间位置处于所述当前标签信息的预测空间位置的预设范围内的情况下，将所述第一标记对象与所述当前标签信息进行匹配，获得匹配关系，其中，所述预设范围是根据对所述被摄体的动作轨迹预测所设定的范围；In the case where it is determined that the current spatial position of the first marked object is within a preset range of the predicted spatial position of the current label information, match the first marked object with the current label information to obtain a matching relationship , wherein the preset range is a range set according to the motion trajectory prediction of the subject;

根据所述匹配关系，确定所述第一标记对象对应的所述当前标签信息。According to the matching relationship, the current tag information corresponding to the first tag object is determined.
如权利要求4所述的方法，其特征在于，所述利用所述当前空间位置以及所述当前标签信息对所述被摄体对应的所述基准姿态模型进行调整获取被摄体的当前姿态模型以用于捕捉所述被摄体的当前姿态包括：The method according to claim 4, wherein the reference attitude model corresponding to the subject is adjusted by using the current spatial position and the current label information to obtain the current attitude model of the subject The current pose for capturing the subject includes:

在所述基准姿态模型的形体参数确定的情况下，通过不断调整所述动作参数使所有所述标签信息的虚拟空间位置与所有所述标签信息对应的所述第一标记对象的空间位置距离之和最小，获取所述当前姿态模型，以用于捕捉所述被摄体的姿态。When the physical parameters of the reference pose model are determined, the distance between the virtual space positions of all the tag information and the spatial position distance of the first marked object corresponding to all the tag information is made by continuously adjusting the action parameters. and the minimum, the current pose model is obtained for capturing the pose of the subject.
如权利要求6所述的方法，其特征在于，还包括：The method of claim 6, further comprising:

利用预先设置的动作库生成的先验动作模型，对所述当前姿态模型进行约束，获取约束后的当前姿态模型。Using a priori action model generated by a preset action library, the current posture model is constrained, and the constrained current posture model is obtained.
如权利要求1至7中的任一权利要求所述的方法，其特征在于，还包括：The method of any one of claims 1 to 7, further comprising:

确定与所述被摄体执行互动的互动道具；determining an interactive prop that interacts with the subject;

通过在所述互动道具上附着的道具标记对象，获取所述互动道具的道具空间位置及道具标签信息；Obtain the prop space position and prop label information of the interactive prop through the prop tag object attached to the interactive prop;

利用所述道具空间位置以及道具标签信息，对与所述互动道具对应的基本道具姿态模型进行调整，生成当前道具姿态模型，以用于捕捉所述互动道具的运动。Using the prop space position and prop label information, the basic prop pose model corresponding to the interactive prop is adjusted to generate a current prop pose model for capturing the movement of the interactive prop.
如权利要求1所述的方法，其特征在于，确定附着在被摄体上的第一标记对象的空间位置包括：The method of claim 1, wherein determining the spatial position of the first marked object attached to the subject comprises:

利用至少两个相机对第一标记对象拍摄的图像，确定第一标记的对象的至少两个二维位置；Determine at least two two-dimensional positions of the first marked object using images captured by at least two cameras of the first marked object;

利用所述至少两个二维位置以及所述至少两个相机的标定信息，确定第一标记对象的空间位置。Using the at least two two-dimensional positions and the calibration information of the at least two cameras, the spatial position of the first marked object is determined.
如权利要求9所述的方法，其特征在于，所述利用所述至少两个二维位置以及所述至少两个相机的标定信息确定第一标记对象的空间位置包括：The method according to claim 9, wherein the determining the spatial position of the first marked object by using the at least two two-dimensional positions and the calibration information of the at least two cameras comprises:

利用所述至少两个二维位置以及所述至少两个相机的标定信息，确定所述至少两个相机对应于第一标记对象的至少两条射线；Using the at least two two-dimensional positions and the calibration information of the at least two cameras, determining that the at least two cameras correspond to at least two rays of the first marked object;

通过第一标记对象在所述空间位置上距所述至少两条射线的距离最小的方式，确定第一标记对象的空间位置。The spatial position of the first marking object is determined in such a way that the distance between the first marking object and the at least two rays at the spatial position is the smallest.
如权利要求1所述的方法，其特征在于，所述空间位置包括第一标记对象在用于捕捉所述被摄体的捕捉空间对应的空间坐标系内的坐标数据。The method of claim 1, wherein the spatial position comprises coordinate data of the first marked object in a spatial coordinate system corresponding to a capture space used to capture the subject.
如权利要求11所述的方法，其特征在于，还包括：The method of claim 11, further comprising:

通过对用于捕捉所述被摄体的所有相机执行标定，获取所有相机的标定信息；Obtain calibration information of all cameras by performing calibration on all cameras used to capture the subject;

根据所述所有相机的标定信息和具有第二标记对象的标记装置设置比例关系；setting a proportional relationship according to the calibration information of all the cameras and the marking device having the second marking object;

利用所述所有相机的标定信息对所述捕捉空间的地面进行标定，确定地面信息以及利用所述地面信息确定的所述空间坐标系。The ground of the captured space is calibrated by using the calibration information of all the cameras, and the ground information and the space coordinate system determined by using the ground information are determined.
如权利要求1所述的方法，其特征在于，所述确定附着在被摄体上的第一标记对象的空间位置以及第一标记对象在所述被摄体的部位的标签信息还包括：对所述第一标记对象和所述标签信息进行匹配，获得所述第一标记对象和所述标签信息的对应关系。The method according to claim 1, wherein the determining the spatial position of the first marking object attached to the subject and the label information of the first marking object on the part of the subject further comprises: The first marked object and the label information are matched to obtain the corresponding relationship between the first marked object and the label information.
一种姿态捕捉方法，其特征在于，包括：A gesture capturing method, comprising:

响应于被摄体的动作，确定附着在所述被摄体上的第一标记对象的当前空间位置以及用于描述第一标记对象在所述被摄体的部位的标签信息；In response to the action of the subject, determining the current spatial position of the first marking object attached to the subject and label information for describing the position of the first marking object on the subject;

利用第一标记对象的当前空间位置和标签信息，对被摄体的基准姿态模型进行调整，获取被摄体的当前姿态模型，从而捕捉到所述被摄体的当前姿态。Using the current spatial position and label information of the first marked object, the reference attitude model of the subject is adjusted to obtain the current attitude model of the subject, thereby capturing the current attitude of the subject.
一种姿态捕捉装置，其特征在于，包括：A gesture capturing device, comprising:

标签信息确定单元，用于确定附着在被摄体上的第一标记对象的空间位置以及用于描述第一标记对象在所述被摄体的部位的标签信息；A label information determining unit, configured to determine the spatial position of the first marked object attached to the subject and the label information used to describe the position of the first marked object on the subject;

基准姿态模型生成单元，用于利用第一标记对象的所述空间位置以及所述标签信息对初始姿态模型进行调整，生成与所述被摄体对应的基准姿态模型。The reference pose model generating unit is configured to adjust the initial pose model by using the spatial position of the first marked object and the label information to generate a reference pose model corresponding to the subject.
一种电子设备，包括：An electronic device comprising:

一个或多个处理器；one or more processors;

存储器；以及memory; and

一个或多个程序，其中所述一个或多个程序存储在所述存储器中并被配置为由所述一个或多个处理器执行，所述一个或多个程序包括用于执行根据权利要求1-13中任一权利要求所述的方法。One or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs comprising a program for performing the method according to claim 1 - The method of any one of claims 13.
一种存储一个或多个程序的计算机可读存储介质，所述一个或多个程序包括指令，所述指令当由计算设备执行时，使得所述计算设备执行根据权利要求1-13中的任一权利要求所述的方法。A computer-readable storage medium storing one or more programs comprising instructions that, when executed by a computing device, cause the computing device to perform any of the methods according to claims 1-13. A method as claimed in claim.