WO2024114000A1 - Motion-capture-system-based virtual human body driving method and apparatus in electric power scene - Google Patents

Motion-capture-system-based virtual human body driving method and apparatus in electric power scene Download PDF

Info

Publication number
WO2024114000A1
WO2024114000A1 PCT/CN2023/116033 CN2023116033W WO2024114000A1 WO 2024114000 A1 WO2024114000 A1 WO 2024114000A1 CN 2023116033 W CN2023116033 W CN 2023116033W WO 2024114000 A1 WO2024114000 A1 WO 2024114000A1
Authority
WO
WIPO (PCT)
Prior art keywords
motion capture
data
driving
human body
virtual
Prior art date
Application number
PCT/CN2023/116033
Other languages
French (fr)
Chinese (zh)
Inventor
孔吉宏
雒文琦
罗茜
吕毅松
王佶
胖永新
杨斌
施涛
Original Assignee
南方电网储能股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 南方电网储能股份有限公司 filed Critical 南方电网储能股份有限公司
Publication of WO2024114000A1 publication Critical patent/WO2024114000A1/en

Links

Definitions

  • the present invention relates to the field of virtual reality technology, and in particular to a method and device for driving a virtual human body in an electric scene based on a motion capture system.
  • “Motion capture” is to record the motion information of the actors through a specific capture device as raw data for animation production. These raw data will be used to drive the virtual 3D animation character model. Compared with directly drawing the motion of 3D animation characters, the above method reduces the dependence on the work experience of artists and reduces the labor cost of related animation production; on the other hand, it also makes the produced animation more realistic and smooth. At present, there are more and more software for making virtual characters, and the technology is maturing, so more and more people can participate in the construction of virtual characters.
  • Xsens MVN is an easy-to-use, high-end inertial motion capture system.
  • the MVN motion capture system uses the latest inertial sensing technology, has powerful and reliable hardware, and can generate data for use. Both hardware performance and software functions have been greatly improved. It is suitable for 3D character animation production, game development, virtual idols, game film art, human-computer interaction, human-computer ergonomics, virtual reality, motion analysis, training and simulation.
  • the present invention provides a method and device for driving a virtual human body in an electric scene based on a motion capture system, so as to solve at least one of the above-mentioned technical problems.
  • a method for driving a virtual human body in an electric scene based on a motion capture system comprising:
  • the virtual human is driven to perform real-time operations through motion capture data.
  • the method also includes: walking, squatting, running and jumping instructions for human body execution tasks based on the ergonomics in the existing virtual reality editor.
  • the method also includes: establishing a formatted data-driven configuration of the motion capture system according to the skeleton ID of the ergonomic human body in the virtual reality editor, and establishing a matching connection between the ergonomic human body skeleton component structure drive node in the virtual reality editor and the data structure node of the motion capture system.
  • the method further includes: when driven by motion capture, using motion capture head tracking data to drive the helmet's first-person viewpoint to perform visibility verification.
  • the method also includes: when the human body wears the motion capture system, initializing a preset posture to obtain initial posture data, and importing the initial posture data into a virtual reality editor to trigger the start of a real-time drive interface.
  • a virtual human driving device for electric scene based on a motion capture system for implementing the method described, the device comprising:
  • connection setting module is used to establish a communication connection between the Xsens motion capture system and the VR editor and set up the real-time driver interface;
  • Courseware loading module used to load courseware scene data in the virtual reality editor
  • Data capture module used to execute equipment maintenance plans and obtain motion capture data
  • the human body driving module is used to drive the virtual human to perform real-time operations based on the real-time driving interface through motion capture data.
  • an electronic device comprising a processor, a memory, and a computer program stored in the memory and executable by the processor, wherein when the computer program is executed by the processor, the steps of the method for driving a virtual human body in an electric scene based on a motion capture system are implemented.
  • a computer-readable storage medium characterized in that a computer program is stored on the computer-readable storage medium, wherein when the computer program is executed by a processor, the steps of the method for driving a virtual human body in an electric power scene based on a motion capture system are implemented.
  • the present invention has the following beneficial effects:
  • the present invention customizes and develops the software interface of the Xsens MVN BIOMECHLINK motion capture system, utilizes the existing virtual reality editor MakeReal3D, and combines the maintenance courseware data to develop a motion capture interface for interactive courseware.
  • the interface supports the operator wearing a motion capture device to drive a simulated person in a virtual scene, and can execute commands such as walking, squatting, running and jumping of the character.
  • the ergonomics module it can verify the maintenance plan of the hydropower plant; according to the safety operating rules, it can conduct safety operation emergency drills and complete the recording and calling of various types of character action animations.
  • FIG. 1 is a flow chart of a method for driving a virtual human body in an electric power scene based on a motion capture system according to an embodiment of the present invention.
  • FIG. 2 is a schematic diagram of a software interface implementation of a motion capture system according to an embodiment of the present invention.
  • a method for driving a virtual human body in an electric power scene based on a motion capture system customizes and develops the software interface of the Xsens MVN BIOMECHLINK motion capture system, utilizes the existing virtual reality editor MakeReal3D, and combines maintenance courseware data to develop a motion capture interface for interactive courseware.
  • the interface supports an operator wearing a motion capture device to drive a virtual person in a virtual scene, and can execute commands such as walking, squatting, running and jumping of the character.
  • the maintenance plan of the hydropower plant can be verified.
  • the method of the present invention can also perform safety operation emergency drills and complete the recording and calling of various character action animations according to safety regulations and operating rules.
  • the method comprises:
  • Step 1 Establish a communication connection between the Xsens motion capture system and the VR editor and set up the real-time driver interface.
  • Step 2 Load the courseware scene data in the virtual reality editor.
  • Step 3 Execute the equipment maintenance plan and obtain motion capture data.
  • Step 4 Based on the real-time driving interface, the virtual human is driven to perform real-time operations through the motion capture data.
  • the present invention is customized and developed based on the software interface of the Xsens MVN BIOMECHLINK motion capture system, and utilizes the existing editor to realize the ergonomic human body motion capture data collection, maintenance plan verification, safe operation emergency drills according to safety regulations and operating rules, and complete the recording and calling functions of various character action animations.
  • the communication interface is developed through the API of the motion capture system driver interface to transmit the data captured by the motion capture system to the MakeReal3D virtual reality editor in real time.
  • the hardware data of the Xsens MVN BIOMECHLINK motion capture system is developed through the driver's API interface to establish a communication interface in MakeReal3D, that is, real-time communication.
  • the raw data of the Xsens MVN BIOMECHLINK motion capture system is obtained through the real-time communication switch, data reception, and real-time storage.
  • the obtained raw data is then parsed and developed, and the parsed data is converted and developed into real-time drive data for the MakeReal3D virtual reality editor.
  • the real-time driver interface mainly provides a driver interface for user settings and operations.
  • the driver interface generally includes the development of settings such as data connection, data calibration, data setting, and data closing.
  • the Xsens MVN BIOMECHLINK motion capture system software interface drives the ergonomic human body to execute commands such as walking, squatting, running and jumping. It needs to be implemented based on the ergonomic human body in the existing MakeReal3D virtual reality editor.
  • the implementation process is as follows:
  • a formatted data driven configuration of the motion capture system is established according to the skeleton ID of the functional human body in the virtual reality editor, and a matching connection is established between the functional human body skeleton component structure driving node and the data structure node of the motion capture system in the virtual reality editor.
  • MakeReal3D real-time ergonomic analysis and verification mainly includes ergonomic analysis functions such as visual and achievable comfort. To drive in real time and implement the above ergonomic verification on the human body, it is necessary to call this function in the interactive scene, that is, to add an ergonomic entrance in the interactive scene.
  • Visibility is mainly based on subjective human visual judgment.
  • the head tracking data of motion capture is used to drive the first-person viewpoint of the helmet for visibility.
  • an entry interaction UI under VR interaction needs to be added for entry, and ergonomic analysis verification can be called through the menu.
  • the operator wears this full-body motion capture system to drive the virtual character in the scene to perform 1:1 motion simulation.
  • this interface is used to simulate the human-machine efficiency of the equipment maintenance plan.
  • the motion capture system drives the virtual person to perform real-time operations, and drives the visual accessibility and comfort analysis to simulate the human-machine efficiency.
  • the operator can drive the virtual character in the scene to perform 1:1 motion simulation, truly simulate the actual operation process, and verify whether the operator's field of vision accessibility and activity space comfort meet the specifications.
  • the operator wears the device to truly simulate the entire operation process in the actual working conditions in the virtual scene, and quickly and conveniently implement simulation training for new employees. If the scene is selected in the generator motor layer, the power cabinet in the scene is used for operation simulation verification.
  • the operator manipulates the virtual character, it is a 1:1 real-time action.
  • the operator can manipulate the virtual character in the scene to walk, half squat, squat, etc.
  • the program displays the virtual character's field of view interface in real time to verify the world accessibility during the operation process and avoid invisible or difficult-to-see equipment operations during the actual operation process; at the same time, when the operator operates the equipment in different positions, he can personally feel the human body comfort during the operation process and verify the operation comfort level in the actual operation.
  • operators can manipulate virtual characters in the scene to interact with the models in the scene at any time and place. For example, opening the cabinet door, operating the power equipment buttons in the cabinet, etc. With the assistance of the xsens full-body motion capture system, operators can manipulate virtual characters in the scene to simulate the real operation process, learn and master the operation process, and assess the operator's operation level.
  • a virtual human driving device for electric scene based on a motion capture system, for implementing the method described, the device comprising:
  • connection setting module is used to establish a communication connection between the Xsens motion capture system and the VR editor and set up the real-time driver interface;
  • Courseware loading module used to load courseware scene data in the virtual reality editor
  • Data capture module used to execute equipment maintenance plans and obtain motion capture data
  • the human body driving module is used to drive the virtual human to perform real-time operations based on the real-time driving interface through motion capture data.
  • the apparatus may be implemented using a method embodiment.
  • the present invention further provides an electronic device, which may be an industrial computer, a server or a computer terminal.
  • the electronic device includes a processor, a memory, and a computer program stored in the memory and executable by the processor, wherein when the computer program is executed by the processor, the steps of the power scene virtual human body driving method based on the motion capture system are implemented.
  • the electronic device includes a processor, a memory and a network interface connected via a system bus, wherein the memory may include a non-volatile storage medium and an internal memory.
  • the non-volatile storage medium can store an operating system and a computer program.
  • the computer program includes program instructions, and when the program instructions are executed, the processor can execute any power scene virtual human driving method based on the motion capture system.
  • the processor is used to provide computing and control capabilities to support the operation of the entire electronic device.
  • the internal memory provides an environment for the operation of the computer program in the non-volatile storage medium.
  • the processor can execute any power scene virtual human driving method based on the motion capture system.
  • the network interface is used for network communication, such as sending assigned tasks.
  • the processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.
  • the general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.
  • the processor is used to run a computer program stored in the memory to implement the following steps:
  • the virtual human is driven to perform real-time operations through motion capture data.
  • the present invention further provides a computer-readable storage medium, on which a computer program is stored.
  • the computer program is executed by a processor, the steps of the method for driving a virtual human body in an electric power scene based on a motion capture system are implemented.

Landscapes

  • Processing Or Creating Images (AREA)

Abstract

Disclosed in the present invention are a motion-capture-system-based virtual human body driving method and apparatus in an electric power system. The method comprises: establishing a communication connection between an Xsens motion capture system and a virtual reality editor, and setting a real-time driving interface; loading courseware scene data in the virtual reality editor; executing a device maintenance scheme, and acquiring motion capture data; and on the basis of the real-time driving interface, driving, by means of the motion capture data, a virtual person to perform a real-time operation. In the present invention, by means of customizing and developing an Xsens motion capture system software interface, a motion capture interface for interactive courseware is developed by using the existing virtual reality editor MakeReal3D in combination with maintenance courseware data, the interface supports an operator in wearing a motion capture apparatus to drive a simulated person in a virtual scene, instructions for the person such as walking, squatting, running and jumping can be executed, and the maintenance scheme of a hydraulic power plant can be verified in cooperation with a man-machine ergonomics module; and according to safety regulations and operation procedures, safety operation emergency drills can be performed, and recording, calling, etc., of various character action animations can be completed.

Description

基于动捕***的电力场景虚拟人体驱动方法及装置Method and device for driving virtual human body in power scene based on motion capture system 技术领域Technical Field
本发明涉及虚拟现实技术领域，尤其涉及一种基于动捕***的电力场景虚拟人体驱动方法及装置。The present invention relates to the field of virtual reality technology, and in particular to a method and device for driving a virtual human body in an electric scene based on a motion capture system.
背景技术Background technique
“动作捕捉”就是通过特定的捕捉设备将动捕演员的运动信息记录下来，作为制作动画用的原始数据。这些原始数据将用来驱动虚拟的三维动画角色模型。相对于直接绘制三维动画角色的运动，上述方法一方面降低了对美术人员工作经验的依赖，降低了相关动画制作的人力成本；另一方面它也使得所制作的动画更为逼真流畅。目前，制作虚拟角色的软件越来越多，技术也走向成熟，越来越多的人能够参与到虚拟角色的构建工作中。"Motion capture" is to record the motion information of the actors through a specific capture device as raw data for animation production. These raw data will be used to drive the virtual 3D animation character model. Compared with directly drawing the motion of 3D animation characters, the above method reduces the dependence on the work experience of artists and reduces the labor cost of related animation production; on the other hand, it also makes the produced animation more realistic and smooth. At present, there are more and more software for making virtual characters, and the technology is maturing, so more and more people can participate in the construction of virtual characters.
传统的全身动捕设备由于包含大量的高精度器件，因此价格高昂，一般需要数万元甚至几十万、几百万，因此不利于低成本制作。另外大部分动捕设备在结合手部动作时效果较差，只能实现手掌的方向，不能生成手指精细的动作数据，难以结合虚拟场景做出手部交互的效果。Traditional full-body motion capture equipment contains a large number of high-precision components, so it is expensive, generally costing tens of thousands or even hundreds of thousands or millions of dollars, which is not conducive to low-cost production. In addition, most motion capture equipment has poor results when combined with hand movements. It can only realize the direction of the palm, and cannot generate fine finger movement data, making it difficult to combine with virtual scenes to create hand interaction effects.
Xsens MVN是一款便于操作、配置高端的惯性动作捕捉***。MVN动作捕捉***采用最新的惯性传感技术，具有强大且可靠的硬件，并可生成用于使用的数据，硬件效能与软件功能方面均获得大幅的提升，适用于3D人物动画制作、游戏开发、虚拟偶像、游戏电影艺术、人机交互、人机工效、虚拟现实，运动分析、训练和模拟等领域。Xsens MVN is an easy-to-use, high-end inertial motion capture system. The MVN motion capture system uses the latest inertial sensing technology, has powerful and reliable hardware, and can generate data for use. Both hardware performance and software functions have been greatly improved. It is suitable for 3D character animation production, game development, virtual idols, game film art, human-computer interaction, human-computer ergonomics, virtual reality, motion analysis, training and simulation.
发明内容Summary of the invention
为克服上述现有技术的不足，本发明提供一种基于动捕***的电力场景虚拟人体驱动方法及装置，用以解决上述至少一个技术问题。In order to overcome the above-mentioned deficiencies in the prior art, the present invention provides a method and device for driving a virtual human body in an electric scene based on a motion capture system, so as to solve at least one of the above-mentioned technical problems.
根据本发明说明书的一方面，提供一种基于动捕***的电力场景虚拟人体驱动方法，包括：According to one aspect of the present invention, a method for driving a virtual human body in an electric scene based on a motion capture system is provided, comprising:
在Xsens动捕***与虚拟现实编辑器之间建立通信连接并进行实时驱动接口设置；Establish communication connection between Xsens motion capture system and VR editor and set up real-time driver interface;
在虚拟现实编辑器中加载课件场景数据；Load the courseware scene data in the virtual reality editor;
执行设备检修方案并获取动作捕捉数据；Execute equipment maintenance plans and obtain motion capture data;
基于所述实时驱动接口，通过动作捕捉数据驱动虚拟人进行实时操作。Based on the real-time driving interface, the virtual human is driven to perform real-time operations through motion capture data.
作为进一步的技术方案，所述方法还包括：基于现有虚拟现实编辑器中的功效学人体执行任务的行走、蹲起、跑跳指令。 As a further technical solution, the method also includes: walking, squatting, running and jumping instructions for human body execution tasks based on the ergonomics in the existing virtual reality editor.
作为进一步的技术方案，所述方法还包括：根据虚拟现实编辑器中功效学人体的骨骼ID建立动捕***的格式化数据驱动配置，在虚拟现实编辑器中功效学人体骨架组成结构驱动节点与动捕***的数据结构节点之间建立匹配连接。As a further technical solution, the method also includes: establishing a formatted data-driven configuration of the motion capture system according to the skeleton ID of the ergonomic human body in the virtual reality editor, and establishing a matching connection between the ergonomic human body skeleton component structure drive node in the virtual reality editor and the data structure node of the motion capture system.
作为进一步的技术方案，所述方法还包括：在动作捕捉驱动时，采用动作捕捉的头部跟踪数据驱动头盔第一人称视点进行可视性验证。As a further technical solution, the method further includes: when driven by motion capture, using motion capture head tracking data to drive the helmet's first-person viewpoint to perform visibility verification.
作为进一步的技术方案，所述方法还包括：在人体穿戴动作捕捉***时，初始化预设姿态以得到初始位姿数据，并将初始位姿数据导入虚拟现实编辑器，以触发实时驱动接口启动。As a further technical solution, the method also includes: when the human body wears the motion capture system, initializing a preset posture to obtain initial posture data, and importing the initial posture data into a virtual reality editor to trigger the start of a real-time drive interface.
根据本发明说明书的一方面，提供一种基于动捕***的电力场景虚拟人体驱动装置，用于实现所述的方法，所述装置包括：According to one aspect of the present invention, a virtual human driving device for electric scene based on a motion capture system is provided, for implementing the method described, the device comprising:
连接设置模块，用于在Xsens动捕***与虚拟现实编辑器之间建立通信连接并进行实时驱动接口设置；The connection setting module is used to establish a communication connection between the Xsens motion capture system and the VR editor and set up the real-time driver interface;
课件加载模块，用于在虚拟现实编辑器中加载课件场景数据；Courseware loading module, used to load courseware scene data in the virtual reality editor;
数据捕获模块，用于执行设备检修方案并获取动作捕捉数据；Data capture module, used to execute equipment maintenance plans and obtain motion capture data;
人体驱动模块，用于基于所述实时驱动接口，通过动作捕捉数据驱动虚拟人进行实时操作。The human body driving module is used to drive the virtual human to perform real-time operations based on the real-time driving interface through motion capture data.
根据本发明说明书的一方面，提供一种电子设备，所述电子设备包括处理器、存储器，以及存储在所述存储器上并可被所述处理器执行的计算机程序，其中所述计算机程序被所述处理器执行时，实现所述的基于动捕***的电力场景虚拟人体驱动方法的步骤。According to one aspect of the present specification, there is provided an electronic device, comprising a processor, a memory, and a computer program stored in the memory and executable by the processor, wherein when the computer program is executed by the processor, the steps of the method for driving a virtual human body in an electric scene based on a motion capture system are implemented.
根据本发明说明书的一方面，提供一种计算机可读存储介质，其特征在于，所述计算机可读存储介质上存储有计算机程序，其中所述计算机程序被处理器执行时，实现所述的基于动捕***的电力场景虚拟人体驱动方法的步骤。According to one aspect of the present specification, a computer-readable storage medium is provided, characterized in that a computer program is stored on the computer-readable storage medium, wherein when the computer program is executed by a processor, the steps of the method for driving a virtual human body in an electric power scene based on a motion capture system are implemented.
与现有技术相比，本发明的有益效果在于：Compared with the prior art, the present invention has the following beneficial effects:
本发明通过对Xsens MVN BIOMECHLINK动捕***软件接口定制开发，利用现有虚拟现实编辑器MakeReal3D，结台检修课件数据，开发用于交互式课件的动作捕捉接口，该接口支持操作者身穿动作捕捉装置，驱动虚拟场景中的模似人，可执行人物的行走、蹲起、跑跳等指令，配合人机工效模块，可进行水电厂检修方案验证；根据安规操作守则，可进行安全操作应急演练以及完成各类人物动作动画的录制调用等功能。The present invention customizes and develops the software interface of the Xsens MVN BIOMECHLINK motion capture system, utilizes the existing virtual reality editor MakeReal3D, and combines the maintenance courseware data to develop a motion capture interface for interactive courseware. The interface supports the operator wearing a motion capture device to drive a simulated person in a virtual scene, and can execute commands such as walking, squatting, running and jumping of the character. In conjunction with the ergonomics module, it can verify the maintenance plan of the hydropower plant; according to the safety operating rules, it can conduct safety operation emergency drills and complete the recording and calling of various types of character action animations.
附图说明BRIEF DESCRIPTION OF THE DRAWINGS
图1为根据本发明实施例的基于动捕***的电力场景虚拟人体驱动方法的流程图。 FIG. 1 is a flow chart of a method for driving a virtual human body in an electric power scene based on a motion capture system according to an embodiment of the present invention.
图2为根据本发明实施例的动捕***软件接口实现示意图。FIG. 2 is a schematic diagram of a software interface implementation of a motion capture system according to an embodiment of the present invention.
具体实施方式Detailed ways
以下将结合附图对本发明各实施例的技术方案进行清楚、完整的描述，显然，所描述发实施例仅仅是本发明的一部分实施例，而不是全部的实施例。基于本发明的实施例，本领域普通技术人员在没有做出创造性劳动的前提下所得到的所有其它实施例，都属于本发明所保护的范围。The following will clearly and completely describe the technical solutions of various embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.
根据本发明说明书的一方面，提供一种基于动捕***的电力场景虚拟人体驱动方法，该方法通过对Xsens MVN BIOMECHLINK动捕***软件接口定制开发，利用现有虚拟现实编辑器MakeReal3D，结台检修课件数据，开发用于交互式课件的动作捕捉接口，该接口支持操作者身穿动作捕捉装置，驱动虚拟场景中的模似人，可执行人物的行走、蹲起、跑跳等指令，配合人机工效模块，可进行水电厂检修方案验证。According to one aspect of the present invention specification, there is provided a method for driving a virtual human body in an electric power scene based on a motion capture system. The method customizes and develops the software interface of the Xsens MVN BIOMECHLINK motion capture system, utilizes the existing virtual reality editor MakeReal3D, and combines maintenance courseware data to develop a motion capture interface for interactive courseware. The interface supports an operator wearing a motion capture device to drive a virtual person in a virtual scene, and can execute commands such as walking, squatting, running and jumping of the character. In conjunction with the human-machine ergonomics module, the maintenance plan of the hydropower plant can be verified.
同时，本发明所述方法还能够根据安规操作守则，进行安全操作应急演练以及完成各类人物动作动画的录制调用等功能。At the same time, the method of the present invention can also perform safety operation emergency drills and complete the recording and calling of various character action animations according to safety regulations and operating rules.
如图1所示，所述方法包括：As shown in FIG1 , the method comprises:
步骤1，在Xsens动捕***与虚拟现实编辑器之间建立通信连接并进行实时驱动接口设置。Step 1: Establish a communication connection between the Xsens motion capture system and the VR editor and set up the real-time driver interface.
步骤2，在虚拟现实编辑器中加载课件场景数据。Step 2: Load the courseware scene data in the virtual reality editor.
步骤3，执行设备检修方案并获取动作捕捉数据。Step 3: Execute the equipment maintenance plan and obtain motion capture data.
步骤4，基于所述实时驱动接口，通过动作捕捉数据驱动虚拟人进行实时操作。Step 4: Based on the real-time driving interface, the virtual human is driven to perform real-time operations through the motion capture data.
本发明基于Xsens MVN BIOMECHLINK动捕***软件接口定制开发，利用现有的编辑器实现基于工效学人体的动作捕捉数据采集、检修方案验证、根据安规操作守则进行安全操作应急演练以及完成各类人物动作动画的录制调用功能。The present invention is customized and developed based on the software interface of the Xsens MVN BIOMECHLINK motion capture system, and utilizes the existing editor to realize the ergonomic human body motion capture data collection, maintenance plan verification, safe operation emergency drills according to safety regulations and operating rules, and complete the recording and calling functions of various character action animations.
如图2所示，Xsens MVN BIOMECHLINK动捕***软件接口定制开发的功能实现流程，基本实现方法如下：As shown in Figure 2, the function implementation process of the customized development of the Xsens MVN BIOMECHLINK motion capture system software interface is as follows:
通过动捕***驱动接口的API进行通信接口开发，以将动捕***捕获的数据实时传输给MakeReal3D虚拟现实编辑器。The communication interface is developed through the API of the motion capture system driver interface to transmit the data captured by the motion capture system to the MakeReal3D virtual reality editor in real time.
Xsens MVN BIOMECHLINK动捕***的硬件数据通过驱动的API接口开发在MakeReal3D中建立通信接口即实时通信，通过实时通信的开关、数据接收、实时存储获得Xsens MVN BIOMECHLINK动捕***的原始数据，然后通过获得的原始数据进行解析开发，将解析开发的数据进行转换开发为MakeReal3D虚拟现实编辑器实时驱动数据。 The hardware data of the Xsens MVN BIOMECHLINK motion capture system is developed through the driver's API interface to establish a communication interface in MakeReal3D, that is, real-time communication. The raw data of the Xsens MVN BIOMECHLINK motion capture system is obtained through the real-time communication switch, data reception, and real-time storage. The obtained raw data is then parsed and developed, and the parsed data is converted and developed into real-time drive data for the MakeReal3D virtual reality editor.
可选地，实时驱动接口主要为用户设定和操作提供驱动接口，驱动接口一般包括数据连接、数据校准、数据设定、数据关闭等设定的开发。Optionally, the real-time driver interface mainly provides a driver interface for user settings and operations. The driver interface generally includes the development of settings such as data connection, data calibration, data setting, and data closing.
在接口开发阶段，除了解决基于Xsens MVN BIOMECHLINK的数据通信、体实时驱动接口开发、交互接口等，另外需要基于现有的MakeReal3D虚拟现实人机工效学分析验证功能和工效学人体进行开发，并开发工效学人体的人物动作录制与调用功能。During the interface development phase, in addition to solving data communication based on Xsens MVN BIOMECHLINK, real-time drive interface development, interactive interface, etc., it is also necessary to develop based on the existing MakeReal3D virtual reality ergonomics analysis and verification functions and ergonomic human body, and develop character motion recording and calling functions of the ergonomic human body.
Xsens MVN BIOMECHLINK动捕***软件接口驱动工效学人体，执行人物的行走、蹲起、跑跳等指令，需要基于现有的MakeReal3D虚拟现实编辑器中的工效学人体进行实现，其实现流程如下：The Xsens MVN BIOMECHLINK motion capture system software interface drives the ergonomic human body to execute commands such as walking, squatting, running and jumping. It needs to be implemented based on the ergonomic human body in the existing MakeReal3D virtual reality editor. The implementation process is as follows:
根据虚拟现实编辑器中功效学人体的骨骼ID建立动捕***的格式化数据驱动配置，在虚拟现实编辑器中功效学人体骨架组成结构驱动节点与动捕***的数据结构节点之间建立匹配连接。A formatted data driven configuration of the motion capture system is established according to the skeleton ID of the functional human body in the virtual reality editor, and a matching connection is established between the functional human body skeleton component structure driving node and the data structure node of the motion capture system in the virtual reality editor.
MakeReal3D实时人体工效学分析验证主要为可视可达舒适度等工效学分析功能，实时驱动并在人体实现以上人体工效学验证，是需要在交互场景中调用该功能，即需在交互场景中增加人机工效学入口。MakeReal3D real-time ergonomic analysis and verification mainly includes ergonomic analysis functions such as visual and achievable comfort. To drive in real time and implement the above ergonomic verification on the human body, it is necessary to call this function in the interactive scene, that is, to add an ergonomic entrance in the interactive scene.
可视性主要以人体主观的视觉判断，动作捕捉驱动的时，采用动作捕捉的头部跟踪数据驱动头盔第一人称视点进行可视性。Visibility is mainly based on subjective human visual judgment. When driven by motion capture, the head tracking data of motion capture is used to drive the first-person viewpoint of the helmet for visibility.
可达性、舒适性等人机工效需增加VR交互下的入***互UI进行进入，通过菜单调用人机工效学分析验证。For ergonomics such as accessibility and comfort, an entry interaction UI under VR interaction needs to be added for entry, and ergonomic analysis verification can be called through the menu.
作为一种实施方式，操作者穿戴此全身动作捕捉***，驱动场景中的虚拟人物进行1：1运动模拟，在现有模型资源场景中，利用此接口，对设备检修方案进行人机功效仿真，通过动作捕捉***驱动虚拟人进行实时操作，驱动可视可达性以及舒适度分析进行人机功效模拟。As an implementation method, the operator wears this full-body motion capture system to drive the virtual character in the scene to perform 1:1 motion simulation. In the existing model resource scene, this interface is used to simulate the human-machine efficiency of the equipment maintenance plan. The motion capture system drives the virtual person to perform real-time operations, and drives the visual accessibility and comfort analysis to simulate the human-machine efficiency.
通过Xsens全身动作捕捉***，操作者可以驱动场景中的虚拟人物进行1：1运动模拟，真实模拟实际操作过程，验证操作者的视界可达性、活动空间舒适性是否符合规范。同时，操作者穿戴设备，可以在虚拟场景中真实模拟实际工况中的全部操作过程，方便快捷的实现对新员工的模拟训练。如场景选在发电电动机层，利用场景中的电力柜进行操作模拟验证。Through the Xsens full-body motion capture system, the operator can drive the virtual character in the scene to perform 1:1 motion simulation, truly simulate the actual operation process, and verify whether the operator's field of vision accessibility and activity space comfort meet the specifications. At the same time, the operator wears the device to truly simulate the entire operation process in the actual working conditions in the virtual scene, and quickly and conveniently implement simulation training for new employees. If the scene is selected in the generator motor layer, the power cabinet in the scene is used for operation simulation verification.
穿戴好全身动作捕捉***设备，调试完成之后进入程序，利用设备操纵场景中的虚拟人物进行动作，其中包括：Put on the full-body motion capture system equipment, enter the program after debugging, and use the equipment to manipulate the virtual characters in the scene to perform actions, including:
1、视界可达性验证与动作舒适性验证 1. Verification of visual accessibility and movement comfort
操作者操纵虚拟人物时，是1：1实时动作。操作者可以操纵场景中的虚拟人物进行行走、半蹲、全蹲等动作，程序实时显示虚拟人物的视界界面，验证操作过程中的世界可达性，避免实际操作过程中，有不可见或者不易见的设备操作；同时，操作者在不同位置操作设备时，可以亲身感受操作过程中的人体舒适度，验证实际操作中的操作舒适程度。When the operator manipulates the virtual character, it is a 1:1 real-time action. The operator can manipulate the virtual character in the scene to walk, half squat, squat, etc. The program displays the virtual character's field of view interface in real time to verify the world accessibility during the operation process and avoid invisible or difficult-to-see equipment operations during the actual operation process; at the same time, when the operator operates the equipment in different positions, he can personally feel the human body comfort during the operation process and verify the operation comfort level in the actual operation.
2、拾取、放置、拆装2. Pick up, place, disassemble and assemble
在装配、检修场景中，操作者可以在任意时间、场合操纵场景中的虚拟人物对场景中的模型进行交互。例如，打开机柜柜门，操作机柜中的电力设备按钮等。在xsens全身动作捕捉***的辅助下，操作者可以在场景中操作虚拟人物进行真实的操作过程模拟，学***。In assembly and maintenance scenarios, operators can manipulate virtual characters in the scene to interact with the models in the scene at any time and place. For example, opening the cabinet door, operating the power equipment buttons in the cabinet, etc. With the assistance of the xsens full-body motion capture system, operators can manipulate virtual characters in the scene to simulate the real operation process, learn and master the operation process, and assess the operator's operation level.
根据本发明说明书的一方面，还提供一种基于动捕***的电力场景虚拟人体驱动装置，用于实现所述的方法，所述装置包括：According to one aspect of the present invention, there is also provided a virtual human driving device for electric scene based on a motion capture system, for implementing the method described, the device comprising:
连接设置模块，用于在Xsens动捕***与虚拟现实编辑器之间建立通信连接并进行实时驱动接口设置；The connection setting module is used to establish a communication connection between the Xsens motion capture system and the VR editor and set up the real-time driver interface;
课件加载模块，用于在虚拟现实编辑器中加载课件场景数据；Courseware loading module, used to load courseware scene data in the virtual reality editor;
数据捕获模块，用于执行设备检修方案并获取动作捕捉数据；Data capture module, used to execute equipment maintenance plans and obtain motion capture data;
人体驱动模块，用于基于所述实时驱动接口，通过动作捕捉数据驱动虚拟人进行实时操作。The human body driving module is used to drive the virtual human to perform real-time operations based on the real-time driving interface through motion capture data.
所述装置可利用方法实施方式来实现。The apparatus may be implemented using a method embodiment.
本发明一方面还提供一种电子设备，该电子设备可以为工控机、服务器或计算机终端。In one aspect, the present invention further provides an electronic device, which may be an industrial computer, a server or a computer terminal.
所述电子设备包括处理器、存储器，以及存储在所述存储器上并可被所述处理器执行的计算机程序，其中所述计算机程序被所述处理器执行时，实现所述的基于动捕***的电力场景虚拟人体驱动方法的步骤。The electronic device includes a processor, a memory, and a computer program stored in the memory and executable by the processor, wherein when the computer program is executed by the processor, the steps of the power scene virtual human body driving method based on the motion capture system are implemented.
该电子设备包括通过***总线连接的处理器、存储器和网络接口，其中，存储器可以包括非易失性存储介质和内存储器。The electronic device includes a processor, a memory and a network interface connected via a system bus, wherein the memory may include a non-volatile storage medium and an internal memory.
非易失性存储介质可存储操作***和计算机程序。该计算机程序包括程序指令，该程序指令被执行时，可使得处理器执行任意一种基于动捕***的电力场景虚拟人体驱动方法。The non-volatile storage medium can store an operating system and a computer program. The computer program includes program instructions, and when the program instructions are executed, the processor can execute any power scene virtual human driving method based on the motion capture system.
处理器用于提供计算和控制能力，支撑整个电子设备的运行。内存储器为非易失性存储介质中的计算机程序的运行提供环境，该计算机程序被处理器执行时，可使得处理器执行任意一种基于动捕***的电力场景虚拟人体驱动方法。 The processor is used to provide computing and control capabilities to support the operation of the entire electronic device. The internal memory provides an environment for the operation of the computer program in the non-volatile storage medium. When the computer program is executed by the processor, the processor can execute any power scene virtual human driving method based on the motion capture system.
该网络接口用于进行网络通信，如发送分配的任务等。The network interface is used for network communication, such as sending assigned tasks.
应当理解的是，处理器可以是中央处理单元(CentralProcessingUnit，CPU)，该处理器还可以是其他通用处理器、数字信号处理器(DigitalSignalProcessor，DSP)、专用集成电路(ApplicationSpecificIntegratedCircuit，ASIC)、现场可编程门阵列(Field-ProgrammableGateArray，FPGA)或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件等。其中，通用处理器可以是微处理器或者该处理器也可以是任何常规的处理器等。It should be understood that the processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.
其中，在一个实施例中，所述处理器用于运行存储在存储器中的计算机程序，以实现如下步骤：In one embodiment, the processor is used to run a computer program stored in the memory to implement the following steps:
在Xsens动捕***与虚拟现实编辑器之间建立通信连接并进行实时驱动接口设置；Establish communication connection between Xsens motion capture system and VR editor and set up real-time driver interface;
在虚拟现实编辑器中加载课件场景数据；Load the courseware scene data in the virtual reality editor;
执行设备检修方案并获取动作捕捉数据；Execute equipment maintenance plans and obtain motion capture data;
基于所述实时驱动接口，通过动作捕捉数据驱动虚拟人进行实时操作。Based on the real-time driving interface, the virtual human is driven to perform real-time operations through motion capture data.
本发明一方面还提供一种计算机可读存储介质，所述计算机可读存储介质上存储有计算机程序，其中所述计算机程序被处理器执行时，实现所述的基于动捕***的电力场景虚拟人体驱动方法的步骤。On one hand, the present invention further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the method for driving a virtual human body in an electric power scene based on a motion capture system are implemented.
在本说明书的描述中，参考术语“一个实施方式”、“某些实施方式”、“示意性实施方式”、“示例”、“具体示例”、或“一些示例”等的描述意指结合所述实施方式或示例描述的具体特征、结构、材料或者特点包含于本发明的至少一个实施方式或示例中。在本说明书中，对上述术语的示意性表述不一定指的是相同的实施方式或示例。而且，描述的具体特征、结构、材料或者特点可以在任何的一个或多个实施方式或示例中以合适的方式结合。In the description of this specification, the description with reference to the terms "one embodiment", "certain embodiments", "illustrative embodiments", "examples", "specific examples", or "some examples" means that the specific features, structures, materials, or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.
最后应说明的是：以上实施例仅用以说明本发明的技术方案，而非对其限制；尽管参照前述实施例对本发明进行了详细的说明，本领域的普通技术人员应当理解：其依然可以对前述实施例所记载的技术方案进行修改，或者对其中部分或者全部技术特征进行等同替换；而这些修改或者替换，并不使相应技术方案的本质脱离本发明实施例技术方案。 Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the technical solutions of the embodiments of the present invention.

Claims (8)

  1. 基于动捕***的电力场景虚拟人体驱动方法，其特征在于，包括：A method for driving a virtual human body in an electric scene based on a motion capture system is characterized by comprising:
    在Xsens动捕***与虚拟现实编辑器之间建立通信连接并进行实时驱动接口设置；Establish communication connection between Xsens motion capture system and VR editor and set up real-time driver interface;
    在虚拟现实编辑器中加载课件场景数据；Load the courseware scene data in the virtual reality editor;
    执行设备检修方案并获取动作捕捉数据；Execute equipment maintenance plans and obtain motion capture data;
    基于所述实时驱动接口，通过动作捕捉数据驱动虚拟人进行实时操作。Based on the real-time driving interface, the virtual human is driven to perform real-time operations through motion capture data.
  2. 根据权利要求1所述基于动捕***的电力场景虚拟人体驱动方法，其特征在于，所述方法还包括：基于现有虚拟现实编辑器中的功效学人体执行任务的行走、蹲起、跑跳指令。According to the method for driving a virtual human body in an electric power scene based on a motion capture system according to claim 1, it is characterized in that the method further comprises: walking, squatting, running and jumping instructions for performing tasks based on the ergonomic human body in an existing virtual reality editor.
  3. 根据权利要求2所述基于动捕***的电力场景虚拟人体驱动方法，其特征在于，所述方法还包括：根据虚拟现实编辑器中功效学人体的骨骼ID建立动捕***的格式化数据驱动配置，在虚拟现实编辑器中功效学人体骨架组成结构驱动节点与动捕***的数据结构节点之间建立匹配连接。According to claim 2, the method for driving a virtual human body in an electric scene based on a motion capture system is characterized in that the method further comprises: establishing a formatted data driving configuration of the motion capture system according to the skeleton ID of the ergonomic human body in the virtual reality editor, and establishing a matching connection between the ergonomic human body skeleton composition structure driving node and the data structure node of the motion capture system in the virtual reality editor.
  4. 根据权利要求2所述基于动捕***的电力场景虚拟人体驱动方法，其特征在于，所述方法还包括：在动作捕捉驱动时，采用动作捕捉的头部跟踪数据驱动头盔第一人称视点进行可视性验证。According to the method for driving a virtual human body in an electric power scene based on a motion capture system in claim 2, the method further comprises: during motion capture driving, using the motion capture head tracking data to drive the first-person viewpoint of the helmet for visibility verification.
  5. 根据权利要求1所述基于动捕***的电力场景虚拟人体驱动方法，其特征在于，所述方法还包括：在人体穿戴动作捕捉***时，初始化预设姿态以得到初始位姿数据，并将初始位姿数据导入虚拟现实编辑器，以触发实时驱动接口启动。According to the method for driving a virtual human body in an electric power scene based on a motion capture system according to claim 1, the method further comprises: when the human body wears the motion capture system, initializing a preset posture to obtain initial posture data, and importing the initial posture data into a virtual reality editor to trigger the start of a real-time driving interface.
  6. 基于动捕***的电力场景虚拟人体驱动装置，用于实现权利要求1-5任一项所述的方法，其特征在于，所述装置包括：A virtual human driving device for electric scenes based on a motion capture system, used to implement the method described in any one of claims 1 to 5, characterized in that the device comprises:
    连接设置模块，用于在Xsens动捕***与虚拟现实编辑器之间建立通信连接并进行实时驱动接口设置；The connection setting module is used to establish a communication connection between the Xsens motion capture system and the VR editor and set up the real-time driver interface;
    课件加载模块，用于在虚拟现实编辑器中加载课件场景数据；Courseware loading module, used to load courseware scene data in the virtual reality editor;
    数据捕获模块，用于执行设备检修方案并获取动作捕捉数据；Data capture module, used to execute equipment maintenance plans and obtain motion capture data;
    人体驱动模块，用于基于所述实时驱动接口，通过动作捕捉数据驱动虚拟人进行实时操作。The human body driving module is used to drive the virtual human to perform real-time operations based on the real-time driving interface through motion capture data.
  7. 一种电子设备，其特征在于，所述电子设备包括处理器、存储器，以及存储在所述存储器上并可被所述处理器执行的计算机程序，其中所述计算机程序被所述处理器执行时，实现如权利要求1至5中任一项所述的基于动捕***的电力场景虚拟人体驱动方法的步骤。 An electronic device, characterized in that the electronic device comprises a processor, a memory, and a computer program stored in the memory and executable by the processor, wherein when the computer program is executed by the processor, the steps of the method for driving a virtual human body in an electric scene based on a motion capture system as described in any one of claims 1 to 5 are implemented.
  8. 一种计算机可读存储介质，其特征在于，所述计算机可读存储介质上存储有计算机程序，其中所述计算机程序被处理器执行时，实现如权利要求1至5中任一项所述的基于动捕***的电力场景虚拟人体驱动方法的步骤。 A computer-readable storage medium, characterized in that a computer program is stored on the computer-readable storage medium, wherein when the computer program is executed by a processor, the steps of the method for driving a virtual human body in an electric scene based on a motion capture system as described in any one of claims 1 to 5 are implemented.
PCT/CN2023/116033 2022-11-28 2023-08-31 Motion-capture-system-based virtual human body driving method and apparatus in electric power scene WO2024114000A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202211508395.7 2022-11-28
CN202211508395.7A CN115861496A (en) 2022-11-28 2022-11-28 Power scene virtual human body driving method and device based on dynamic capture system

Publications (1)

Publication Number Publication Date
WO2024114000A1 true WO2024114000A1 (en) 2024-06-06

Family

ID=85667577

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2023/116033 WO2024114000A1 (en) 2022-11-28 2023-08-31 Motion-capture-system-based virtual human body driving method and apparatus in electric power scene

Country Status (2)

Country Link
CN (1) CN115861496A (en)
WO (1) WO2024114000A1 (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115861496A (en) * 2022-11-28 2023-03-28 南方电网储能股份有限公司 Power scene virtual human body driving method and device based on dynamic capture system

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2013096403A1 (en) * 2011-12-21 2013-06-27 Twentieth Century Fox Film Corporation System, method and apparatus for rapid film pre-visualization
CN104573230A (en) * 2015-01-06 2015-04-29 北京卫星环境工程研究所 Virtual human work task simulation analyzing system and method for spacecraft repair
CN115113725A (en) * 2022-01-06 2022-09-27 南方电网调峰调频发电有限公司 Power professional virtual reality post evaluation and assessment method and system based on dynamic capturing system
CN115220578A (en) * 2022-06-30 2022-10-21 华东交通大学 Interactive VR system and method based on optical motion capture
CN115861496A (en) * 2022-11-28 2023-03-28 南方电网储能股份有限公司 Power scene virtual human body driving method and device based on dynamic capture system

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2013096403A1 (en) * 2011-12-21 2013-06-27 Twentieth Century Fox Film Corporation System, method and apparatus for rapid film pre-visualization
CN104573230A (en) * 2015-01-06 2015-04-29 北京卫星环境工程研究所 Virtual human work task simulation analyzing system and method for spacecraft repair
CN115113725A (en) * 2022-01-06 2022-09-27 南方电网调峰调频发电有限公司 Power professional virtual reality post evaluation and assessment method and system based on dynamic capturing system
CN115220578A (en) * 2022-06-30 2022-10-21 华东交通大学 Interactive VR system and method based on optical motion capture
CN115861496A (en) * 2022-11-28 2023-03-28 南方电网储能股份有限公司 Power scene virtual human body driving method and device based on dynamic capture system

Also Published As

Publication number Publication date
CN115861496A (en) 2023-03-28

Similar Documents

Publication Publication Date Title
CN110930483B (en) Role control method, model training method and related device
CN103513992B (en) A kind of general Edutainment robot application software development platform
CN104111861B (en) Unmanned aerial vehicle simulation training system and control method thereof
US11823315B2 (en) Animation making method and apparatus, computing device, and storage medium
WO2024114000A1 (en) Motion-capture-system-based virtual human body driving method and apparatus in electric power scene
CN110928418A (en) Aviation cable auxiliary assembly method and system based on MR
CN105374251A (en) Mine virtual reality training system based on immersion type input and output equipment
CN111223171A (en) Image processing method, image processing device, electronic equipment and storage medium
CN110335359B (en) Distribution board fire accident emergency drilling simulation method based on virtual reality technology
CN114372356B (en) Artificial enhancement method, device and medium based on digital twins
CN108153932B (en) Desktop type three-dimensional virtual maintenance model modeling system
CN111341166A (en) Control method and system of emergency drilling system
CN105336241A (en) Dangerous situation survival simulation training system and establishment method thereof
CN105138315B (en) A kind of virtual assembly system based on Visual Basic and EON Studio
WO2024114001A1 (en) Data glove interface implementation method and apparatus for power interaction courseware, and device
CN112988267A (en) Loading method and device, storage medium and electronic equipment
CN110457227B (en) System and method for assisting developers or students in VR application development
Strazdins et al. Virtual reality using gesture recognition for deck operation training
CN209486946U (en) Protection simulation training platform system based on VR technology
CN115797517A (en) Data processing method, device, equipment and medium of virtual model
CN115115814A (en) Information processing method, information processing apparatus, readable storage medium, and electronic apparatus
Giunchi et al. DreamCodeVR: Towards Democratizing Behavior Design in Virtual Reality with Speech-Driven Programming
CN102929632B (en) Control action executing method
KR101965732B1 (en) the method for controling motion platform using the authoring tool
CN111600891A (en) Exoskeleton robot communication protocol, man-machine interaction system and multi-platform interaction software