WO2022021782A1 - Method and system for automatically generating six-dimensional posture data set, and terminal and storage medium - Google Patents

Method and system for automatically generating six-dimensional posture data set, and terminal and storage medium Download PDF

Info

Publication number
WO2022021782A1
WO2022021782A1 PCT/CN2020/139761 CN2020139761W WO2022021782A1 WO 2022021782 A1 WO2022021782 A1 WO 2022021782A1 CN 2020139761 W CN2020139761 W CN 2020139761W WO 2022021782 A1 WO2022021782 A1 WO 2022021782A1
Authority
WO
WIPO (PCT)
Prior art keywords
dimensional
data set
model
dimensional model
image
Prior art date
Application number
PCT/CN2020/139761
Other languages
French (fr)
Chinese (zh)
Inventor
张耕慎
宁立
张涌
Original Assignee
中国科学院深圳先进技术研究院
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 中国科学院深圳先进技术研究院 filed Critical 中国科学院深圳先进技术研究院
Publication of WO2022021782A1 publication Critical patent/WO2022021782A1/en

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
    • G06T7/75—Determining position or orientation of objects or cameras using feature-based methods involving models
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T19/00—Manipulating 3D models or images for computer graphics
    • G06T19/20—Editing of 3D images, e.g. changing shapes or colours, aligning objects or positioning parts
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10024—Color image
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10028—Range image; Depth image; 3D point clouds

Definitions

  • the present application belongs to the technical field of six-dimensional attitude estimation, and in particular, relates to a method, system, terminal and storage medium for automatically generating a six-dimensional attitude data set.
  • FIG. 1 it is a flow chart of obtaining a six-dimensional pose data set in a traditional method.
  • deep learning datasets usually require a large number of images and accurate labels
  • traditional dataset acquisition methods require a lot of time, manpower, money, etc., which are cumbersome and computationally complex.
  • the present application provides a method, system, terminal and storage medium for automatically generating a six-dimensional pose data set, aiming to solve the problems of existing data set acquisition methods that require a lot of time, manpower, money, tedious operations and high computational complexity technical issues.
  • Step a build a data set automatic generation platform based on OpenGL virtual environment
  • Step b obtaining the three-dimensional model of the object and the six-dimensional attitude parameters of the three-dimensional model, and importing the three-dimensional model and the six-dimensional attitude parameters into the data set automatic generation platform;
  • Step c The data set automatic generation platform controls the three-dimensional model to change the posture in the OpenGL virtual environment according to the six-dimensional attitude parameters, and collects the three-dimensional model in each six-dimensional environment through the virtual camera in the OpenGL virtual environment.
  • the RGB color image and the corresponding depth image of the object in each six-dimensional attitude are generated, and the six-dimensional attitude label of the image is marked.
  • the technical solutions adopted in the embodiments of the present application further include: in the step a, the acquiring the three-dimensional model of the object and the six-dimensional attitude parameters of the three-dimensional model further include:
  • the three-dimensional model and six-dimensional pose parameters are converted into a unified file format imported into the data set automatic generation platform.
  • the technical solution adopted in the embodiment of the present application further includes: in the step a, the importing the three-dimensional model and the six-dimensional pose parameters into the data set automatic generation platform further includes:
  • the technical solution adopted in the embodiment of the present application further includes: in the step c, the shape of the three-dimensional model in each six-dimensional pose is the multiplication of the coordinates (x, y, z) of each vertex in the initial shape of the three-dimensional model
  • the technical solution adopted in the embodiment of the present application further includes: in the step c, the generating an RGB color image of the object under each six-dimensional pose includes:
  • the object between the near section and the far section is orthographically projected onto the near section, and stored in the color buffer as a binary integer number;
  • the technical solution adopted in the embodiment of the present application further includes: in the step c, the generating a depth image of the object under each six-dimensional pose includes:
  • Allocate a depth value between the near section and the far section obtain the depth value of the front surface of the object, and store the depth value in the depth buffer as a floating-point value;
  • the technical solution adopted in the embodiment of the present application further includes: in the step c, generating an RGB color image and a corresponding depth image of the object under each six-dimensional posture, and labeling the six-dimensional posture label of the image and further include:
  • the RGB color image and the depth image collected by the virtual camera are saved, and the corresponding six-dimensional attitude when the image is collected is used as the six-dimensional attitude label of the RGB color image and the depth image.
  • a system for automatically generating a six-dimensional attitude data set comprising:
  • Platform building module used to build a data set automatic generation platform based on OpenGL virtual environment
  • Model and parameter acquisition module used to acquire the three-dimensional model of the object and the six-dimensional attitude parameters of the three-dimensional model
  • Data import module used to import the three-dimensional model and six-dimensional attitude parameters into the data set automatic generation platform
  • Image acquisition module used to control the posture of the three-dimensional model to change in the OpenGL virtual environment according to the six-dimensional posture parameters, and collect the form of the three-dimensional model in each six-dimensional posture through the virtual camera in the OpenGL virtual environment , generate an RGB color image and a corresponding depth image of the object under each six-dimensional pose, and mark the six-dimensional pose label of the image.
  • a terminal includes a processor and a memory coupled to the processor, wherein,
  • the memory stores program instructions for realizing the method for automatically generating the six-dimensional attitude data set
  • the processor is configured to execute the program instructions stored in the memory to control the automatic generation of a six-dimensional pose data set.
  • a storage medium storing program instructions executable by a processor, where the program instructions are used to execute the method for automatically generating a six-dimensional attitude data set.
  • the beneficial effects of the embodiments of the present application are: the method, system, terminal and storage medium for automatically generating a six-dimensional attitude data set according to the embodiments of the present application are based on OpenGL by importing the three-dimensional model and six-dimensional attitude parameters of the object into The data set of the virtual environment automatically generates a platform, which controls the three-dimensional model to continuously change the posture in the virtual environment according to the six-dimensional posture parameters, and the RGB color image and the corresponding depth image of the object under each six-dimensional posture are automatically captured by the virtual camera. , and automatically annotate 6D pose labels.
  • the embodiment of the present application can efficiently generate RGB color images and depth images and six-dimensional attitude labels in different six-dimensional attitudes of the three-dimensional model through simple operations, which is less time-consuming and saves manpower and money. cost consumption.
  • Fig. 1 is the flow chart of obtaining the six-dimensional attitude data set in the traditional method
  • FIG. 2 is a flowchart of a method for automatically generating a six-dimensional attitude dataset according to an embodiment of the present application
  • Fig. 3 is the working principle diagram of the data set automatic generation platform of the embodiment of the present application.
  • FIG. 4 is a schematic diagram of a virtual camera setting according to an embodiment of the present application.
  • FIG. 5 is a schematic structural diagram of a system for automatically generating a six-dimensional attitude data set according to an embodiment of the present application
  • FIG. 6 is a schematic diagram of the experimental results of the three-dimensional model of the stl format according to the embodiment of the application; wherein, (a) is a real object; (b) is the RGB obtained by importing the stl three-dimensional model of the real object into the automatic generation platform of the embodiment of the application Color image, (c) is the RGB color image obtained by performing certain pose transformation on the 3D model in (b); (d) is the front view and left view of the real 3D point cloud of the stl 3D model of the object; ( e) is the depth image obtained by importing the stl three-dimensional model of the real object into the automatic generation platform of the embodiment of the application, and then converting it into a front view and a left view of a three-dimensional point cloud;
  • FIG. 7 is a schematic diagram of an experimental result of a three-dimensional model in ply format according to an embodiment of the application; wherein (a) is a real object, (b) is a ply three-dimensional model captured by a virtual camera, and (c) is a three-dimensional model in (b) The RGB color image obtained by the model performing a certain attitude transformation; (d) is the front view and left view of the real 3D point cloud of the ply 3D model of the object; (e) is the ply 3D model of the object imported into the embodiment of the application The depth image obtained by the automatic generation platform is converted into the front view and left view of the 3D point cloud;
  • FIG. 8 is a schematic structural diagram of a terminal according to an embodiment of the present application.
  • FIG. 9 is a schematic structural diagram of a storage medium according to an embodiment of the present application.
  • the method for automatically generating a six-dimensional pose data set constructs a data set automatic generation platform based on an OpenGL (Open Graphics Library) virtual environment, and combines the three-dimensional model of the object with the Relative to the six-dimensional attitude parameters of the virtual camera, the platform is imported into the platform, and the platform controls the three-dimensional model to continuously change the attitude in the virtual environment according to the six-dimensional attitude parameters, and the virtual camera automatically captures the RGB color image of the object under each six-dimensional attitude and the corresponding The depth image is automatically annotated with 6D pose labels.
  • OpenGL Open Graphics Library
  • FIG. 2 is a flowchart of a method for automatically generating a six-dimensional pose data set according to an embodiment of the present application.
  • the method for automatically generating a six-dimensional pose data set according to the embodiment of the present application includes the following steps:
  • Step 100 build a data set automatic generation platform based on the OpenGL virtual environment
  • Step 200 obtaining a three-dimensional model of the object and a six-dimensional attitude parameter of the three-dimensional model relative to the virtual camera;
  • t] is three-dimensional translation and three-dimensional rotation relative to the photographing device.
  • Three-dimensional translation is the distance from the origin of the coordinate system of the photographing device to the origin of the coordinate system of the observed object along the X, Y, and Z axes.
  • 3D translation can be represented as a 3D vector
  • the three-dimensional rotation can be regarded as the angle of rotation from the coordinate system of the shooting device to the coordinate system of the observed object around the X, Y, and Z axes, which can be represented by a 3 ⁇ 3 matrix, that is, R ⁇ SO(3 ).
  • Step 300 Preprocess the three-dimensional model and the six-dimensional attitude parameters, and convert them into a unified file format for importing into the data set automatic generation platform;
  • the file format of the 3D model includes stl, ply, obj, etc. Since the various information modes of the 3D model stored in different file formats are different, it is inconvenient to process the data set automatic generation platform.
  • a 3D model in stl format is a common format for 3D printing.
  • the file in this format uses triangular patches one by one to form a triangular mesh to represent a 3D object, and saves the 3D model in the form of a text file or binary file in turn.
  • the 3D model in ply format depicts 3D objects through a collection of polygonal patches, and saves the x, y, z coordinates, normal coordinates, color, transparency, etc. of each vertex in the form of text files or binary files, and The number of vertices per polygon and the number of each vertex.
  • main information such as the x, y, z coordinates, normal coordinates and color information of the patch vertices in different format files in the order of one patch. Therefore, in the embodiment of the present application, by converting the three-dimensional models and six-dimensional attitude parameters of different formats into a unified file format, it is convenient for the three-dimensional models of different formats to be imported into the virtual environment and the subsequent attitude change control.
  • Step 400 import the preprocessed three-dimensional model and six-dimensional attitude parameters into the data set automatic generation platform, and set the three-dimensional model and virtual camera in the OpenGL virtual environment;
  • step 400 is a working principle diagram of the data set automatic generation platform according to the embodiment of the present application.
  • a large three-dimensional coordinate system in the OpenGL virtual environment that is, the world coordinate system, through which the virtual camera is set. Specifically include:
  • Step 500 Control the three-dimensional model to cyclically change the posture in the OpenGL virtual environment according to each six-dimensional posture parameter, obtain the three-dimensional model under each six-dimensional posture, and collect the form of the three-dimensional model under each six-dimensional posture through a virtual camera, and generate each six-dimensional posture.
  • the shape of the three-dimensional model in each six-dimensional pose is the coordinates (x, y, z) of each vertex in the initial shape of the three-dimensional model multiplied by the six-dimensional pose [R
  • RGB color image and depth image generation include:
  • the object between the near section and the far section is orthographically projected onto the near section, and stored in the color buffer as a binary integer number;
  • the floating-point values read from the depth buffer are arranged according to the depth value of each pixel from left to right and bottom to top in a picture, the floating-point values are assigned in the same order as they are sorted.
  • Step 600 Save the RGB color image and the depth image collected by the virtual camera, and use the corresponding six-dimensional pose [R
  • FIG. 5 is a schematic structural diagram of a system for automatically generating a six-dimensional pose data set according to an embodiment of the present application.
  • the six-dimensional pose data set automatic generation system of the embodiment of the present application includes:
  • Platform building module used to build a data set automatic generation platform based on OpenGL virtual environment
  • Model and parameter acquisition module used to acquire the three-dimensional model of the object and the six-dimensional attitude parameters of the three-dimensional model relative to the virtual camera;
  • t] is the three-dimensional translation and three-dimensional rotation relative to the photographing device.
  • Three-dimensional translation is the distance from the origin of the coordinate system of the photographing device to the origin of the coordinate system of the observed object along the X, Y, and Z axes.
  • 3D translation can be represented as a 3D vector
  • the three-dimensional rotation can be regarded as the angle of rotation from the coordinate system of the shooting device to the coordinate system of the observed object around the X, Y, and Z axes, which can be represented by a 3 ⁇ 3 matrix, that is, R ⁇ SO(3 ).
  • Data preprocessing module It is used to preprocess the 3D model and 6D pose parameters, and convert them into a unified file format for importing into the data set automatic generation platform; among them, the file formats of the 3D model include stl, ply, obj, etc.
  • the various information modes of the 3D models stored in different file formats are different, which is not convenient for the processing of the data set automatic generation platform.
  • a 3D model in stl format is a common format for 3D printing.
  • the file in this format uses triangular patches one by one to form a triangular mesh to represent a 3D object, and saves the 3D model in the form of a text file or binary file in turn.
  • the 3D model in ply format depicts 3D objects through a collection of polygonal patches, and saves the x, y, z coordinates, normal coordinates, color, transparency, etc. of each vertex in the form of text files or binary files, and The number of vertices per polygon and the number of each vertex.
  • main information such as the x, y, z coordinates, normal coordinates and color information of the patch vertices in different format files in the order of one patch. Therefore, in the embodiment of the present application, by converting the three-dimensional models and six-dimensional attitude parameters of different formats into a unified file format, it is convenient for the three-dimensional models of different formats to be imported into the virtual environment and the subsequent attitude change control.
  • Data import module It is used to import the preprocessed 3D model and 6D pose parameters into the data set automatic generation platform, and set the 3D model and virtual camera in the OpenGL virtual environment; among them, in the OpenGL virtual environment, there is a large
  • the three-dimensional coordinate system that is, the world coordinate system, through which the virtual camera is set. Specifically include:
  • Image acquisition module It is used to control the three-dimensional model to cyclically change the attitude in the OpenGL virtual environment according to each six-dimensional attitude parameter, obtain the three-dimensional model under each six-dimensional attitude, and collect the shape of the three-dimensional model under each six-dimensional attitude through the virtual camera, Generate the RGB color image and the corresponding depth image of the object under each six-dimensional pose; wherein, the shape of the three-dimensional model under each six-dimensional pose is the coordinate (x, y, z) of each vertex in the initial shape of the three-dimensional model multiplied by The six-dimensional pose [R
  • RGB color image and depth image generation include:
  • the object between the near section and the far section is orthographically projected onto the near section, and stored in the color buffer as a binary integer number;
  • the floating-point values read from the depth buffer are arranged according to the depth value of each pixel from left to right and bottom to top in a picture, the floating-point values are assigned in the same order as they are sorted.
  • Image saving module used to save the RGB color image and depth image collected by the virtual camera, and use the corresponding six-dimensional pose [R
  • the front view and the left view of the three-dimensional point cloud; (e) is the depth image obtained by importing the stl three-dimensional model of the real object into the automatic generation platform of the embodiment of the application, and then converting it into the front view and the left view of the three-dimensional point cloud, which can be It can be seen that the three-dimensional shape presented by the real three-dimensional point cloud in (d) is basically the same, but since the depth image only obtains the depth value of the front surface of the object (the part of the surface of the object facing the virtual camera), the converted point cloud is only the object. The front surface is a little cloudy.
  • Figure 7 is the experimental result of the 3D model in ply format, in which (a) is a real object, (b) is a ply 3D model captured by a virtual camera, and (c) is a certain pose performed by the 3D model in (b)
  • the RGB color image obtained by the transformation is the front view and left view of the real 3D point cloud of the ply 3D model of the object; (e) is that the ply 3D model of the object is imported into the automatic generation platform of the embodiment of the application to obtain , and then transform it into the front and left views of the 3D point cloud.
  • the experimental results show that the embodiments of the present application can accurately obtain RGB color images and depth images in different six-dimensional poses for three-dimensional models in stl and ply formats.
  • FIG. 8 is a schematic structural diagram of a terminal according to an embodiment of the present application.
  • the terminal 50 includes a processor 51 and a memory 52 coupled to the processor 51 .
  • the memory 52 stores program instructions for implementing the above-mentioned method for automatically generating a six-dimensional pose data set.
  • the processor 51 is configured to execute program instructions stored in the memory 52 to control the automatic generation of the six-dimensional pose data set.
  • the processor 51 may also be referred to as a CPU (Central Processing Unit, central processing unit).
  • the processor 51 may be an integrated circuit chip with signal processing capability.
  • the processor 51 may also be a general purpose processor, digital signal processor (DSP), application specific integrated circuit (ASIC), off-the-shelf programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic device, discrete hardware component .
  • DSP digital signal processor
  • ASIC application specific integrated circuit
  • FPGA off-the-shelf programmable gate array
  • a general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
  • FIG. 9 is a schematic structural diagram of a storage medium according to an embodiment of the present application.
  • the storage medium of this embodiment of the present application stores a program file 61 capable of implementing all the above methods, wherein the program file 61 may be stored in the above-mentioned storage medium in the form of a software product, and includes several instructions to make a computer device (which may It is a personal computer, a server, or a network device, etc.) or a processor that executes all or part of the steps of the methods of the various embodiments of the present invention.
  • a computer device which may It is a personal computer, a server, or a network device, etc.
  • a processor that executes all or part of the steps of the methods of the various embodiments of the present invention.
  • the aforementioned storage medium includes: U disk, mobile hard disk, Read-Only Memory (ROM, Read-Only Memory), Random Access Memory (RAM, Random Access Memory), magnetic disk or optical disk and other media that can store program codes , or terminal devices such as computers, servers, mobile phones, and tablets.
  • the method, system, terminal, and storage medium for automatically generating a six-dimensional attitude dataset generate a platform automatically by importing the three-dimensional model and six-dimensional attitude parameters of an object into a dataset based on an OpenGL virtual environment.
  • the 3D model is controlled to continuously change its posture in the virtual environment, and the RGB color image and the corresponding depth image of the object under each 6D posture are automatically captured by the virtual camera, and the 6D posture label is automatically marked.
  • the embodiment of the present application can efficiently generate RGB color images and depth images and six-dimensional attitude labels in different six-dimensional attitudes of the three-dimensional model through simple operations, which is less time-consuming and saves manpower and money. cost consumption.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Graphics (AREA)
  • Software Systems (AREA)
  • Geometry (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Architecture (AREA)
  • Computer Hardware Design (AREA)
  • General Engineering & Computer Science (AREA)
  • Processing Or Creating Images (AREA)
  • Image Analysis (AREA)
  • Image Processing (AREA)

Abstract

The present application relates to a method and system for automatically generating a six-dimensional posture data set, and a terminal and a storage medium. The method comprises: constructing an automatic data set generation platform based on an OpenGL virtual environment; acquiring a three-dimensional model of an object and six-dimensional posture parameters of the three-dimensional model; importing the three-dimensional model and the six-dimensional posture parameters into the automatic data set generation platform; the automatic data set generation platform controlling, according to the six-dimensional posture parameters, the three-dimensional model to change a posture in the OpenGL virtual environment, and collecting the shape of the three-dimensional model under each six-dimensional posture by means of a virtual camera to generate an RGB color image and a corresponding depth image of the object under each six-dimensional posture; and marking six-dimensional posture labels of the images. In the present application, by means of simple operations, RGB color images and depth images of an object under different six-dimensional postures, and six-dimensional posture labels are efficiently generated, such that little time is consumed, and the cost consumption of labor and money is reduced.

Description

六维姿态数据集自动生成方法、***、终端以及存储介质Method, system, terminal and storage medium for automatic generation of six-dimensional pose data set 技术领域technical field
本申请属于六维姿态估计技术领域，特别涉及一种六维姿态数据集自动生成方法、***、终端以及存储介质。The present application belongs to the technical field of six-dimensional attitude estimation, and in particular, relates to a method, system, terminal and storage medium for automatically generating a six-dimensional attitude data set.
背景技术Background technique
在计算机视觉中的六维姿态估计的深度学习任务中，需要有大量RGB彩色图像、深度图像及对应的六维姿态标签的数据集来进行训练。传统方法中，获取数据集的方式为：通过人工手持相机、手机等拍摄设备围着物体从不同的角度进行拍摄，然后计算出每张照片中被拍摄物体相对于拍摄设备的六维姿态标签，以获取到用于训练的数据集。具体如图1所示，为传统方法中获取六维姿态数据集的流程图。然而，由于深度学习的数据集通常需要大量的图像以及精准的标签，传统的数据集获取方法需要花费大量的时间、人力、金钱等，操作繁琐且计算复杂度高。In the deep learning task of 6D pose estimation in computer vision, a large number of datasets of RGB color images, depth images and corresponding 6D pose labels are required for training. In the traditional method, the way to obtain the data set is to take pictures from different angles around the object by hand-held camera, mobile phone and other shooting equipment, and then calculate the six-dimensional pose label of the object to be photographed in each photo relative to the shooting device. to obtain a dataset for training. Specifically, as shown in FIG. 1 , it is a flow chart of obtaining a six-dimensional pose data set in a traditional method. However, since deep learning datasets usually require a large number of images and accurate labels, traditional dataset acquisition methods require a lot of time, manpower, money, etc., which are cumbersome and computationally complex.
发明内容SUMMARY OF THE INVENTION
本申请提供了一种六维姿态数据集自动生成方法、***、终端以及存储介质，旨在解决现有数据集获取方法存在的需要花费大量的时间、人力、金钱，操作繁琐且计算复杂度高的技术问题。The present application provides a method, system, terminal and storage medium for automatically generating a six-dimensional pose data set, aiming to solve the problems of existing data set acquisition methods that require a lot of time, manpower, money, tedious operations and high computational complexity technical issues.
为了解决上述问题，本申请提供了如下技术方案：In order to solve the above problems, the application provides the following technical solutions:
一种六维姿态数据集自动生成方法，包括以下步骤：A method for automatically generating a six-dimensional pose dataset, comprising the following steps:
步骤a：构建基于OpenGL虚拟环境的数据集自动生成平台；Step a: build a data set automatic generation platform based on OpenGL virtual environment;
步骤b：获取物体的三维模型以及所述三维模型的六维姿态参数，并将所述三维模型以及六维姿态参数导入所述数据集自动生成平台；Step b: obtaining the three-dimensional model of the object and the six-dimensional attitude parameters of the three-dimensional model, and importing the three-dimensional model and the six-dimensional attitude parameters into the data set automatic generation platform;
步骤c：所述数据集自动生成平台根据所述六维姿态参数控制所述三维模型在OpenGL虚拟环境中变动姿态，并通过所述OpenGL虚拟环境中的虚拟相机采集所述三维模型在各个六维姿态下的形态，生成各个六维姿态下所述物体的RGB彩色图像和对应的深度图像，并标注所述图像的六维姿态标签。Step c: The data set automatic generation platform controls the three-dimensional model to change the posture in the OpenGL virtual environment according to the six-dimensional attitude parameters, and collects the three-dimensional model in each six-dimensional environment through the virtual camera in the OpenGL virtual environment. The RGB color image and the corresponding depth image of the object in each six-dimensional attitude are generated, and the six-dimensional attitude label of the image is marked.
本申请实施例采取的技术方案还包括：所述步骤a中，所述获取物体的三维模型以及所述三维模型的六维姿态参数还包括：The technical solutions adopted in the embodiments of the present application further include: in the step a, the acquiring the three-dimensional model of the object and the six-dimensional attitude parameters of the three-dimensional model further include:
将所述三维模型和六维姿态参数转换成导入所述数据集自动生成平台的统一文件格式。The three-dimensional model and six-dimensional pose parameters are converted into a unified file format imported into the data set automatic generation platform.
本申请实施例采取的技术方案还包括：在所述步骤a中，所述将所述三维模型以及六维姿态参数导入所述数据集自动生成平台还包括：The technical solution adopted in the embodiment of the present application further includes: in the step a, the importing the three-dimensional model and the six-dimensional pose parameters into the data set automatic generation platform further includes:
设置所述三维模型在所述OpenGL虚拟环境中的世界坐标系中的初始位置，并根据每个面片顶点的x，y，z坐标、法线坐标和颜色将所述三维模型的初始形态绘制在所述世界坐标系中；Set the initial position of the 3D model in the world coordinate system in the OpenGL virtual environment, and draw the initial shape of the 3D model according to the x, y, z coordinates, normal coordinates and color of each patch vertex in said world coordinate system;
设置所述虚拟相机在所述OpenGL虚拟环境中的世界坐标系中的位置、朝向、法线、视角、拍摄图片的长宽比、近截面以及远截面。Set the position, orientation, normal, angle of view, aspect ratio, near section and far section of the virtual camera in the world coordinate system in the OpenGL virtual environment.
本申请实施例采取的技术方案还包括：在所述步骤c中，所述三维模型在各个六维姿态下的形态为所述三维模型初始形态下各顶点的坐标(x,y,z)乘上六维姿态[R|t]，即
Figure PCTCN2020139761-appb-000001
The technical solution adopted in the embodiment of the present application further includes: in the step c, the shape of the three-dimensional model in each six-dimensional pose is the multiplication of the coordinates (x, y, z) of each vertex in the initial shape of the three-dimensional model The upper six-dimensional pose [R|t], namely
Figure PCTCN2020139761-appb-000001
本申请实施例采取的技术方案还包括：在所述步骤c中，所述生成各个六维姿态下所述物体的RGB彩色图像包括：The technical solution adopted in the embodiment of the present application further includes: in the step c, the generating an RGB color image of the object under each six-dimensional pose includes:
利用所述虚拟相机的成像原理，将所述近截面与远截面之间的物体正射投影到所述近截面上，并以二进制整型数字存储到颜色缓存中；Using the imaging principle of the virtual camera, the object between the near section and the far section is orthographically projected onto the near section, and stored in the color buffer as a binary integer number;
获取所述颜色缓存中的二进制整型数字，并将所述二进制整型数字按顺序分配为每个像素上的R、G、B值，生成所述RGB彩色图像。Acquire the binary integer number in the color buffer, and assign the binary integer number as R, G, and B values on each pixel in sequence to generate the RGB color image.
本申请实施例采取的技术方案还包括：在所述步骤c中，所述生成各个六维姿态下所述物体的深度图像包括：The technical solution adopted in the embodiment of the present application further includes: in the step c, the generating a depth image of the object under each six-dimensional pose includes:
对所述近截面到远截面之间分配深度值，得到所述物体前表面的深度值，并将所述深度值以浮点型数值存储到深度缓存中；Allocate a depth value between the near section and the far section, obtain the depth value of the front surface of the object, and store the depth value in the depth buffer as a floating-point value;
获取所述深度缓存中的浮点型数值，对所述浮点型数值进行计算，得到所述三维模型的前表面距所述虚拟相机的真实距离，并将所述浮点型数值按顺序分配到每个像素上，得到对应的深度图像。Obtain the floating-point value in the depth buffer, calculate the floating-point value, obtain the real distance between the front surface of the 3D model and the virtual camera, and assign the floating-point value in order To each pixel, the corresponding depth image is obtained.
本申请实施例采取的技术方案还包括：在所述步骤c中，所述生成各个六维姿态下所述物体的RGB彩色图像和对应的深度图像，并标注所述图像的六维姿态标签还包括：The technical solution adopted in the embodiment of the present application further includes: in the step c, generating an RGB color image and a corresponding depth image of the object under each six-dimensional posture, and labeling the six-dimensional posture label of the image and further include:
保存所述虚拟相机采集的RGB彩色图像和深度图像，并将采集图像时对应的六维姿态作为所述RGB彩色图像和深度图像的六维姿态标签。The RGB color image and the depth image collected by the virtual camera are saved, and the corresponding six-dimensional attitude when the image is collected is used as the six-dimensional attitude label of the RGB color image and the depth image.
本申请实施例采取的另一技术方案为：一种六维姿态数据集自动生成***，包括：Another technical solution adopted by the embodiments of the present application is: a system for automatically generating a six-dimensional attitude data set, comprising:
平台构建模块：用于构建基于OpenGL虚拟环境的数据集自动生成平台；Platform building module: used to build a data set automatic generation platform based on OpenGL virtual environment;
模型及参数获取模块：用于获取物体的三维模型以及所述三维模型的六维姿态参数；Model and parameter acquisition module: used to acquire the three-dimensional model of the object and the six-dimensional attitude parameters of the three-dimensional model;
数据导入模块：用于将所述三维模型以及六维姿态参数导入所述数据集自动生成平台；Data import module: used to import the three-dimensional model and six-dimensional attitude parameters into the data set automatic generation platform;
图像采集模块：用于根据所述六维姿态参数控制所述三维模型在OpenGL虚拟环境中变动姿态，并通过所述OpenGL虚拟环境中的虚拟相机采集所述三维模型在各个六维姿态下的形态，生成各个六维姿态下所述物体的RGB彩色图像和对应的深度图像，并标注所述图像的六维姿态标签。Image acquisition module: used to control the posture of the three-dimensional model to change in the OpenGL virtual environment according to the six-dimensional posture parameters, and collect the form of the three-dimensional model in each six-dimensional posture through the virtual camera in the OpenGL virtual environment , generate an RGB color image and a corresponding depth image of the object under each six-dimensional pose, and mark the six-dimensional pose label of the image.
本申请实施例采取的又一技术方案为：一种终端，所述终端包括处理器、与所述处理器耦接的存储器，其中，Another technical solution adopted by the embodiments of the present application is: a terminal, the terminal includes a processor and a memory coupled to the processor, wherein,
所述存储器存储有用于实现所述六维姿态数据集自动生成方法的程序指令；The memory stores program instructions for realizing the method for automatically generating the six-dimensional attitude data set;
所述处理器用于执行所述存储器存储的所述程序指令以控制六维姿态数据集自动生成。The processor is configured to execute the program instructions stored in the memory to control the automatic generation of a six-dimensional pose data set.
本申请实施例采取的又一技术方案为：一种存储介质，存储有处理器可运行的程序指令，所述程序指令用于执行所述六维姿态数据集自动生成方法。Another technical solution adopted by the embodiments of the present application is: a storage medium storing program instructions executable by a processor, where the program instructions are used to execute the method for automatically generating a six-dimensional attitude data set.
相对于现有技术，本申请实施例产生的有益效果在于：本申请实施例的六维姿态数据集自动生成方法、***、终端以及存储介质通过将物体的三维 模型和六维姿态参数导入基于OpenGL虚拟环境的数据集自动生成平台，该平台根据六维姿态参数控制三维模型在虚拟环境中不断变动姿态，并由虚拟相机自动拍摄每个六维姿态下该物体的RGB彩色图像和对应的深度图像，并自动标注六维姿态标签。相对于现有技术，本申请实施例通过简单的操作以高效的生成三维模型不同的六维姿态下的RGB彩色图像和深度图像以及六维姿态标签，耗时低，并节约了人力和金钱的成本消耗。Compared with the prior art, the beneficial effects of the embodiments of the present application are: the method, system, terminal and storage medium for automatically generating a six-dimensional attitude data set according to the embodiments of the present application are based on OpenGL by importing the three-dimensional model and six-dimensional attitude parameters of the object into The data set of the virtual environment automatically generates a platform, which controls the three-dimensional model to continuously change the posture in the virtual environment according to the six-dimensional posture parameters, and the RGB color image and the corresponding depth image of the object under each six-dimensional posture are automatically captured by the virtual camera. , and automatically annotate 6D pose labels. Compared with the prior art, the embodiment of the present application can efficiently generate RGB color images and depth images and six-dimensional attitude labels in different six-dimensional attitudes of the three-dimensional model through simple operations, which is less time-consuming and saves manpower and money. cost consumption.
附图说明Description of drawings
图1为传统方法中获取六维姿态数据集的流程图；Fig. 1 is the flow chart of obtaining the six-dimensional attitude data set in the traditional method;
图2是本申请实施例的六维姿态数据集自动生成方法的流程图；2 is a flowchart of a method for automatically generating a six-dimensional attitude dataset according to an embodiment of the present application;
图3是本申请实施例的数据集自动生成平台的工作原理图；Fig. 3 is the working principle diagram of the data set automatic generation platform of the embodiment of the present application;
图4是本申请实施例的虚拟相机设置示意图；4 is a schematic diagram of a virtual camera setting according to an embodiment of the present application;
图5为本申请实施例的六维姿态数据集自动生成***结构示意图；5 is a schematic structural diagram of a system for automatically generating a six-dimensional attitude data set according to an embodiment of the present application;
图6为本申请实施例的stl格式的三维模型的实验结果示意图；其中，(a)为真实物体；(b)是将真实物体的stl三维模型导入本申请实施例的自动生成平台得到的RGB彩色图像，(c)是由(b)中的三维模型进行一定的姿态变换所得到的RGB彩色图像；(d)是该物体的stl三维模型的真实三维点云的正视图和左视图；(e)是真实物体的stl三维模型导入本申请实施例的自动生成平台得到的深度图像，再将其转化成三维点云的正视图和左视图；6 is a schematic diagram of the experimental results of the three-dimensional model of the stl format according to the embodiment of the application; wherein, (a) is a real object; (b) is the RGB obtained by importing the stl three-dimensional model of the real object into the automatic generation platform of the embodiment of the application Color image, (c) is the RGB color image obtained by performing certain pose transformation on the 3D model in (b); (d) is the front view and left view of the real 3D point cloud of the stl 3D model of the object; ( e) is the depth image obtained by importing the stl three-dimensional model of the real object into the automatic generation platform of the embodiment of the application, and then converting it into a front view and a left view of a three-dimensional point cloud;
图7为本申请实施例的ply格式的三维模型的实验结果示意图；其中，(a)为真实物体，(b)为虚拟相机拍摄的ply三维模型，(c)是由(b)中的三维模型进行一定的姿态变换所得到的RGB彩色图像；(d)是该物体的ply三 维模型的真实三维点云的正视图和左视图；(e)是该物体的ply三维模型导入本申请实施例的自动生成平台得到的深度图像，再将其转化成三维点云的正视图和左视图；7 is a schematic diagram of an experimental result of a three-dimensional model in ply format according to an embodiment of the application; wherein (a) is a real object, (b) is a ply three-dimensional model captured by a virtual camera, and (c) is a three-dimensional model in (b) The RGB color image obtained by the model performing a certain attitude transformation; (d) is the front view and left view of the real 3D point cloud of the ply 3D model of the object; (e) is the ply 3D model of the object imported into the embodiment of the application The depth image obtained by the automatic generation platform is converted into the front view and left view of the 3D point cloud;
图8为本申请实施例的终端结构示意图；FIG. 8 is a schematic structural diagram of a terminal according to an embodiment of the present application;
图9为本申请实施例的存储介质的结构示意图。FIG. 9 is a schematic structural diagram of a storage medium according to an embodiment of the present application.
具体实施方式detailed description
为了使本申请的目的、技术方案及优点更加清楚明白，以下结合附图及实施例，对本申请进行进一步详细说明。应当理解，此处所描述的具体实施例仅用以解释本申请，并不用于限定本申请。In order to make the purpose, technical solutions and advantages of the present application more clearly understood, the present application will be described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application, but not to limit the present application.
为了解决现有技术的不足，本申请实施例的六维姿态数据集自动生成方法通过构建一个基于OpenGL(Open Graphics Library，开放图形库)虚拟环境的数据集自动生成平台，将物体的三维模型和相对于虚拟相机的六维姿态参数导入该平台，该平台根据六维姿态参数控制三维模型在虚拟环境中不断变动姿态，由虚拟相机自动拍摄每个六维姿态下该物体的RGB彩色图像和对应的深度图像，并自动标注六维姿态标签。In order to solve the deficiencies of the prior art, the method for automatically generating a six-dimensional pose data set according to the embodiment of the present application constructs a data set automatic generation platform based on an OpenGL (Open Graphics Library) virtual environment, and combines the three-dimensional model of the object with the Relative to the six-dimensional attitude parameters of the virtual camera, the platform is imported into the platform, and the platform controls the three-dimensional model to continuously change the attitude in the virtual environment according to the six-dimensional attitude parameters, and the virtual camera automatically captures the RGB color image of the object under each six-dimensional attitude and the corresponding The depth image is automatically annotated with 6D pose labels.
具体的，请参阅图2，是本申请实施例的六维姿态数据集自动生成方法的流程图。本申请实施例的六维姿态数据集自动生成方法包括以下步骤：Specifically, please refer to FIG. 2 , which is a flowchart of a method for automatically generating a six-dimensional pose data set according to an embodiment of the present application. The method for automatically generating a six-dimensional pose data set according to the embodiment of the present application includes the following steps:
步骤100：构建基于OpenGL虚拟环境的数据集自动生成平台；Step 100: build a data set automatic generation platform based on the OpenGL virtual environment;
步骤200：获取物体的三维模型以及该三维模型相对于虚拟相机的六维姿态参数；Step 200: obtaining a three-dimensional model of the object and a six-dimensional attitude parameter of the three-dimensional model relative to the virtual camera;
步骤200中，六维姿态[R|t]是相对于拍摄设备的三维平移和三维旋转。三维平移是从拍摄设备坐标系原点开始沿着X、Y、Z轴移动到所观察物体的坐标系原点之间的距离。三维平移可以表示为一个三维的向量
Figure PCTCN2020139761-appb-000002
而三维旋转可以看作是从拍摄设备坐标系到所观察物体的坐标系分别绕着X、Y、Z轴所旋转的角度，可以用一个3×3的矩阵来表示，即R∈SO(3)。假设该三维模型绕其原始的X、Y、Z轴所旋转的角度分别为γ、β、α，而沿其原始的X、Y、Z轴平移的距离为a、b、c，则该三维模型的六维姿态[R|t]可以表示为： In step 200, the six-dimensional pose [R|t] is three-dimensional translation and three-dimensional rotation relative to the photographing device. Three-dimensional translation is the distance from the origin of the coordinate system of the photographing device to the origin of the coordinate system of the observed object along the X, Y, and Z axes. 3D translation can be represented as a 3D vector
Figure PCTCN2020139761-appb-000002
The three-dimensional rotation can be regarded as the angle of rotation from the coordinate system of the shooting device to the coordinate system of the observed object around the X, Y, and Z axes, which can be represented by a 3×3 matrix, that is, R∈SO(3 ). Assuming that the rotation angles of the three-dimensional model around its original X, Y, and Z axes are γ, β, and α, respectively, and the translation distances along its original X, Y, and Z axes are a, b, and c, then the three-dimensional model The six-dimensional pose [R|t] of the model can be expressed as:
Figure PCTCN2020139761-appb-000003
Figure PCTCN2020139761-appb-000003
通过设置γ、β、α、a、b、c的取值范围以及取值间隔，即可得到一系列的六维姿态参数值。By setting the value ranges and value intervals of γ, β, α, a, b, and c, a series of six-dimensional attitude parameter values can be obtained.
步骤300：对三维模型和六维姿态参数进行预处理，将其转换成为导入数据集自动生成平台的统一文件格式；Step 300: Preprocess the three-dimensional model and the six-dimensional attitude parameters, and convert them into a unified file format for importing into the data set automatic generation platform;
步骤300中，三维模型的文件格式包括stl、ply、obj等，由于不同文件格式所存储的三维模型的各种信息模式都不一样，不便于数据集自动生成平台的处理。例如，stl格式的三维模型是三维打印的通用格式，该格式的文件用一个一个的三角面片组合成三角网格来表现三维物体，以文本文件或者二进制文件的形式依次保存构成该三维模型的每一个三角面片的法线坐标和其三个顶点的x，y，z坐标。而ply格式的三维模型，通过多边形面片的集合来描绘三维物体，并以文本文件或者二进制文件的形式保存了每个顶点的x，y，z坐标、法线坐标、颜色、透明度等，以及每个多边形的顶点数和每个顶点 的编号。为了在虚拟环境中绘制出三维模型，需要按照一个一个面片的顺序提取出不同格式文件中面片顶点的x，y，z坐标，法线坐标以及颜色信息等等主要信息。因此本申请实施例通过将不同格式的三维模型和六维姿态参数转换为统一的文件格式，便于不同格式的三维模型导入虚拟环境以及后续的姿态变动控制。In step 300, the file format of the 3D model includes stl, ply, obj, etc. Since the various information modes of the 3D model stored in different file formats are different, it is inconvenient to process the data set automatic generation platform. For example, a 3D model in stl format is a common format for 3D printing. The file in this format uses triangular patches one by one to form a triangular mesh to represent a 3D object, and saves the 3D model in the form of a text file or binary file in turn. The normal coordinates of each triangular patch and the x, y, z coordinates of its three vertices. The 3D model in ply format depicts 3D objects through a collection of polygonal patches, and saves the x, y, z coordinates, normal coordinates, color, transparency, etc. of each vertex in the form of text files or binary files, and The number of vertices per polygon and the number of each vertex. In order to draw a 3D model in a virtual environment, it is necessary to extract the main information such as the x, y, z coordinates, normal coordinates and color information of the patch vertices in different format files in the order of one patch. Therefore, in the embodiment of the present application, by converting the three-dimensional models and six-dimensional attitude parameters of different formats into a unified file format, it is convenient for the three-dimensional models of different formats to be imported into the virtual environment and the subsequent attitude change control.
步骤400：将预处理后的三维模型和六维姿态参数导入数据集自动生成平台，对OpenGL虚拟环境中的三维模型和虚拟相机进行设置；Step 400: import the preprocessed three-dimensional model and six-dimensional attitude parameters into the data set automatic generation platform, and set the three-dimensional model and virtual camera in the OpenGL virtual environment;
步骤400中，如图3所示，是本申请实施例的数据集自动生成平台的工作原理图。在OpenGL虚拟环境中有一个大的三维坐标系，即世界坐标系，通过该坐标系进行虚拟相机的设置。具体包括：In step 400, as shown in FIG. 3, it is a working principle diagram of the data set automatic generation platform according to the embodiment of the present application. There is a large three-dimensional coordinate system in the OpenGL virtual environment, that is, the world coordinate system, through which the virtual camera is set. Specifically include:
1、设置三维模型在该坐标系中的初始位置(通常初始位置为该坐标系的原点处)，并根据每个面片顶点的x，y，z坐标、法线坐标和颜色等将三维模型的初始形态绘制在该坐标系中。1. Set the initial position of the 3D model in the coordinate system (usually the initial position is the origin of the coordinate system), and convert the 3D model according to the x, y, z coordinates, normal coordinates and color of each patch vertex. The initial form of is drawn in this coordinate system.
2、设置虚拟相机在该坐标系中的位置(通常设置为该坐标系的原点处)、朝向、法线、视角、拍摄图片的长宽比以及近截面、远截面等，具体如图4所示。2. Set the position of the virtual camera in the coordinate system (usually set to the origin of the coordinate system), orientation, normal, angle of view, aspect ratio of the captured picture, near section, far section, etc., as shown in Figure 4. Show.
步骤500：根据各个六维姿态参数控制三维模型在OpenGL虚拟环境中循环变动姿态，得到各个六维姿态下的三维模型，并通过虚拟相机采集三维模型在各个六维姿态下的形态，生成各个六维姿态下物体的RGB彩色图像和对应的深度图像；Step 500: Control the three-dimensional model to cyclically change the posture in the OpenGL virtual environment according to each six-dimensional posture parameter, obtain the three-dimensional model under each six-dimensional posture, and collect the form of the three-dimensional model under each six-dimensional posture through a virtual camera, and generate each six-dimensional posture. The RGB color image and the corresponding depth image of the object under the 3D pose;
步骤500中，三维模型在各个六维姿态下的形态即为该三维模型初始形态下各顶点的坐标(x,y,z)乘上六维姿态[R|t]，即
Figure PCTCN2020139761-appb-000004
In step 500, the shape of the three-dimensional model in each six-dimensional pose is the coordinates (x, y, z) of each vertex in the initial shape of the three-dimensional model multiplied by the six-dimensional pose [R|t], that is,
Figure PCTCN2020139761-appb-000004
RGB彩色图像和深度图像生成原理具体包括：The principles of RGB color image and depth image generation include:
1、得到不同姿态下的三维模型后，利用虚拟相机的成像原理，将近截面与远截面之间的物体正射投影到近截面上，并以二进制整型数字存储到颜色缓存中；1. After obtaining the 3D models in different poses, using the imaging principle of the virtual camera, the object between the near section and the far section is orthographically projected onto the near section, and stored in the color buffer as a binary integer number;
2、获取颜色缓存中的二进制整型数字，并将二进制整型数字按顺序分配为每个像素上的R、G、B值，组成RGB彩色图像；其中，由于从颜色缓存中读取的二进制整型数字是按照一张图片中从左到右、从下到上的每个像素点的R，G，B值进行排列的，因此，二进制整型数字的分配顺序与其排列顺序相同。2. Obtain the binary integer numbers in the color buffer, and assign the binary integer numbers to the R, G, and B values on each pixel in order to form an RGB color image; Integer numbers are arranged according to the R, G, B value of each pixel in a picture from left to right and bottom to top, so binary integer numbers are assigned in the same order as they are arranged.
3、对近截面到远截面之间分配深度值，其中近截面深度值为0，远截面深度值为1，从而得到物体前表面(物体朝向虚拟相机的面为前表面，背向虚拟相机的面为后表面)的深度值，并将深度值以浮点型数值存储到深度缓存中；3. Assign a depth value between the near section and the far section, where the depth value of the near section is 0, and the depth value of the far section is 1, so as to obtain the front surface of the object (the surface of the object facing the virtual camera is the front surface, and the surface facing away from the virtual camera is the front surface. The depth value of the surface is the back surface), and the depth value is stored in the depth buffer as a floating-point value;
4、获取深度缓存中的浮点型数值，对该浮点型数值进行计算，得到三维模型的前表面距所述虚拟相机的真实距离，并将浮点型数值按顺序分配到每个像素上，得到对应的深度图像；其中，真实距离计算公式为：4. Obtain the floating-point value in the depth buffer, calculate the floating-point value, obtain the real distance between the front surface of the 3D model and the virtual camera, and assign the floating-point value to each pixel in sequence , the corresponding depth image is obtained; among them, the formula for calculating the true distance is:
Figure PCTCN2020139761-appb-000005
Figure PCTCN2020139761-appb-000005
上述中，其中，由于从深度缓存中读取的浮点型数值是按照一张图片中从左到右、从下到上的每个像素点的深度值进行排列的，因此，浮点型数值的分配顺序与其排列顺序相同。In the above, since the floating-point values read from the depth buffer are arranged according to the depth value of each pixel from left to right and bottom to top in a picture, the floating-point values are assigned in the same order as they are sorted.
步骤600：保存虚拟相机采集的RGB彩色图像和深度图像，并将采集图像时对应的六维姿态[R|t]作为RGB彩色图像和深度图像的六维姿态标签。Step 600: Save the RGB color image and the depth image collected by the virtual camera, and use the corresponding six-dimensional pose [R|t] when the image is collected as the six-dimensional pose label of the RGB color image and the depth image.
请参阅图5，是本申请实施例的六维姿态数据集自动生成***的结构示意图。本申请实施例的六维姿态数据集自动生成***包括：Please refer to FIG. 5 , which is a schematic structural diagram of a system for automatically generating a six-dimensional pose data set according to an embodiment of the present application. The six-dimensional pose data set automatic generation system of the embodiment of the present application includes:
平台构建模块：用于构建基于OpenGL虚拟环境的数据集自动生成平台；Platform building module: used to build a data set automatic generation platform based on OpenGL virtual environment;
模型及参数获取模块：用于获取物体的三维模型以及该三维模型相对于虚拟相机的六维姿态参数；Model and parameter acquisition module: used to acquire the three-dimensional model of the object and the six-dimensional attitude parameters of the three-dimensional model relative to the virtual camera;
其中，六维姿态[R|t]是相对于拍摄设备的三维平移和三维旋转。三维平移是从拍摄设备坐标系原点开始沿着X、Y、Z轴移动到所观察物体的坐标系原点之间的距离。三维平移可以表示为一个三维的向量
Figure PCTCN2020139761-appb-000006
而三维旋转可以看作是从拍摄设备坐标系到所观察物体的坐标系分别绕着X、Y、Z轴所旋转的角度，可以用一个3×3的矩阵来表示，即R∈SO(3)。假设该三维模型绕其原始的X、Y、Z轴所旋转的角度分别为γ、β、α，而沿其原始的X、Y、Z轴平移的距离为a、b、c，则该三维模型的六维姿态[R|t]可以表示为： Among them, the six-dimensional pose [R|t] is the three-dimensional translation and three-dimensional rotation relative to the photographing device. Three-dimensional translation is the distance from the origin of the coordinate system of the photographing device to the origin of the coordinate system of the observed object along the X, Y, and Z axes. 3D translation can be represented as a 3D vector
Figure PCTCN2020139761-appb-000006
The three-dimensional rotation can be regarded as the angle of rotation from the coordinate system of the shooting device to the coordinate system of the observed object around the X, Y, and Z axes, which can be represented by a 3×3 matrix, that is, R∈SO(3 ). Assuming that the rotation angles of the three-dimensional model around its original X, Y, and Z axes are γ, β, and α, respectively, and the translation distances along its original X, Y, and Z axes are a, b, and c, then the three-dimensional model The six-dimensional pose [R|t] of the model can be expressed as:
Figure PCTCN2020139761-appb-000007
Figure PCTCN2020139761-appb-000007
通过设置γ、β、α、a、b、c的取值范围以及取值间隔，即可得到一系列的六维姿态参数值。By setting the value ranges and value intervals of γ, β, α, a, b, and c, a series of six-dimensional attitude parameter values can be obtained.
数据预处理模块：用于对三维模型和六维姿态参数进行预处理，将其转换成为导入数据集自动生成平台的统一文件格式；其中，三维模型的文件格式包括stl、ply、obj等，由于不同文件格式所存储的三维模型的各种信息模式都不一样，不便于数据集自动生成平台的处理。例如，stl格式的三维模型是三维打印的通用格式，该格式的文件用一个一个的三角面片组合成三角网格来表现三维物体，以文本文件或者二进制文件的形式依次保存构成该三维模型的每一个三角面片的法线坐标和其三个顶点的x，y，z坐标。而ply格式的三维模型，通过多边形面片的集合来描绘三维物体，并以文本文件或者二进制文件的形式保存了每个顶点的x，y，z坐标、法线坐标、颜色、透明度等，以及每个多边形的顶点数和每个顶点的编号。为了在虚拟环境中绘制出三维模型，需要按照一个一个面片的顺序提取出不同格式文件中面片顶点的x，y，z坐标，法线坐标以及颜色信息等等主要信息。因此本申请实施例通过将不同格式的三维模型和六维姿态参数转换为统一的文件格式，便于不同格式的三维模型导入虚拟环境以及后续的姿态变动控制。Data preprocessing module: It is used to preprocess the 3D model and 6D pose parameters, and convert them into a unified file format for importing into the data set automatic generation platform; among them, the file formats of the 3D model include stl, ply, obj, etc. The various information modes of the 3D models stored in different file formats are different, which is not convenient for the processing of the data set automatic generation platform. For example, a 3D model in stl format is a common format for 3D printing. The file in this format uses triangular patches one by one to form a triangular mesh to represent a 3D object, and saves the 3D model in the form of a text file or binary file in turn. The normal coordinates of each triangular patch and the x, y, z coordinates of its three vertices. The 3D model in ply format depicts 3D objects through a collection of polygonal patches, and saves the x, y, z coordinates, normal coordinates, color, transparency, etc. of each vertex in the form of text files or binary files, and The number of vertices per polygon and the number of each vertex. In order to draw a 3D model in a virtual environment, it is necessary to extract the main information such as the x, y, z coordinates, normal coordinates and color information of the patch vertices in different format files in the order of one patch. Therefore, in the embodiment of the present application, by converting the three-dimensional models and six-dimensional attitude parameters of different formats into a unified file format, it is convenient for the three-dimensional models of different formats to be imported into the virtual environment and the subsequent attitude change control.
数据导入模块：用于将预处理后的三维模型和六维姿态参数导入数据集自动生成平台，对OpenGL虚拟环境中的三维模型和虚拟相机进行设置；其中，在OpenGL虚拟环境中有一个大的三维坐标系，即世界坐标系，通过该坐标系进行虚拟相机的设置。具体包括：Data import module: It is used to import the preprocessed 3D model and 6D pose parameters into the data set automatic generation platform, and set the 3D model and virtual camera in the OpenGL virtual environment; among them, in the OpenGL virtual environment, there is a large The three-dimensional coordinate system, that is, the world coordinate system, through which the virtual camera is set. Specifically include:
1、设置三维模型在该坐标系中的初始位置(通常初始位置为该坐标系的原点处)，并根据每个面片顶点的x，y，z坐标、法线坐标和颜色等将三维模型的初始形态绘制在该坐标系中。1. Set the initial position of the 3D model in the coordinate system (usually the initial position is the origin of the coordinate system), and convert the 3D model according to the x, y, z coordinates, normal coordinates and color of each patch vertex. The initial form of is drawn in this coordinate system.
2、设置虚拟相机在该坐标系中的位置(通常设置为该坐标系的原点处)、朝向、法线、视角、拍摄图片的长宽比以及近截面、远截面等，具体如图4所示。2. Set the position of the virtual camera in the coordinate system (usually set to the origin of the coordinate system), orientation, normal, angle of view, aspect ratio of the captured picture, near section, far section, etc., as shown in Figure 4. Show.
图像采集模块：用于根据各个六维姿态参数控制三维模型在OpenGL虚拟环境中循环变动姿态，得到各个六维姿态下的三维模型，并通过虚拟相机采集三维模型在各个六维姿态下的形态，生成各个六维姿态下物体的RGB彩色图像和对应的深度图像；其中，三维模型在各个六维姿态下的形态即为该三维模型初始形态下各顶点的坐标(x,y,z)乘上六维姿态[R|t]，即
Figure PCTCN2020139761-appb-000008
Figure PCTCN2020139761-appb-000009
Image acquisition module: It is used to control the three-dimensional model to cyclically change the attitude in the OpenGL virtual environment according to each six-dimensional attitude parameter, obtain the three-dimensional model under each six-dimensional attitude, and collect the shape of the three-dimensional model under each six-dimensional attitude through the virtual camera, Generate the RGB color image and the corresponding depth image of the object under each six-dimensional pose; wherein, the shape of the three-dimensional model under each six-dimensional pose is the coordinate (x, y, z) of each vertex in the initial shape of the three-dimensional model multiplied by The six-dimensional pose [R|t], namely
Figure PCTCN2020139761-appb-000008
Figure PCTCN2020139761-appb-000009
RGB彩色图像和深度图像生成原理具体包括：The principles of RGB color image and depth image generation include:
1、得到不同姿态下的三维模型后，利用虚拟相机的成像原理，将近截面与远截面之间的物体正射投影到近截面上，并以二进制整型数字存储到颜色缓存中；1. After obtaining the 3D models in different poses, using the imaging principle of the virtual camera, the object between the near section and the far section is orthographically projected onto the near section, and stored in the color buffer as a binary integer number;
2、获取颜色缓存中的二进制整型数字，并将二进制整型数字按顺序分配为每个像素上的R、G、B值，组成RGB彩色图像；其中，由于从颜色缓存中读取的二进制整型数字是按照一张图片中从左到右、从下到上的每个像素点的R，G，B值进行排列的，因此，二进制整型数字的分配顺序与其排列顺序相同。2. Obtain the binary integer numbers in the color buffer, and assign the binary integer numbers to the R, G, and B values on each pixel in order to form an RGB color image; Integer numbers are arranged according to the R, G, B value of each pixel in a picture from left to right and bottom to top, so binary integer numbers are assigned in the same order as they are arranged.
3、对近截面到远截面之间分配深度值，其中近截面深度值为0，远截面深度值为1，从而得到物体前表面(物体朝向虚拟相机的面为前表面，背向虚拟相机的面为后表面)的深度值，并将深度值以浮点型数值存储到深度缓存中；3. Assign a depth value between the near section and the far section, where the depth value of the near section is 0, and the depth value of the far section is 1, so as to obtain the front surface of the object (the surface of the object facing the virtual camera is the front surface, and the surface facing away from the virtual camera is the front surface. The depth value of the surface is the back surface), and the depth value is stored in the depth buffer as a floating-point value;
4、获取深度缓存中的浮点型数值，对该浮点型数值进行计算，得到三维模型的前表面距所述虚拟相机的真实距离，并将浮点型数值按顺序分配到每个像素上，得到对应的深度图像；其中，真实距离计算公式为：4. Obtain the floating-point value in the depth buffer, calculate the floating-point value, obtain the real distance between the front surface of the 3D model and the virtual camera, and assign the floating-point value to each pixel in sequence , the corresponding depth image is obtained; among them, the formula for calculating the true distance is:
Figure PCTCN2020139761-appb-000010
Figure PCTCN2020139761-appb-000010
上述中，其中，由于从深度缓存中读取的浮点型数值是按照一张图片中从左到右、从下到上的每个像素点的深度值进行排列的，因此，浮点型数值的分配顺序与其排列顺序相同。In the above, since the floating-point values read from the depth buffer are arranged according to the depth value of each pixel from left to right and bottom to top in a picture, the floating-point values are assigned in the same order as they are sorted.
图像保存模块：用于保存虚拟相机采集的RGB彩色图像和深度图像，并将采集图像时对应的六维姿态[R|t]作为RGB彩色图像和深度图像的六维姿态标签。Image saving module: used to save the RGB color image and depth image collected by the virtual camera, and use the corresponding six-dimensional pose [R|t] when collecting the image as the six-dimensional pose label of the RGB color image and the depth image.
为了验证本申请实施例的可行性和有效性，以下分别对stl，ply格式的三维模型进行了实验。如图6所示，为stl格式的三维模型的实验结果，(a)为真实物体；(b)是将真实物体的stl三维模型导入本申请实施例的自动生成平台得到的RGB彩色图像，可以看出其和(a)中的真实物体基本一样；(c)是由(b)中的三维模型进行一定的姿态变换所得到的RGB彩色图像；(d)是该物体的stl三维模型的真实三维点云的正视图和左视图；(e)是真实物体的stl三维模型导入本申请实施例的自动生成平台得到的深度图像，再将其转化 成三维点云的正视图和左视图，可以看出与(d)中的真实三维点云所呈现的三维形状基本一致，但由于深度图像只获取物体前表面(物体朝向虚拟相机的那部分表面)的深度值，因而转换的点云只有物体前表面有点云。In order to verify the feasibility and effectiveness of the embodiments of the present application, experiments are carried out on the three-dimensional models in stl and ply formats respectively below. As shown in Figure 6, it is the experimental result of the three-dimensional model of stl format, (a) is the real object; (b) is the RGB color image obtained by importing the stl three-dimensional model of the real object into the automatic generation platform of the embodiment of the present application, which can be It can be seen that it is basically the same as the real object in (a); (c) is the RGB color image obtained by performing a certain attitude transformation of the three-dimensional model in (b); (d) is the real stl three-dimensional model of the object. The front view and the left view of the three-dimensional point cloud; (e) is the depth image obtained by importing the stl three-dimensional model of the real object into the automatic generation platform of the embodiment of the application, and then converting it into the front view and the left view of the three-dimensional point cloud, which can be It can be seen that the three-dimensional shape presented by the real three-dimensional point cloud in (d) is basically the same, but since the depth image only obtains the depth value of the front surface of the object (the part of the surface of the object facing the virtual camera), the converted point cloud is only the object. The front surface is a little cloudy.
图7为ply格式的三维模型的实验结果图，其中，(a)为真实物体，(b)为虚拟相机拍摄的ply三维模型，(c)是由(b)中的三维模型进行一定的姿态变换所得到的RGB彩色图像；(d)是该物体的ply三维模型的真实三维点云的正视图和左视图；(e)是该物体的ply三维模型导入本申请实施例的自动生成平台得到的深度图像，再将其转化成三维点云的正视图和左视图。实验结果表明，本申请实施例对stl，ply格式的三维模型都能准确得到不同六维姿态下的RGB彩色图像和深度图像。Figure 7 is the experimental result of the 3D model in ply format, in which (a) is a real object, (b) is a ply 3D model captured by a virtual camera, and (c) is a certain pose performed by the 3D model in (b) The RGB color image obtained by the transformation; (d) is the front view and left view of the real 3D point cloud of the ply 3D model of the object; (e) is that the ply 3D model of the object is imported into the automatic generation platform of the embodiment of the application to obtain , and then transform it into the front and left views of the 3D point cloud. The experimental results show that the embodiments of the present application can accurately obtain RGB color images and depth images in different six-dimensional poses for three-dimensional models in stl and ply formats.
请参阅图8，为本申请实施例的终端结构示意图。该终端50包括处理器51、与处理器51耦接的存储器52。Please refer to FIG. 8 , which is a schematic structural diagram of a terminal according to an embodiment of the present application. The terminal 50 includes a processor 51 and a memory 52 coupled to the processor 51 .
存储器52存储有用于实现上述六维姿态数据集自动生成方法的程序指令。The memory 52 stores program instructions for implementing the above-mentioned method for automatically generating a six-dimensional pose data set.
处理器51用于执行存储器52存储的程序指令以控制六维姿态数据集自动生成。The processor 51 is configured to execute program instructions stored in the memory 52 to control the automatic generation of the six-dimensional pose data set.
其中，处理器51还可以称为CPU(Central Processing Unit，中央处理单元)。处理器51可能是一种集成电路芯片，具有信号的处理能力。处理器51还可以是通用处理器、数字信号处理器(DSP)、专用集成电路(ASIC)、现成可编程门阵列(FPGA)或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件。通用处理器可以是微处理器或者该处理器也可以是任何常规的处理器等。The processor 51 may also be referred to as a CPU (Central Processing Unit, central processing unit). The processor 51 may be an integrated circuit chip with signal processing capability. The processor 51 may also be a general purpose processor, digital signal processor (DSP), application specific integrated circuit (ASIC), off-the-shelf programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic device, discrete hardware component . A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
请参阅图9，为本申请实施例的存储介质的结构示意图。本申请实施例的存储介质存储有能够实现上述所有方法的程序文件61，其中，该程序文件61可以以软件产品的形式存储在上述存储介质中，包括若干指令用以使得一台计算机设备(可以是个人计算机，服务器，或者网络设备等)或处理器(processor)执行本发明各个实施方式方法的全部或部分步骤。而前述的存储介质包括：U盘、移动硬盘、只读存储器(ROM，Read-Only Memory)、随机存取存储器(RAM，Random Access Memory)、磁碟或者光盘等各种可以存储程序代码的介质，或者是计算机、服务器、手机、平板等终端设备。Please refer to FIG. 9 , which is a schematic structural diagram of a storage medium according to an embodiment of the present application. The storage medium of this embodiment of the present application stores a program file 61 capable of implementing all the above methods, wherein the program file 61 may be stored in the above-mentioned storage medium in the form of a software product, and includes several instructions to make a computer device (which may It is a personal computer, a server, or a network device, etc.) or a processor that executes all or part of the steps of the methods of the various embodiments of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, Read-Only Memory (ROM, Read-Only Memory), Random Access Memory (RAM, Random Access Memory), magnetic disk or optical disk and other media that can store program codes , or terminal devices such as computers, servers, mobile phones, and tablets.
本申请实施例的六维姿态数据集自动生成方法、***、终端以及存储介质通过将物体的三维模型和六维姿态参数导入基于OpenGL虚拟环境的数据集自动生成平台，该平台根据六维姿态参数控制三维模型在虚拟环境中不断变动姿态，并由虚拟相机自动拍摄每个六维姿态下该物体的RGB彩色图像和对应的深度图像，并自动标注六维姿态标签。相对于现有技术，本申请实施例通过简单的操作以高效的生成三维模型不同的六维姿态下的RGB彩色图像和深度图像以及六维姿态标签，耗时低，并节约了人力和金钱的成本消耗。The method, system, terminal, and storage medium for automatically generating a six-dimensional attitude dataset according to the embodiments of the present application generate a platform automatically by importing the three-dimensional model and six-dimensional attitude parameters of an object into a dataset based on an OpenGL virtual environment. The 3D model is controlled to continuously change its posture in the virtual environment, and the RGB color image and the corresponding depth image of the object under each 6D posture are automatically captured by the virtual camera, and the 6D posture label is automatically marked. Compared with the prior art, the embodiment of the present application can efficiently generate RGB color images and depth images and six-dimensional attitude labels in different six-dimensional attitudes of the three-dimensional model through simple operations, which is less time-consuming and saves manpower and money. cost consumption.
对所公开的实施例的上述说明，使本领域专业技术人员能够实现或使用本申请。对这些实施例的多种修改对本领域的专业技术人员来说将是显而易见的，本申请中所定义的一般原理可以在不脱离本申请的精神或范围的情况下，在其它实施例中实现。因此，本申请将不会被限制于本申请所示的这些实施例，而是要符合与本申请所公开的原理和新颖特点相一致的最宽的范围。The above description of the disclosed embodiments enables any person skilled in the art to make or use the present application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined in this application may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims (10)

  1. 一种六维姿态数据集自动生成方法，其特征在于，包括以下步骤：A method for automatically generating a six-dimensional attitude dataset, comprising the following steps:
    步骤a：构建基于OpenGL虚拟环境的数据集自动生成平台；Step a: build a data set automatic generation platform based on OpenGL virtual environment;
    步骤b：获取物体的三维模型以及所述三维模型的六维姿态参数，并将所述三维模型以及六维姿态参数导入所述数据集自动生成平台；Step b: obtaining the three-dimensional model of the object and the six-dimensional attitude parameters of the three-dimensional model, and importing the three-dimensional model and the six-dimensional attitude parameters into the data set automatic generation platform;
    步骤c：所述数据集自动生成平台根据所述六维姿态参数控制所述三维模型在OpenGL虚拟环境中变动姿态，并通过所述OpenGL虚拟环境中的虚拟相机采集所述三维模型在各个六维姿态下的形态，生成各个六维姿态下所述物体的RGB彩色图像和对应的深度图像，并标注所述图像的六维姿态标签。Step c: The data set automatic generation platform controls the three-dimensional model to change the posture in the OpenGL virtual environment according to the six-dimensional attitude parameters, and collects the three-dimensional model in each six-dimensional environment through the virtual camera in the OpenGL virtual environment. The RGB color image and the corresponding depth image of the object in each six-dimensional attitude are generated, and the six-dimensional attitude label of the image is marked.
  2. 根据权利要求1所述的六维姿态数据集自动生成方法，其特征在于，所述步骤a中，所述获取物体的三维模型以及所述三维模型的六维姿态参数还包括：The method for automatically generating a six-dimensional attitude data set according to claim 1, wherein, in the step a, the acquiring the three-dimensional model of the object and the six-dimensional attitude parameters of the three-dimensional model further comprise:
    将所述三维模型和六维姿态参数转换成导入所述数据集自动生成平台的统一文件格式。The three-dimensional model and six-dimensional pose parameters are converted into a unified file format imported into the data set automatic generation platform.
  3. 根据权利要求1所述的六维姿态数据集自动生成方法，其特征在于，在所述步骤a中，所述将所述三维模型以及六维姿态参数导入所述数据集自动生成平台还包括：The method for automatically generating a six-dimensional attitude dataset according to claim 1, wherein in the step a, the step of importing the three-dimensional model and six-dimensional attitude parameters into the automatic generating platform for the dataset further comprises:
    设置所述三维模型在所述OpenGL虚拟环境中的世界坐标系中的初始位置，并根据每个面片顶点的x，y，z坐标、法线坐标和颜色将所述三维模型的初始形态绘制在所述世界坐标系中；Set the initial position of the 3D model in the world coordinate system in the OpenGL virtual environment, and draw the initial shape of the 3D model according to the x, y, z coordinates, normal coordinates and color of each patch vertex in said world coordinate system;
    设置所述虚拟相机在所述OpenGL虚拟环境中的世界坐标系中的位置、朝向、法线、视角、拍摄图片的长宽比、近截面以及远截面。Set the position, orientation, normal, angle of view, aspect ratio, near section and far section of the virtual camera in the world coordinate system in the OpenGL virtual environment.
  4. 根据权利要求3所述的六维姿态数据集自动生成方法，其特征在于，在所述步骤c中，所述三维模型在各个六维姿态下的形态为所述三维模型初始形态下各顶点的坐标(x,y,z)乘上六维姿态[R|t]，即
    Figure PCTCN2020139761-appb-100001
    The method for automatically generating a six-dimensional pose data set according to claim 3, wherein in the step c, the shape of the three-dimensional model under each six-dimensional pose is the shape of each vertex in the initial shape of the three-dimensional model. The coordinates (x, y, z) are multiplied by the six-dimensional attitude [R|t], that is
    Figure PCTCN2020139761-appb-100001
  5. 根据权利要求4所述的六维姿态数据集自动生成方法，其特征在于，在所述步骤c中，所述生成各个六维姿态下所述物体的RGB彩色图像包括：The method for automatically generating a six-dimensional pose data set according to claim 4, wherein in the step c, the generating the RGB color image of the object under each six-dimensional pose comprises:
    利用所述虚拟相机的成像原理，将所述近截面与远截面之间的物体正射投影到所述近截面上，并以二进制整型数字存储到颜色缓存中；Using the imaging principle of the virtual camera, the object between the near section and the far section is orthographically projected onto the near section, and stored in the color buffer as a binary integer number;
    获取所述颜色缓存中的二进制整型数字，并将所述二进制整型数字按顺序分配为每个像素上的R、G、B值，生成所述RGB彩色图像。Acquire the binary integer number in the color buffer, and assign the binary integer number as R, G, and B values on each pixel in sequence to generate the RGB color image.
  6. 根据权利要求5所述的六维姿态数据集自动生成方法，其特征在于，在所述步骤c中，所述生成各个六维姿态下所述物体的深度图像包括：The method for automatically generating a six-dimensional pose data set according to claim 5, wherein in the step c, the generating the depth image of the object under each six-dimensional pose comprises:
    对所述近截面到远截面之间分配深度值，得到所述物体前表面的深度值，并将所述深度值以浮点型数值存储到深度缓存中；Allocate a depth value between the near section and the far section, obtain the depth value of the front surface of the object, and store the depth value in the depth buffer as a floating-point value;
    获取所述深度缓存中的浮点型数值，对所述浮点型数值进行计算，得到所述三维模型的前表面距所述虚拟相机的真实距离，并将所述浮点型数值按顺序分配到每个像素上，得到对应的深度图像。Obtain the floating-point value in the depth buffer, calculate the floating-point value, obtain the real distance between the front surface of the 3D model and the virtual camera, and assign the floating-point value in order To each pixel, the corresponding depth image is obtained.
  7. 根据权利要求1至6任一项所述的六维姿态数据集自动生成方法，其特征在于，在所述步骤c中，所述生成各个六维姿态下所述物体的RGB彩色图像和对应的深度图像，并标注所述图像的六维姿态标签还包括：The method for automatically generating a six-dimensional pose data set according to any one of claims 1 to 6, wherein in the step c, the RGB color image of the object and the corresponding RGB color image of the object under each six-dimensional pose are generated depth image, and annotating the image's six-dimensional pose label also includes:
    保存所述虚拟相机采集的RGB彩色图像和深度图像，并将采集图像时对应的六维姿态作为所述RGB彩色图像和深度图像的六维姿态标签。The RGB color image and the depth image collected by the virtual camera are saved, and the corresponding six-dimensional attitude when the image is collected is used as the six-dimensional attitude label of the RGB color image and the depth image.
  8. 一种六维姿态数据集自动生成***，其特征在于，包括：An automatic generation system for a six-dimensional attitude data set, characterized in that it includes:
    平台构建模块：用于构建基于OpenGL虚拟环境的数据集自动生成平台；Platform building module: used to build a data set automatic generation platform based on OpenGL virtual environment;
    模型及参数获取模块：用于获取物体的三维模型以及所述三维模型的六维姿态参数；Model and parameter acquisition module: used to acquire the three-dimensional model of the object and the six-dimensional attitude parameters of the three-dimensional model;
    数据导入模块：用于将所述三维模型以及六维姿态参数导入所述数据集自动生成平台；Data import module: used to import the three-dimensional model and six-dimensional attitude parameters into the data set automatic generation platform;
    图像采集模块：用于根据所述六维姿态参数控制所述三维模型在OpenGL虚拟环境中变动姿态，并通过所述OpenGL虚拟环境中的虚拟相机采集所述三维模型在各个六维姿态下的形态，生成各个六维姿态下所述物体的RGB彩色图像和对应的深度图像，并标注所述图像的六维姿态标签。Image acquisition module: used to control the posture of the three-dimensional model to change in the OpenGL virtual environment according to the six-dimensional posture parameters, and collect the form of the three-dimensional model in each six-dimensional posture through the virtual camera in the OpenGL virtual environment , generate an RGB color image and a corresponding depth image of the object under each six-dimensional pose, and mark the six-dimensional pose label of the image.
  9. 一种终端，其特征在于，所述终端包括处理器、与所述处理器耦接的存储器，其中，A terminal, characterized in that the terminal includes a processor and a memory coupled to the processor, wherein,
    所述存储器存储有用于实现权利要求1-7任一项所述的六维姿态数据集自动生成方法的程序指令；The memory stores program instructions for realizing the method for automatically generating a six-dimensional attitude data set according to any one of claims 1-7;
    所述处理器用于执行所述存储器存储的所述程序指令以控制六维姿态数据集自动生成。The processor is configured to execute the program instructions stored in the memory to control the automatic generation of a six-dimensional pose data set.
  10. 一种存储介质，其特征在于，存储有处理器可运行的程序指令，所述程序指令用于执行权利要求1至7任一项所述六维姿态数据集自动生成方法。A storage medium, characterized in that it stores program instructions executable by a processor, and the program instructions are used to execute the method for automatically generating a six-dimensional attitude data set according to any one of claims 1 to 7.
PCT/CN2020/139761 2020-07-27 2020-12-26 Method and system for automatically generating six-dimensional posture data set, and terminal and storage medium WO2022021782A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202010734849.7A CN111951333A (en) 2020-07-27 2020-07-27 Automatic six-dimensional attitude data set generation method, system, terminal and storage medium
CN202010734849.7 2020-07-27

Publications (1)

Publication Number Publication Date
WO2022021782A1 true WO2022021782A1 (en) 2022-02-03

Family

ID=73339629

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/139761 WO2022021782A1 (en) 2020-07-27 2020-12-26 Method and system for automatically generating six-dimensional posture data set, and terminal and storage medium

Country Status (2)

Country Link
CN (1) CN111951333A (en)
WO (1) WO2022021782A1 (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114683288A (en) * 2022-05-07 2022-07-01 法奥意威(苏州)机器人***有限公司 Robot display and control method and device and electronic equipment
CN116502548A (en) * 2023-06-29 2023-07-28 湖北工业大学 Three-dimensional toy design method based on deep learning

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111951333A (en) * 2020-07-27 2020-11-17 中国科学院深圳先进技术研究院 Automatic six-dimensional attitude data set generation method, system, terminal and storage medium

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2014162615A1 (en) * 2013-04-05 2014-10-09 Molecular Design Frontier Co. Ltd. Analysis of biomolecular solvation sites by the 3d-rism theory.
CN109636856A (en) * 2019-01-17 2019-04-16 天津大学 Object 6 DOF degree posture information union measuring method based on HOG Fusion Features operator
CN110660101A (en) * 2019-08-19 2020-01-07 浙江理工大学 Object 6D posture prediction method based on RGB image and coordinate system transformation
CN111951333A (en) * 2020-07-27 2020-11-17 中国科学院深圳先进技术研究院 Automatic six-dimensional attitude data set generation method, system, terminal and storage medium

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111161387B (en) * 2019-12-31 2023-05-30 华东理工大学 Method and system for synthesizing images in stacked scene, storage medium and terminal equipment
CN111415389B (en) * 2020-03-18 2023-08-29 清华大学 Label-free six-dimensional object posture prediction method and device based on reinforcement learning

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2014162615A1 (en) * 2013-04-05 2014-10-09 Molecular Design Frontier Co. Ltd. Analysis of biomolecular solvation sites by the 3d-rism theory.
CN109636856A (en) * 2019-01-17 2019-04-16 天津大学 Object 6 DOF degree posture information union measuring method based on HOG Fusion Features operator
CN110660101A (en) * 2019-08-19 2020-01-07 浙江理工大学 Object 6D posture prediction method based on RGB image and coordinate system transformation
CN111951333A (en) * 2020-07-27 2020-11-17 中国科学院深圳先进技术研究院 Automatic six-dimensional attitude data set generation method, system, terminal and storage medium

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
CHEN YIKUAN: "6D Pose Estimation of Industrial Parts Based on Random Ferns Regression", MASTER THESIS, TIANJIN POLYTECHNIC UNIVERSITY, CN, no. 8, 15 August 2018 (2018-08-15), CN , XP055891292, ISSN: 1674-0246 *

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114683288A (en) * 2022-05-07 2022-07-01 法奥意威(苏州)机器人***有限公司 Robot display and control method and device and electronic equipment
CN114683288B (en) * 2022-05-07 2023-05-30 法奥意威(苏州)机器人***有限公司 Robot display and control method and device and electronic equipment
CN116502548A (en) * 2023-06-29 2023-07-28 湖北工业大学 Three-dimensional toy design method based on deep learning
CN116502548B (en) * 2023-06-29 2023-09-15 湖北工业大学 Three-dimensional toy design method based on deep learning

Also Published As

Publication number Publication date
CN111951333A (en) 2020-11-17

Similar Documents

Publication Publication Date Title
WO2022021782A1 (en) Method and system for automatically generating six-dimensional posture data set, and terminal and storage medium
CN108764048B (en) Face key point detection method and device
CN107330439B (en) Method for determining posture of object in image, client and server
CN108509848A (en) The real-time detection method and system of three-dimension object
JP6560480B2 (en) Image processing system, image processing method, and program
CN111401266B (en) Method, equipment, computer equipment and readable storage medium for positioning picture corner points
CN108010123B (en) Three-dimensional point cloud obtaining method capable of retaining topology information
CN109711472B (en) Training data generation method and device
Pan et al. Rapid scene reconstruction on mobile phones from panoramic images
US10977767B2 (en) Propagation of spot healing edits from one image to multiple images
JP2013050947A (en) Method for object pose estimation, apparatus for object pose estimation, method for object estimation pose refinement and computer readable medium
JP2010079453A (en) Apparatus for generating position-and-orientation estimation model, position and orientation calculation device, image processor, and method for the same
WO2023284713A1 (en) Three-dimensional dynamic tracking method and apparatus, electronic device and storage medium
US10169891B2 (en) Producing three-dimensional representation based on images of a person
WO2021027585A1 (en) Human face image processing method and electronic device
Jin et al. Robust 3D face modeling and reconstruction from frontal and side images
WO2023093739A1 (en) Multi-view three-dimensional reconstruction method
CN115880415A (en) Three-dimensional reconstruction method and device, electronic equipment and storage medium
WO2021142843A1 (en) Image scanning method and device, apparatus, and storage medium
CN115008454A (en) Robot online hand-eye calibration method based on multi-frame pseudo label data enhancement
WO2024037562A1 (en) Three-dimensional reconstruction method and apparatus, and computer-readable storage medium
US11080920B2 (en) Method of displaying an object
CN111460937B (en) Facial feature point positioning method and device, terminal equipment and storage medium
CN113362467A (en) Point cloud preprocessing and ShuffleNet-based mobile terminal three-dimensional pose estimation method
CN109166176B (en) Three-dimensional face image generation method and device

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20947756

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20947756

Country of ref document: EP

Kind code of ref document: A1

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 10/07/2023)