WO2021120846A1

WO2021120846A1 - Three-dimensional reconstruction method and device, and computer readable medium

Info

Publication number: WO2021120846A1
Application number: PCT/CN2020/123377
Authority: WO
Inventors: 郭林杰; 马岳文; 郁树达; 李思琪; 邹成
Original assignee: 支付宝(杭州)信息技术有限公司
Priority date: 2019-12-20
Filing date: 2020-10-23
Publication date: 2021-06-24
Also published as: CN111127633A

Abstract

A three-dimensional reconstruction scheme. The scheme comprises: first scanning a target object from multiple angles of view by means of a scanning device, so as to acquire an initial geometric model of the target object, texture pictures at multiple angles of view and camera poses of the texture pictures with respect to the initial geometric model, and then acquiring a complete geometric model of the target object in other manners, and after the complete geometric model is registered with the initial geometric model, performing texture mapping on the complete geometric model of the target object according to the texture pictures and camera poses of the texture pictures with respect to the complete geometric model, thereby obtaining a three-dimensional model of the target object. An additionally acquired complete geometric model can be used to perform registration with an initial geometric model, therefore, even if the initial geometric model is deficient due to reasons such as high light and transparency of an object surface, the initial geometric model can be compensated by the complete geometric model, and a complete three-dimensional model with a real scale can be obtained after texture mapping is performed.

Description

三维重建方法、设备以及计算机可读介质Three-dimensional reconstruction method, equipment and computer readable medium

技术领域Technical field

本申请涉及信息技术领域，尤其涉及一种三维重建方法、设备以及计算机可读介质。This application relates to the field of information technology, and in particular to a three-dimensional reconstruction method, device, and computer-readable medium.

背景技术Background technique

三维重建技术是物体对象在三维空间中数据表示的处理过程，三维模型可以使用采集物体的三维空间点构成点云来进行表示，点云可以使用三角网格、线、多边形网格来进行连接重建模型的表面。三维模型可以使用在在电影、游戏、制造等领域，三维建模技术属于一个多学科交叉的研究领域，是计算机图形学和图像处理在工程中的重要应用。Three-dimensional reconstruction technology is the process of data representation of objects in three-dimensional space. The three-dimensional model can use the three-dimensional space points of the collected objects to form a point cloud for representation, and the point cloud can be connected and reconstructed using triangular grids, lines, and polygonal grids. The surface of the model. Three-dimensional models can be used in movies, games, manufacturing and other fields. Three-dimensional modeling technology belongs to a multidisciplinary research field, and is an important application of computer graphics and image processing in engineering.

当前三维重建行业中主要有以下几种三维重建方案：There are mainly the following 3D reconstruction solutions in the current 3D reconstruction industry:

1、基于背景纹影技术的透明物体三维表面重建方案，这种方案通过获取光源穿过透明物体后透射于背景图像后的折射率场来对透明物体进行三维表面重建。其缺点在主要在于，仅能够对整体透明的物体进行三维建模，对于同时存在透明与不透明部分的物体(如塑料饮料瓶等)，只能获取到透明部分的三维点云数据，由此重建的三维模型依然是残缺的。1. A three-dimensional surface reconstruction scheme for transparent objects based on background schlieren technology. This scheme performs three-dimensional surface reconstruction on transparent objects by obtaining the refractive index field after the light source passes through the transparent object and is transmitted to the background image. The main disadvantage is that it can only perform three-dimensional modeling of objects that are transparent as a whole. For objects that have both transparent and opaque parts (such as plastic beverage bottles, etc.), only the three-dimensional point cloud data of the transparent part can be obtained for reconstruction. The 3D model is still incomplete.

2、一种通过扫描物体的深度数据和色彩数据进行三维重建方案，该方案通过RGB-D相机采集物体的彩色数据和深度数据，并基于彩色数据和深度数据进行几何建模、纹理贴图等，从而完整物体表面三维重建。其缺点在于，对于一些表面存在透明或者高光部分的物体，如同时存在透明与不透明部分的物体，由于用于采集深度数据主动光会因物体表面的透明或高光而无法被摄像头有效获取，导致深度数据缺失，无法重建获得完整的三维模型。2. A three-dimensional reconstruction scheme by scanning the depth data and color data of the object. The scheme collects the color data and depth data of the object through an RGB-D camera, and performs geometric modeling and texture mapping based on the color data and depth data, etc. Thereby the three-dimensional reconstruction of the complete object surface. The disadvantage is that for some objects with transparent or high-light parts on the surface, such as objects with both transparent and opaque parts, the active light used to collect depth data will not be effectively captured by the camera due to the transparency or high light on the surface of the object, resulting in depth The data is missing, and a complete three-dimensional model cannot be reconstructed.

3、基于色彩图像的多视角立体几何三维重建方案，该方案通过输入密集的RGB图像，而后通过多视角立体几何的方法得到每幅图像对应的位姿，然后根据图像纹理提取出稀疏点云，进而得到稠密点云并进行纹理贴图，以获得三维模型。其缺点在于，对于纹理不丰富的区域，由于无法提取出有效的特征点，立体几何的方法失去效果，无法得到用于重建的点云数据，导致无法重建完整的三维模型。并且，通过该方案得到的三维模型没有真实的尺度数据，其重建获得的模型数值无法表示物体的真实大小。3. Multi-view stereo geometry 3D reconstruction scheme based on color images. This scheme inputs dense RGB images, and then obtains the corresponding pose of each image through the method of multi-view stereo geometry, and then extracts sparse point clouds according to the image texture. Then get dense point cloud and perform texture mapping to obtain a three-dimensional model. The disadvantage is that for regions with insufficient texture, effective feature points cannot be extracted, and the solid geometry method loses its effect, and the point cloud data for reconstruction cannot be obtained, resulting in the inability to reconstruct a complete three-dimensional model. In addition, the three-dimensional model obtained by this scheme has no real scale data, and the model value obtained by its reconstruction cannot represent the real size of the object.

由此可知，现有的各个三维方案各个方案的适用范围较小，仅对特定类型的物体的三维重建具有较好的重建效果，没有一种能够广泛适用于各种不同类型的物体的三维重建方法。It can be seen that the scope of application of each of the existing three-dimensional schemes is relatively small, and only has a good reconstruction effect for the three-dimensional reconstruction of specific types of objects, and none of them can be widely applied to the three-dimensional reconstruction of various types of objects. method.

发明内容Summary of the invention

本申请的一个目的是提供一种三维重建方法、设备以及计算机可读介质，用以解决现有方案适用范围较小的问题。One purpose of the present application is to provide a three-dimensional reconstruction method, equipment and computer readable medium to solve the problem of the small applicable scope of the existing solutions.

本申请实施例中提供了一种三维重建方法，该方法包括：An embodiment of the present application provides a three-dimensional reconstruction method, which includes:

通过扫描设备对目标物体进行多视角扫描，获取目标物体的初始几何模型、多个视角的纹理图片以及所述纹理图片关于所述初始几何模型的相机位姿，其中，所述多个视角的纹理图片至少覆盖所述目标物体的表面；Multi-view scanning of the target object is performed by the scanning device to obtain the initial geometric model of the target object, the texture picture of the multiple viewing angles, and the camera pose of the texture picture with respect to the initial geometric model, wherein the texture of the multiple viewing angles The picture covers at least the surface of the target object;

获取所述目标物体的完整几何模型；Acquiring a complete geometric model of the target object;

将所述完整几何模型与所述初始几何模型进行配准，确定所述纹理图片关于完整几何模型的相机位姿；Registering the complete geometric model with the initial geometric model, and determining the camera pose of the texture picture with respect to the complete geometric model;

根据多个视角的纹理图片以及所述纹理图片关于完整几何模型的相机位姿，对所述目标物体的完整几何模型进行纹理贴图，获得目标物体的三维模型。Perform texture mapping on the complete geometric model of the target object according to texture pictures from multiple perspectives and the camera pose of the texture picture with respect to the complete geometric model to obtain a three-dimensional model of the target object.

本申请实施例还提供了一种三维重建设备，该设备包括：The embodiment of the present application also provides a three-dimensional reconstruction device, which includes:

扫描处理模块，用于通过扫描设备对目标物体进行多视角扫描，获取目标物体的初始几何模型、多个视角的纹理图片以及所述纹理图片关于所述初始几何模型的相机位姿，其中，所述多个视角的纹理图片至少覆盖所述目标物体的表面；The scanning processing module is used to perform multi-view scanning of the target object through the scanning device to obtain the initial geometric model of the target object, the texture picture of the multiple viewpoints, and the camera pose of the texture picture with respect to the initial geometric model. The texture pictures of the multiple viewing angles cover at least the surface of the target object;

模型获取模块，用于获取所述目标物体的完整几何模型；A model acquisition module for acquiring a complete geometric model of the target object;

模型配准模块，用于将所述完整几何模型与所述初始几何模型进行配准，确定所述纹理图片关于完整几何模型的相机位姿；A model registration module, configured to register the complete geometric model with the initial geometric model, and determine the camera pose of the texture picture with respect to the complete geometric model;

贴图处理模块，用于根据多个视角的纹理图片以及所述纹理图片关于完整几何模型的相机位姿，对所述目标物体的完整几何模型进行纹理贴图，获得目标物体的三维模型。The texture processing module is used to perform texture mapping on the complete geometric model of the target object according to texture pictures from multiple perspectives and the camera pose of the texture picture with respect to the complete geometric model to obtain a three-dimensional model of the target object.

本申请的一些实施例还提供了一种计算设备，其中，该设备包括用于存储计算机程序指令的存储器和用于执行计算机程序指令的处理器，其中，当该计算机程序指令被该处理器执行时，触发所述设备执行前述的三维重建方法。Some embodiments of the present application also provide a computing device, wherein the device includes a memory for storing computer program instructions and a processor for executing computer program instructions, wherein when the computer program instructions are executed by the processor At this time, the device is triggered to execute the aforementioned three-dimensional reconstruction method.

本申请的另一些实施例还提供了一种计算机可读介质，其上存储有计算机程序指令，所述计算机可读指令可被处理器执行以实现所述三维重建方法。Other embodiments of the present application also provide a computer-readable medium on which computer program instructions are stored, and the computer-readable instructions can be executed by a processor to implement the three-dimensional reconstruction method.

本申请实施例提供方案中，首先通过扫描设备对目标物体进行多视角扫描，获取目标物体的初始几何模型、多个视角的纹理图片以及所述纹理图片关于所述初始几何模型的相机位姿，而后通过其它方式获取所述目标物体的完整几何模型，与所述初始几何模型进行配准之后，可以根据纹理图片以及纹理图片关于完整几何模型的相机位姿，对所述目标物体的完整几何模型进行纹理贴图，从而获得目标物体的三维模型。由于可以使用另外获取的完整几何模型与初始几何模型进行配准，即使初始几何模型因为物体表面的高光、透明等原因而导致残缺，可以由完整几何模型进行弥补，由此进行纹理贴图之后即可获得完整且具有真实尺度的三维模型。In the solution provided in the embodiments of the present application, a scanning device is used to scan a target object from multiple perspectives to obtain an initial geometric model of the target object, a texture picture of multiple viewing angles, and a camera pose of the texture picture with respect to the initial geometric model. Then obtain the complete geometric model of the target object by other means, and after registering with the initial geometric model, the complete geometric model of the target object can be determined according to the texture picture and the camera pose of the texture picture with respect to the complete geometric model. Perform texture mapping to obtain a three-dimensional model of the target object. Since the separately obtained complete geometric model can be used for registration with the initial geometric model, even if the initial geometric model is incomplete due to the high light and transparency of the object surface, it can be compensated by the complete geometric model, which can be used after texture mapping. Obtain a complete and true-scale three-dimensional model.

附图说明Description of the drawings

通过阅读参照以下附图所作的对非限制性实施例所作的详细描述，本申请的其它特征、目的和优点将会变得更明显：By reading the detailed description of the non-limiting embodiments with reference to the following drawings, other features, purposes and advantages of the present application will become more apparent:

图1为本申请实施例提供的一种三维重建方法的处理流程图；FIG. 1 is a processing flowchart of a three-dimensional reconstruction method provided by an embodiment of the application;

图2为本申请实施例中对深度数据和色彩数据处理时的流程示意图；2 is a schematic diagram of the process of processing depth data and color data in an embodiment of the application;

图3为采用本申请实施例中的方案实现一般物体三维重建时的处理过程示意图；FIG. 3 is a schematic diagram of the processing process when the solution in the embodiment of the present application is used to realize three-dimensional reconstruction of a general object;

图4为本申请实施例提供的一种三维重建设备的结构示意图；4 is a schematic structural diagram of a three-dimensional reconstruction device provided by an embodiment of this application;

图5为本申请实施例提供的一种用于实现三维重建的计算设备的结构示意图；FIG. 5 is a schematic structural diagram of a computing device for realizing three-dimensional reconstruction provided by an embodiment of the application;

附图中相同或相似的附图标记代表相同或相似的部件。The same or similar reference signs in the drawings represent the same or similar components.

具体实施方式Detailed ways

下面结合附图对本申请作进一步详细描述。The application will be further described in detail below in conjunction with the accompanying drawings.

在本申请一个典型的配置中，终端、服务网络的设备均包括一个或多个处理器(CPU)、输入/输出接口、网络接口和内存。In a typical configuration of this application, both the terminal and the equipment serving the network include one or more processors (CPU), input/output interfaces, network interfaces, and memory.

内存可能包括计算机可读介质中的非永久性存储器，随机存取存储器(RAM)和/或非易失性内存等形式，如只读存储器(ROM)或闪存(flash RAM)。内存是计算机可读介质的示例。Memory may include non-permanent memory in computer-readable media, random access memory (RAM) and/or non-volatile memory, such as read-only memory (ROM) or flash memory (flash RAM). Memory is an example of computer readable media.

计算机可读介质包括永久性和非永久性、可移动和非可移动媒体，可以由任何方法或技术来实现信息存储。信息可以是计算机可读指令、数据结构、程序的装置或其他数据。计算机的存储介质的例子包括，但不限于相变内存(PRAM)、静态随机存取存储器(SRAM)、动态随机存取存储器(DRAM)、其他类型的随机存取存储器(RAM)、只读存储器(ROM)、电可擦除可编程只读存储器(EEPROM)、快闪记忆体或其他内存技术、只读光盘(CD-ROM)、数字多功能光盘(DVD)或其他光学存储、磁盒式磁带，磁带磁盘存储或其他磁性存储设备或任何其他非传输介质，可用于存储可以被计算设备访问的信息。Computer-readable media includes permanent and non-permanent, removable and non-removable media, and information storage can be realized by any method or technology. The information can be computer readable instructions, data structures, program devices, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, CD-ROM, digital versatile disc (DVD) or other optical storage, magnetic cassette type Magnetic tape, magnetic tape disk storage or other magnetic storage devices or any other non-transmission media can be used to store information that can be accessed by computing devices.

本申请实施例提供了一种三维重建方法，该方法可以使用另外获取的完整几何模型与初始几何模型进行配准，即使初始几何模型因为物体表面的高光、透明等原因而导致残缺，可以由完整几何模型进行弥补，由此进行纹理贴图之后即可获得完整且具有真实尺度的三维模型。The embodiment of the application provides a three-dimensional reconstruction method, which can use the complete geometric model obtained separately to register with the initial geometric model. Even if the initial geometric model is incomplete due to the high light and transparency of the surface of the object, the complete The geometric model is compensated, and a complete and true-scale 3D model can be obtained after texture mapping.

在实际场景中，该方法的执行主体可以是用户设备、网络设备或者用户设备与网络设备通过网络相集成所构成的设备，此外也可以是运行于上述设备中的程序。所述用户设备包括但不限于计算机、手机、平板电脑等各类终端设备；所述网络设备包括但不限于如网络主机、单个网络服务器、多个网络服务器集或基于云计算的计算机集合等实现。在此，云由基于云计算(Cloud Computing)的大量主机或网络服务器构成，其中，云计算是分布式计算的一种，由一群松散耦合的计算机集组成的一个虚拟计算机。In actual scenarios, the execution subject of this method may be user equipment, network equipment, or a device formed by the integration of user equipment and network equipment through a network, and may also be a program running in the above-mentioned equipment. The user equipment includes, but is not limited to, various terminal devices such as computers, mobile phones, and tablet computers; the network equipment includes, but is not limited to, implementations such as a network host, a single network server, a set of multiple network servers, or a set of computers based on cloud computing, etc. . Here, the cloud is composed of a large number of hosts or network servers based on Cloud Computing. Cloud computing is a type of distributed computing, a virtual computer composed of a group of loosely coupled computer sets.

图1示出了本申请实施提供的一种三维重建方法的处理流程，该方法至少包括以下处理步骤：Fig. 1 shows the processing flow of a three-dimensional reconstruction method provided by the implementation of this application. The method includes at least the following processing steps:

步骤S101，通过扫描设备对目标物体进行多视角扫描，获取目标物体的初始几何模型、多个视角的纹理图片以及所述纹理图片关于所述初始几何模型的相机位姿。Step S101: Perform multi-view scanning of a target object by a scanning device to obtain an initial geometric model of the target object, a texture picture of multiple viewing angles, and a camera pose of the texture picture with respect to the initial geometric model.

所述目标物体不限于特定的类型，可以是同时存在透明与不透明部分的物体，也可以是表面存在透明或者高光部分的物体、表面纹理不丰富的物体等。由此，通过扫描设备对目标物体进行多视角扫描所获取初始几何模型可能是残缺的几何模型，例如对于一个表面存在高光部分的目标物体，可能会因为缺少该高光部分的点云数据(point cloud data)，而导致初始几何模型中缺少该高光部分，或者该高光部分的数据有误，此时该初始几何模型即为残缺的几何模型。The target object is not limited to a specific type, and may be an object with both transparent and opaque parts, an object with transparent or high-gloss parts on the surface, and an object with insufficient surface texture. Therefore, the initial geometric model obtained by scanning the target object through the multi-view scanning device may be a broken geometric model. For example, for a target object with a highlight part on its surface, it may be due to the lack of point cloud data of the highlight part. data), resulting in the lack of the highlight part in the initial geometric model, or the error in the data of the highlight part, at this time the initial geometric model is the incomplete geometric model.

在本申请的一些实施例中，所述扫描设备可以是任意能够获取色彩数据和深度数据的设备，所述色彩数据用于表示图像中各个像素点的色彩，例如可以采用RGB的表示方式记录每个像素点的RGB值，所述深度数据用于表示图像中各个像素点与深度数据的采集器之间的距离，可以反映图像中物体可见表面的几何形状。由此，可以通过扫描设备对目标物体进行多视角扫描，获取关于所述目标物体的多个视角的深度数据和色彩数据，而后根据多个视角的深度数据和色彩数据获取目标物体的初始几何模型、多个视角的纹理图片以及所述纹理图片关于所述初始几何模型的相机位姿。In some embodiments of the present application, the scanning device may be any device that can obtain color data and depth data. The color data is used to represent the color of each pixel in the image. For example, RGB can be used to record each pixel. The RGB value of each pixel, the depth data is used to represent the distance between each pixel in the image and the depth data collector, and can reflect the geometric shape of the visible surface of the object in the image. As a result, the target object can be scanned from multiple perspectives by the scanning device to obtain depth data and color data from multiple perspectives of the target object, and then the initial geometric model of the target object can be acquired based on the depth data and color data of the multiple perspectives. , Texture pictures of multiple viewing angles and camera poses of the texture pictures with respect to the initial geometric model.

为了能够获得深度数据和色彩数据，所述扫描设备可以是RGB-D摄像头等能够同时对物体的色彩和深度进行采集的设备。并且在进行多角度扫描时，需要保证角度能够覆盖整个物体，以使得由此获得的多个视角的纹理图片至少覆盖所述目标物体的表面，避免在进行后续的贴图时导致表面某部分的纹理缺失。In order to be able to obtain depth data and color data, the scanning device may be a device capable of simultaneously collecting the color and depth of an object, such as an RGB-D camera. And when performing multi-angle scanning, it is necessary to ensure that the angle can cover the entire object, so that the texture images obtained from multiple perspectives at least cover the surface of the target object, so as to avoid the texture of a certain part of the surface during subsequent mapping. Missing.

在实际场景中，进行多视角扫描时，可以至少包括6个视角的扫描。当采用最少的6个视角时，可以从目标物体的正面、反面、左右两个侧面以及顶面和底面获取此6个视角的色彩数据和深度数据，对应于目标物体的正六视图，或者也可以沿物体的周围按照间隔60度的方式设定6个扫描的视角。当采用6个以上的视角时，可以根据实际需要调整这些视角的分布，例如可以设置多个视角采集目标物体表面上几何形状较为复杂的区域，从而确保能够更加有效地获取各个目标物体的深度数据和色彩数据。In an actual scene, when scanning with multiple viewing angles, at least 6 viewing angles can be included. When using a minimum of 6 viewing angles, the color data and depth data of these 6 viewing angles can be obtained from the front, back, left and right sides, top and bottom surfaces of the target object, corresponding to the front six views of the target object, or it can be Set 6 scan angles along the periphery of the object at intervals of 60 degrees. When more than 6 viewing angles are used, the distribution of these viewing angles can be adjusted according to actual needs. For example, multiple viewing angles can be set to collect areas with complex geometric shapes on the surface of the target object, so as to ensure that the depth data of each target object can be obtained more effectively And color data.

本申请的一些实施例中，在根据多个视角的深度数据和色彩数据获取目标物体的初始几何模型、多个视角的纹理图片以及所述纹理图片关于所述初始几何模型的相机位姿时，可以采用如图2所示的处理流程，至少包括了以下的处理步骤：In some embodiments of the present application, when acquiring the initial geometric model of the target object, the texture picture of the multiple perspectives, and the camera pose of the texture picture with respect to the initial geometric model according to the depth data and color data of multiple viewing angles, The processing flow shown in Figure 2 can be used, which includes at least the following processing steps:

步骤S201，根据多个视角的深度数据获得不同视角的点云数据，并对不同视角的点云数据进行配准，获得目标物体的点云数据。Step S201: Obtain point cloud data of different viewing angles according to the depth data of multiple viewing angles, and register the point cloud data of different viewing angles to obtain the point cloud data of the target object.

在获取不同视角的点云数据时，可以基于每个视角的深度数据进行单独处理。由于深度数据表示图像中的各个像素点与深度数据的采集器之间的距离，能够反映出物体的几何形状，因此深度数据通过坐标转换可以计算为点云数据。所述点云数据是指三维坐标系下的点的集合，当点的数量足够多时可以用于描述物体的几何形状。When acquiring point cloud data from different perspectives, it can be processed separately based on the depth data of each perspective. Since the depth data represents the distance between each pixel in the image and the depth data collector, and can reflect the geometric shape of the object, the depth data can be calculated as point cloud data through coordinate conversion. The point cloud data refers to a collection of points in a three-dimensional coordinate system, which can be used to describe the geometric shape of an object when the number of points is sufficient.

在实际场景中，通过扫描设备对目标物体进行多视角扫描时，环境因素难以处于完全理想的状态，因此扫描获得的深度数据中不可避免地会存在一些会对处理产生干扰的噪声数据。在进行后续处理之前，对这些噪声数据进行过滤，可以使得处理结果更加准确。在本申请的一些实施例中，可以先根据深度阈值确定多个视角的深度数据中的背景深度数据，并过滤所述背景深度数据，获得目标物体的本体深度数据。In actual scenes, when multi-view scanning of a target object is performed by a scanning device, environmental factors are difficult to be in a completely ideal state. Therefore, there will inevitably be some noise data in the depth data obtained by scanning that will interfere with the processing. Before performing subsequent processing, filtering the noise data can make the processing result more accurate. In some embodiments of the present application, the background depth data in the depth data of multiple viewing angles may be determined according to the depth threshold first, and the background depth data may be filtered to obtain the body depth data of the target object.

例如，本实施例中采用16位的深度值，可以表示65536(2 ¹⁶)个不同的深度值，用于标定目标物体的各个点与采集器之间的距离。若该16位的深度值分别用于表示0～65535毫米的距离，在扫描时可以将目标物体与扫描设备的深度采集器之间的放置一定的举例，例如0.6～1.2米之间。因此，采集到的深度数据中，过大或者过小的深度值都不可能是目标物体上的点所对应的深度值，而是属于环境背景中其它物体的深度值。由此，可以过滤掉背景深度数据，获得目标物体的本体深度数据，并进一步根据多个视角的本体深度数据获得不同视角的点云数据。 For example, in this embodiment, a 16-bit depth value is used, which can represent 65536 (2 ¹⁶ ) different depth values, which are used to calibrate the distance between each point of the target object and the collector. If the 16-bit depth value is used to represent a distance of 0 to 65535 mm, during scanning, the target object and the depth collector of the scanning device can be placed between a certain example, for example, between 0.6 to 1.2 meters. Therefore, in the collected depth data, the depth value that is too large or too small cannot be the depth value corresponding to the point on the target object, but belong to the depth value of other objects in the environmental background. In this way, the background depth data can be filtered out, the body depth data of the target object can be obtained, and the point cloud data of different perspectives can be further obtained according to the body depth data of multiple perspectives.

根据多个视角的本体深度数据获得不同视角的点云数据的处理过程，可以理解为一个坐标转换的过程，每个视角的本体深度数据即为像素坐标系下的坐标值，而对应视角下的点云数据即为世界坐标系下的坐标值，通过将本体深度数据从像素坐标系换算至世界坐标系，即可获得不同视角下点云数据。The process of obtaining point cloud data from different perspectives based on the body depth data of multiple perspectives can be understood as a process of coordinate conversion. The body depth data of each perspective is the coordinate value in the pixel coordinate system, and the corresponding perspective The point cloud data is the coordinate value in the world coordinate system. By converting the body depth data from the pixel coordinate system to the world coordinate system, point cloud data in different perspectives can be obtained.

由于不同视角的点云数据之间会存在公共部分，对不同视角的点云数据进行配准之后即可获得整个目标物体的点云数据。该配准过程中，需要对各个视角的点云数据进行分析，求解相互之间的变换参数，进而根据变换参数将所有的视角下的点云数据映射到同一个坐标系中。Since there will be common parts between the point cloud data of different perspectives, the point cloud data of the entire target object can be obtained after the point cloud data of different perspectives are registered. In the registration process, it is necessary to analyze the point cloud data of each view to solve the mutual transformation parameters, and then map the point cloud data from all the views to the same coordinate system according to the transformation parameters.

在本申请的一些实施例中，可以采用如下的方式对不同视角的点云数据进行配准。首先，对不同视角的点云数据进行粗糙配准，提取其中具有公共部分的两个视角的点云数据之间的特征点，这种特征点可以是直线、拐点、曲线曲率等显式特征，也可以是自定义的符号、旋转图形、轴心等类型的特征。而后根据这些特征对其中一个点云数据的坐标系进行变换，使得两个视角的点云数据处于同一参考坐标系内，以此获得变换估计值。采用上述方式，对所有视角的点云数据两两进行粗糙配准，直至将所有视角的点云数据变换至同一参考坐标系内，获取在此过程中所对应的所有变换估计值。In some embodiments of the present application, the following methods may be used to register point cloud data from different perspectives. First, perform rough registration of point cloud data from different perspectives, and extract feature points between the point cloud data of two perspectives that have a common part. Such feature points can be explicit features such as straight lines, inflection points, and curve curvatures. It can also be a type of feature such as custom symbols, rotating graphics, and pivots. Then, the coordinate system of one of the point cloud data is transformed according to these characteristics, so that the point cloud data of the two viewing angles are in the same reference coordinate system, so as to obtain the transformation estimation value. Using the above method, the point cloud data of all the viewing angles are paired with rough registration until the point cloud data of all viewing angles are transformed into the same reference coordinate system, and all the corresponding transformation estimated values in this process are obtained.

对所有具有公共部分的点云数据完成粗糙配准之后，可以进一步进行精细配准。经过前一步粗糙配准之后，可以得到变换估计值，将此变换估计值作为初始值，在经过不断收敛与迭代的精细配准后，达到更加精准的效果。例如，以ICP(Iterative Closest Point，迭代最近点)算法为例，该算法首先计算初始点云上所有点与目标点云的距离，保证这些点和目标点云的最近点相互对应，同时构造残差平方和的目标函数。基于最小二乘法对目标函数进行最小化处理，经过反复迭代，直到均方误差小于设定的阈值，由此可以获得最优的变换估计值，以用于将所有视角的点云数据变换至同一参考坐标系内，从而获得目标物体的点云数据。After rough registration is completed for all point cloud data with common parts, fine registration can be further performed. After the rough registration in the previous step, the transformed estimated value can be obtained. The transformed estimated value is used as the initial value. After the fine registration of continuous convergence and iteration, a more accurate effect can be achieved. For example, take the ICP (Iterative Closest Point) algorithm as an example. The algorithm first calculates the distance between all points on the initial point cloud and the target point cloud to ensure that these points correspond to the closest points of the target point cloud, and at the same time construct the residual The objective function of the sum of squared differences. The objective function is minimized based on the least squares method. After repeated iterations, until the mean square error is less than the set threshold, the optimal transformation estimate can be obtained to transform the point cloud data of all perspectives to the same In the reference coordinate system, the point cloud data of the target object can be obtained.

步骤S202，根据所述目标物体的点云数据计算目标物体的面片数据，确定目标物体的初始几何模型。其中，所述面片数据用于表示由点云数据所表示的点所构成的面，在实际场景中，可以采用Marching Cube算法实现面片数据的计算，该算法首先将点云数据中八个位置相邻的点分别存放在一个四面体体元的八个顶点处。对于一个边界体素上一条棱边的两个端点而言，当其值一个大于给定的常数T，另一个小于T时，则这条棱边上一定有等值面的一个顶点。然后计算该体元中十二条棱和等值面的交点，并构造体元中的三角面片，所有的三角面片把体元分成了等值面内与等值面外两块区域。最后连接其中的所有体元的三角面片，构成等值面，合并所有立方体的等值面便可生成完整的三维表面，从而确定目标物体的初始几何模型。Step S202: Calculate the patch data of the target object according to the point cloud data of the target object, and determine the initial geometric model of the target object. Wherein, the patch data is used to represent the surface formed by the points represented by the point cloud data. In the actual scene, the Marching Cube algorithm can be used to calculate the patch data. The algorithm first combines eight points in the point cloud data. The adjacent points are stored at the eight vertices of a tetrahedron. For the two endpoints of an edge on a boundary voxel, when one of its values is greater than a given constant T and the other is less than T, there must be a vertex of an isosurface on this edge. Then calculate the intersection points of the twelve edges and the isosurface in the voxel, and construct the triangular faces in the voxel. All the triangular faces divide the voxel into two regions, namely the inner and outer isosurfaces. Finally, the triangular faces of all the voxels are connected to form an isosurface, and the isosurface of all the cubes can be combined to generate a complete three-dimensional surface, thereby determining the initial geometric model of the target object.

步骤S203，根据多个视角的色彩数据获得不同视角的纹理图片。Step S203: Obtain texture pictures of different viewing angles according to the color data of multiple viewing angles.

步骤S204，根据纹理图片的视角与初始几何模型之间的空间映射关系，确定所述纹理图片关于所述初始几何模型的相机位姿。由于生成纹理图片的色彩数据的视角与生成初始几何模型中相应的深度数据的视角是一一对应的，因此可以确定纹理图片的视角与初始几何模型之间的空间映射关系，并由此确定所述纹理图片关于所述初始几何模型的相机位姿。Step S204: Determine the camera pose of the texture picture with respect to the initial geometric model according to the spatial mapping relationship between the perspective of the texture picture and the initial geometric model. Since the perspective of the color data of the texture image is one-to-one corresponding to the perspective of the corresponding depth data in the initial geometric model, the spatial mapping relationship between the perspective of the texture image and the initial geometric model can be determined, and the resulting image can be determined accordingly. The texture picture is related to the camera pose of the initial geometric model.

在实际场景中，实际采集的深度数据和色彩数据的采集器是不同的，例如RGB数据是通过RGB摄像头采集，而深度数据是由深度摄像头采集，采集到的深度数据和色彩数据的坐标系会存在差异。因此，需要先将各个视角的色彩数据与对应视角的深度数据进行对其处理，例如实际场景中可以调用OpenNI库的GetAIternativeViewPointCap函数自动对齐色彩数据和深度数据。In actual scenes, the actual depth data and color data collectors are different. For example, RGB data is collected by an RGB camera, while depth data is collected by a depth camera. The coordinate system of the collected depth data and color data will be different. has a difference. Therefore, it is necessary to process the color data of each perspective and the depth data of the corresponding perspective first. For example, in the actual scene, the GetAIternativeViewPointCap function of the OpenNI library can be called to automatically align the color data and the depth data.

步骤S102，获取所述目标物体的完整几何模型。其中，所述完整几何模型是指通过其它方式获得的，包含没有点云数据缺失的几何模型。在实际场景中，所述完整几何模型可以从已有的模型库中下载获得，也可以是由用户通过人工方式设计获得。在通过人工方式设计获得完整几何模型时，可以采用各类三维设计程序根据目标物体的实际几何尺寸进行设计，从而获取目标物体的完整几何模型。此外，也可以通过扫描的方式获得目标物体的完整集合模型。Step S102: Obtain a complete geometric model of the target object. Wherein, the complete geometric model refers to a geometric model obtained by other means, including a geometric model without missing point cloud data. In an actual scene, the complete geometric model can be downloaded from an existing model library, or it can be manually designed by the user. When the complete geometric model is obtained by manual design, various three-dimensional design programs can be used to design according to the actual geometric size of the target object, so as to obtain the complete geometric model of the target object. In addition, a complete collection model of the target object can also be obtained by scanning.

步骤S103，将所述完整几何模型与所述初始几何模型进行配准，确定所述纹理图片关于完整几何模型的相机位姿。In step S103, the complete geometric model is registered with the initial geometric model, and the camera pose of the texture image with respect to the complete geometric model is determined.

在配准过程中，可以采用ICP或者其它变种的三维配准算法将所述完整几何模型与所述初始几何模型进行配准，以使所述完整几何模型替换所述初始几何模型，而后根据所述纹理图片关于所述初始几何模型的相机位姿，确定所述纹理图片关于替换后的完整几何模型的相机位姿。During the registration process, ICP or other variants of the three-dimensional registration algorithm can be used to register the complete geometric model with the initial geometric model, so that the complete geometric model replaces the initial geometric model, and then according to all Determining the camera pose of the texture picture relative to the initial geometric model, and determining the camera pose of the texture picture relative to the replaced complete geometric model.

步骤S104，根据多个视角的纹理图片以及所述纹理图片关于完整几何模型的相机位姿，对所述目标物体的完整几何模型进行纹理贴图，获得目标物体的三维模型。In step S104, texture mapping is performed on the complete geometric model of the target object according to the texture pictures of multiple perspectives and the camera pose of the texture picture with respect to the complete geometric model to obtain a three-dimensional model of the target object.

图3示出了采用本申请实施例中的方案实现一般物体三维重建时的处理过程，首先对通过扫描仪对一般物体310进行扫描，获得初始数据320。该初始数据包括两部分，即初始几何模型以及多视角的纹理图片以及相对初始几何模型的相机位姿。而后获取到完整几何模型330，并基于该完整几何模型进行配准处理340，该配准处理用于将完整几何模型配准到初始几何模型，并将其替换。在完成配准处理之后，即可将初始数据320更新为完整数据350，完整数据包括两部分，即完整几何模型以及多视角的纹理图片以及相对完整几何模型的相机位姿。最后，基于完整数据进行纹理贴图，获得具有纹理图片的完整三维模型360。FIG. 3 shows the processing procedure when the solution in the embodiment of the present application is used to realize the three-dimensional reconstruction of a general object. First, the general object 310 is scanned by a scanner to obtain initial data 320. The initial data includes two parts, namely the initial geometric model and the multi-view texture picture and the camera pose relative to the initial geometric model. Then the complete geometric model 330 is obtained, and a registration process 340 is performed based on the complete geometric model. The registration process is used to register the complete geometric model to the initial geometric model and replace it. After the registration process is completed, the initial data 320 can be updated to the complete data 350. The complete data includes two parts, namely, a complete geometric model, a multi-view texture image, and a camera pose of a relatively complete geometric model. Finally, texture mapping is performed based on the complete data to obtain a complete three-dimensional model 360 with texture pictures.

基于同一发明构思，本申请实施例中还提供了一种三维重建设备，所述设备对应的方法是前述实施例中的三维重建方法，并且其解决问题的原理与该方法相似。Based on the same inventive concept, an embodiment of the present application also provides a three-dimensional reconstruction device, the method corresponding to the device is the three-dimensional reconstruction method in the foregoing embodiment, and the principle of solving the problem is similar to the method.

本申请实施例提供的一种三维重建设备可以使用另外获取的完整几何模型与初始几何模型进行配准，即使初始几何模型因为物体表面的高光、透明等原因而导致残缺，可以由完整几何模型进行弥补，由此进行纹理贴图之后即可获得完整且具有真实尺度的三维模型。The three-dimensional reconstruction equipment provided by the embodiment of the application can use the complete geometric model obtained separately to register with the initial geometric model. Even if the initial geometric model is incomplete due to the high light and transparency of the surface of the object, it can be performed by the complete geometric model. Remedy, after texture mapping, a complete and true-scale three-dimensional model can be obtained.

在实际场景中，该三维重建设备可以是用户设备、网络设备或者用户设备与网络设备通过网络相集成所构成的设备。所述用户设备包括但不限于计算机、手机、平板电脑等各类终端设备；所述网络设备包括但不限于如网络主机、单个网络服务器、多个网络服务器集或基于云计算的计算机集合等实现。在此，云由基于云计算(Cloud Computing)的大量主机或网络服务器构成，其中，云计算是分布式计算的一种，由一群松散耦合的计算机集组成的一个虚拟计算机。In an actual scene, the three-dimensional reconstruction device may be user equipment, network equipment, or a device formed by integrating user equipment and network equipment through a network. The user equipment includes, but is not limited to, various terminal devices such as computers, mobile phones, and tablet computers; the network equipment includes, but is not limited to, implementations such as a network host, a single network server, a set of multiple network servers, or a set of computers based on cloud computing, etc. . Here, the cloud is composed of a large number of hosts or network servers based on Cloud Computing. Cloud computing is a type of distributed computing, a virtual computer composed of a group of loosely coupled computer sets.

图4示出了本申请实施提供的一种三维重建设备的结构，该设备至少包括扫描处理模块410、模型获取模块420、模型配准模块430和贴图处理模块440。其中，扫描处理模块410用于通过扫描设备对目标物体进行多视角扫描，获取目标物体的初始几何模型、多个视角的纹理图片以及所述纹理图片关于所述初始几何模型的相机位姿。模型获取模块420用于获取所述目标物体的完整几何模型。模型配准模块430用于将所述完整几何模型与所述初始几何模型进行配准，确定所述纹理图片关于完整几何模型的相机位姿。贴图处理模块440用于根据多个视角的纹理图片以及所述纹理图片关于完整几何模型的相机位姿，对所述目标物体的完整几何模型进行纹理贴图，获得目标物体的三维模型。FIG. 4 shows the structure of a three-dimensional reconstruction device provided by the implementation of this application. The device at least includes a scan processing module 410, a model acquisition module 420, a model registration module 430, and a texture processing module 440. Wherein, the scanning processing module 410 is used to scan a target object from multiple perspectives through a scanning device to obtain an initial geometric model of the target object, a texture picture of multiple viewing angles, and a camera pose of the texture picture with respect to the initial geometric model. The model obtaining module 420 is used to obtain a complete geometric model of the target object. The model registration module 430 is configured to register the complete geometric model with the initial geometric model, and determine the camera pose of the texture picture with respect to the complete geometric model. The texture processing module 440 is configured to perform texture mapping on the complete geometric model of the target object according to the texture pictures from multiple perspectives and the camera pose of the texture picture with respect to the complete geometric model to obtain a three-dimensional model of the target object.

本申请的一些实施例中，扫描处理模块410在根据多个视角的深度数据和色彩数据获取目标物体的初始几何模型、多个视角的纹理图片以及所述纹理图片关于所述初始几何模型的相机位姿时，可以采用如图2所示的处理流程，至少包括了以下的处理步骤：In some embodiments of the present application, the scanning processing module 410 obtains the initial geometric model of the target object, the texture image of the multiple perspectives, and the camera of the texture image with respect to the initial geometric model according to the depth data and color data of multiple viewing angles. For poses, the processing flow shown in Figure 2 can be used, which includes at least the following processing steps:

在实际场景中，通过扫描设备对目标物体进行多视角扫描时，环境因素难以处于完全理想的状态，因此扫描获得的深度数据中不可避免地会存在一些会对处理产生干扰的噪声数据。在进行后续处理之前，对这些噪声数据进行过滤，可以使得处理结果更加准确。在本申请的一些实施例中，可以先根据深度阈值确定多个视角的深度数据中的背景深度数据，并过滤所述背景深度数据，获得目标物体的本体深度数据。In actual scenes, when multi-view scanning of a target object is performed by a scanning device, environmental factors are difficult to be in a completely ideal state. Therefore, there will inevitably be some noise data in the depth data obtained by scanning that will interfere with the processing. Before performing subsequent processing, filtering the noise data can make the processing result more accurate. In some embodiments of the present application, the background depth data in the depth data of multiple viewing angles may be determined first according to the depth threshold, and the background depth data may be filtered to obtain the body depth data of the target object.

模型获取模块在获取所述目标物体的完整几何模型时，其所获取的完整几何模型是指通过其它方式获得的，包含没有点云数据缺失的几何模型。在实际场景中，所述完整几何模型可以从已有的模型库中下载获得，也可以是由用户通过人工方式设计获得。When the model obtaining module obtains the complete geometric model of the target object, the obtained complete geometric model refers to a geometric model obtained by other means, including a geometric model without missing point cloud data. In an actual scene, the complete geometric model can be downloaded from an existing model library, or it can be manually designed by the user.

模型配准模块在配准过程中，可以采用ICP或者其它变种的三维配准算法将所述完整几何模型与所述初始几何模型进行配准，以使所述完整几何模型替换所述初始几何模型，而后根据所述纹理图片关于所述初始几何模型的相机位姿，确定所述纹理图片关于替换后的完整几何模型的相机位姿。During the registration process, the model registration module may use ICP or other variants of three-dimensional registration algorithms to register the complete geometric model with the initial geometric model, so that the complete geometric model replaces the initial geometric model , And then determine the camera pose of the texture picture with respect to the replaced complete geometric model according to the camera pose of the texture picture with respect to the initial geometric model.

另外，本申请的一部分可被应用为计算机程序产品，例如计算机程序指令，当其被计算机执行时，通过该计算机的操作，可以调用或提供根据本申请的方法和/或技术方案。而调用本申请的方法的程序指令，可能被存储在固定的或可移动的记录介质中，和/或通过广播或其他信号承载媒体中的数据流而被传输，和/或被存储在根据程序指令运行的计算机设备的工作存储器中。在此，根据本申请的一些实施例包括一个如图5所示的计算设备，该设备包括存储有计算机可读指令的一个或多个存储器510和用于执行计算机可读指令的处理器520，其中，当该计算机可读指令被该处理器执行时，使得所述设备执行基于前述本申请的多个实施例的方法和/或技术方案。In addition, a part of this application can be applied as a computer program product, such as a computer program instruction, when it is executed by a computer, through the operation of the computer, the method and/or technical solution according to this application can be invoked or provided. The program instructions for invoking the method of the present application may be stored in a fixed or removable recording medium, and/or transmitted through data streams in broadcast or other signal-bearing media, and/or stored in accordance with the program In the working memory of the computer equipment on which the instructions are executed Here, some embodiments according to the present application include a computing device as shown in FIG. 5, which includes one or more memories 510 storing computer-readable instructions and a processor 520 for executing computer-readable instructions, Wherein, when the computer-readable instruction is executed by the processor, the device is caused to execute the method and/or technical solution based on the foregoing multiple embodiments of the present application.

此外，本申请的一些实施例还提供了一种计算机可读介质，其上存储有计算机程序指令，所述计算机可读指令可被处理器执行以实现前述本申请的多个实施例的方法和/或技术方案。In addition, some embodiments of the present application also provide a computer-readable medium on which computer program instructions are stored, and the computer-readable instructions can be executed by a processor to implement the methods and methods of the foregoing multiple embodiments of the present application. / Or technical solutions.

综上所述，本申请实施例提供方案中，首先通过扫描设备对目标物体进行多视角扫描，获取目标物体的初始几何模型、多个视角的纹理图片以及所述纹理图片关于所述初始几何模型的相机位姿，而后通过其它方式获取所述目标物体的完整几何模型，与所述初始几何模型进行配准之后，可以根据纹理图片以及纹理图片关于完整几何模型的相机位姿，对所述目标物体的完整几何模型进行纹理贴图，从而获得目标物体的三维模型。由于可以使用另外获取的完整几何模型与初始几何模型进行配准，即使初始几何模型因为物体表面的高光、透明等原因而导致残缺，可以由完整几何模型进行弥补，由此进行纹理贴图之后即可获得完整且具有真实尺度的三维模型。To sum up, in the solution provided in the embodiments of the present application, the target object is scanned from multiple perspectives by the scanning device to obtain the initial geometric model of the target object, the texture image of multiple perspectives, and the texture image relative to the initial geometric model. Then, the complete geometric model of the target object is obtained by other means, and after registration with the initial geometric model, the texture picture and the camera pose of the texture picture with respect to the complete geometric model can be compared to the target object. The complete geometric model of the object is subjected to texture mapping to obtain a three-dimensional model of the target object. Since the separately obtained complete geometric model can be used for registration with the initial geometric model, even if the initial geometric model is incomplete due to the high light and transparency of the object surface, it can be compensated by the complete geometric model, which can be used after texture mapping. Obtain a complete and true-scale three-dimensional model.

需要注意的是，本申请可在软件和/或软件与硬件的组合体中被实施，例如，可采用专用集成电路(ASIC)、通用目的计算机或任何其他类似硬件设备来实现。在一些实施例中，本申请的软件程序可以通过处理器执行以实现上文步骤或功能。同样地，本申请的软件程序(包括相关的数据结构)可以被存储到计算机可读记录介质中，例如，RAM存储器，磁或光驱动器或软磁盘及类似设备。另外，本申请的一些步骤或功能可采用硬件来实现，例如，作为与处理器配合从而执行各个步骤或功能的电路。It should be noted that this application can be implemented in software and/or a combination of software and hardware. For example, it can be implemented using an application specific integrated circuit (ASIC), a general purpose computer or any other similar hardware device. In some embodiments, the software program of the present application may be executed by a processor to realize the above steps or functions. Likewise, the software program (including related data structures) of the present application can be stored in a computer-readable recording medium, for example, RAM memory, magnetic or optical drives or floppy disks and similar devices. In addition, some steps or functions of the present application may be implemented by hardware, for example, as a circuit that cooperates with a processor to execute each step or function.

对于本领域技术人员而言，显然本申请不限于上述示范性实施例的细节，而且在不背离本申请的精神或基本特征的情况下，能够以其他的具体形式实现本申请。因此，无论从哪一点来看，均应将实施例看作是示范性的，而且是非限制性的，本申请的范围由所附权利要求而不是上述说明限定，因此旨在将落在权利要求的等同要件的含义和范围内的所有变化涵括在本申请内。不应将权利要求中的任何附图标记视为限制所涉及的权利要求。此外，显然“包括”一词不排除其他单元或步骤，单数不排除复数。装置权利要求中陈述的多个单元或装置也可以由一个单元或装置通过软件或者硬件来实现。第一，第二等词语用来表示名称，而并不表示任何特定的顺序。For those skilled in the art, it is obvious that the present application is not limited to the details of the foregoing exemplary embodiments, and the present application can be implemented in other specific forms without departing from the spirit or basic characteristics of the application. Therefore, no matter from which point of view, the embodiments should be regarded as exemplary and non-limiting. The scope of this application is defined by the appended claims rather than the above description, and therefore it is intended to fall into the claims. All changes in the meaning and scope of the equivalent elements of are included in this application. Any reference signs in the claims should not be regarded as limiting the claims involved. In addition, it is obvious that the word "including" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices stated in the device claims can also be implemented by one unit or device through software or hardware. Words such as first and second are used to denote names, but do not denote any specific order.

Claims

一种三维重建方法，其中，该方法包括：A three-dimensional reconstruction method, wherein the method includes:

通过扫描设备对目标物体进行多视角扫描，获取目标物体的初始几何模型、多个视角的纹理图片以及所述纹理图片关于所述初始几何模型的相机位姿，其中，所述多个视角的纹理图片至少覆盖所述目标物体的表面；Multi-view scanning of the target object is performed by the scanning device to obtain the initial geometric model of the target object, the texture picture of the multiple viewing angles, and the camera pose of the texture picture with respect to the initial geometric model, wherein the texture of the multiple viewing angles The picture covers at least the surface of the target object;

获取所述目标物体的完整几何模型；Acquiring a complete geometric model of the target object;

将所述完整几何模型与所述初始几何模型进行配准，确定所述纹理图片关于完整几何模型的相机位姿；Registering the complete geometric model with the initial geometric model, and determining the camera pose of the texture picture with respect to the complete geometric model;

根据多个视角的纹理图片以及所述纹理图片关于完整几何模型的相机位姿，对所述目标物体的完整几何模型进行纹理贴图，获得目标物体的三维模型。Perform texture mapping on the complete geometric model of the target object according to texture pictures from multiple perspectives and the camera pose of the texture picture with respect to the complete geometric model to obtain a three-dimensional model of the target object.
根据权利要求1所述的方法，其中，通过扫描设备对目标物体进行多视角扫描，获取目标物体的初始几何模型、多个视角的纹理图片以及所述纹理图片关于所述初始几何模型的相机位姿，包括：The method according to claim 1, wherein the target object is scanned from multiple perspectives by a scanning device to obtain an initial geometric model of the target object, a texture picture of multiple perspectives, and a camera position of the texture picture with respect to the initial geometric model. Posture, including:

通过扫描设备对目标物体进行多视角扫描，获取关于所述目标物体的多个视角的深度数据和色彩数据；Perform multi-view scanning of the target object by the scanning device, and obtain depth data and color data of multiple views of the target object;

根据多个视角的深度数据和色彩数据获取目标物体的初始几何模型、多个视角的纹理图片以及所述纹理图片关于所述初始几何模型的相机位姿。The initial geometric model of the target object, the texture picture of the multiple viewing angles, and the camera pose of the texture picture with respect to the initial geometric model are acquired according to the depth data and color data of multiple viewing angles.
根据权利要求2所述的方法，其中，根据多个视角的深度数据和色彩数据获取目标物体的初始几何模型、多个视角的纹理图片以及所述纹理图片关于所述初始几何模型的相机位姿，包括：The method according to claim 2, wherein the initial geometric model of the target object, the texture picture of the multiple perspectives, and the camera pose of the texture picture with respect to the initial geometric model are acquired according to the depth data and color data of multiple viewing angles. ,include:

根据多个视角的深度数据获得不同视角的点云数据，并对不同视角的点云数据进行配准，获得目标物体的点云数据；Obtain point cloud data of different perspectives according to the depth data of multiple perspectives, and register the point cloud data of different perspectives to obtain the point cloud data of the target object;

根据所述目标物体的点云数据计算目标物体的面片数据，确定目标物体的初始几何模型；Calculating the patch data of the target object according to the point cloud data of the target object, and determining the initial geometric model of the target object;

根据多个视角的色彩数据获得不同视角的纹理图片；Obtain texture pictures of different viewing angles according to the color data of multiple viewing angles;

根据纹理图片的视角与初始几何模型之间的空间映射关系，确定所述纹理图片关于所述初始几何模型的相机位姿。According to the spatial mapping relationship between the perspective of the texture picture and the initial geometric model, the camera pose of the texture picture with respect to the initial geometric model is determined.
根据权利要求3所述的方法，其中，根据多个视角的深度数据获得不同视角的点云数据，包括：The method according to claim 3, wherein obtaining point cloud data of different viewing angles according to the depth data of multiple viewing angles comprises:

根据深度阈值确定多个视角的深度数据中的背景深度数据，并过滤所述背景深度数据，获得目标物体的本体深度数据；Determine the background depth data in the depth data of multiple viewing angles according to the depth threshold, and filter the background depth data to obtain the body depth data of the target object;

根据多个视角的本体深度数据获得不同视角的点云数据。Obtain point cloud data from different perspectives according to the depth data of the ontology from multiple perspectives.
根据权利要求1所述的方法，其中，通过扫描设备对目标物体进行多视角扫描，包括：The method according to claim 1, wherein the multi-view scanning of the target object by the scanning device comprises:

通过扫描设备对目标物体进行至少6个视角的扫描，以使所述至少6个视角的纹理图片至少覆盖所述目标物体的表面。Scanning the target object from at least 6 viewing angles by the scanning device, so that the texture image of the at least 6 viewing angles at least covers the surface of the target object.
根据权利要求1所述的方法，其中，获取所述目标物体的完整几何模型，包括：The method according to claim 1, wherein obtaining a complete geometric model of the target object comprises:

从已有的模型库中下载获取所述目标物体的完整几何模型，或通过人工方式设计获取所述目标物体的完整几何模型，或通过扫描获得目标物体的完整几何模型。Download and obtain the complete geometric model of the target object from an existing model library, or obtain the complete geometric model of the target object through manual design, or obtain the complete geometric model of the target object through scanning.
根据权利要求1所述的方法，其中，将所述完整几何模型与所述初始几何模型进行配准，确定所述纹理图片关于完整几何模型的相机位姿，包括：The method according to claim 1, wherein registering the complete geometric model with the initial geometric model to determine the camera pose of the texture picture with respect to the complete geometric model comprises:

将所述完整几何模型与所述初始几何模型进行配准，以使所述完整几何模型替换所述初始几何模型；Registering the complete geometric model with the initial geometric model, so that the complete geometric model replaces the initial geometric model;

根据所述纹理图片关于所述初始几何模型的相机位姿，确定所述纹理图片关于替换后的完整几何模型的相机位姿。According to the camera pose of the texture picture with respect to the initial geometric model, the camera pose of the texture picture with respect to the replaced complete geometric model is determined.
一种三维重建设备，其中，该设备包括：A three-dimensional reconstruction device, wherein the device includes:

扫描处理模块，用于通过扫描设备对目标物体进行多视角扫描，获取目标物体的初始几何模型、多个视角的纹理图片以及所述纹理图片关于所述初始几何模型的相机位姿，其中，所述多个视角的纹理图片至少覆盖所述目标物体的表面；The scanning processing module is used to perform multi-view scanning of the target object through the scanning device, to obtain the initial geometric model of the target object, the texture picture of multiple views, and the camera pose of the texture picture with respect to the initial geometric model, wherein The texture pictures of the multiple viewing angles cover at least the surface of the target object;

模型获取模块，用于获取所述目标物体的完整几何模型；A model acquisition module for acquiring a complete geometric model of the target object;

模型配准模块，用于将所述完整几何模型与所述初始几何模型进行配准，确定所述纹理图片关于完整几何模型的相机位姿；A model registration module, configured to register the complete geometric model with the initial geometric model, and determine the camera pose of the texture picture with respect to the complete geometric model;

贴图处理模块，用于根据多个视角的纹理图片以及所述纹理图片关于完整几何模型的相机位姿，对所述目标物体的完整几何模型进行纹理贴图，获得目标物体的三维模型。The texture processing module is used to perform texture mapping on the complete geometric model of the target object according to texture pictures from multiple perspectives and the camera pose of the texture picture with respect to the complete geometric model to obtain a three-dimensional model of the target object.
根据权利要求8所述的设备，其中，所述扫描处理模块，用于通过扫描设备对目标物体进行多视角扫描，获取关于所述目标物体的多个视角的深度数据和色彩数据；根据多个视角的深度数据和色彩数据获取目标物体的初始几何模型、多个视角的纹理图片以及所述纹理图片关于所述初始几何模型的相机位姿。8. The device according to claim 8, wherein the scanning processing module is configured to scan a target object from multiple perspectives by a scanning device to obtain depth data and color data from multiple perspectives of the target object; The depth data and the color data of the viewing angle obtain the initial geometric model of the target object, the texture pictures of multiple viewing angles, and the camera pose of the texture picture with respect to the initial geometric model.
根据权利要求9所述的设备，其中，所述扫描处理模块，用于根据多个视角的深度数据获得不同视角的点云数据，并对不同视角的点云数据进行配准，获得目标物体的点云数据；根据所述目标物体的点云数据计算目标物体的面片数据，确定目标物体的初始几何模型；根据多个视角的色彩数据获得不同视角的纹理图片；以及根据纹理图片的视角与初始几何模型之间的空间映射关系，确定所述纹理图片关于所述初始几何模型的相机位姿。The device according to claim 9, wherein the scanning processing module is configured to obtain point cloud data of different perspectives according to the depth data of multiple perspectives, and to register the point cloud data of different perspectives to obtain the target object Point cloud data; calculate the patch data of the target object according to the point cloud data of the target object to determine the initial geometric model of the target object; obtain texture pictures of different perspectives according to the color data of multiple perspectives; and according to the perspective and The spatial mapping relationship between the initial geometric models determines the camera pose of the texture picture with respect to the initial geometric model.
根据权利要求10所述的设备，其中，所述扫描处理模块，用于根据深度阈值确定多个视角的深度数据中的背景深度数据，并过滤所述背景深度数据，获得目标物体的本体深度数据；以及根据多个视角的本体深度数据获得不同视角的点云数据。The device according to claim 10, wherein the scanning processing module is configured to determine the background depth data in the depth data of the multiple viewing angles according to the depth threshold, and filter the background depth data to obtain the body depth data of the target object ; And according to the depth data of the ontology of multiple perspectives to obtain point cloud data of different perspectives.
根据权利要求8所述的设备，其中，所述扫描处理模块，用于通过扫描设备对目标物体进行至少6个视角的扫描，以使所述至少6个视角的纹理图片至少覆盖所述目标物体的表面。8. The device according to claim 8, wherein the scanning processing module is configured to scan the target object in at least 6 viewing angles through the scanning device, so that the texture image of the at least 6 viewing angles at least covers the target object s surface.
根据权利要求8所述的设备，其中，所述模型获取模块，用于从已有的模型库中下载获取所述目标物体的完整几何模型，或通过人工方式设计获取所述目标物体的完整几何模型，或通过扫描获得目标物体的完整几何模型。The device according to claim 8, wherein the model acquisition module is configured to download and acquire the complete geometric model of the target object from an existing model library, or obtain the complete geometric model of the target object through manual design Model, or obtain a complete geometric model of the target object through scanning.
根据权利要求8所述的设备，其中，所述模型配准模块，用于将所述完整几何模型与所述初始几何模型进行配准，以使所述完整几何模型替换所述初始几何模型；以及根据所述纹理图片关于所述初始几何模型的相机位姿，确定所述纹理图片关于替换后的完整几何模型的相机位姿。8. The device according to claim 8, wherein the model registration module is configured to register the complete geometric model with the initial geometric model, so that the complete geometric model replaces the initial geometric model; And according to the camera pose of the texture picture with respect to the initial geometric model, the camera pose of the texture picture with respect to the replaced complete geometric model is determined.
一种计算装置，其中，该设备包括用于存储计算机程序指令的存储器和用于执行计算机程序指令的处理器，其中，当该计算机程序指令被该处理器执行时，触发所述设备执行权利要求1至7中任一项所述的方法。A computing device, wherein the device includes a memory for storing computer program instructions and a processor for executing the computer program instructions, wherein when the computer program instructions are executed by the processor, the device is triggered to execute the claims The method of any one of 1 to 7.
一种计算机可读介质，其上存储有计算机程序指令，所述计算机可读指令可被处理器执行以实现如权利要求1至7中任一项所述的方法。A computer-readable medium having computer program instructions stored thereon, and the computer-readable instructions can be executed by a processor to implement the method according to any one of claims 1 to 7.