WO2020151078A1 - 一种三维重建的方法和装置 - Google Patents

一种三维重建的方法和装置 Download PDF

Info

Publication number
WO2020151078A1
WO2020151078A1 PCT/CN2019/079686 CN2019079686W WO2020151078A1 WO 2020151078 A1 WO2020151078 A1 WO 2020151078A1 CN 2019079686 W CN2019079686 W CN 2019079686W WO 2020151078 A1 WO2020151078 A1 WO 2020151078A1
Authority
WO
WIPO (PCT)
Prior art keywords
position point
reconstruction
images
reconstruction space
space
Prior art date
Application number
PCT/CN2019/079686
Other languages
English (en)
French (fr)
Inventor
王淞
张锋
于新然
Original Assignee
北京极智无限科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 北京极智无限科技有限公司 filed Critical 北京极智无限科技有限公司
Priority to US17/250,707 priority Critical patent/US11954832B2/en
Priority to JP2021540143A priority patent/JP7398819B2/ja
Publication of WO2020151078A1 publication Critical patent/WO2020151078A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/50—Image enhancement or restoration using two or more images, e.g. averaging or subtraction
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/50—Depth or shape recovery
    • G06T7/55—Depth or shape recovery from multiple images
    • G06T7/564—Depth or shape recovery from multiple images from contours
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/50—Depth or shape recovery
    • G06T7/55—Depth or shape recovery from multiple images
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30204—Marker
    • G06T2207/30208—Marker matrix

Definitions

  • This application relates to the technical field of image analysis and processing, and in particular to a method and device for three-dimensional reconstruction.
  • 3D Reconstruction International: 3D Reconstruction; abbreviated: 3D reconstruction
  • 3D reconstruction The current mainstream 3D reconstruction technology mainly includes direct 3D reconstruction technology and indirect 3D reconstruction technology.
  • the direct 3D reconstruction technology refers to the actual measurement of the 3D shape and 3D structure of the object by using a special measuring device with laser ranging;
  • the indirect 3D reconstruction technology refers to the use of images of different angles of the object to extract the image
  • the 3D shape and 3D structure of the object are finally generated by matching the feature points on the feature points.
  • the technical problem to be solved by this application is to provide a three-dimensional reconstruction method and device, which can quickly complete three-dimensional reconstruction of objects entering the reconstruction space in real time, which is extremely suitable for scenes that require real-time three-dimensional reconstruction.
  • an embodiment of the present application provides a three-dimensional reconstruction method, which includes:
  • N is a positive integer greater than or equal to 2, and the field of view of the camera covers the reconstruction space;
  • the front background difference corresponding to each position point of the N current images is obtained, and the N initial background images are obtained when there is no object in the reconstruction space. Images taken by the N cameras in the reconstruction space;
  • the front background difference corresponding to each position point of the N current images is correspondingly merged to obtain the corresponding position point of the reconstruction space.
  • each position point in the reconstruction space is selected to reconstruct the object in three dimensions.
  • each position point of the N current images and each position point of the reconstruction space is determined by the position information and angle information of the N cameras fixedly arranged at the edge of the reconstruction space.
  • the obtaining the front background difference corresponding to each position point of the N current images according to the N current images and the corresponding N initial background images includes:
  • the corresponding pixel value of each position point of the N initial background image and the difference function obtain the front background difference value corresponding to each position point of the N current image .
  • the difference function is determined according to a Gaussian mixture model modeling; or, the difference function is determined according to a vector distance formula.
  • the vector distance formula includes Euclidean distance formula, Manhattan distance formula or cosine distance formula.
  • each position point of the N current images correspondingly fuse the front background difference corresponding to each position point of the N current images to obtain the reconstruction
  • the front background difference corresponding to each position point in the space including:
  • the front background difference corresponding to each position point in the reconstruction space is obtained.
  • the fusion function includes an addition function or a multiplication function.
  • the boundary of the reconstruction space adopts a deep and shallow texture.
  • Optional also includes:
  • an embodiment of the present application provides a three-dimensional reconstruction device, which includes:
  • An acquiring unit configured to acquire, when an object enters the reconstruction space, images taken by N cameras in the reconstruction space as N current images, where N is a positive integer greater than or equal to 2, and the field of view of the camera covers the reconstruction space;
  • the first obtaining unit is configured to obtain the front background difference corresponding to each position point of the N current images according to the N current images and the corresponding N initial background images, and the N initial background images are the Images taken by the N cameras in the reconstruction space when there is no object in the reconstruction space;
  • the second obtaining unit is configured to correspondingly fuse the front background difference corresponding to each position point of the N current images based on the corresponding relationship between each position point of the N current images and each position point of the reconstruction space to obtain the The front and background difference corresponding to each position point in the reconstruction space;
  • the reconstruction unit is configured to filter each position point in the reconstruction space to reconstruct the object in three dimensions according to the front background difference corresponding to each position point in the reconstruction space and a preset front background threshold.
  • an embodiment of the present application provides a terminal device, which includes a processor and a memory:
  • the memory is used to store program code and transmit the program code to the processor
  • the processor is configured to execute the three-dimensional reconstruction method according to any one of the foregoing first aspects according to instructions in the program code.
  • an embodiment of the present application provides a computer-readable storage medium, the computer-readable storage medium is used to store program code, and the program code is used to execute the three-dimensional reconstruction method of any one of the above-mentioned first aspects .
  • the captured images of the N cameras covering the reconstruction space are acquired as N current images; then, according to the N current images and the corresponding N
  • the initial background image obtains the front background difference corresponding to each position of the N current image.
  • the N initial background images are the images taken by the N cameras in the reconstruction space when there are no objects in the reconstruction space; secondly, based on the position points of the N current images Corresponding relationship with each position point in the reconstruction space, correspondingly merge the front background difference corresponding to each position point of the N current image to obtain the front background difference corresponding to each position point in the reconstruction space; finally, according to the front background corresponding to each position point in the reconstruction space
  • the difference and the preset front background threshold filter the three-dimensional reconstruction objects at various locations in the reconstruction space. It can be seen that when the object enters the reconstruction space, the front and background separation of the three-dimensional reconstruction object is carried out through the N current images and the corresponding N initial background images.
  • the front background separation operation is relatively simple and convenient, and the three-dimensional object entering the reconstruction space can be completed quickly and in real time. Reconstruction, this method is extremely suitable for scenes that require real-time 3D reconstruction.
  • FIG. 1 is a schematic diagram of a system framework involved in an application scenario in an embodiment of this application;
  • FIG. 2 is a schematic flowchart of a three-dimensional reconstruction method provided by an embodiment of the application
  • FIG. 3 is a schematic diagram of a reconstruction space and N cameras structure provided by an embodiment of the application.
  • FIG. 4 is a schematic diagram of a three-dimensional reconstruction plane of a pure black cylinder in a pure white reconstruction space provided by an embodiment of the application;
  • Figure 5 is a black and white dark and light texture provided by an embodiment of the application.
  • FIG. 6 is a schematic structural diagram of a three-dimensional reconstruction device provided by an embodiment of the application.
  • the mainstream 3D reconstruction technology mainly includes two kinds, one is the direct 3D reconstruction technology, which actually measures the 3D shape and 3D structure of the object by using a special measuring device with laser ranging; the other is the indirect 3D reconstruction technology , By taking images of objects from different angles, using algorithms to extract feature points on the images, and finally generating the 3D shape and 3D structure of the object through the comparison and matching of feature points.
  • the direct 3D reconstruction technology takes a lot of time to perform a comprehensive laser ranging of the object, and the indirect 3D reconstruction technology compares the feature points in the object images at different angles with more complicated calculations and a large amount of calculation.
  • 3D reconstruction it takes several minutes or even hours to complete a 3D reconstruction, which is not suitable for scenes that require real-time 3D reconstruction, for example, scenes that need to capture human movements in real time, such as smart vending machines to determine the actions or actions of customers taking goods.
  • Production equipment checks whether workers' operations comply with safety regulations, etc.; another example is scenes that need to capture object movement in real time, such as detecting the transportation of parts and accessories on a factory production line.
  • the captured images of the N cameras covering the reconstruction space are obtained as N current images; then, according to the N current images and The corresponding N initial background images obtain the front background difference corresponding to each position of the N current images.
  • the N initial background images are the images taken by N cameras in the reconstruction space when there are no objects in the reconstruction space; secondly, based on the N current images Correspondence between each position point of the image and each position point in the reconstruction space, correspondingly fuse the front background difference corresponding to each position point of the N current images to obtain the front background difference corresponding to each position point in the reconstruction space; finally, according to each position point in the reconstruction space
  • the corresponding front background difference value and the preset front background threshold filter the three-dimensional reconstruction objects at various position points in the reconstruction space. It can be seen that when the object enters the reconstruction space, the front and background separation of the three-dimensional reconstruction object is carried out through the N current images and the corresponding N initial background images.
  • the front background separation operation is relatively simple and convenient, and the three-dimensional object entering the reconstruction space can be completed quickly and in real time. Reconstruction, this method is extremely suitable for scenes that require real-time 3D reconstruction.
  • the scenario includes N cameras 101, a processor 102, and a user terminal 103, where N is greater than or equal to 2.
  • N is greater than or equal to 2.
  • the field of view of the camera covers the reconstruction space.
  • N cameras 101 photograph the reconstructed space to obtain the captured image and send it to the processor 102, which uses it as N initial background images.
  • the N cameras 101 photograph the reconstruction space to obtain the captured images and send them to the processor 102, and the processor 102 uses them as N current images.
  • the processor 102 obtains the front background difference corresponding to each position point of the N current images according to the N current images and the corresponding N initial background images.
  • the processor 102 correspondingly fuses the front background difference values corresponding to each position point of the N current images based on the corresponding relationship between each position point of the N current image and each position point of the reconstruction space, and obtains the front background difference value corresponding to each position point of the reconstruction space.
  • the processor 102 screens the 3D reconstructed objects at each position point in the reconstruction space according to the front background difference corresponding to each position point in the reconstruction space and the preset front background threshold, and obtains the 3D reconstructed image of the object and sends it to the user terminal 103.
  • the user terminal 103 displays the 3D reconstruction of the object Image to the user.
  • FIG. 2 shows a schematic flowchart of a three-dimensional reconstruction method in an embodiment of the present application.
  • the method may include the following steps, for example:
  • Step 201 When an object enters the reconstruction space, acquire images taken by N cameras of the reconstruction space as N current images, where N is a positive integer greater than or equal to 2, and the field of view of the camera covers the reconstruction space.
  • the reconstruction space refers to the space where the entering object needs to be reconstructed in three dimensions.
  • the shape of the reconstruction space is not limited, and it can be a square, a cylinder, or a sphere.
  • the prerequisite for implementing the embodiments of this application is that the cameras need to be arranged at the edge of the reconstruction space. Assuming that N cameras are arranged, and N is a positive integer greater than or equal to 2, the N cameras should be scattered at the edge of the reconstruction space, and the lens of each camera faces the reconstruction. Inside the space, and the view covers the entire reconstruction space. In addition, in the implementation of this application, there are no other rigid restrictions on the number of cameras and the location of the cameras.
  • the number of cameras will not affect or change the operation and calculation of the subsequent steps.
  • all N cameras need to be fixed. During the application process, it must be ensured that each camera does not move or rotate.
  • the reconstruction space and the structure diagram of N cameras For example, as shown in Figure 3, the reconstruction space and the structure diagram of N cameras. If the reconstruction space is a cube, 4 cameras can be arranged at the midpoints of its specific four edges, and N is 4; or 8 cameras are arranged at its eight vertices, and N is 8. If the reconstruction space is a cylinder, 4 cameras that are staggered from each other can be arranged on the top and bottom surfaces, and N is 4.
  • N cameras capture the reconstruction space to obtain N captured images, and the processor obtains the above N captured images as N current images.
  • the N current images all include partial images of the object, which are used as foreground images, and the remaining partial images are used as background images.
  • Step 202 Obtain the front background difference corresponding to each position point of the N current images according to the N current images and the corresponding N initial background images, where the N initial background images do not exist in the reconstruction space
  • the object is an image taken by the N cameras in the reconstruction space.
  • the N initial background images are actually after the N cameras are arranged and fixed.
  • the N cameras are started to capture the reconstruction space to obtain N captured images, and the processor obtains N captured images. of.
  • the processor obtains N captured images. of.
  • N initial background images are stored in the processor, and each initial background image does not have any partial image of the object entering the reconstruction space, only the background image of the pure reconstruction space.
  • each initial background image in the N initial background images does not have any partial images of objects that enter the reconstruction space, that is, all partial images in each initial background image belong to background images.
  • the current image can be obtained by comparing the current image with the corresponding initial background image. The difference between the front and background corresponding to each position point is used to separate the foreground image from the current image for subsequent three-dimensional reconstruction of the object.
  • the difference between the pixel value of the current image and the corresponding initial background image can be used to represent the current image
  • the front background difference corresponding to the position point.
  • the pixel value difference can be obtained as the front background difference value by using a predefined difference function.
  • the larger the front background difference value the greater the probability that the corresponding position point belongs to the foreground image, the smaller the front background difference value, the greater the probability that the corresponding position point belongs to the background image. Therefore, in some implementations of the embodiments of the present application, the step 202 may include the following steps, for example:
  • Step A Obtain the pixel value of each position point of the N current images and the corresponding pixel value of each position point of the N initial background images;
  • Step B According to the pixel value of each position point of the N current images, the corresponding pixel value of each position point of the N initial background image, and the difference function, obtain the previous corresponding to each position point of the N current image Background difference.
  • the pre-defined difference function can be various, and in the application embodiment, two ways can be used to determine the defined difference function.
  • the first is more classic, the difference function can be determined according to the Gaussian mixture model modeling; the second, the pixel value can be regarded as a 3-dimensional vector, and the difference function can be determined according to the classic vector distance formula.
  • the common vector distance formulas are Euclidean distance formula, Manhattan distance formula and cosine distance formula, etc.
  • the difference function can be determined by selecting any one of the distance formulas.
  • the difference function is determined according to the Gaussian mixture model modeling; or, the difference function is determined according to a vector distance formula; the vector distance formula includes Euclidean Distance formula, Manhattan distance formula or cosine distance formula.
  • the front background difference Diff corresponding to each position point of the current image is:
  • the current position of the image points corresponding to respective front of the background difference Diff: Diff
  • the front background difference Diff corresponding to each position point of the current image is:
  • Step 203 Based on the correspondence between each position point of the N current images and each position point of the reconstruction space, correspondingly fuse the front background difference corresponding to each position point of the N current images to obtain each position of the reconstruction space The difference of the front background corresponding to the point.
  • step 202 After obtaining the front background difference corresponding to each position point of the N current image in step 202, which indicates the probability of each position point of the N current image belonging to the foreground image, it is necessary to obtain the front background difference corresponding to each position point in the reconstruction space.
  • the background difference is used to determine the probability that each position point in the reconstruction space belongs to the object.
  • the step 203 may include the following steps, for example:
  • Step C Based on the corresponding relationship between each position point of the N current images and each position point of the reconstruction space, determine the corresponding position of the N current image position points corresponding to each position point in the reconstruction space Front background difference;
  • Step D Obtain the front background difference corresponding to each position point in the reconstruction space according to the front background difference and the fusion function corresponding to the N current image position points corresponding to each position point in the reconstruction space .
  • the pre-defined fusion function be R
  • Diff 3D R(Diff 1 ,Diff 2 ,... ., Diff N ).
  • the front background difference corresponding to the N current image location points corresponding to each location point in the reconstruction space is first based on the correspondence between each location point of the N current image and each location point in the reconstruction space The relationship determines the corresponding position point of each position point in the reconstruction space in the N current images, and then determines its corresponding front background difference.
  • N current images are taken by N cameras fixedly arranged at the edge of the reconstruction space, the position information and angle information of the N cameras are fixed.
  • the principle of camera imaging is to project different points in the reconstruction space to An imaging plane generates a captured image, (x, y, z) represents a position point in the reconstruction space.
  • the position corresponding to the above position point on the current image corresponding to the camera can be obtained Point (u, v), that is, the position information and angle information of the camera can determine the correspondence between the current image position point (u, v) and the reconstruction space position point (x, y, z). Therefore, in some implementations of the embodiments of the present application, the corresponding relationship between each position point of the N current images and each position point of the reconstruction space is determined by the N cameras fixedly arranged at the edge of the reconstruction space. Location information and angle information are determined.
  • the position point A and the position point B on the same ray entering the camera correspond to the same position point on the current image corresponding to the camera, if (x A ,y A ,z A ) Represents the position point A in the reconstruction space, if let (x B , y B , z B ) denote the position point B in the reconstruction space, then the (u 1 , v 1 ) position point in the current image corresponding to the camera is obtained Corresponding to the location point A and location point B in the reconstruction space.
  • the pre-defined fusion function can also be various. Since the function of the fusion function is to fuse the front background difference corresponding to the N current image position points corresponding to each position point in the reconstruction space, the front background difference corresponding to the N current image position points can be added together. Fusion, you can also take the multiplication method for fusion.
  • the following two functions can be used as the fusion function. The first is the addition function, and the second is the multiplication function. Any one of the above two functions can be used as the fusion function. Therefore, in some implementations of the embodiments of the present application, the fusion function includes an addition function or a multiplication function.
  • the front background difference Diff 3D corresponding to each position point in the reconstruction space is
  • the front background difference Diff 3D corresponding to each position point in the reconstruction space is
  • Step 204 According to the front background difference corresponding to each position point in the reconstruction space and a preset front background threshold, select each position point in the reconstruction space to reconstruct the object in three dimensions.
  • each position point in the reconstruction space can indicate the probability of each position point belonging to the object, that is, the larger the front background difference corresponding to each position point in the reconstruction space, it means that each position point belongs to The greater the probability of the object.
  • each position point in the reconstruction space can be filtered by a preset front background threshold. For example, if the front background difference value corresponding to a certain position point in the reconstruction space is greater than or equal to the preset The front background threshold is considered to belong to the object, and the object can be reconstructed in three dimensions by filtering out the position points belonging to the object in each position point in the reconstruction space.
  • the space boundary reconstruction texture pure white
  • the front background difference Diff can be calculated as follows: If the current image location point belongs to the background image, the corresponding front background difference Diff can be calculated as follows: As shown in FIG. 4, a schematic diagram of a three-dimensional reconstruction plane of a pure black cylinder in a pure white reconstruction space. For the convenience of description, a top view is used to perform three-dimensional reconstruction on a plane. Among them, the reconstruction space as shown on the left is a cube. The top view of the reconstruction space is divided into four vertices with 4 cameras fixedly arranged.
  • the area between the two rays emitted by each camera in the figure is an object, and the rest is the background.
  • the position point in the figure is located between the two rays of a certain camera, and the front background difference Diff corresponding to the current image position point corresponding to this camera is 441, otherwise it is 0.
  • the front background difference Diff 3D corresponding to each position point in the reconstruction space is
  • the preset front background threshold is 1764, then compare the Diff 3D of each location point in area 1, area 2 and area 3 with the size of the preset front background threshold 1764, if the Diff 3D of a certain location point is greater than or equal to the preset front background
  • the threshold 1764 considers that the location point belongs to the object, and each location point in the area 3 belongs to the object, and the polygon shown in the area 3 is obtained by three-dimensional reconstruction. It should be noted that since the entering object is actually a cylinder, its top view is circular, and the shape constructed by the 3D reconstruction is a polygon with a certain error.
  • the shape of the three-dimensional reconstruction structure is closer to a circle, that is, the more cameras arranged, the subsequent usable The more data, the higher the accuracy of the 3D reconstruction.
  • the texture color of the outer surface of the object entering the reconstruction space is uncertain, and it can be either a dark texture or a light texture.
  • the pair of objects on the outer surface of dark texture and the initial background image of light texture is relatively strong, and the pair of objects on the outer surface of light texture and the initial background image of dark texture are relatively strong.
  • the outer surface of the object can also be reconstructed by using dark and light texture layouts, that is, the reconstruction space boundary can also use dark and light textures to achieve better separation of the front background. effect. Therefore, in some implementation manners of the embodiments of the present application, the reconstruction space boundary adopts a deep and shallow texture. For example, the black and white dark and light texture arrangement as shown in FIG. 5 is used to reconstruct the space boundary.
  • the initial background image obtained in advance is still suitable for the reconstruction space, and the initial background image does not need to be updated and can be used all the time. If the illumination of the reconstruction space changes and the initial background image obtained in advance is no longer applicable to the reconstruction space after the illumination changes, the initial background image needs to be updated.
  • the principle of updating the initial background image is to update the initial background image by judging whether there is an object in the current reconstruction space. If it does not exist, each camera will take another shot of the reconstruction space to obtain a captured image to update the original background image. Therefore, in some implementations of the embodiments of the present application, for example, it may further include step E: when the illumination changes and there are no objects in the reconstruction space, acquiring the updated information of the captured images of the N cameras in the reconstruction space Describe N initial background images.
  • the captured images of the N cameras covering the reconstruction space are acquired as N current images; then, according to the N current images and the corresponding N initial background images obtain the front background difference corresponding to each position of the N current images.
  • the N initial background images are the images taken by N cameras in the reconstruction space when there are no objects in the reconstruction space; secondly, based on each of the N current images The corresponding relationship between the position point and each position point in the reconstruction space, corresponding to the fusion of the front background difference corresponding to each position point of the N current images to obtain the front background difference corresponding to each position point in the reconstruction space; finally, according to the corresponding position point in the reconstruction space
  • the front background difference and the preset front background threshold filter the three-dimensional reconstruction objects at various locations in the reconstruction space. It can be seen that when the object enters the reconstruction space, the front and background separation of the three-dimensional reconstruction object is carried out through the N current images and the corresponding N initial background images.
  • the front background separation operation is relatively simple and convenient, and the three-dimensional object entering the reconstruction space can be completed quickly and in real time. Reconstruction, this method is extremely suitable for scenes that require real-time 3D reconstruction.
  • the device may specifically include:
  • the acquiring unit 601 is configured to acquire, when an object enters the reconstruction space, images taken by N cameras in the reconstruction space as N current images, where N is a positive integer greater than or equal to 2, and the field of view of the camera covers the reconstruction space ;
  • the first obtaining unit 602 is configured to obtain the front background difference corresponding to each position point of the N current images according to the N current images and the corresponding N initial background images, and the N initial background images are all Images taken by the N cameras in the reconstruction space when there are no objects in the reconstruction space;
  • the second obtaining unit 603 is configured to correspondingly fuse the front background difference corresponding to each position point of the N current image based on the corresponding relationship between each position point of the N current images and each position point of the reconstruction space to obtain the The front background difference corresponding to each position point in the reconstruction space;
  • the reconstruction unit 604 is configured to filter each position point in the reconstruction space to reconstruct the object in three dimensions according to the front background difference corresponding to each position point in the reconstruction space and a preset front background threshold.
  • each position point of the N current images and each position point of the reconstruction space is determined by the N cameras fixedly arranged at the edge of the reconstruction space. Location information and angle information are determined.
  • the first obtaining unit 602 includes:
  • the first obtaining subunit is configured to obtain the pixel value of each position point of the N current images and the corresponding pixel value of each position point of the N initial background images;
  • the second obtaining subunit is configured to obtain each of the N current images according to the pixel value of each position point of the N current images, the corresponding pixel value of each position point of the N initial background images, and the difference function.
  • the front background difference corresponding to the position point.
  • the difference function is determined according to a Gaussian mixture model modeling; or, the difference function is determined according to a vector distance formula.
  • the vector distance formula includes Euclidean distance formula, Manhattan distance formula, or cosine distance formula.
  • the second obtaining unit 603 includes:
  • a determining subunit configured to determine the N current image positions corresponding to each position point in each position point in the reconstruction space based on the correspondence between each position point of the N current images and each position point in the reconstruction space The front background difference corresponding to the point;
  • the third obtaining subunit is configured to obtain the corresponding front background difference and fusion function of the N current image position points corresponding to each position point in the reconstruction space. The difference of the front background.
  • the fusion function includes an addition function or a multiplication function.
  • the reconstruction space boundary adopts a deep and shallow texture.
  • the updating unit is configured to obtain and update the N initial background images taken by the N cameras in the reconstruction space when the illumination changes and there are no objects in the reconstruction space.
  • the acquiring unit acquires the captured images of N cameras covering the reconstruction space as N current images when the object enters the reconstruction space; the first acquiring unit uses the N current images and The corresponding N initial background images obtain the front background difference corresponding to each position of the N current images.
  • the N initial background images are the images taken by N cameras in the reconstruction space when there are no objects in the reconstruction space; the second acquisition unit is based on N The corresponding relationship between each position point of the current image and each position point in the reconstruction space, and correspondingly fuse the front background difference corresponding to each position point of the N current images to obtain the front background difference corresponding to each position point in the reconstruction space;
  • the front background difference corresponding to the position point and the preset front background threshold filter the three-dimensional reconstruction object at each position point in the reconstruction space. It can be seen that when the object enters the reconstruction space, the front and background separation of the three-dimensional reconstruction object is carried out through the N current images and the corresponding N initial background images.
  • the front background separation operation is relatively simple and convenient, and the three-dimensional object entering the reconstruction space can be completed quickly and in real time. Reconstruction, this method is extremely suitable for scenes that require real-time 3D reconstruction.
  • an embodiment of the present application also provides a terminal device, which includes a processor and a memory:
  • the memory is used to store program code and transmit the program code to the processor
  • the processor is configured to execute the three-dimensional reconstruction method according to any one of the foregoing method embodiments according to instructions in the program code.
  • an embodiment of the present application also provides a computer-readable storage medium, characterized in that the computer-readable storage medium is used to store program code, and the program code is used to execute any one of the above method embodiments. 3D reconstruction method.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Computer Graphics (AREA)
  • Geometry (AREA)
  • Software Systems (AREA)
  • Image Analysis (AREA)
  • Image Processing (AREA)
  • Length Measuring Devices By Optical Means (AREA)

Abstract

本申请公开了一种三维重建的方法和装置，该方法包括：在物体进入重建空间时获取N台视野覆盖重建空间的摄像机对重建空间的拍摄图像作为N个当前图像；根据N个当前图像和重建空间不存在物体时N台摄像机对重建空间的拍摄图像得到的N个初始背景图像，获得N个当前图像各个位置点对应的前背景差值；基于N个当前图像各个位置点与重建空间各个位置点的对应关系，对应融合上述前背景差值获得重建空间各个位置点对应的前背景差值；根据重建空间各个位置点对应的前背景差值和预设前背景阈值筛选重建空间各个位置点三维重建物体。可见，通过N个当前图像和对应的N个初始背景图像进行较为简单的前背景分离，快速实时完成进入重建空间物体的三维重建。

Description

一种三维重建的方法和装置
本申请要求于2019年01月25日提交中国专利局、申请号为201910074343.5、发明名称为“一种三维重建的方法和装置”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及图像分析处理技术领域，尤其涉及一种三维重建的方法和装置。
背景技术
随着科技的快速发展，越来越多的场景需要应用三维重建(英文：3D Reconstruction；简称：3D重建)技术，以对物体建立适合计算机表示和处理的数学模型，是在计算机中建立表达客观世界的虚拟现实的关键技术。目前主流的3D重建技术主要包括直接3D重建技术和间接3D重建技术。
其中，直接3D重建技术是指通过使用专门的带有激光测距的测量设备实际测量出物体的3D形状和3D结构；间接3D重建技术是指通过对物体拍摄不同角度的图像，利用算法提取图像上的特征点，通过特征点的对比匹配最终生成物体的3D形状和3D结构。
发明人经过研究发现，直接3D重建技术对物体进行全面的激光测距耗费大量时间，间接3D重建技术对比不同角度物体图像中特征点运算较为复杂且计算量大，即，上述两种3D重建技术均需要数分钟甚至数小时才能完成一次3D重建。则上述两种3D重建技术无法适用于需要实时3D重建的场景，比如对人的动作的捕捉、物体运动捕捉等。
发明内容
本申请所要解决的技术问题是，提供一种三维重建的方法和装置，能够快速实时完成进入重建空间物体的三维重建，极其适用于需要实时三维重建的场景。
第一方面，本申请实施例提供了一种三维重建的方法，该方法包括：
物体进入重建空间时获取N台摄像机对所述重建空间的拍摄图像作为N个当前图像，所述N为大于等于2的正整数，所述摄像机的视野覆盖所述重 建空间；
根据所述N个当前图像和对应的N个初始背景图像，获得所述N个当前图像各个位置点对应的前背景差值，所述N个初始背景图像为所述重建空间不存在物体时所述N台摄像机对所述重建空间的拍摄图像；
基于所述N个当前图像各个位置点与所述重建空间各个位置点的对应关系，对应融合所述N个当前图像各个位置点对应的前背景差值，获得所述重建空间各个位置点对应的前背景差值；
根据所述重建空间各个位置点对应的前背景差值和预设前背景阈值，筛选所述重建空间各个位置点三维重建所述物体。
可选的，所述N个当前图像各个位置点与所述重建空间各个位置点的对应关系是由固定布置在所述重建空间边缘的所述N台摄像机的位置信息和角度信息确定的。
可选的，所述根据所述N个当前图像和对应的N个初始背景图像，获得所述N个当前图像各个位置点对应的前背景差值，包括：
获得所述N个当前图像各个位置点的像素值和对应的所述N个初始背景图像各个位置点的像素值；
根据所述N个当前图像各个位置点的像素值、对应的所述N个初始背景图像各个位置点的像素值和差值函数，获得所述N个当前图像各个位置点对应的前背景差值。
可选的，所述差值函数是根据混合高斯模型建模确定的；或，所述差值函数是根据向量距离公式确定的。
可选的，所述向量距离公式包括欧式距离公式、曼哈顿距离公式或余弦距离公式。
可选的，所述基于所述N个当前图像各个位置点与所述重建空间各个位置点的对应关系，对应融合所述N个当前图像各个位置点对应的前背景差值，获得所述重建空间各个位置点对应的前背景差值，包括：
基于所述N个当前图像各个位置点与所述重建空间各个位置点的对应关系，确定所述重建空间各个位置点中每个位置点对应的所述N个当前图像位置点对应的前背景差值；
根据所述重建空间各个位置点中每个位置点对应的所述N个当前图像位置点对应的前背景差值和融合函数，获得所述重建空间各个位置点对应的前背景差值。
可选的，所述融合函数包括相加函数或相乘函数。
可选的，所述重建空间边界采用深浅相间纹理。
可选的，还包括：
当光照发生变化且所述重建空间不存在物体时，获取所述N台摄像机对所述重建空间的拍摄图像更新所述N个初始背景图。
第二方面，本申请实施例提供了一种三维重建的装置，该装置包括：
获取单元，用于物体进入重建空间时获取N台摄像机对所述重建空间的拍摄图像作为N个当前图像，所述N为大于等于2的正整数，所述摄像机的视野覆盖所述重建空间；
第一获得单元，用于根据所述N个当前图像和对应的N个初始背景图像，获得所述N个当前图像各个位置点对应的前背景差值，所述N个初始背景图像为所述重建空间不存在物体时所述N台摄像机对所述重建空间的拍摄图像；
第二获得单元，用于基于所述N个当前图像各个位置点与所述重建空间各个位置点的对应关系，对应融合所述N个当前图像各个位置点对应的前背景差值，获得所述重建空间各个位置点对应的前背景差值；
重建单元，用于根据所述重建空间各个位置点对应的前背景差值和预设前背景阈值，筛选所述重建空间各个位置点三维重建所述物体。
第三方面，本申请实施例提供了一种终端设备，该终端设备包括处理器以及存储器：
所述存储器用于存储程序代码，并将所述程序代码传输给所述处理器；
所述处理器用于根据所述程序代码中的指令执行上述第一方面任一项所述的三维重建方法。
第四方面，本申请实施例提供了一种计算机可读存储介质，该计算机可读存储介质用于存储程序代码，所述程序代码用于执行上述第一方面任一项所述的三维重建方法。
与现有技术相比，本申请至少具有以下优点：
采用本申请实施例的技术方案，首先，在物体进入重建空间时获取N台视野覆盖重建空间的摄像机对重建空间的拍摄图像作为N个当前图像；然后，根据N个当前图像和对应的N个初始背景图像获得N个当前图像各个位置点对应的前背景差值，N个初始背景图像为重建空间不存在物体时N台摄像机对重建空间的拍摄图像；其次，基于N个当前图像各个位置点与重建空间各个位置点的对应关系，对应融合N个当前图像各个位置点对应的前背景差值获得重建空间各个位置点对应的前背景差值；最后，根据重建空间各个位置点对应的前背景差值和预设前背景阈值筛选重建空间各个位置点三维重建物体。由此可见，当物体进入重建空间时，通过N个当前图像和对应的N个初始背景图像进行前背景分离三维重建物体，前背景分离运算较为简单方便，能够快速实时完成进入重建空间物体的三维重建，该方法极其适用于需要实时三维重建的场景。
附图说明
为了更清楚地说明本申请实施例的技术方案，下面将对本申请实施例描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本申请中记载的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其它的附图。
图1为本申请实施例中一种应用场景所涉及的***框架示意图；
图2为本申请实施例提供的一种三维重建的方法的流程示意图；
图3为本申请实施例提供的一种重建空间和N台摄像机结构示意图；
图4为本申请实施例提供的纯白色重建空间中纯黑色圆柱体三维重建平面示意图；
图5为本申请实施例提供的黑白深浅相间纹理；
图6为本申请实施例提供的一种三维重建的装置的结构示意图。
具体实施方式
为了使本技术领域的人员更好地理解本申请方案，下面将结合本申请实施例中的附图，对本申请实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例仅是本申请一部分实施例，而不是全部的实施例。基于本申 请中的实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例，都属于本申请保护的范围。
目前主流的3D重建技术主要包括两种，一种是直接3D重建技术，通过使用专门的带有激光测距的测量设备实际测量出物体的3D形状和3D结构；另一种是间接3D重建技术，通过对物体拍摄不同角度的图像，利用算法提取图像上的特征点，通过特征点的对比匹配最终生成物体的3D形状和3D结构。但是，发明人经过研究发现，直接3D重建技术对物体进行全面的激光测距耗费大量时间，间接3D重建技术对比不同角度物体图像中特征点运算较为复杂且计算量大。即，均需要数分钟甚至数小时才能完成一次3D重建，无法适用于需要实时3D重建的场景，例如，需要实时捕捉人的动作的场景，比如智能自贩机来判断客户拿取商品的动作或生产设备检测工人操作是否符合安全规范等；又如，需要实时捕捉物体运动的场景，比如检测工厂生产线上零配件的运送情况等。
为了解决这一问题，在本申请实施例中，首先，在物体进入重建空间时获取N台视野覆盖重建空间的摄像机对重建空间的拍摄图像作为N个当前图像；然后，根据N个当前图像和对应的N个初始背景图像获得N个当前图像各个位置点对应的前背景差值，N个初始背景图像为重建空间不存在物体时N台摄像机对重建空间的拍摄图像；其次，基于N个当前图像各个位置点与重建空间各个位置点的对应关系，对应融合N个当前图像各个位置点对应的前背景差值获得重建空间各个位置点对应的前背景差值；最后，根据重建空间各个位置点对应的前背景差值和预设前背景阈值筛选重建空间各个位置点三维重建物体。由此可见，当物体进入重建空间时，通过N个当前图像和对应的N个初始背景图像进行前背景分离三维重建物体，前背景分离运算较为简单方便，能够快速实时完成进入重建空间物体的三维重建，该方法极其适用于需要实时三维重建的场景。
举例来说，本申请实施例的场景之一，可以是应用到如图1所示的场景中，该场景包括N台摄像机101、处理器102和用户终端103，其中，N为大于等于2的正整数，摄像机的视野覆盖重建空间。
当重建空间不存在物体时，N台摄像机101对重建空间进行拍摄得到拍 摄图像发送至处理器102，处理器102将其作为N个初始背景图像。当物体进入重建空间时，N台摄像机101对重建空间进行拍摄得到拍摄图像发送至处理器102，处理器102将其作为N个当前图像。处理器102根据N个当前图像和对应的N个初始背景图像，获得N个当前图像各个位置点对应的前背景差值。处理器102基于N个当前图像各个位置点与重建空间各个位置点的对应关系，对应融合N个当前图像各个位置点对应的前背景差值，获得重建空间各个位置点对应的前背景差值。处理器102根据重建空间各个位置点对应的前背景差值和预设前背景阈值，筛选重建空间各个位置点三维重建物体，得到物体三维重建图像发送至用户终端103，用户终端103显示物体三维重建图像给用户。
可以理解的是，在上述应用场景中，虽然将本申请实施方式的动作描述由处理器102执行，但是，本申请在执行主体方面不受限制，只要执行了本申请实施方式所公开的动作即可。
可以理解的是，上述场景仅是本申请实施例提供的一个场景示例，本申请实施例并不限于此场景。
下面结合附图，通过实施例来详细说明本申请实施例中三维重建的方法和装置的具体实现方式。
示例性方法
参见图2，示出了本申请实施例中一种三维重建的方法的流程示意图。在本实施例中，所述方法例如可以包括以下步骤：
步骤201：物体进入重建空间时获取N台摄像机对所述重建空间的拍摄图像作为N个当前图像，所述N为大于等于2的正整数，所述摄像机的视野覆盖所述重建空间。
其中，重建空间是指需要对进入物体进行三维重建的空间，在本申请实施中，并不限定该重建空间的形状，既可以是四方体，也可以是圆柱体，还可以球体等等。实现本申请实施例的前提是需要在重建空间边缘进行摄像机布置，假设布置N台摄像机，N为大于等于2的正整数，N台摄像机应分散布置在重建空间边缘，每台摄像机的镜头朝向重建空间内部，且视野覆盖整 个重建空间。除此之外，在本申请实施中对摄像机数量多少和摄像机布置位置无其他硬性限定，摄像机数量的多少不会影响和改变后续步骤的操作和计算，布置的摄像机越多则可后续可使用的数据越多，三维重建的精确度则越高。需要注意的是，摄像机布置完毕后，N台摄像机均需进行固定，在应用过程中必须确保每台摄像机不发生任何移动或转动等。
例如，如图3所示的重建空间和N台摄像机结构示意图。若重建空间为立方体，则可在其特定四个棱边中点布置4台摄像机，则N为4；或者在其八个顶点布置8台摄像机，则N为8。若重建空间为圆柱体，可以在其顶面和底面布置相互错开的4台摄像机，则N为4。
可以理解的是，当物体进入重建空间时，表示该重建空间存在需要进行三维重建的物体，此时，N台摄像机对重建空间进行拍摄得到N个拍摄图像，处理器获取上述N个拍摄图像作为N个当前图像。其中，N个当前图像均包括物体部分图像，将其作为前景图像，其余部分图像作为背景图像。
步骤202：根据所述N个当前图像和对应的N个初始背景图像，获得所述N个当前图像各个位置点对应的前背景差值，所述N个初始背景图像为所述重建空间不存在物体时所述N台摄像机对所述重建空间的拍摄图像。
其中，N个初始背景图像实际上是在N台摄像机布置固定完毕后，当重建空间不存在物体时，启动N台摄像机对重建空间进行拍摄获得N个拍摄图像，处理器获取N个拍摄图像得到的。获取初始背景图时，需确保重建区域内没有任何其他物体进入。N个初始背景图像存储在处理器中，且每个初始背景图像上不存在任何进入重建空间的物体部分图像，只有纯重建空间的背景图像。
可以理解的是，由于N个初始背景图中每个初始背景图像上不存在任何进入重建空间的物体部分图像，即，每个初始背景图像中所有部分图像都属于背景图像。对于每台摄像机来说，由于摄像机是固定的，当前图像各个位置点与对应的初始背景图各个位置点存在一一对应关系，通过比对当前图像和对应的初始背景图，即可得到当前图像各个位置点对应的前背景差值，以便将前景部分图像从当前图像中分离出来用于后续的物体三维重建。
需要说明的是，对于当前图像与对应的初始背景图中任意一个位置点， 可以用当前图像该位置点的像素值与对应的初始背景图该位置点的像素值之间的差值表示当前图像该位置点对应的前背景差值。其中，在明确当前图像各个位置点的像素值与对应的初始背景图各个位置点的像素值后，采用预先定义的差值函数即可获得像素值差值作为前背景差值。前背景差值越大表示其对应的位置点属于前景图像概率越大，前背景差值越小表示其对应的位置点属于背景图像概率越大。因此，在本申请实施例的一些实施方式中，所述步骤202例如可以包括以下步骤：
步骤A：获得所述N个当前图像各个位置点的像素值和对应的所述N个初始背景图像各个位置点的像素值；
步骤B：根据所述N个当前图像各个位置点的像素值、对应的所述N个初始背景图像各个位置点的像素值和差值函数，获得所述N个当前图像各个位置点对应的前背景差值。
例如，令(u,v)表示当前图像与对应的初始背景图中一个位置点，令(R C,G C,B C)表示当前图像该位置点的像素值，令(R B,G B,B B)表示对应的初始背景图像该位置点的像素值，令预先定义的差值函数为F，则当前图像该位置点对应的前背景差值Diff为：Diff＝F(R B,G B,B B,R C,G C,B C)。若该位置点在当前图像中并不属于前景图像而是背景图像，则(R C,G C,B C)＝(R B,G B,B B)，当前图像该位置点对应的前背景差值Diff为0。
需要说明的是，预先定义的差值函数可以是各种各样的，在申请实施例中可以采用两种方式确定定义差值函数。第一种较为经典的，可以根据混合高斯模型建模确定差值函数；第二种，将像素值可以看做3维向量，根据经典的向量距离公式即可确定差值函数。其中，常见的向量距离公式为欧氏距离公式、曼哈顿距离公式和余弦距离公式等，选取其中任何一种距离公式均可确定差值函数。因此，在本申请实施例的一些实施方式中，所述差值函数是根据混合高斯模型建模确定的；或，所述差值函数是根据向量距离公式确定的；所述向量距离公式包括欧式距离公式、曼哈顿距离公式或余弦距离公式。
例如，当采用欧式距离公式确定差值函数F时，当前图像各个位置点对应的前背景差值Diff为：
Figure PCTCN2019079686-appb-000001
当采用曼哈顿距离公式确定差值函数F时，当前图像各个位置点对应的前背景差值 Diff为：Diff＝|R B-R C|+|G B-G C|+|B B-B C|；当采用余弦距离公式确定差值函数F时，当前图像各个位置点对应的前背景差值Diff为：
Figure PCTCN2019079686-appb-000002
步骤203：基于所述N个当前图像各个位置点与所述重建空间各个位置点的对应关系，对应融合所述N个当前图像各个位置点对应的前背景差值，获得所述重建空间各个位置点对应的前背景差值。
需要说明的是，在步骤202获得N个当前图像各个位置点对应的前背景差值，表示N个当前图像各个位置点属于前景图像的概率大小程度之后，需要获得重建空间各个位置点对应的前背景差值，以确定重建空间各个位置点属于物体的概率大小程度。对于重建空间各个位置点中任意一个位置点，其在N个当前图像中均有对应的位置点，而每个当前图像中对应的位置点均对应一个表示其属于前景图像概率大小程度的前背景差值，采用融合函数将N个当前图像中对应的位置点所对应的前背景差值进行融合，即可得到表示重建空间该位置点属于物体概率大小程度的前背景差值。因此，在本申请实施例的一些实施方式中，所述步骤203例如可以包括以下步骤：
步骤C：基于所述N个当前图像各个位置点与所述重建空间各个位置点的对应关系，确定所述重建空间各个位置点中每个位置点对应的所述N个当前图像位置点对应的前背景差值；
步骤D：根据所述重建空间各个位置点中每个位置点对应的所述N个当前图像位置点对应的前背景差值和融合函数，获得所述重建空间各个位置点对应的前背景差值。
例如，令(x,y,z)表示重建空间一个位置点，令Diff i表示对应该位置点的处理器从第i台摄像机获取的当前图像位置点对应的前背景差值，其中，i＝1，2,3...，N，令预先定义的融合函数为R，则当重建空间该位置点对应的前背景差值Diff 3D为：Diff 3D＝R(Diff 1,Diff 2,...，Diff N)。
需要说明的是，重建空间各个位置点中每个位置点对应的所述N个当前图像位置点对应的前背景差值，首先是根据N个当前图像各个位置点与重建空间各个位置点的对应关系确定重建空间每个位置点在N个当前图像中对应的位置点，再确定其对应的前背景差值。其中，由于N个当前图像是由固定 布置在重建空间边缘的N台摄像机拍摄得到，N台摄像机的位置信息和角度信息的是固定的，摄像机成像原理是将重建空间中的不同位置点投射到一个成像平面并生成一幅拍摄图像，(x,y,z)表示重建空间一个位置点，基于摄像机固定的位置信息和角度信息，即可得到该摄像机对应的当前图像上对应上述位置点的位置点(u,v)，也就是说，摄像机的位置信息和角度信息可以确定当前图像位置点(u,v)与重建空间位置点(x,y,z)的对应关系。因此，在本申请实施例的一些实施方式中，所述N个当前图像各个位置点与所述重建空间各个位置点的对应关系是由固定布置在所述重建空间边缘的所述N台摄像机的位置信息和角度信息确定的。
需要注意的是，由于摄像机成像时每条射入摄像机光线上的所有位置点对应于拍摄图像上的同一个点，则处在进入摄像机这条光线上的所有位置点在对应的当前图像上的位置点是一样的。例如，处在进入摄像机同一条光线上的位置点A和位置点B，则位置点A和位置点B对应该摄像机对应的当前图像上同一位置点，若令(x A,y A,z A)表示重建空间中的位置点A，若令(x B,y B,z B)表示重建空间中的位置点B，则得到该摄像机对应的当前图像中的(u 1,v 1)位置点对应重建空间中的位置点A和位置点B。
需要说明的是，类似于差值函数，预先定义的融合函数也可以是各种各样的。由于融合函数的作用是融合重建空间每个位置点对应的N个当前图像位置点对应的前背景差值，则对于N个当前图像位置点对应的前背景差值既可以采取相加的方式进行融合，也可以采取相乘的方式进行融合。则在申请实施例中可以采用以下两种函数作为融合函数，第一种是相加函数，第二种是相乘函数，上述两种函数中任意一种函数均可以作为融合函数。因此，在本申请实施例的一些实施方式中，所述融合函数包括相加函数或相乘函数。
例如，当采用相加函数作为融合函数R时，重建空间各个位置点对应的前背景差值Diff 3D为
Figure PCTCN2019079686-appb-000003
当采用相乘函数作为融合函数R时，重建空间各个位置点对应的前背景差值Diff 3D为
Figure PCTCN2019079686-appb-000004
步骤204：根据所述重建空间各个位置点对应的前背景差值和预设前背景阈值，筛选所述重建空间各个位置点三维重建所述物体。
可以理解的是，由于重建空间各个位置点对应的前背景差值可以表示各 个位置点属于物体的概率大小程度，即，重建空间各个位置点对应的前背景差值越大，表示各个位置点属于物体的概率越大。在步骤203获得重建空间各个位置点对应的前背景差值之后，可以通过预设前背景阈值筛选重建空间各个位置点，例如，若重建空间某个位置点对应的前背景差值大于等于预设前背景阈值，则认为该位置点属于物体，将重建空间各个位置点中属于物体的位置点筛选出来即可三维重建物体。
例如，重建空间边界采用纯白色纹理，则N个初始背景图像各个位置点的像素值为(R B,G B,B B)＝(255，255，255)，若一个纯黑色的圆柱体进入重建空间，则在N个当前图像中物体部分图像(前景图像)各个位置点的像素值为(R C,G C,B C)＝(0，0，0)，其余部分图像(背景图像)各个位置点的像素值为(R C,G C,B C)＝(255，255，255)，当采用欧式距离公式确定差值函数F时，若当前图像位置点属于前景图像，其对应的前背景差值Diff如下可以计算得到：
Figure PCTCN2019079686-appb-000005
若当前图像位置点属于背景图像，其对应的前背景差值Diff如下可以计算得到：
Figure PCTCN2019079686-appb-000006
如图4所示的纯白色重建空间中纯黑色圆柱体三维重建平面示意图，为了描述方便采用俯视图在平面上进行三维重建。其中，如左图所示重建空间为正方体，固定布置4台摄像机在重建空间俯视图分四个顶点上，图中每台摄像机射出的两条射线之间区域为物体，其余区域为背景。则图中位置点位于某台摄像机的两条射线之间，该位置点在这台摄像机对应的当前图像位置点对应的前背景差值Diff为441，反之则为0。当采用相加函数作为融合函数R时，重建空间各个位置点对应的前背景差值Diff 3D为
Figure PCTCN2019079686-appb-000007
对于区域1各个位置点，其Diff 1＝0，Diff 2＝0，Diff 3＝0，Diff 4＝0，则其Diff 3D＝0；对于区域2各个位置点，其Diff 1＝441，Diff 2＝0，Diff 3＝0，Diff 4＝441，则Diff 3D＝882；对于区域3各个位置点，其Diff 1＝441，Diff 2＝441，Diff 3＝441，Diff 4＝441，则其Diff 3D＝1764。另预设前背景阈值为1764，则比较区域1、区域2和区域3中各个位置点的Diff 3D与预设前背景阈值1764的大小，若某个位置点的Diff 3D大于等于预设前背景阈值1764则认为该位置点属于物体，则区域3中各个位置点属于物体，三维重建得到 区域3所示的多边形。需要注意的是，由于进入的物体实际上是圆柱体，其俯视图为圆形，三维重建构造出来的形状是多边形存在一定误差。则例如图4中的右图所示，如果将固定布置的4台摄像机增加到8台摄像机，三维重建构造出来的形状更加接近于圆形，即，布置的摄像机越多则可后续可使用的数据越多，三维重建的精确度则越高。
需要说明的是，在实际应用中，进入重建空间的物体外表面纹理颜色是不确定的，既可以是深色纹理，也可以是浅色纹理。其中，深色纹理外表面的物体与浅色纹理初始背景图对比较强，浅色纹理外表面的物体与深色纹理初始背景图对比较强。为了能够同时兼容深色和浅色纹理两种情况外表面的物体，还可以采用深浅相间纹理布置重建空间边界，即，重建空间边界还可以采用深浅相间纹理，以便后续达到更好的前背景分离效果。因此，在本申请实施例的一些实施方式中，所述重建空间边界采用深浅相间纹理。例如，采用如图5所示的黑白深浅相间纹理布置重建空间边界。
需要说明的是，若重建空间的光照等对稳定，那么预先获得的初始背景图仍然适用于重建空间，初始背景图不需要更新可以一直沿用。若重建空间的光照发生变化，预先获得的初始背景图已不再适用于光照变化后的重建空间，那么需要对初始背景图进行更新。初始背景图更新的原则是通过判断当前重建空间是否还存在物体，如果不存在，则每台摄像机都对重建空间进行再次拍摄得到拍摄图像以更新初始背景图。因此，在本申请实施例的一些实施方式中，例如还可以包括步骤E：当光照发生变化且所述重建空间不存在物体时，获取所述N台摄像机对所述重建空间的拍摄图像更新所述N个初始背景图。
通过本实施例提供的各种实施方式，首先，在物体进入重建空间时获取N台视野覆盖重建空间的摄像机对重建空间的拍摄图像作为N个当前图像；然后，根据N个当前图像和对应的N个初始背景图像获得N个当前图像各个位置点对应的前背景差值，N个初始背景图像为重建空间不存在物体时N台摄像机对重建空间的拍摄图像；其次，基于N个当前图像各个位置点与重建空间各个位置点的对应关系，对应融合N个当前图像各个位置点对应的前背景差值获得重建空间各个位置点对应的前背景差值；最后，根据重建空间各 个位置点对应的前背景差值和预设前背景阈值筛选重建空间各个位置点三维重建物体。由此可见，当物体进入重建空间时，通过N个当前图像和对应的N个初始背景图像进行前背景分离三维重建物体，前背景分离运算较为简单方便，能够快速实时完成进入重建空间物体的三维重建，该方法极其适用于需要实时三维重建的场景。
示例性装置
参见图6，示出了本申请实施例中一种三维重建的装置的结构示意图。在本实施例中，所述装置例如具体可以包括：
获取单元601，用于物体进入重建空间时获取N台摄像机对所述重建空间的拍摄图像作为N个当前图像，所述N为大于等于2的正整数，所述摄像机的视野覆盖所述重建空间；
第一获得单元602，用于根据所述N个当前图像和对应的N个初始背景图像，获得所述N个当前图像各个位置点对应的前背景差值，所述N个初始背景图像为所述重建空间不存在物体时所述N台摄像机对所述重建空间的拍摄图像；
第二获得单元603，用于基于所述N个当前图像各个位置点与所述重建空间各个位置点的对应关系，对应融合所述N个当前图像各个位置点对应的前背景差值，获得所述重建空间各个位置点对应的前背景差值；
重建单元604，用于根据所述重建空间各个位置点对应的前背景差值和预设前背景阈值，筛选所述重建空间各个位置点三维重建所述物体。
在本申请实施例一种可选的方式中，所述N个当前图像各个位置点与所述重建空间各个位置点的对应关系是由固定布置在所述重建空间边缘的所述N台摄像机的位置信息和角度信息确定的。
在本申请实施例一种可选的方式中，所述第一获得单元602包括：
第一获得子单元，用于获得所述N个当前图像各个位置点的像素值和对应的所述N个初始背景图像各个位置点的像素值；
第二获得子单元，用于根据所述N个当前图像各个位置点的像素值、对应的所述N个初始背景图像各个位置点的像素值和差值函数，获得所述N个 当前图像各个位置点对应的前背景差值。
在本申请实施例一种可选的方式中，所述差值函数是根据混合高斯模型建模确定的；或，所述差值函数是根据向量距离公式确定的。
在本申请实施例一种可选的方式中，所述向量距离公式包括欧式距离公式、曼哈顿距离公式或余弦距离公式。
在本申请实施例一种可选的方式中，所述第二获得单元603包括：
确定子单元，用于基于所述N个当前图像各个位置点与所述重建空间各个位置点的对应关系，确定所述重建空间各个位置点中每个位置点对应的所述N个当前图像位置点对应的前背景差值；
第三获得子单元，用于根据所述重建空间各个位置点中每个位置点对应的所述N个当前图像位置点对应的前背景差值和融合函数，获得所述重建空间各个位置点对应的前背景差值。
在本申请实施例一种可选的方式中，所述融合函数包括相加函数或相乘函数。
在本申请实施例一种可选的方式中，所述重建空间边界采用深浅相间纹理。
在本申请实施例一种可选的方式中，还包括：
更新单元，用于当光照发生变化且所述重建空间不存在物体时，获取所述N台摄像机对所述重建空间的拍摄图像更新所述N个初始背景图。
通过本实施例提供的各种实施方式，获取单元在物体进入重建空间时获取N台视野覆盖重建空间的摄像机对重建空间的拍摄图像作为N个当前图像；第一获得单元根据N个当前图像和对应的N个初始背景图像获得N个当前图像各个位置点对应的前背景差值，N个初始背景图像为重建空间不存在物体时N台摄像机对重建空间的拍摄图像；第二获得单元基于N个当前图像各个位置点与重建空间各个位置点的对应关系，对应融合N个当前图像各个位置点对应的前背景差值获得重建空间各个位置点对应的前背景差值；重建单元根据重建空间各个位置点对应的前背景差值和预设前背景阈值筛选重建空间各个位置点三维重建物体。由此可见，当物体进入重建空间时，通过N个当前图像和对应的N个初始背景图像进行前背景分离三维重建物体，前背 景分离运算较为简单方便，能够快速实时完成进入重建空间物体的三维重建，该方法极其适用于需要实时三维重建的场景。
另本申请实施例还提供了一种终端设备，所述终端设备包括处理器以及存储器：
所述存储器用于存储程序代码，并将所述程序代码传输给所述处理器；
所述处理器用于根据所述程序代码中的指令执行上述方法实施例任一项所述的三维重建方法。
此外，本申请实施例还提供了一种计算机可读存储介质，其特征在于，所述计算机可读存储介质用于存储程序代码，所述程序代码用于执行上述方法实施例任一项所述的三维重建方法。
本说明书中各个实施例采用递进的方式描述，每个实施例重点说明的都是与其他实施例的不同之处，各个实施例之间相同相似部分互相参见即可。对于实施例公开的装置而言，由于其与实施例公开的方法相对应，所以描述的比较简单，相关之处参见方法部分说明即可。
专业人员还可以进一步意识到，结合本文中所公开的实施例描述的各示例的单元及算法步骤，能够以电子硬件、计算机软件或者二者的结合来实现，为了清楚地说明硬件和软件的可互换性，在上述说明中已经按照功能一般性地描述了各示例的组成及步骤。这些功能究竟以硬件还是软件方式来执行，取决于技术方案的特定应用和设计约束条件。专业技术人员可以对每个特定的应用来使用不同方法来实现所描述的功能，但是这种实现不应认为超出本申请的范围。
需要说明的是，在本文中，诸如第一和第二等之类的关系术语仅仅用来将一个实体或者操作与另一个实体或操作区分开来，而不一定要求或者暗示这些实体或操作之间存在任何这种实际的关系或者顺序。术语“包括”、“包含”或者其任何其他变体意在涵盖非排他性的包含，从而使得包括一系列要素的过程、方法、物品或者设备不仅包括那些要素，而且还包括没有明确列出的其他要素，或者是还包括为这种过程、方法、物品或者设备所固有的要 素。在没有更多限制的情况下，由语句“包括一个……”限定的要素，并不排除在包括所述要素的过程、方法、物品或者设备中还存在另外的相同要素。
以上所述，仅是本申请的较佳实施例而已，并非对本申请作任何形式上的限制。虽然本申请已以较佳实施例揭露如上，然而并非用以限定本申请。任何熟悉本领域的技术人员，在不脱离本申请技术方案范围情况下，都可利用上述揭示的方法和技术内容对本申请技术方案做出许多可能的变动和修饰，或修改为等同变化的等效实施例。因此，凡是未脱离本申请技术方案的内容，依据本申请的技术实质对以上实施例所做的任何简单修改、等同变化及修饰，均仍属于本申请技术方案保护的范围内。

Claims (12)

  1. 一种三维重建的方法，其特征在于，包括：
    物体进入重建空间时获取N台摄像机对所述重建空间的拍摄图像作为N个当前图像，所述N为大于等于2的正整数，所述摄像机的视野覆盖所述重建空间；
    根据所述N个当前图像和对应的N个初始背景图像，获得所述N个当前图像各个位置点对应的前背景差值，所述N个初始背景图像为所述重建空间不存在物体时所述N台摄像机对所述重建空间的拍摄图像；
    基于所述N个当前图像各个位置点与所述重建空间各个位置点的对应关系，对应融合所述N个当前图像各个位置点对应的前背景差值，获得所述重建空间各个位置点对应的前背景差值；
    根据所述重建空间各个位置点对应的前背景差值和预设前背景阈值，筛选所述重建空间各个位置点三维重建所述物体。
  2. 根据权利要求1所述的方法，其特征在于，所述N个当前图像各个位置点与所述重建空间各个位置点的对应关系是由固定布置在所述重建空间边缘的所述N台摄像机的位置信息和角度信息确定的。
  3. 根据权利要求1所述的方法，其特征在于，所述根据所述N个当前图像和对应的N个初始背景图像，获得所述N个当前图像各个位置点对应的前背景差值，包括：
    获得所述N个当前图像各个位置点的像素值和对应的所述N个初始背景图像各个位置点的像素值；
    根据所述N个当前图像各个位置点的像素值、对应的所述N个初始背景图像各个位置点的像素值和差值函数，获得所述N个当前图像各个位置点对应的前背景差值。
  4. 根据权利要求3所述的方法，其特征在于，所述差值函数是根据混合高斯模型建模确定的；或，所述差值函数是根据向量距离公式确定的。
  5. 根据权利要求4所述的方法，其特征在于，所述向量距离公式包括欧式距离公式、曼哈顿距离公式或余弦距离公式。
  6. 根据权利要求1所述的方法，其特征在于，所述基于所述N个当前 图像各个位置点与所述重建空间各个位置点的对应关系，对应融合所述N个当前图像各个位置点对应的前背景差值，获得所述重建空间各个位置点对应的前背景差值，包括：
    基于所述N个当前图像各个位置点与所述重建空间各个位置点的对应关系，确定所述重建空间各个位置点中每个位置点对应的所述N个当前图像位置点对应的前背景差值；
    根据所述重建空间各个位置点中每个位置点对应的所述N个当前图像位置点对应的前背景差值和融合函数，获得所述重建空间各个位置点对应的前背景差值。
  7. 根据权利要求6所述的方法，其特征在于，所述融合函数包括相加函数或相乘函数。
  8. 根据权利要求1所述的方法，其特征在于，所述重建空间边界采用深浅相间纹理。
  9. 根据权利要求1所述的方法，其特征在于，还包括：
    当光照发生变化且所述重建空间不存在物体时，获取所述N台摄像机对所述重建空间的拍摄图像更新所述N个初始背景图。
  10. 一种三维重建的装置，其特征在于，包括：
    获取单元，用于物体进入重建空间时获取N台摄像机对所述重建空间的拍摄图像作为N个当前图像，所述N为大于等于2的正整数，所述摄像机的视野覆盖所述重建空间；
    第一获得单元，用于根据所述N个当前图像和对应的N个初始背景图像，获得所述N个当前图像各个位置点对应的前背景差值，所述N个初始背景图像为所述重建空间不存在物体时所述N台摄像机对所述重建空间的拍摄图像；
    第二获得单元，用于基于所述N个当前图像各个位置点与所述重建空间各个位置点的对应关系，对应融合所述N个当前图像各个位置点对应的前背景差值，获得所述重建空间各个位置点对应的前背景差值；
    重建单元，用于根据所述重建空间各个位置点对应的前背景差值和预设前背景阈值，筛选所述重建空间各个位置点三维重建所述物体。
  11. 一种终端设备，其特征在于，所述终端设备包括处理器以及存储器：
    所述存储器用于存储程序代码，并将所述程序代码传输给所述处理器；
    所述处理器用于根据所述程序代码中的指令执行权利要求1-9任一项所述的三维重建方法。
  12. 一种计算机可读存储介质，其特征在于，所述计算机可读存储介质用于存储程序代码，所述程序代码用于执行权利要求1-9任一项所述的三维重建方法。
PCT/CN2019/079686 2019-01-25 2019-03-26 一种三维重建的方法和装置 WO2020151078A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
US17/250,707 US11954832B2 (en) 2019-01-25 2019-03-26 Three-dimensional reconstruction method and apparatus
JP2021540143A JP7398819B2 (ja) 2019-01-25 2019-03-26 三次元再構成の方法及び装置

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201910074343.5A CN109785429B (zh) 2019-01-25 2019-01-25 一种三维重建的方法和装置
CN201910074343.5 2019-01-25

Publications (1)

Publication Number Publication Date
WO2020151078A1 true WO2020151078A1 (zh) 2020-07-30

Family

ID=66502627

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2019/079686 WO2020151078A1 (zh) 2019-01-25 2019-03-26 一种三维重建的方法和装置

Country Status (4)

Country Link
US (1) US11954832B2 (zh)
JP (1) JP7398819B2 (zh)
CN (1) CN109785429B (zh)
WO (1) WO2020151078A1 (zh)

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11205310B1 (en) * 2021-03-08 2021-12-21 Htc Corporation Background skybox constructing method and electronic device
CN113298934B (zh) * 2021-05-26 2022-07-08 重庆邮电大学 一种基于双向匹配的单目视觉图像三维重建方法及***
CN113873156A (zh) * 2021-09-27 2021-12-31 北京有竹居网络技术有限公司 图像处理方法、装置和电子设备

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20110191070A1 (en) * 2010-02-01 2011-08-04 Ramalingam Sathyakumar A Method and Apparatus to Create 3-Dimensional Computer Models of Persons From Specially Created 2-Dimensional images
CN105979203A (zh) * 2016-04-29 2016-09-28 中国石油大学(北京) 一种多摄像机协同监控方法及装置
CN107170037A (zh) * 2016-03-07 2017-09-15 深圳市鹰眼在线电子科技有限公司 一种基于多摄像机的实时三维点云重建方法和***
CN108694741A (zh) * 2017-04-07 2018-10-23 杭州海康威视数字技术股份有限公司 一种三维重建方法及装置

Family Cites Families (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1480903A (zh) * 1996-08-29 2004-03-10 ������������ʽ���� 特征信息赋予、物体抽取和立体模型生成方法及其装置
JP2000172878A (ja) 1998-12-09 2000-06-23 Sony Corp 情報処理装置および情報処理方法、並びに提供媒体
JP2000324509A (ja) 1999-05-11 2000-11-24 For-A Co Ltd 画像合成におけるカメラパラメ−タ推定方法
KR100748719B1 (ko) * 2005-07-14 2007-08-13 연세대학교 산학협력단 다중 스테레오 카메라를 이용한 3차원 모델링 장치 및 그방법
JP4865328B2 (ja) * 2005-12-28 2012-02-01 セコム株式会社 画像センサ
JP2014164525A (ja) * 2013-02-25 2014-09-08 Nippon Telegr & Teleph Corp <Ntt> 物体数推定方法、物体数推定装置、及び物体数推定プログラム
US10157327B2 (en) * 2014-08-06 2018-12-18 Sony Semiconductor Solutions Corporation Image processing device, image processing method, and program
CN104574311B (zh) * 2015-01-06 2017-08-11 华为技术有限公司 图像处理方法和装置
CN105374019B (zh) * 2015-09-30 2018-06-19 华为技术有限公司 一种多深度图融合方法及装置
JP6914734B2 (ja) * 2017-05-30 2021-08-04 Kddi株式会社 シルエット抽出装置、方法およびプログラム

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20110191070A1 (en) * 2010-02-01 2011-08-04 Ramalingam Sathyakumar A Method and Apparatus to Create 3-Dimensional Computer Models of Persons From Specially Created 2-Dimensional images
CN107170037A (zh) * 2016-03-07 2017-09-15 深圳市鹰眼在线电子科技有限公司 一种基于多摄像机的实时三维点云重建方法和***
CN105979203A (zh) * 2016-04-29 2016-09-28 中国石油大学(北京) 一种多摄像机协同监控方法及装置
CN108694741A (zh) * 2017-04-07 2018-10-23 杭州海康威视数字技术股份有限公司 一种三维重建方法及装置

Also Published As

Publication number Publication date
US20210192704A1 (en) 2021-06-24
CN109785429A (zh) 2019-05-21
CN109785429B (zh) 2020-08-21
US11954832B2 (en) 2024-04-09
JP7398819B2 (ja) 2023-12-15
JP2022518402A (ja) 2022-03-15

Similar Documents

Publication Publication Date Title
US11838606B2 (en) Methods and systems for large-scale determination of RGBD camera poses
CN106228507B (zh) 一种基于光场的深度图像处理方法
CN102834845B (zh) 用于多摄像装置校准的方法与装置
Saxena et al. Make3d: Learning 3d scene structure from a single still image
Waechter et al. Virtual rephotography: Novel view prediction error for 3D reconstruction
US20130335535A1 (en) Digital 3d camera using periodic illumination
US8917317B1 (en) System and method for camera calibration
KR100681320B1 (ko) 헬름홀츠 교환조건으로부터 유도되는 편미분 방정식의레벨셋 풀이 방법을 이용한 물체의 3차원 형상 모델링 방법
EP2064675A1 (en) Method for determining a depth map from images, device for determining a depth map
WO2020151078A1 (zh) 一种三维重建的方法和装置
JP2016537901A (ja) ライトフィールド処理方法
JP7479729B2 (ja) 三次元表現方法及び表現装置
CN115439607A (zh) 一种三维重建方法、装置、电子设备及存储介质
CN112802208B (zh) 一种航站楼内三维可视化方法及装置
Song et al. Volumetric stereo and silhouette fusion for image-based modeling
CN109064533B (zh) 一种3d漫游方法及***
TW201707437A (zh) 畫像處理裝置及畫像處理方法
CN112802081A (zh) 一种深度检测方法、装置、电子设备及存储介质
CN116051736A (zh) 一种三维重建方法、装置、边缘设备和存储介质
JP2020098421A (ja) 三次元形状モデル生成装置、三次元形状モデル生成方法、及びプログラム
Coorg Pose imagery and automated three-dimensional modeling of urban environments
Grochulla et al. Combining photometric normals and multi-view stereo for 3d reconstruction
WO2022142311A1 (zh) 图像中目标物体的分割方法、三维重建方法及相关装置
Zhang et al. A Robust Multi‐View System for High‐Fidelity Human Body Shape Reconstruction
Song et al. ConcatNet: A deep architecture of concatenation-assisted network for dense facial landmark alignment

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19911020

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2021540143

Country of ref document: JP

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 19911020

Country of ref document: EP

Kind code of ref document: A1