WO2021139549A1 - 一种平面检测方法及装置、平面跟踪方法及装置 - Google Patents

一种平面检测方法及装置、平面跟踪方法及装置 Download PDF

Info

Publication number
WO2021139549A1
WO2021139549A1 PCT/CN2020/139837 CN2020139837W WO2021139549A1 WO 2021139549 A1 WO2021139549 A1 WO 2021139549A1 CN 2020139837 W CN2020139837 W CN 2020139837W WO 2021139549 A1 WO2021139549 A1 WO 2021139549A1
Authority
WO
WIPO (PCT)
Prior art keywords
plane
image
effective
current frame
frame image
Prior art date
Application number
PCT/CN2020/139837
Other languages
English (en)
French (fr)
Inventor
王宝林
曹世明
周锋宜
吴涛
Original Assignee
青岛小鸟看看科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 青岛小鸟看看科技有限公司 filed Critical 青岛小鸟看看科技有限公司
Publication of WO2021139549A1 publication Critical patent/WO2021139549A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/22—Matching criteria, e.g. proximity measures
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/46—Descriptors for shape, contour or point-related descriptors, e.g. scale invariant feature transform [SIFT] or bags of words [BoW]; Salient regional features
    • G06V10/462—Salient features, e.g. scale invariant feature transforms [SIFT]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20112—Image segmentation details
    • G06T2207/20164—Salient point detection; Corner detection

Definitions

  • the present disclosure relates to the field of image processing technology, and more specifically, the present disclosure relates to a plane detection method and device, and a plane tracking method and device.
  • AR augmented reality
  • mobile devices have become a reality, and are favored by major mobile device manufacturers and users.
  • AR augmented reality
  • plane detection is an important function.
  • An object of the present disclosure is to provide a new technical solution for plane detection and tracking.
  • a plane detection method including:
  • the sparse point data set including the three-dimensional coordinate information of a plurality of the feature points;
  • plane detection is performed on the current frame image to obtain an optimal effective plane of the current frame image, where the optimal effective plane is a plane containing the most feature points.
  • the current frame image includes a first target image and a second target image
  • the feature points include a first feature point and a second feature point
  • the dividing the current frame image of the environment scene collected by the binocular camera, and detecting the feature points corresponding to each sub-image block obtained after the division includes:
  • the method before the dividing the first target image to obtain a plurality of first sub-image blocks, the method further includes:
  • the pixel state of the dots whose Euclidean distance is greater than the first set pixel is set to the second state, and the pixel state of the remaining points in the first target image is set to the first status.
  • the detecting the first feature point corresponding to each of the first sub-image blocks includes:
  • the pixel state of the target point in the first sub-image block is the first state, determining the target point in the first sub-image block as the first feature point;
  • feature point detection is performed on the next first sub-image block until each first sub-image block is traversed.
  • the method further includes:
  • the pixel state of the point with the Euclidean distance less than the second set pixel is set to the second state.
  • the performing plane detection on the current frame image based on the sparse point data set to obtain the optimal effective plane of the current frame image includes:
  • S2 Determine the current effective plane according to the normal vector of the current plane and the number of first interior points, where the first interior point is a feature point whose distance from the current plane is less than a first set distance;
  • S3 Iterate S1-S2 repeatedly until the preset number of iterations is satisfied, and update the current effective plane obtained in the last iteration to the optimal effective plane of the current frame image.
  • the S3 further includes:
  • Every iteration S1-S2, the number of iterations is updated according to the proportion of the first interior points of the current effective plane, where the proportion of the first interior points is the number of the first interior points and the feature point concentration feature The ratio of the total number of points.
  • the method further includes:
  • S5 Use the starting point as a seed point, and determine the neighboring points of the starting point among the feature points included in the optimal effective plane according to a set radius;
  • S6 Use the neighboring points as seed points, and determine the neighboring points of the neighboring points from the feature points included in the optimal effective plane until the end point is determined, and the end point is a feature point without neighboring points;
  • S7 Calculate the number of the seed points until the number of the seed points is greater than the preset minimum number of clustering points, and output an effective plane;
  • S8 Re-select a feature from the remaining feature points of the optimal effective plane as the starting point, and execute S5-S7 until all feature points contained in the optimal effective plane are traversed, and multiple effective points of the current frame image are output. flat.
  • a planar tracking method including:
  • the multiple effective planes of the current frame image are merged with multiple effective planes of the previous frame image to determine the tracking plane of the continuous multiple frame images; wherein, the current frame is obtained according to the optimal effective plane of the current frame image Multiple effective planes of the image, and the plane frame number of the effective plane is consistent with the frame number of the corresponding image.
  • the fusion of multiple effective planes of the current frame of image with multiple effective planes of the previous frame of image to determine the tracking plane of consecutive multiple frames of image includes:
  • the plane that satisfies the plane fusion condition in the current frame image is fused with the corresponding plane of the previous frame image to obtain The first plane, and update the plane frame number of the first plane to the frame number of the current frame image;
  • the effective plane of the last frame of image is updated according to the first plane and the second plane, and the tracking plane of the continuous multiple frames of images is determined.
  • the plane fusion condition includes:
  • the height difference between the effective plane of the current frame image and any effective plane of the previous frame image is less than the first set height
  • the angle between the normal vector of the effective plane of the current frame image and the normal vector of any effective plane of the previous frame image is smaller than the first set angle
  • the ratio of points falling in any one of the effective planes of the last frame of image to all points in the effective plane of the current frame of image is greater than the first set ratio.
  • the fusing a plane that meets a plane fusion condition in the current frame of image with a corresponding plane of the previous frame of image to obtain the first plane includes:
  • a point whose distance to any point in the corresponding plane of the last frame of image is greater than the second set distance is added to the corresponding plane to obtain the first plane.
  • the method before the updating the effective plane of the last frame of image according to the first plane and the second plane, and determining the tracking plane of the continuous multiple frames of images, the method further includes:
  • the invalid planes in the first plane and the second plane are eliminated; wherein, the invalid plane is a plane whose aspect ratio is less than a preset aspect ratio threshold.
  • the method before the updating the effective plane of the last frame of image according to the first plane and the second plane, and determining the tracking plane of the continuous multiple frames of images, the method further includes:
  • the lost planes in the first plane and the second plane are eliminated; wherein the lost plane is that the difference between the plane frame number and the frame number of the current frame image is greater than the set frame Number of planes.
  • the method further includes:
  • the first plane and any unfused effective plane in the current frame image meet the plane fusion condition, the first plane and the corresponding effective plane in the current frame image are plane fused.
  • the method further includes:
  • the center point of the tracking plane is eliminated, and the edge points of the tracking plane are retained.
  • a plane detection device comprising:
  • a memory configured to store computer instructions
  • a processor configured to call the computer instructions from the memory, and execute the method according to any one of the first aspects of the present disclosure under the control of the computer instructions.
  • a planar tracking device comprising:
  • a memory configured to store computer instructions
  • a processor configured to call the computer instructions from the memory, and execute the method according to any one of the second aspects of the present disclosure under the control of the computer instructions.
  • plane detection is performed on the current frame image based on a sparse point data set without storing a large amount of point cloud data, which can increase the speed of plane detection, thereby improving the real-time performance of plane detection, and is based on the sparse point data set
  • the detection can be applied to mobile devices with limited computing power, avoiding the delay of the display screen, and improving the user experience.
  • FIG. 1 shows a schematic diagram of the hardware configuration of an electronic device provided by an embodiment of the present disclosure
  • FIG. 2 shows a first schematic flowchart of a plane detection method provided by an embodiment of the present disclosure
  • Fig. 3 shows a schematic flow chart of a plane detection method according to an example of the present disclosure
  • FIG. 4 shows a second schematic flowchart of a plane detection method provided by an embodiment of the present disclosure
  • FIG. 5 shows a schematic flow chart 1 of a planar tracking method provided by an embodiment of the present disclosure
  • FIG. 6 shows a second schematic flowchart of a planar tracking method provided by an embodiment of the present disclosure
  • FIG. 7 shows a block diagram of a plane detection device provided by an embodiment of the present disclosure
  • Fig. 8 shows a block diagram of a planar tracking device provided by an embodiment of the present disclosure.
  • FIG. 1 is a block diagram of a hardware configuration of an electronic device 100 that can be used to implement an embodiment of the present disclosure.
  • the electronic device may be an intelligent device such as a virtual reality (VR) device, an augmented reality (Augmented Reality, AR) device, or a mixed reality (Mixed Reality) device.
  • VR virtual reality
  • AR Augmented Reality
  • Mixed Reality Mixed reality
  • the electronic device 100 may include a processor 101, a memory 102, an interface device 103, a communication device 104, a display device 105, an input device 106, an audio device 107, a sensor 108, a camera 109, etc. .
  • the processor 101 may include, but is not limited to, a central processing unit (CPU), a microprocessor MCU, and the like.
  • the processor 101 may also include a GPU (Graphics Processing Unit) and the like.
  • the memory 102 may include, but is not limited to, ROM (Read Only Memory), RAM (Random Access Memory), non-volatile memory such as a hard disk, and the like.
  • the interface device 103 may include, but is not limited to, a USB interface, a serial interface, a parallel interface, an infrared interface, and the like.
  • the communication device 104 can perform wired or wireless communication, for example, and specifically may include WiFi communication, Bluetooth communication, 2G/3G/4G/5G communication, and the like.
  • the display device 105 is, for example, a liquid crystal display, an LED display, a touch display, or the like.
  • the input device 106 may include, but is not limited to, a touch screen, a keyboard, a somatosensory input, and the like.
  • the audio device 107 may be configured to input/output voice information.
  • the sensor 108 is, for example, an image sensor, an infrared sensor, a laser sensor, a pressure sensor, a gyroscope sensor, a magnetic sensor, an acceleration sensor, a distance sensor, a proximity light sensor, an ambient light sensor, a fingerprint sensor, a touch sensor, a temperature sensor, etc.
  • the sensor 108 can be It is configured to measure the posture change of the electronic device 100.
  • the camera 109 may be configured to obtain image information, and the camera 109 may be, for example, a binocular camera.
  • the electronic device 100 is used to obtain an image of an environmental scene to perform plane detection and plane tracking on the image.
  • the electronic device 100 shown in FIG. 1 is only illustrative and does not mean any limitation to the embodiment of the present specification, its application or use. Those skilled in the art should understand that although multiple devices of the electronic device 100 are described above, the embodiments of this specification may only involve some of them. Those skilled in the art can design instructions according to the scheme disclosed in the embodiments of this specification, and how the instructions control the processor to operate is a well-known technology in the art, so it will not be described in detail here.
  • Fig. 2 is a schematic diagram of a plane detection method provided by an embodiment of the present specification.
  • the plane detection method provided in this embodiment is implemented by computer technology, and can be implemented by the electronic device 100 described in FIG. 1.
  • the plane detection method provided in this embodiment includes steps S2100-S2300.
  • step S2100 the current frame image of the environment scene collected by the binocular camera is divided, and the feature points corresponding to each sub-image block obtained by the division are detected.
  • the environmental scene may be, for example, a scene including the ground, a table, a platform, and the like.
  • the binocular camera is used to shoot the environment scene, the current frame image collected by the binocular camera is segmented, the feature extraction is performed on each sub-image block obtained after the segmentation, and the feature point corresponding to each sub-image block is obtained.
  • the feature point may be a relatively prominent point in the current frame image, such as a contour point in the current frame image, a bright spot in a darker area, a dark point in a brighter area, etc.
  • the feature point is a point used for plane detection.
  • the binocular camera may be provided on a head-mounted display device, for example, and there is no limitation here.
  • the binocular camera can be a high-resolution camera, a low-resolution camera, and the binocular camera can also be a fisheye camera.
  • the current frame image includes a first target image and a second target image
  • the feature points include a plurality of first feature points corresponding to the first target image and a plurality of second feature points corresponding to the second target image.
  • the first feature point is used for plane detection
  • the second feature point is used for calculating the three-dimensional coordinate information of the first feature point.
  • a binocular camera includes a left-eye camera and a right-eye camera. The current frame image collected by the left-eye camera is used as the first-eye image, and the current frame image collected by the right-eye camera is the second-eye image.
  • the step of dividing the current frame image of the environment scene collected by the binocular camera, and detecting the feature point corresponding to each sub-image block obtained by the division may further include: steps S2110-S2140.
  • Step S2110 Divide the first target image to obtain multiple first sub-image blocks.
  • the divided first sub-image blocks may be image blocks of the same size, or image blocks of different sizes.
  • the first target image is divided according to a preset division method to obtain a plurality of first sub-image blocks.
  • the first target image is divided into i*j sub-image blocks of the same size.
  • Step S2120 Detect the first feature point corresponding to each first sub-image block.
  • FAST Feature from Accelerated Segment Test, accelerated segmentation test feature
  • SIFT Scale Invariant Feature Transform, scale invariant feature transform
  • ORB Oriented FAST and Rotated Brief, fast feature point extraction and description
  • FAST corner detection is performed on the first sub-image block, and the corner point with the highest Harris response in the sub-image block is selected as the first feature point corresponding to the first sub-image block.
  • Step S2130 Use the epipolar matching method to perform feature point matching in the second target image, and obtain a second feature point in the second target image that matches the first feature point.
  • Step S2140 using the first feature point and the second feature point to calculate the three-dimensional coordinate information of the first feature point.
  • the method before dividing the first target image to obtain a plurality of first sub-image blocks, the method further includes: taking the center of the first target image as the origin, and dividing the point whose Euclidean distance is greater than the first set pixel The pixel state is set to the second state, and the pixel state of the remaining dots in the first target image is set to the first state.
  • the pixel state of the dot is set according to the position of the dot in the first target image.
  • the pixel state includes a first state and a second state. According to the degree of distortion of the points in the first target image, set the pixel state of the less distorted point in the first target image to the first state, and set the pixel state of the more distorted point in the first target image to the second state . For example, the first state is true and the second state is false.
  • the first setting pixel can be set according to engineering experience or experimental simulation results. For example, taking the center of the first target image as the origin, the pixel status of the points whose Euclidean distance is greater than m pixels is set to false, and the pixel status of the remaining points in the first target image is set to true.
  • the pixel state of the point in the first target image is set, which can eliminate the severely distorted points in the image and avoid the plane calculated by using the three-dimensional coordinate information of the points.
  • the error of the parameter is relatively large, combined with the subsequent steps, the accuracy of plane detection can be improved.
  • the severely distorted points in the image are excluded, so that this method can be applied to a fisheye camera, and a larger field of view can be obtained.
  • the step of detecting the first feature point corresponding to each first sub-image block may further include: steps S2121-S2124.
  • Step S2121 Perform feature point detection on the first sub-image block, and determine the target point in each first sub-image block.
  • FAST corner detection is performed on the first sub-image block, and the corner point with the highest Harris response in the sub-image block is selected as the target point corresponding to the first sub-image block.
  • Step S2122 Determine the pixel state of the target point in the first sub-image block, and the pixel state includes a first state and a second state.
  • Step S2123 if the pixel state of the target point in the first sub-image block is the first state, determine the target point in the first sub-image block as the first feature point;
  • Step S2124 if the pixel state of the target point in the first sub-image block is in the second state, perform feature point detection on the next first sub-image block until each first sub-image block is traversed.
  • the first feature point corresponding to the first sub-image block is determined according to the pixel state of the point in the first target image, which can prevent the severely distorted point from affecting the accuracy of calculating the plane parameters.
  • the accuracy of plane detection can be improved.
  • the method further includes: taking the first feature point obtained from the first sub-image block as the origin, and reducing the Euclidean distance to be smaller than the second feature point.
  • the pixel state of the point of the set pixel is set to the second state.
  • the second setting pixel can be set according to engineering experience or experimental simulation results. For example, taking the first feature point corresponding to a first sub-image block as the origin, the pixel state of the point with the Euclidean distance less than n pixels is set to false.
  • the pixel state of the point in the first target image is set again, and the pixel state of the points around the first feature point is set to the first feature point.
  • the two states can prevent the detected first feature points from being too dense, and the detected first feature points are more evenly distributed, thereby improving the accuracy of detection.
  • the step of dividing the current frame image of the environment scene collected by the binocular camera and detecting the feature points corresponding to each sub-image block obtained after the division may further include the following steps S301 to S308.
  • step S301 the center of the first target image is used as the origin, the pixel state of the points whose Euclidean distance is greater than m pixels is set to false, and the pixel state of the remaining points in the first target image is set to true.
  • Step S302 Divide the first target image into i*j first sub-image blocks of the same size, and number the first sub-image blocks, and the number num of the first sub-image blocks is 0 ⁇ i*j-1.
  • Step S303 Perform FAST corner detection on the first sub-image block numbered num, and select the corner point with the highest Harris response in the first sub-image block as the target point corresponding to the first sub-image block.
  • Step S304 It is judged whether the pixel state of the target point in the first sub-image block is true, if yes, go to step S305, otherwise, go to step S307.
  • Step S305 Determine the target point as the first feature point corresponding to the first sub-image block numbered num.
  • step S306 the first feature point corresponding to the first sub-image block numbered num is used as the origin, and the pixel state of the point with the Euclidean distance less than n pixels is set to false.
  • step S307 it is judged whether the number of the first sub-image is less than i*j-1, if yes, go to step S303, otherwise, go to step S308.
  • step S308 the epipolar line matching method is used to perform feature point matching in the second target image, and a second feature point matching the first feature point in the second target image is obtained.
  • Step S2200 Obtain a sparse point data set of the current frame image according to the three-dimensional coordinate information of the feature point in the world coordinate system.
  • the sparse point data set includes three-dimensional coordinate information of multiple feature points.
  • the feature points of the current frame image include multiple first feature points in the first target image.
  • obtaining the sparse point data set of the current frame image according to the three-dimensional coordinate information of the feature point in the world coordinate system may further include: steps S2210-S2230.
  • step S2210 the three-dimensional coordinate information of the first feature point of the first eye image under the reference system of the first eye camera is calculated according to the first feature point and the second feature point through triangulation.
  • Step S2220 Obtain the pose of the binocular camera when the current frame of image is collected.
  • the pose of the binocular camera may be 6Dof information of the binocular camera.
  • the pose of the binocular camera can be acquired through a SLAM (Simultaneous Localization And Mapping, which can be translated as a synchronous positioning and mapping) system or an optical tracking system.
  • SLAM Simultaneous Localization And Mapping
  • Step S2230 Perform coordinate system conversion according to the pose of the binocular camera when the current frame image is collected, and obtain the three-dimensional coordinate information of the first feature point of the first eye image in the current frame image in the world coordinate system.
  • the three-dimensional coordinate information of the first feature point of the first item image in the world coordinate system constitutes a sparse point data set.
  • the pose of the binocular camera when collecting the current frame image is acquired through the SLAM system, and the coordinate system conversion is performed according to the pose of the binocular camera, combined with subsequent steps, the horizontal plane or the vertical plane can be accurately detected.
  • Step S2300 Perform plane detection on the current frame image based on the sparse point data set, and obtain the optimal effective plane of the current frame image.
  • the optimal effective plane is the plane containing the most feature points.
  • the plane detection of the current frame image based on the sparse point data set can increase the speed of plane detection, thereby improving the real-time performance of the plane detection, and the detection based on the sparse point data set can be suitable for limited computing power. Mobile devices to avoid display delays and improve user experience.
  • the step of performing plane detection on the current frame image based on the sparse point data set to obtain the optimal effective plane of the current frame image may further include: S2310-S2330.
  • Step S2310 Determine the current plane according to three feature points randomly selected from the sparse point data set.
  • three feature points are randomly selected from the sparse point data set. According to the three-dimensional coordinate information of these three feature points, it is judged whether the selected three feature points are collinear. If the selected three feature points are not Collinear, the current plane is determined according to the three feature points. If the selected three feature points are collinear, the feature points are re-extracted from the sparse point data set.
  • Step S2320 Determine the current effective plane according to the normal vector of the current plane and the number of first interior points, where the first interior point is a feature point whose distance from the current plane is less than a first set distance.
  • the normal vector of the current plane can be calculated according to the three-dimensional coordinate information of the feature point. For example, calculate the covariance matrix of all interior points in the current plane, and use the eigenvector corresponding to the smallest eigenvalue among all interior points in the current plane as the normal vector of the current plane.
  • the current effective plane is determined according to the normal vector of the current plane. Specifically, it is determined whether the current plane is the current effective plane according to the calculated angle between the normal vector of the current plane and the reference normal vector.
  • the reference normal vector can be determined according to the type of plane to be detected. For example, when detecting a horizontal plane, the reference normal vector is the gravity direction vector.
  • Determine the current effective plane according to the number of first interior points of the current plane specifically, determine the first interior point according to the distance from the point in the current frame image to the current plane, and determine whether the current plane is the current plane according to the number of first interior points Effective plane.
  • Step S2330 Iterate steps S2310-S2320 repeatedly until the preset number of iterations is satisfied, and update the current effective plane obtained in the last iteration to the optimal effective plane of the current frame image.
  • step S2330 it further includes: every iteration S1-S2, updating the number of iterations according to the proportion of the first interior point of the current effective plane, wherein the proportion of the first interior point is the first interior point.
  • the number of iterations is updated according to the proportion of the first interior point of the current effective plane, which can improve the efficiency of plane detection and quickly detect the effective plane with the most feature points.
  • the step of performing plane detection on the current frame image based on the sparse point data set to obtain the optimal effective plane point of the current frame image may further include the following steps S401 to S408.
  • Step S401 randomly extract three feature points from the sparse point data set of the current frame image
  • Step S402 according to the three-dimensional coordinate information of the three characteristic points, judge whether the selected three characteristic points are collinear, if yes, go to step S403, if not, go to step S401;
  • Step S403 Determine the current plane according to the selected three characteristic points, and calculate the normal vector of the current plane
  • Step S404 it is judged whether the normal vector of the current plane meets the requirements, if yes, go to step S405, if not, go to step S401;
  • the angle between the normal vector of the current plane and the reference normal vector is calculated. If the angle between the normal vector of the current plane and the reference normal vector is less than the second set angle, it is judged that the normal vector of the current plane meets the requirements If the angle between the normal vector of the current plane and the reference normal vector is not less than the second set angle, it is judged that the normal vector of the current plane does not meet the requirements.
  • the reference normal vector can be determined according to the type of plane to be detected, and the second setting angle can be set according to engineering experience or experimental simulation results. For example, when detecting the horizontal plane, the reference normal vector is the gravity direction vector, and the second set angle is 10°. If the angle between the normal vector of the current plane and the gravity direction vector is less than 10°, it is judged that the normal vector of the current plane meets the requirements . When detecting a vertical plane, the reference normal vector is the gravity direction vector, and the first set angle is 80°. If the angle between the normal vector of the current plane and the gravity direction vector is greater than 80°, it is judged that the normal vector of the current plane meets the requirements.
  • Step S405 In addition to the selected three feature points, calculate the distance from all the feature points in the sparse point data set to the current plane, and set the feature points whose distance from the current plane is less than the first set distance to the first inside of the current plane point;
  • the first set distance can be set based on engineering experience or experimental simulation results. For example, 2cm.
  • Step S406 Determine whether the number of first interior points of the current plane is greater than the number of first interior points of the last effective plane, if yes, calculate the normal vector of the current effective plane according to the first interior points of the current plane. If not, perform step S401 ;
  • Step S407 recalculate the normal vector of the current plane according to the first interior point
  • Step S408 it is judged whether the normal vector of the current plane meets the requirements, if it is, the current plane is taken as the current effective plane, if not, step S401 is executed;
  • Step S409 Iterate S1-S7 repeatedly until the preset number of iterations is met, and update the current effective plane to the optimal effective plane, which is the plane containing the most feature points.
  • the method after performing plane detection on the current frame image based on the sparse point data set, after obtaining the optimal effective plane of the current frame image, the method further includes: performing plane clustering on the obtained optimal effective plane, and outputting the current frame Multiple effective planes of the image. As shown in Figure 5, it specifically includes steps S2400-S2800.
  • step S2400 a feature point is selected arbitrarily from the feature points included in the optimal effective plane as a starting point.
  • step S2500 the starting point is used as a seed point, and the adjacent points of the starting point are determined among the characteristic points contained in the optimal effective plane according to the set radius.
  • step S2600 the neighboring points are used as seed points, and the neighboring points of the neighboring points are determined from the feature points included in the optimal effective plane until the end point is determined, and the end point is a feature point without neighboring points.
  • Step S2700 Calculate the number of seed points until the number of seed points is greater than the preset minimum number of clustering points, and output an effective plane.
  • Step S2800 reselect a feature from the remaining feature points of the optimal effective plane as the starting point, and execute S2500-S2700 until all feature points included in the optimal effective plane are traversed, and multiple effective planes of the current frame image are output.
  • the remaining feature points are the feature points that were not selected when outputting the last effective plane.
  • plane detection is performed on the current frame image based on a sparse point data set without storing a large amount of point cloud data, which can improve the speed of plane detection, thereby improving the real-time performance of plane detection, and performing plane detection based on the sparse point data set.
  • the detection can be applied to mobile devices with limited computing power, avoiding the delay of displaying images and improving user experience.
  • Fig. 5 is a schematic diagram of a plane tracking method provided by an embodiment of the present specification.
  • the plane tracking method provided in this embodiment is implemented by computer technology, and can be implemented by the electronic device 100 described in FIG. 1.
  • the plane tracking method provided in this embodiment is based on the plane detection method provided above, and the method includes steps S3100-S3200.
  • Step S3100 Acquire continuous multiple frames of images of the environment scene collected by the binocular camera.
  • step S3200 according to the optimal effective plane of the current frame image, multiple effective planes of the current frame image are obtained, and the plane frame number of the effective plane is consistent with the frame number of the corresponding image.
  • plane clustering is performed on the optimal effective plane of the current frame image to obtain multiple effective planes of the current frame image, and the plane frame number of the effective plane of the current frame image is consistent with the frame number of the corresponding image.
  • step S3300 the multiple effective planes of the current frame of image and the multiple effective planes of the previous frame of image are merged to determine the tracking plane of the continuous multiple frames of image.
  • the multiple effective planes of each frame of image are detected, and the plane fusion between the multiple frames of images is performed based on the effective plane of each frame of image, which can improve the stability of plane fusion and expansion.
  • the edge of the plane has better fit, and further as the environment changes, the plane can be tracked based on the fusion plane, so that as the user's perspective changes, the plane expands steadily in all directions, thereby improving user experience.
  • the step of fusing multiple effective planes of the current frame image with multiple effective planes of the previous frame image to determine the tracking plane of the continuous multiple frame images may further include step S3310 -S3350.
  • step S3310 the effective plane of the current frame of image is sequentially compared with multiple effective planes of the previous frame of image.
  • Step S3320 It is judged whether there is a plane that meets the plane fusion condition among the multiple effective planes of the current frame image.
  • the plane fusion conditions include:
  • the height difference between the effective plane of the current frame image and any effective plane of the previous frame image is less than the first set height; between the normal vector of the effective plane of the current frame image and the normal vector of any effective plane of the previous frame image
  • the included angle of is smaller than the first set angle; and, the ratio of points falling in any effective plane of the previous frame of image to all points in the effective plane of the current frame of image is greater than the first set ratio.
  • the first set height can be set based on engineering experience or experimental simulation results.
  • the first setting angle can be set according to engineering experience or experimental simulation results.
  • the first setting ratio can be set according to engineering experience or experimental simulation results. If the above three plane fusion conditions are met at the same time, it is determined that there are planes that meet the plane fusion conditions among the multiple effective planes of the current frame image.
  • Step S3330 in the case that there are planes that meet the plane fusion condition in the multiple effective planes of the current frame image, merge the plane that meets the plane fusion condition in the current frame image with the corresponding plane of the previous frame image to obtain the first plane , And update the plane frame number of the first plane to the frame number of the current frame image.
  • the step of fusing the plane that meets the plane fusion condition in the current frame of image with the corresponding plane of the previous frame of image may further include: steps S3331-S3332.
  • Step S3331 Calculate the distance from all points in the plane that meets the plane fusion condition in the current frame of image to any point in the corresponding plane of the previous frame of image.
  • step S3332 a point whose distance to any point in the corresponding plane of the previous frame of image is greater than the second set distance is added to the corresponding plane to obtain the first plane.
  • a point in the current frame of image that satisfies the plane fusion condition and the distance from any point in the corresponding plane of the previous frame of image is not greater than the second set distance is considered to be the coincident point in the two frames of images.
  • the distance of any point in the corresponding plane of the last frame of image, the point fusion between the two planes, can avoid too many repeated points.
  • Step S3340 in the case that there is no plane that meets the plane fusion condition among the multiple effective planes of the current frame image, use the plane that does not meet the plane fusion condition in the current frame image as the second plane.
  • step S3350 the effective plane of the previous frame of image is updated according to the first plane and the second plane, and the tracking plane of the continuous multiple frames of image is determined.
  • the tracking planes of consecutive multiple frames of images include the first plane, the second plane, and the plane in the previous frame of image that is not fused with any plane in the current frame of image.
  • the last frame image includes a1 effective plane and a2 effective plane
  • the current frame image includes b1 effective plane and b2 effective plane
  • the b1 effective plane of the current frame image and the a1 effective plane r of the previous frame image are merged into the c1 effective plane.
  • the b2 effective plane of the current frame image and the a2 effective plane r of the previous frame image are merged into the c2 effective plane
  • the tracking plane of the continuous multiple frames of images includes the c1 effective plane and the c2 effective plane.
  • the previous frame image includes a1 effective plane and a2 effective plane
  • the current frame image includes b1 effective plane and b2 effective plane
  • the b1 effective plane of the current frame image and the a1 effective plane r of the previous frame image are merged into the c1 effective plane.
  • the b2 effective plane of the current frame of image does not merge with any effective plane of the previous frame of image
  • the tracking planes of consecutive multiple frames of images include the c1 effective plane, a2 effective plane, and b2 effective plane.
  • the plane tracking method may further include: steps S4100-S4200 .
  • Step S4100 Calculate the plane parameters of the first plane and the second plane according to the three-dimensional coordinate information of the feature points in the sparse point data set.
  • the plane parameters include plane height, plane length-to-width ratio, and so on.
  • the plane parameters are calculated according to the three-dimensional coordinate information of all points in the plane.
  • the plane height is the average plane height. Specifically, the plane height corresponding to each point is calculated according to the points contained in the plane, and then the average plane height corresponding to all points is calculated, and the average value is the plane height corresponding to the plane.
  • the aspect ratio of the plane can be calculated according to the three-dimensional coordinate information of the four corner points of the plane.
  • Step S4200 eliminate invalid planes in the first plane and the second plane; wherein the invalid plane is a plane whose aspect ratio is less than a preset aspect ratio threshold.
  • the invalid plane may be, for example, a long and narrow plane or a too small plane, and the invalid plane affects the generation of virtual objects.
  • Invalid plane can be judged according to the ratio of plane length to width.
  • the preset aspect ratio threshold can be set according to engineering experience or experimental simulation results.
  • invalid planes are eliminated according to the plane parameters to avoid the influence of invalid planes on the generation of virtual objects and ensure the effectiveness of the tracking plane.
  • the plane tracking method may further include: step S5100.
  • Step S5100 according to the plane frame number, remove the lost planes in the first plane and the second plane; wherein, the lost plane is the one where the difference between the plane frame number and the frame number of the current frame image is greater than the set number of frames flat.
  • the environment scene changes
  • the detected plane also changes.
  • the lost plane is eliminated according to the plane frame number, which reduces the amount of data processing and improves the effectiveness of tracking the plane.
  • the plane tracking method may further include: steps S6100-S6200 .
  • Step S6100 Calculate the number of points in the tracking plane.
  • step S6200 if the number of points in the tracking plane exceeds the first set number, the center point of the tracking plane is removed, and the edge points of the tracking plane are retained.
  • the first set number can be set based on engineering experience or experimental simulation results.
  • the number of points contained in the plane continues to increase, and the storage space occupied by the data becomes larger and larger.
  • the center point of the plane is removed and only the edge points of the plane are retained. It can reduce the storage space occupied by data, thereby increasing the processing speed, realizing the speed of plane detection, fusion and expansion, and the user experience is better.
  • the plane tracking method may further include: step S7100 -S7200.
  • Step S7100 It is judged whether the first plane and any unfused effective plane in the current frame of image meet the plane fusion condition.
  • Step S7200 in the case that the first plane and any unfused effective plane in the current frame image meet the plane fusion condition, perform plane fusion on the first plane and the corresponding effective plane in the current frame image.
  • the effective planes that meet the plane fusion conditions in the current frame image change. After the effective planes are fused, it is judged again whether there are planes that can be fused between the multiple effective planes of the current frame image after the update, and the fusionable planes are merged to avoid plane overlap.
  • the multiple effective planes of each frame of image are detected, based on the effective plane of each frame of image, the plane fusion between the multiple frames of images can be improved, and the stability of plane fusion and expansion can be improved.
  • the edge of the rear plane has better fit.
  • the plane can be tracked based on the merged plane, so that as the user's perspective changes, the plane expands steadily in all directions, thereby improving the user experience.
  • a plane detection device 700 is provided, and the plane detection device 700 may be the electronic device 100 shown in FIG. 1.
  • the plane detection device 700 includes a processor 710 and a memory 720.
  • the memory 720 may be configured to store executable instructions
  • the processor 710 may be configured to execute the plane detection method as provided in the foregoing embodiment of the present disclosure according to the control of the executable instruction.
  • the processor 710 may be configured to execute steps S2100-S2300, steps S2110-S2140, steps S2121-S2124, steps S2210-S2230, S2310- in the foregoing embodiments of the present disclosure under the control of executable instructions. S2330 and/or steps S2400-S2800.
  • plane detection is performed on the current frame image based on a sparse point data set without storing a large amount of point cloud data, which can improve the speed of plane detection, thereby improving the real-time performance of plane detection, and performing plane detection based on the sparse point data set.
  • the detection can be applied to mobile devices with limited computing power, avoiding the delay of displaying images and improving user experience.
  • planar tracking device 800 is provided, and the planar tracking device 800 may be the electronic device 100 shown in FIG. 1.
  • the planar tracking device 800 includes a processor 810 and a memory 820.
  • the memory 820 may be configured to store executable instructions
  • the processor 810 may be configured to execute the planar tracking method as provided in the foregoing embodiment of the present disclosure according to the control of the executable instruction.
  • the processor 810 may be configured to execute steps S3100-S3300, steps S3310-S3350, steps S3331-S3332, steps S4100-S4200, and step S5100 in the foregoing embodiments of the present disclosure according to the control of executable instructions. , Steps S6100-S6200 and/or steps S7100-S7200.
  • the multiple effective planes of each frame of image are detected, based on the effective plane of each frame of image, the plane fusion between the multiple frames of images can be improved, and the stability of plane fusion and expansion can be improved.
  • the edge of the rear plane has better fit.
  • the plane can be tracked based on the merged plane, so that as the user's perspective changes, the plane expands steadily in all directions, thereby improving the user experience.
  • This embodiment provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, the method described in any one of the above method embodiments is implemented.
  • the present disclosure may be a system, method and/or computer program product.
  • the computer program product may include a computer-readable storage medium loaded with computer-readable program instructions for enabling a processor to implement various aspects of the present disclosure.
  • the computer-readable storage medium may be a tangible device that can hold and store instructions used by the instruction execution device.
  • the computer-readable storage medium may be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing.
  • Non-exhaustive list of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM) Or flash memory), static random access memory (SRAM), portable compact disk read-only memory (CD-ROM), digital versatile disk (DVD), memory stick, floppy disk, mechanical encoding device, such as a printer with instructions stored thereon
  • RAM random access memory
  • ROM read-only memory
  • EPROM erasable programmable read-only memory
  • flash memory flash memory
  • SRAM static random access memory
  • CD-ROM compact disk read-only memory
  • DVD digital versatile disk
  • memory stick floppy disk
  • mechanical encoding device such as a printer with instructions stored thereon
  • the computer-readable storage medium used here is not interpreted as the instantaneous signal itself, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (for example, light pulses through fiber optic cables), or through wires Transmission of electrical signals.
  • the computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to various computing/processing devices, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and/or a wireless network.
  • the network may include copper transmission cables, optical fiber transmission, wireless transmission, routers, firewalls, switches, gateway computers, and/or edge servers.
  • the network adapter card or network interface in each computing/processing device receives computer-readable program instructions from the network, and forwards the computer-readable program instructions for storage in the computer-readable storage medium in each computing/processing device .
  • the computer program instructions used to perform the operations of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state setting data, or in one or more programming languages.
  • Source code or object code written in any combination, the programming language includes object-oriented programming languages such as Smalltalk, C++, etc., and conventional procedural programming languages such as "C" language or similar programming languages.
  • Computer-readable program instructions can be executed entirely on the user's computer, partly on the user's computer, executed as a stand-alone software package, partly on the user's computer and partly executed on a remote computer, or entirely on the remote computer or server carried out.
  • the remote computer can be connected to the user's computer through any kind of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (for example, using an Internet service provider to access the Internet connection).
  • LAN local area network
  • WAN wide area network
  • an electronic circuit such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), can be customized by using the status information of the computer-readable program instructions.
  • the computer-readable program instructions are executed to realize various aspects of the present disclosure.
  • These computer-readable program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine that makes these instructions when executed by the processor of the computer or other programmable data processing device , A device that implements the functions/actions specified in one or more blocks in the flowcharts and/or block diagrams is produced. It is also possible to store these computer-readable program instructions in a computer-readable storage medium. These instructions make computers, programmable data processing apparatuses, and/or other devices work in a specific manner. Thus, the computer-readable medium storing the instructions includes An article of manufacture, which includes instructions for implementing various aspects of the functions/actions specified in one or more blocks in the flowcharts and/or block diagrams.
  • each block in the flowchart or block diagram may represent a module, program segment, or part of an instruction, and the module, program segment, or part of an instruction contains one or more components for realizing the specified logical function.
  • Executable instructions may also occur in a different order from the order marked in the drawings. For example, two consecutive blocks can actually be executed substantially in parallel, or they can sometimes be executed in the reverse order, depending on the functions involved.
  • each block in the block diagram and/or flowchart, and the combination of the blocks in the block diagram and/or flowchart can be implemented by a dedicated hardware-based system that performs the specified functions or actions Or it can be realized by a combination of dedicated hardware and computer instructions. It is well known to those skilled in the art that implementation through hardware, implementation through software, and implementation through a combination of software and hardware are all equivalent.
  • plane detection is performed on the current frame image based on the sparse point data set without storing a large amount of point cloud data, which can increase the speed of plane detection, thereby improving the real-time performance of plane detection, and perform detection based on the sparse point data set It can be applied to mobile devices with limited computing power, avoiding display delays and improving user experience. Therefore, the present disclosure has strong practicability.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • General Physics & Mathematics (AREA)
  • Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Evolutionary Biology (AREA)
  • Evolutionary Computation (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • General Engineering & Computer Science (AREA)
  • Artificial Intelligence (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Multimedia (AREA)
  • Quality & Reliability (AREA)
  • Image Analysis (AREA)

Abstract

一种平面检测方法及装置、平面跟踪方法及装置，该平面检测方法包括：对双目摄像头采集环境场景的当前帧图像进行划分，检测划分后得到的每个子图像块对应的特征点（S2100）；根据特征点在世界坐标系下的三维坐标信息，获得当前帧图像的稀疏点数据集（S2200），稀疏点数据集包括多个特征点的三维坐标信息；基于稀疏点数据集对当前帧图像进行平面检测，获取当前帧图像的最优有效平面（S2300），最优有效平面为包含特征点最多的平面；该平面跟踪方法包括：根据当前帧图像的最优有效平面，获取当前帧图像的多个有效平面，有效平面的平面帧号与对应的图像的帧号一致（S3200）；将当前帧图像的多个有效平面与上一帧图像的多个有效平面进行融合，确定连续多帧图像的跟踪平面（S3300）。

Description

一种平面检测方法及装置、平面跟踪方法及装置
本公开要求于2020年01月07日提交中国专利局，申请号为202010025935.0，申请名称为“一种平面检测方法及装置、平面跟踪方法及装置”的中国专利申请的优先权，其全部内容通过引用结合在本公开中。
技术领域
本公开涉及图像处理技术领域，更具体地，本公开涉及一种平面检测方法及装置、一种平面跟踪方法及装置。
背景技术
目前，基于移动设备的增强现实(augmented reality，AR)应用已经成为现实，而且备受各大移动设备厂商和广大用户的青睐。在将虚拟对象融合到真实环境时，需要基于平面确定虚拟对象的锚点，从而在确定出的锚点处渲染出虚拟对象。因此，在基于移动设备的增强现实应用的计算过程中，平面检测是一项重要的功能。
而现有技术中，基于slam***对多幅图像进行处理，获得稠密点云，然后通过点云数据进行平面检测，获得全局平面。由于稠密点云是对多幅图像进行特征点提取得到的，稠密点云包含的特征点较多。这种方式计算复杂度高，对于硬件要求较高，成本高，制约了其使用范围。
但是，移动设备的计算能力有限，采用复杂度较高的算法进行平面检测，速度慢，用户体验较差。
发明内容
本公开的一个目的在于提供一种平面检测和跟踪的新技术方案。
根据本公开的第一方面，提供了一种平面检测方法，所述方法包括：
对双目摄像头采集环境场景的当前帧图像进行划分，检测划分后得到的每个子图像块对应的特征点；
根据所述特征点在世界坐标系下的三维坐标信息，获得所述当前帧图像的稀疏点数据集，所述稀疏点数据集包括多个所述特征点的三维坐标信息；
基于所述稀疏点数据集，对所述当前帧图像进行平面检测，获取所述当前帧图像的最优有效平面，所述最优有效平面为包含特征点最多的平面。
可选地，所述当前帧图像包括第一目图像和第二目图像，所述特征点包括第一特征点和第二特征点；
所述对双目摄像头采集环境场景的当前帧图像进行划分，检测划分后得到的每个子图像块对应的特征点，包括：
对所述第一目图像进行划分，获得多个第一子图像块；
检测出每个所述第一子图像块对应的第一特征点；
采用极线匹配方法在所述第二目图像中做特征点匹配，获取所述第二目图像中与所述第一特征点匹配的第二特征点；
利用所述第一特征点和所述第二特征点，计算所述第一特征点的三维坐标信息。
可选地，所述对所述第一目图像进行划分，获得多个第一子图像块之前，所述方法还包括：
以所述第一目图像的中心为原点，将欧式距离大于第一设定像素的点的像素状态设置为第二状态，将所述第一目图像中剩余的点的像素状态设置为第一状态。
可选地，所述检测出每个所述第一子图像块对应的第一特征点，包括：
对所述第一子图像块进行特征点检测，确定每个所述第一子图像块中目标点；
判断所述第一子图像块中所述目标点的像素状态，所述像素状态包括第一状态和第二状态；
如果所述第一子图像块中所述目标点的像素状态为第一状态，将所述 第一子图像块中的所述目标点确定为所述第一特征点；
如果所述第一子图像块中所述目标点的像素状态为第二状态，对下一个所述第一子图像块进行特征点检测，直至遍历每个所述第一子图像块。
可选地，所述检测出每个所述第一子图像块对应的第一特征点之后，所述方法还包括：
以从所述第一子图像块中获取的第一特征点为原点，将欧式距离小于第二设定像素的点的像素状态设置为第二状态。
可选地，所述基于所述稀疏点数据集，对所述当前帧图像进行平面检测，获取所述当前帧图像的最优有效平面，包括：
S1：根据从所述稀疏点数据集中随机抽取的三个特征点，确定当前平面；
S2：根据所述当前平面的法向量和第一内点的数量，确定当前有效平面，所述第一内点为与所述当前平面的距离小于第一设定距离的特征点；
S3：反复迭代S1-S2，直至满足预设迭代次数，将最后一次迭代得到的所述当前有效平面更新为所述当前帧图像的最优有效平面。
可选地，所述S3还包括：
每迭代S1-S2一次，根据所述当前有效平面的第一内点的占比更新迭代次数，其中，第一内点的占比为所述第一内点的数量与所述特征点集中特征点总数的比值。
可选地，所述基于所述稀疏点数据集，对所述当前帧图像进行平面检测，获取所述当前帧图像的最优有效平面，所述方法还包括：
S4：在所述最优有效平面包含的特征点中任意选取一个特征点，作为起始点；
S5：将所述起始点作为种子点，根据设定半径在所述最优有效平面包含的特征点中确定所述起始点的临近点；
S6：将所述临近点作为种子点，在所述最优有效平面包含的特征点中确定所述临近点的临近点，直至确定结束点，所述结束点为不存在临近点的特征点；
S7：计算所述种子点的数量，直至所述种子点的数量大于预设的最小聚类点数，输出一个有效平面；
S8：在所述最优有效平面的剩余特征点中重新选取一个特点作为起始点，执行S5-S7，直至遍历述最优有效平面包含的所有特征点，输出所述当前帧图像的多个有效平面。
根据本公开的第二方面，提供了一种平面跟踪方法，所述方法包括：
获取双目摄像头采集环境场景的连续多帧图像；
将当前帧图像的多个有效平面与上一帧图像的多个有效平面进行融合，确定连续多帧图像的跟踪平面；其中，根据所述当前帧图像的最优有效平面，获取所述当前帧图像的多个有效平面，所述有效平面的平面帧号与对应的图像的帧号一致。
可选地，所述将当前帧图像的多个有效平面与上一帧图像的多个有效平面进行融合，确定连续多帧图像的跟踪平面，包括：
将当前帧图像的有效平面依次与上一帧图像的多个有效平面进行比较；
判断所述当前帧图像的多个有效平面中是否存在满足平面融合条件的平面；
在所述当前帧图像的多个有效平面中存在满足平面融合条件的平面的情况下，将所述当前帧图像中满足平面融合条件的平面与所述上一帧图像的对应平面进行融合，得到第一平面，并将所述第一平面的平面帧号更新为所述当前帧图像的帧号；
在所述当前帧图像的多个有效平面中不存在满足平面融合条件的平面的情况下，将所述当前帧图像中不满足平面融合条件的平面作为第二平面；
根据所述第一平面和所述第二平面对所述上一帧图像的有效平面进行更新，确定连续多帧图像的跟踪平面。
可选地，所述平面融合条件包括：
所述当前帧图像的有效平面与所述上一帧图像的任意一个有效平面 的高度差小于第一设定高度；
所述当前帧图像的有效平面的法向量与所述上一帧图像的任意一个有效平面的法向量之间的夹角小于第一设定角度；以及，
落入所述上一帧图像的任意一个有效平面内的点占所述当前帧图像的有效平面内所有点的比例大于第一设定比例。
可选地，所述将所述当前帧图像中满足平面融合条件的平面与所述上一帧图像的对应平面进行融合，得到第一平面，包括：
计算所述当前帧图像中满足平面融合条件的平面内的所有点到所述上一帧图像的对应平面内任意一点的距离；
将到所述上一帧图像的对应平面内任意一点的距离大于第二设定距离的点加入所述对应平面，得到第一平面。
可选地，所述根据所述第一平面和所述第二平面对所述上一帧图像的有效平面进行更新，确定连续多帧图像的跟踪平面之前，所述方法还包括：
根据所述稀疏点数据集中特征点的三维坐标信息，计算所述第一平面和所述第二平面的平面参数；
根据所述平面参数，剔除所述第一平面和所述第二平面中的无效平面；其中，所述无效平面为平面长宽比例小于预设的长宽比例阈值的平面。
可选地，所述根据所述第一平面和所述第二平面对所述上一帧图像的有效平面进行更新，确定连续多帧图像的跟踪平面之前，所述方法还包括：
根据平面帧号，剔除所述第一平面和所述第二平面中的已丢失平面；其中，所述已丢失平面为所述平面帧号与当前帧图像的帧号的差值大于设定帧数的平面。
可选地，所述根据所述第一平面和所述第二平面对所述上一帧图像的有效平面进行更新，确定连续多帧图像的跟踪平面之后，所述方法还包括：
判断所述第一平面是否与所述当前帧图像中任意一个未融合的有效平面满足平面融合条件；
在所述第一平面与所述当前帧图像中任意一个未融合的有效平面满足平面融合条件的情况下，将所述第一平面与所述当前帧图像中对应有效 平面进行平面融合。
可选地，所述方法还包括：
计算所述跟踪平面内的点的数量；
如果所述跟踪平面内的点的数量超过第一设定数量，剔除所述跟踪平面的中心点，保留所述跟踪平面的边缘点。
根据本公开的第三方面，提供了一种平面检测装置，所述装置包括：
存储器，所述存储器被配置为存储计算机指令；
处理器，所述处理器被配置为从所述存储器中调用所述计算机指令，并在所述计算机指令的控制下执行如本公开第一方面中任一项所述的方法。
根据本公开的第四方面，提供了一种平面跟踪装置，所述装置包括：
存储器，所述存储器被配置为存储计算机指令；
处理器，所述处理器被配置为从所述存储器中调用所述计算机指令，并在所述计算机指令的控制下执行如本公开第二方面中任一项所述的方法。
根据本公开的一个实施例，基于稀疏点数据集对当前帧图像进行平面检测，不需要存储大量点云数据，能够提高平面检测的速度，从而提高平面检测的实时性，并且基于稀疏点数据集进行检测可以适用于计算能力有限的移动设备，避免显示画面的延时，提高用户体验。
通过以下参照附图对本公开的示例性实施例的详细描述，本公开的其它特征及其优点将会变得清楚。
附图说明
为了更清楚地说明本公开实施例的技术方案，下面将对实施例中所需要使用的附图作简单地介绍。应当理解，以下附图仅示出了本公开的某些实施例，因此不应被看作是对范围的限定。对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他相关的附图。
图1示出了本公开实施例提供的电子设备的硬件配置示意图；
图2示出了本公开实施例提供的一种平面检测方法的流程示意图一；
图3示出了本公开一个例子的一种平面检测方法的流程示意图；
图4示出了本公开实施例提供的一种平面检测方法的流程示意图二；
图5示出了本公开实施例提供的一种平面跟踪方法的流程示意图一；
图6示出了本公开实施例提供的一种平面跟踪方法的流程示意图二；
图7示出了本公开实施例提供的一种平面检测装置的框图；
图8示出了本公开实施例提供的一种平面跟踪装置的框图。
具体实施方式
现在将参照附图来详细描述本公开的各种示例性实施例。应注意到：除非另外具体说明，否则在这些实施例中阐述的部件和步骤的相对布置、数字表达式和数值不限制本公开的范围。
以下对至少一个示例性实施例的描述实际上仅仅是说明性的，决不作为对本公开及其应用或使用的任何限制。
对于相关领域普通技术人员已知的技术、方法和设备可能不作详细讨论，但在适当情况下，所述技术、方法和设备应当被视为说明书的一部分。
在这里示出和讨论的所有例子中，任何具体值应被解释为仅仅是示例性的，而不是作为限制。因此，示例性实施例的其它例子可以具有不同的值。
应注意到：相似的标号和字母在下面的附图中表示类似项，因此，一旦某一项在一个附图中被定义，则在随后的附图中不需要对其进行进一步讨论。
<硬件配置>
图1是可用于实现本公开实施例的电子设备100的硬件配置的框图。
该电子设备可以是虚拟现实(Virtual Reality，VR)设备、增强现实(Augmented Reality，AR)设备或者混合现实(Mixed Reality)设备等智能设备。
在一个例子中，如图1所示，该电子设备100可以包括处理器101、存储器102、接口装置103、通信装置104、显示装置105、输入装置106、 音频装置107、传感器108、摄像头109等。
其中，处理器101可以包括但不限于中央处理器CPU、微处理器MCU等。处理器101还可以包括图像处理器GPU(Graphics Processing Unit)等。存储器102可以包括但不限于ROM(只读存储器)、RAM(随机存取存储器)、诸如硬盘的非易失性存储器等。接口装置103可以包括但不限于USB接口、串行接口、并行接口、红外接口等。通信装置104例如能够进行有线或无线通信，具体的可以包括WiFi通信、蓝牙通信、2G/3G/4G/5G通信等。显示装置105例如是液晶显示屏、LED显示屏、触摸显示屏等。输入装置106可以包括但不限于触摸屏、键盘、体感输入等。音频装置107可以被配置为输入/输出语音信息。传感器108例如是图像传感器、红外传感器、激光传感器、压力传感器、陀螺仪传感器、磁传感器、加速度传感器、距离传感器、接近光传感器、环境光传感器、指纹传感器、触摸传感器、温度传感器等，传感器108可以被配置为测量电子设备100的位姿变化。摄像头109可以被配置为获取图像信息，摄像头109例如可以是双目摄像头。
应用于本说明书的一个实施例中，电子设备100用于获取环境场景的图像对图像进行平面检测和平面跟踪。图1所示的电子设备100仅仅是说明性的并且绝不意味着对本说明书实施例、其应用或使用的任何限制。本领域技术人员应当理解，尽管前面描述了电子设备100的多个装置，但是，本说明书实施例可以仅涉及其中的部分装置。本领域技术人员可以根据本说明书实施例所公开的方案设计指令，指令如何控制处理器进行操作，是本领域公知技术，故在此不再详细描述。
<平面检测方法>
图2是本说明书的一个实施例提供的平面检测方法的示意图。该实施例提供的平面检测方法，通过计算机技术实现，可以由图1所述的电子设备100实施。
该实施例提供的平面检测方法，包括步骤S2100-S2300。
步骤S2100，对双目摄像头采集环境场景的当前帧图像进行划分，检 测划分得到的每个子图像块对应的特征点。
该实施例中，环境场景例如可以是包括地面、桌子、平台等的场景。采用双目摄像头拍摄环境场景，对双目摄像头采集的当前帧图像进行分割，对分割后得到的每个子图像块进行特征提取，获取每个子图像块对应的特征点。该特征点可以是当前帧图像中比较显著的点，例如当前帧图像中的轮廓点、较暗区域中的亮点、较亮区域中的暗点等，该特征点为用于平面检测的点。
在一个实施例中，该双目摄像头例如可以设置在头戴式显示设备上，在此不做限制。该双目摄像头可以是高分辨率摄像头、低分辨率摄像头，双目摄像头也可以是鱼眼摄像头。
在一个实施例中，当前帧图像包括第一目图像和第二目图像，特征点包括与第一目图像对应的多个第一特征点和与第二目图像对应的多个第二特征点。第一特征点用于平面检测，第二特征点用于计算第一特征点的三维坐标信息。例如，双目摄像头包括左目摄像头和右目摄像头，将左目摄像头采集的当前帧图像作为第一目图像，将右目摄像头采集的当前帧图像作为第二目图像。
在该实施例中，对双目摄像头采集环境场景的当前帧图像进行划分，检测划分得到的每个子图像块对应的特征点的步骤，可以进一步包括：步骤S2110-S2140。
步骤S2110，对第一目图像进行划分，获得多个第一子图像块。
该实施例中，划分后的多个第一子图像块可以是大小相同的图像块，也可以是大小不同的图像块。
在一个更具体的例子中，按预设的划分方式对第一目图像进行划分，获得多个第一子图像块。例如，将第一目图像划分为i*j个大小相同的子图像块。
步骤S2120，检测出每个第一子图像块对应的第一特征点。
该实施例中，对每个第一子图像块进行特征点检测，可以采用FAST(Feature from Accelerated Segment Test，加速分割测试特征)角点检测算 法、SIFT(Scale Invariant Feature Transform，尺度不变特征变换)算法、或者ORB(Oriented FAST and Rotated BRIEF，快速特征点提取和描述)算法进行特征点检测，在此不做任何限定。
在一个更具体的例子中，对第一子图像块进行FAST角点检测，选取子图像块中Harris响应最高的角点为该第一子图像块对应的第一特征点。
步骤S2130，采用极线匹配方法在第二目图像中做特征点匹配，获取第二目图像中与第一特征点匹配对应的第二特征点。
步骤S2140，利用第一特征点和第二特征点，计算第一特征点的三维坐标信息。
在一个实施例中，对第一目图像进行划分，获得多个第一子图像块之前，方法还包括：以第一目图像的中心为原点，将欧式距离大于第一设定像素的点的像素状态设置为第二状态，将第一目图像中剩余的点的像素状态设置为第一状态。
该实施例中，根据点在第一目图像中的位置，设置点的像素状态。像素状态包括第一状态和第二状态。根据第一目图像中点的畸变程度，将第一目图像中畸变较小的点的像素状态设置为第一状态，将第一目图像中畸变较大的点的像素状态设置为第二状态。例如，第一状态为true，第二状态为false。
第一设定像素可以根据工程经验或者试验仿真结果来设定的。例如，以第一目图像的中心为原点，将欧式距离大于m个像素的点的像素状态设置为false，将第一目图像中剩余的点的像素状态设置为true。
在本公开的实施例中，在对第一目图像进行划分之前，设置第一目图像中点的像素状态，能够将图像中畸变严重的点排除，避免利用点的三维坐标信息计算出的平面参数的误差较大，结合后续步骤，可以提高平面检测的准确性。此外，排除图像中畸变严重的点，使本方法可以适用于鱼眼摄像头，能够获得更大的视场角。
在一个实施例中，检测出每个第一子图像块对应的第一特征点的步骤，可以进一步包括：步骤S2121-S2124。
步骤S2121，对第一子图像块进行特征点检测，确定每个第一子图像块中目标点。
在一个更具体的例子中，对第一子图像块进行FAST角点检测，选取子图像块中Harris响应最高的角点为该第一子图像块对应的目标点。
步骤S2122，判断第一子图像块中目标点的像素状态，像素状态包括第一状态和第二状态。
步骤S2123，如果第一子图像块中目标点的像素状态为第一状态，将第一子图像块中的目标点确定为第一特征点；
步骤S2124，如果第一子图像块中目标点的像素状态为第二状态，对下一个第一子图像块进行特征点检测，直至遍历每个第一子图像块。
在本公开的实施例中，根据第一目图像中点的像素状态，确定第一子图像块对应的第一特征点，能够避免畸变严重的点影响计算平面参数的准确性，结合后续步骤，可以提高平面检测的准确性。
在一个实施例中，检测出每个第一子图像块对应的第一特征点之后，方法还包括：以从第一子图像块中获取的第一特征点为原点，将欧式距离小于第二设定像素的点的像素状态设置为第二状态。第二设定像素可以根据工程经验或者试验仿真结果来设定的。例如，以一个第一子图像块对应的第一特征点为原点，将欧式距离小于n个像素的点的像素状态设置为false。
在本公开的实施例中，在确定一个第一子图像块对应的第一特征点之后，再次设置第一目图像中点的像素状态，将第一特征点周围的点的像素状态设置为第二状态，能够避免检测出的第一特征点过于密集，检测出的第一特征点分布更均匀，从而能够提高检测的准确性。
<例子一>
如图3所示，该对双目摄像头采集环境场景的当前帧图像进行划分，检测划分后得到的每个子图像块对应的特征点的步骤，可以进一步包括如下步骤S301～S308。
步骤S301，以第一目图像的中心为原点，将欧式距离大于m个像素 的点的像素状态设置为false，将第一目图像中剩余的点的像素状态设置为true。
步骤S302，将第一目图像划分为i*j个大小相同的第一子图像块，并对第一子图像块进行编号，第一子图像块的编号num为0～i*j-1。
步骤S303，对编号为num第一子图像块进行FAST角点检测，选取该第一子图像块中Harris响应最高的角点为该第一子图像块对应的目标点。
步骤S304，判断该第一子图像块中目标点的像素状态是否为true，如果是，进入步骤S305，否则，进入步骤S307。
步骤S305，将该目标点确定为编号为num的第一子图像块对应的第一特征点。
步骤S306，以编号为num的第一子图像块对应的第一特征点为原点，将欧式距离小于n个像素的点的像素状态设置为false。
步骤S307，判断第一子图像的编号是否小于i*j-1，如果是，进入步骤S303，否则，进入步骤S308。
步骤S308，采用极线匹配方法在第二目图像中做特征点匹配，获取第二目图像中与第一特征点匹配的第二特征点。
在对双目摄像头采集环境场景的当前帧图像进行划分，检测划分得到的每个子图像块对应的特征点之后，进入：
步骤S2200，根据特征点在世界坐标系下的三维坐标信息，获得当前帧图像的稀疏点数据集。
本实施例中，稀疏点数据集包括多个特征点的三维坐标信息。该当前帧图像的特征点包括第一目图像中的多个第一特征点。
在一个实施例中，根据特征点在世界坐标系下的三维坐标信息，获得当前帧图像的稀疏点数据集，可以进一步包括：步骤S2210-S2230。
步骤S2210，通过三角化，根据第一特征点和第二特征点，计算第一目摄像头的参考系下第一目图像的第一特征点的三维坐标信息。
步骤S2220，获取采集当前帧图像时双目摄像头的位姿。
在一个更具体的例子中，双目摄像头的位姿可以是双目摄像头的6Dof信息。双目摄像头的位姿可以通过SLAM(Simultaneous Localization And Mapping，可译为同步定位与建图)***或者光学追踪***获取。
步骤S2230，根据采集当前帧图像时双目摄像头的位姿，进行坐标系转换，获得当前帧图像中第一目图像的第一特征点的在世界坐标系下的三维坐标信息。第一目图像的第一特征点的在世界坐标系下的三维坐标信息构成稀疏点数据集。
在本公开的实施例中，通过SLAM***获取采集当前帧图像时双目摄像头的位姿，根据双目摄像头的位姿进行坐标系转换，结合后续步骤，能够准确检测出水平面或者垂直平面。
在根据特征点在世界坐标系下的三维坐标信息，获得当前帧图像的稀疏点数据集之后，进入：
步骤S2300，基于稀疏点数据集对当前帧图像进行平面检测，获取当前帧图像的最优有效平面。
本实施例中，最优有效平面为包含特征点最多的平面。在本公开的实施例中，基于稀疏点数据集对当前帧图像进行平面检测，能够提高平面检测的速度，从而提高平面检测的实时性，并且基于稀疏点数据集进行检测可以适用于计算能力有限的移动设备，避免显示画面的延时，提高用户体验。
在一个实施例中，基于稀疏点数据集对当前帧图像进行平面检测，获取当前帧图像的最优有效平面的步骤，可以进一步包括：S2310-S2330。
步骤S2310，根据从稀疏点数据集中随机抽取的三个特征点，确定当前平面。
在一个更具体的例子中，在稀疏点数据集中随机抽取三个特征点，根据这三个特征点的三维坐标信息，判断选取的三个特征点是否共线，如果选取的三个特征点不共线，则根据这三个特征点确定当前平面，如果选取的三个特征点共线，则在稀疏点数据集中重新抽取特征点。
步骤S2320，根据当前平面的法向量和第一内点的数量，确定当前有效平面，第一内点为与当前平面的距离小于第一设定距离的特征点。
该实施例中，当前平面的法向量可以根据特征点的三维坐标信息进行计算。例如，计算当前平面中的所有内点的协方差矩阵，将当前平面中的所有内点中最小特征值对应的特征向量作为当前平面的法向量。
根据当前平面的法向量确定当前有效平面，具体地，根据计算得到的当前平面的法向量与参考法向量之间夹角的大小，判定当前平面是否为当前有效平面。
该例子中，参考法向量可以根据所要检测的平面的类型确定，例如，检测水平面时，参考法向量为重力方向向量。
根据当前平面的第一内点的数量确定当前有效平面，具体地，根据当前帧图像中的点到当前平面的距离确定第一内点，根据第一内点的数量，判定当前平面是否为当前有效平面。
步骤S2330，反复迭代步骤S2310-S2320，直至满足预设迭代次数，将最后一次迭代得到的当前有效平面更新为当前帧图像的最优有效平面。
该实施例中，每迭代一次步骤S2310-S2320，当前有效平面的第一内点的数量发生变化，需要根据变化后的第一内点重新计算当前有效平面的法向量，以保证检测平面的有效性。
在一个实施例中，在步骤S2330中，还包括：每迭代S1-S2一次，根据当前有效平面的第一内点的占比更新迭代次数，其中，第一内点的占比为第一内点的数量与特征点集中特征点总数的比值。
该实施例中，第一内点的占比越大，迭代次数越少，第一内点的占比越小，迭代次数越多。根据当前有效平面的第一内点的占比更新迭代次数，能够提高平面检测的效率，并且快速检测出特征点最多的有效平面。
<例子二>
该基于稀疏点数据集对当前帧图像进行平面检测，获取当前帧图像的最优有效平面点的步骤，可以进一步包括如下步骤S401～S408。
步骤S401，从当前帧图像的稀疏点数据集中随机抽取三个特征点；
步骤S402，根据这三个特征点的三维坐标信息，判断选取的三个特征点是否共线，若是，进行步骤S403，若否，执行步骤S401；
步骤S403，根据选取的三个特征点确定当前平面，计算当前平面的法向量；
步骤S404，判断当前平面的法向量是否符合要求，若是，执行步骤S405，若否，执行步骤S401；
该步骤中，计算当前平面的法向量与参考法向量之间的夹角，如果当前平面的法向量与参考法向量之间的夹角小于第二设定角度，判断当前平面的法向量符合要求，如果当前平面的法向量与参考法向量之间的夹角不小于第二设定角度，判断当前平面的法向量不符合要求。
参考法向量可以根据所要检测的平面的类型确定，第二设定角度可以根据工程经验或者试验仿真结果来设定。例如，检测水平面时，参考法向量为重力方向向量，第二设定角度为10°，如果当前平面的法向量与重力方向向量之间的夹角小于10°，判断当前平面的法向量符合要求。检测垂直面时，参考法向量为重力方向向量，第一设定角度为80°，如果当前平面的法向量与重力方向向量之间的夹角大于80°，判断当前平面的法向量符合要求。
步骤S405，除选取的三个特征点以外，计算稀疏点数据集中的所有特征点到当前平面的距离，将与当前平面的距离小于第一设定距离的特征点设为当前平面的第一内点；
第一设定距离可以根据工程经验或者试验仿真结果来设定的。例如，2cm。
步骤S406，判断当前平面的第一内点的数量是否大于上一个有效平面的第一内点的数量，若是，根据当前平面的第一内点计算当前有效平面的法向量若否，执行步骤S401；
步骤S407，根据第一内点重新计算当前平面的法向量；
步骤S408，判断当前平面的法向量是否符合要求，若是，将当前平面作为当前有效平面，若否，执行步骤S401；
步骤S409，反复迭代S1-S7，直至满足预设迭代次数，将当前有效平面更新为最优有效平面，最优有效平面为包含特征点最多的平面。
在一个实施例中，基于稀疏点数据集对当前帧图像进行平面检测，获取当前帧图像的最优有效平面之后，该方法还包括：对获取的最优有效平面进行平面聚类，输出当前帧图像的多个有效平面。如图5所示，具体包括步骤S2400-S2800。
步骤S2400，在最优有效平面包含的特征点中任意选取一个特征点，作为起始点。
步骤S2500，将起始点作为种子点，根据设定半径在最优有效平面包含的特征点中确定起始点的临近点。
步骤S2600，将临近点作为种子点，在最优有效平面包含的特征点中确定临近点的临近点，直至确定结束点，结束点为不存在临近点的特征点。
步骤S2700，计算种子点的数量，直至种子点的数量大于预设的最小聚类点数，输出一个有效平面。
步骤S2800，在最优有效平面的剩余特征点中重新选取一个特点作为起始点，执行S2500-S2700，直至遍历述最优有效平面包含的所有特征点，输出当前帧图像的多个有效平面。
剩余特征点为输出上一个有效平面时没有选取的特征点。
根据本公开的实施例，基于稀疏点数据集对当前帧图像进行平面检测，不需要存储大量点云数据，能够提高平面检测的速度，从而提高平面检测的实时性，并且基于稀疏点数据集进行检测可以适用于计算能力有限的移动设备，避免显示画面的延时，提高用户体验。
<平面跟踪方法>
图5是本说明书的一个实施例提供的平面跟踪方法的示意图。该实施例提供的平面跟踪方法，通过计算机技术实现，可以由图1所述的电子设备100实施。
该实施例提供的平面跟踪方法，基于前述提供的平面检测方法，该方法包括步骤S3100-S3200。
步骤S3100，获取双目摄像头采集环境场景的连续多帧图像。
步骤S3200，根据当前帧图像的最优有效平面，获取当前帧图像的多个有效平面，有效平面的平面帧号与对应的图像的帧号一致。
本实施例中，对当前帧图像的最优有效平面进行平面聚类，获取当前帧图像的多个有效平面，当前帧图像的有效平面的平面帧号与对应的图像的帧号一致。
步骤S3300，将当前帧图像的多个有效平面与上一帧图像的多个有效平面进行融合，确定连续多帧图像的跟踪平面。
在本公开的实施例中，对检测出每一帧图像的多个有效平面，基于每一帧图像的有效平面，进行多帧图像之间的平面融合，能够提高平面融合和扩展的稳定性，融合后平面边缘的贴合性更好，进一步随着环境场景的变化，基于融合后的平面可以实现平面跟踪，从而随着用户视角的变化，平面向各个方向稳定扩展，从而提高用户体验。
在一个实施例中，如图6所示，将当前帧图像的多个有效平面与上一帧图像的多个有效平面进行融合，确定连续多帧图像的跟踪平面的步骤，可以进一步包括步骤S3310-S3350。
步骤S3310，将当前帧图像的有效平面依次与上一帧图像的多个有效平面进行比较。
步骤S3320，判断当前帧图像的多个有效平面中是否存在满足平面融合条件的平面。
该例子中，平面融合条件包括：
当前帧图像的有效平面与上一帧图像的任意一个有效平面的高度差小于第一设定高度；当前帧图像的有效平面的法向量与上一帧图像的任意一个有效平面的法向量之间的夹角小于第一设定角度；以及，落入上一帧图像的任意一个有效平面内的点占当前帧图像的有效平面内所有点的比例大于第一设定比例。
该例子中，第一设定高度可以根据工程经验或者试验仿真结果来设定。第一设定角可以根据工程经验或者试验仿真结果来设定。第一设定比例可以根据工程经验或者试验仿真结果来设定。上述三个平面融合条件同时满足，则判定当前帧图像的多个有效平面中存在满足平面融合条件的平面。
步骤S3330，在当前帧图像的多个有效平面中存在满足平面融合条件的平面的情况下，将当前帧图像中满足平面融合条件的平面与上一帧图像的对应平面进行融合，得到第一平面，并将第一平面的平面帧号更新为所述当前帧图像的帧号。
在一个更具体的例子中，将当前帧图像中满足平面融合条件的平面与上一帧图像的对应平面进行融合的步骤，可以进一步包括：步骤S3331-S3332。
步骤S3331，计算当前帧图像中满足平面融合条件的平面内的所有点到上一帧图像的对应平面内任意一点的距离。
步骤S3332，将到上一帧图像的对应平面内任意一点的距离大于第二设定距离的点加入对应平面，得到第一平面。
该例子中，当前帧图像中满足平面融合条件的平面内到上一帧图像的对应平面内任意一点的距离不大于第二设定距离的点被认为是两帧图像中重合的点，根据到上一帧图像的对应平面内任意一点的距离，进行两个平面之间的点融合，能够避免重复点过多。
步骤S3340，在当前帧图像的多个有效平面中不存在满足平面融合条件的平面的情况下，将当前帧图像中不满足平面融合条件的平面作为第二平面。
步骤S3350，根据第一平面和第二平面对上一帧图像的有效平面进行更新，确定连续多帧图像的跟踪平面。
该例子中，连续多帧图像的跟踪平面包括第一平面、第二平面和上一帧图像中未与当前帧图像中的任意平面融合的平面。
例如，上一帧图像包括a1有效平面和a2有效平面，当前帧图像包括b1有效平面和b2有效平面，当前帧图像的b1有效平面与上一帧图像的a1 有效平面r融合成c1有效平面，当前帧图像的b2有效平面与上一帧图像的a2有效平面r融合成c2有效平面，则连续多帧图像的跟踪平面包括c1有效平面和c2有效平面。
例如，上一帧图像包括a1有效平面和a2有效平面，当前帧图像包括b1有效平面和b2有效平面，当前帧图像的b1有效平面与上一帧图像的a1有效平面r融合成c1有效平面，当前帧图像的b2有效平面不与上一帧图像的任一有效平面融合，则连续多帧图像的跟踪平面包括c1有效平面、a2有效平面和b2有效平面。
在一个实施例中，根据第一平面和第二平面对上一帧图像的有效平面进行更新，确定连续多帧图像的跟踪平面的步骤之前，该平面跟踪方法还可以进一步包括：步骤S4100-S4200。
步骤S4100，根据稀疏点数据集中特征点的三维坐标信息，计算第一平面和第二平面的平面参数。
该例子中，平面参数包括平面高度、平面长宽比例等。平面参数根据平面内所有点的三维坐标信息计算得到。平面高度为平面平均高度，具体地，根据平面内包含的点，计算每个点对应的平面高度，再计算所有点对应的平面高度的平均值，该平均值为该平面对应的平面高度。平面长宽比例可以根据平面的四个边角点的三维坐标信息计算得到。
步骤S4200，根据平面参数，剔除第一平面和第二平面中的无效平面；其中，该无效平面为平面长宽比例小于预设的长宽比例阈值的平面。
无效平面例如可以是狭长平面或者过小平面，无效平面影响虚拟对象的生成。无效平面可以根据平面长宽比例判定。预设的长宽比例阈值可以根据工程经验或者试验仿真结果来设定。在本公开的实施例中，根据平面参数剔除无效平面，避免无效平面对虚拟对象的生成造成影响，保证跟踪平面的有效性。
在一个实施例中，根据第一平面和第二平面对上一帧图像的有效平面进行更新，确定连续多帧图像的跟踪平面的步骤之前，该平面跟踪方法还可以进一步包括：步骤S5100。
步骤S5100，根据平面帧号，剔除第一平面和第二平面中的已丢失平面；其中，该已丢失平面为所述平面帧号与当前帧图像的帧号的差值大于设定帧数的平面。
该例子中，随着用户视角的变化环境场景发生变化，检测出的平面也发生变化，根据平面帧号剔除已丢失平面，减少数据处理量，提高跟踪平面的有效性。
在一个实施例中，根据第一平面和第二平面对上一帧图像的有效平面进行更新，确定连续多帧图像的跟踪平面的步骤之前，该平面跟踪方法还可以进一步包括：步骤S6100-S6200。
步骤S6100，计算跟踪平面内的点的数量。
步骤S6200，如果跟踪平面内的点的数量超过第一设定数量，剔除跟踪平面的中心点，保留跟踪平面的边缘点。
第一设定数量可以根据工程经验或者试验仿真结果来设定。
在本公开的实施例中，随着平面的融合和扩展，平面内包含的点的数量不断增大，数据占用的存储空间越来越大，剔除平面的中心点而仅保留平面的边缘点，能够减小数据占用的存储空间，从而提高处理速度，实现平面检测、融合和扩展的速度，用户体验更好。
在一个实施例中，根据第一平面和所述第二平面对上一帧图像的有效平面进行更新，确定连续多帧图像的跟踪平面的步骤之后，该平面跟踪方法还可以进一步包括：步骤S7100-S7200。
步骤S7100，判断第一平面是否与当前帧图像中任意一个未融合的有效平面满足平面融合条件。
步骤S7200，在第一平面与当前帧图像中任意一个未融合的有效平面满足平面融合条件的情况下，将第一平面与所述当前帧图像中对应有效平面进行平面融合。
在本公开的实施例中，将当前帧图像的多个有效平面与上一帧图像的多个有效平面进行融合之后，当前帧图像中满足平面融合条件的有效平面 发生变化，在多帧图像的有效平面融合之后，再次判断更新后当前帧图像的多个有效平面之间是否还存在可以融合的平面，对可以融合的平面进行融合，避免出现平面重叠的现象。
根据本公开的实施例，对检测出每一帧图像的多个有效平面，基于每一帧图像的有效平面，进行多帧图像之间的平面融合，能够提高平面融合和扩展的稳定性，融合后平面边缘的贴合性更好，进一步随着环境场景的变化，基于融合后的平面可以实现平面跟踪，从而随着用户视角的变化，平面向各个方向稳定扩展，从而提高用户体验。
<平面检测装置>
在本实施例中，提供一种平面检测装置700，该平面检测装置700可以是如图1所示的电子设备100。
如图7所示，该平面检测装置700包括处理器710和存储器720。
存储器720，可以被配置为存储可执行的指令；
处理器710，可以被配置为根据可执行的指令的控制，执行如本公开前述实施例中提供的平面检测方法。
示例性的，处理器710可以被配置为根据可执行的指令的控制，执行如本公开前述实施例中的步骤S2100-S2300、步骤S2110-S2140、步骤S2121-S2124、步骤S2210-S2230、S2310-S2330和/或步骤S2400-S2800。
根据本公开的实施例，基于稀疏点数据集对当前帧图像进行平面检测，不需要存储大量点云数据，能够提高平面检测的速度，从而提高平面检测的实时性，并且基于稀疏点数据集进行检测可以适用于计算能力有限的移动设备，避免显示画面的延时，提高用户体验。
<平面跟踪装置>
在本实施例中，提供一种平面跟踪装置800，该平面跟踪装置800可以是如图1所示的电子设备100。
如图8所示，该平面跟踪装置800包括处理器810和存储器820。
存储器820，可以被配置为存储可执行的指令；
处理器810，可以被配置为根据可执行的指令的控制，执行如本公开前述实施例中提供的平面跟踪方法。
示例性的，处理器810可以被配置为根据可执行的指令的控制，执行如本公开前述实施例中的步骤S3100-S3300、步骤S3310-S3350、步骤S3331-S3332、步骤S4100-S4200、步骤S5100、步骤S6100-S6200和/或步骤S7100-S7200。
根据本公开的实施例，对检测出每一帧图像的多个有效平面，基于每一帧图像的有效平面，进行多帧图像之间的平面融合，能够提高平面融合和扩展的稳定性，融合后平面边缘的贴合性更好，进一步随着环境场景的变化，基于融合后的平面可以实现平面跟踪，从而随着用户视角的变化，平面向各个方向稳定扩展，从而提高用户体验。
本实施例提供了一种计算机可读存储介质，其上存储有计算机程序，所述计算机程序在被处理器执行时实现上述方法实施例中任一项所述的方法。
本说明书中的各个实施例均采用递进的方式描述，各个实施例之间相同相似的部分相互参见即可，每个实施例重点说明的都是与其他实施例的不同之处，但本领域技术人员应当清楚的是，上述各实施例可以根据需要单独使用或者相互结合使用。另外，对于装置实施例而言，由于其是与方法实施例相对应，所以描述得比较简单，相关之处参见方法实施例的对应部分的说明即可。以上所描述的***实施例仅仅是示意性的，其中作为分离部件说明的模块可以是或者也可以不是物理上分开的。
本公开可以是***、方法和/或计算机程序产品。计算机程序产品可以包括计算机可读存储介质，其上载有用于使处理器实现本公开的各个方面的计算机可读程序指令。
计算机可读存储介质可以是可以保持和存储由指令执行设备使用的指令的有形设备。计算机可读存储介质例如可以是――但不限于――电存 储设备、磁存储设备、光存储设备、电磁存储设备、半导体存储设备或者上述的任意合适的组合。计算机可读存储介质的更具体的例子(非穷举的列表)包括：便携式计算机盘、硬盘、随机存取存储器(RAM)、只读存储器(ROM)、可擦式可编程只读存储器(EPROM或闪存)、静态随机存取存储器(SRAM)、便携式压缩盘只读存储器(CD-ROM)、数字多功能盘(DVD)、记忆棒、软盘、机械编码设备、例如其上存储有指令的打孔卡或凹槽内凸起结构、以及上述的任意合适的组合。这里所使用的计算机可读存储介质不被解释为瞬时信号本身，诸如无线电波或者其他自由传播的电磁波、通过波导或其他传输媒介传播的电磁波(例如，通过光纤电缆的光脉冲)、或者通过电线传输的电信号。
这里所描述的计算机可读程序指令可以从计算机可读存储介质下载到各个计算/处理设备，或者通过网络、例如因特网、局域网、广域网和/或无线网下载到外部计算机或外部存储设备。网络可以包括铜传输电缆、光纤传输、无线传输、路由器、防火墙、交换机、网关计算机和/或边缘服务器。每个计算/处理设备中的网络适配卡或者网络接口从网络接收计算机可读程序指令，并转发该计算机可读程序指令，以供存储在各个计算/处理设备中的计算机可读存储介质中。
用于执行本公开操作的计算机程序指令可以是汇编指令、指令集架构(ISA)指令、机器指令、机器相关指令、微代码、固件指令、状态设置数据、或者以一种或多种编程语言的任意组合编写的源代码或目标代码，所述编程语言包括面向对象的编程语言—诸如Smalltalk、C++等，以及常规的过程式编程语言—诸如“C”语言或类似的编程语言。计算机可读程序指令可以完全地在用户计算机上执行、部分地在用户计算机上执行、作为一个独立的软件包执行、部分在用户计算机上部分在远程计算机上执行、或者完全在远程计算机或服务器上执行。在涉及远程计算机的情形中，远程计算机可以通过任意种类的网络—包括局域网(LAN)或广域网(WAN)—连接到用户计算机，或者，可以连接到外部计算机(例如利用因特网服务提供商来通过因特网连接)。在一些实施例中，通过利用计算机可读程序指 令的状态信息来个性化定制电子电路，例如可编程逻辑电路、现场可编程门阵列(FPGA)或可编程逻辑阵列(PLA)，该电子电路可以执行计算机可读程序指令，从而实现本公开的各个方面。
这里参照根据本公开实施例的方法、装置(***)和计算机程序产品的流程图和/或框图描述了本公开的各个方面。应当理解，流程图和/或框图的每个方框以及流程图和/或框图中各方框的组合，都可以由计算机可读程序指令实现。
这些计算机可读程序指令可以提供给通用计算机、专用计算机或其它可编程数据处理装置的处理器，从而生产出一种机器，使得这些指令在通过计算机或其它可编程数据处理装置的处理器执行时，产生了实现流程图和/或框图中的一个或多个方框中规定的功能/动作的装置。也可以把这些计算机可读程序指令存储在计算机可读存储介质中，这些指令使得计算机、可编程数据处理装置和/或其他设备以特定方式工作，从而，存储有指令的计算机可读介质则包括一个制造品，其包括实现流程图和/或框图中的一个或多个方框中规定的功能/动作的各个方面的指令。
也可以把计算机可读程序指令加载到计算机、其它可编程数据处理装置、或其它设备上，使得在计算机、其它可编程数据处理装置或其它设备上执行一系列操作步骤，以产生计算机实现的过程，从而使得在计算机、其它可编程数据处理装置、或其它设备上执行的指令实现流程图和/或框图中的一个或多个方框中规定的功能/动作。
附图中的流程图和框图显示了根据本公开的多个实施例的***、方法和计算机程序产品的可能实现的体系架构、功能和操作。在这点上，流程图或框图中的每个方框可以代表一个模块、程序段或指令的一部分，所述模块、程序段或指令的一部分包含一个或多个用于实现规定的逻辑功能的可执行指令。在有些作为替换的实现中，方框中所标注的功能也可以以不同于附图中所标注的顺序发生。例如，两个连续的方框实际上可以基本并行地执行，它们有时也可以按相反的顺序执行，这依所涉及的功能而定。也要注意的是，框图和/或流程图中的每个方框、以及框图和/或流程图中的 方框的组合，可以用执行规定的功能或动作的专用的基于硬件的***来实现，或者可以用专用硬件与计算机指令的组合来实现。对于本领域技术人员来说公知的是，通过硬件方式实现、通过软件方式实现以及通过软件和硬件结合的方式实现都是等价的。
以上已经描述了本公开的各实施例，上述说明是示例性的，并非穷尽性的，并且也不限于所披露的各实施例。在不偏离所说明的各实施例的范围和精神的情况下，对于本技术领域的普通技术人员来说许多修改和变更都是显而易见的。本文中所用术语的选择，旨在最好地解释各实施例的原理、实际应用或对市场中的技术改进，或者使本技术领域的其它普通技术人员能理解本文披露的各实施例。本公开的范围由所附权利要求来限定。
工业实用性
通过本公开实施例，基于稀疏点数据集对当前帧图像进行平面检测，不需要存储大量点云数据，能够提高平面检测的速度，从而提高平面检测的实时性，并且基于稀疏点数据集进行检测可以适用于计算能力有限的移动设备，避免显示画面的延时，提高用户体验。因此本公开具有很强的实用性。

Claims (11)

  1. 一种平面检测方法，所述方法包括：
    对双目摄像头采集环境场景的当前帧图像进行划分，检测划分后得到的每个子图像块对应的特征点；
    根据所述特征点在世界坐标系下的三维坐标信息，获得所述当前帧图像的稀疏点数据集，所述稀疏点数据集包括多个所述特征点的三维坐标信息；
    基于所述稀疏点数据集对当前帧图像进行平面检测，获取所述当前帧图像的最优有效平面，所述最优有效平面为包含特征点最多的平面。
  2. 根据权利要求1所述的方法，其中，所述当前帧图像包括第一目图像和第二目图像，所述特征点包括第一特征点和第二特征点；
    所述对双目摄像头采集环境场景的当前帧图像进行划分，检测划分后得到的每个子图像块对应的特征点，包括：
    对所述第一目图像进行划分，获得多个第一子图像块；
    检测出每个所述第一子图像块对应的第一特征点；
    采用极线匹配方法在所述第二目图像中做特征点匹配，获取所述第二目图像中与所述第一特征点匹配的第二特征点；
    利用所述第一特征点和所述第二特征点，计算所述第一特征点的三维坐标信息。
  3. 根据权利要求1或2所述的方法，其中，所述对所述第一目图像进行划分，获得多个第一子图像块之前，所述方法还包括：
    以所述第一目图像的中心为原点，将欧式距离大于第一设定像素的点的像素状态设置为第二状态，将所述第一目图像中剩余的点的像素状态设置为第一状态。
  4. 根据权利要求1-3任一项所述的方法，其中，所述检测出每个所 述第一子图像块对应的第一特征点，包括：
    对所述第一子图像块进行特征点检测，确定每个所述第一子图像块中目标点；
    判断所述第一子图像块中所述目标点的像素状态，所述像素状态包括第一状态和第二状态；
    如果所述第一子图像块中所述目标点的像素状态为第一状态，将所述第一子图像块中的所述目标点确定为所述第一特征点；
    以从所述第一子图像块中获取的第一特征点为原点，将欧式距离小于第二设定像素的点的像素状态设置为第二状态；
    如果所述第一子图像块中所述目标点的像素状态为第二状态，对下一个所述第一子图像块进行特征点检测，直至遍历每个所述第一子图像块。
  5. 根据权利要求1-4任一项所述的方法，其中，所述基于所述稀疏点数据集对所述当前帧图像进行平面检测，获取所述当前帧图像的最优有效平面，包括：
    S1：根据从所述稀疏点数据集中随机抽取的三个特征点，确定当前平面；
    S2：根据所述当前平面的法向量和第一内点的数量，确定当前有效平面，所述第一内点为与所述当前平面的距离小于第一设定距离的特征点；
    S3：反复迭代S1-S2，直至满足预设迭代次数，将最后一次迭代得到的所述当前有效平面更新为所述当前帧图像的最优有效平面，其中，每迭代S1-S2一次，根据所述当前有效平面的第一内点的占比更新迭代次数，所述第一内点的占比为所述第一内点的数量与所述特征点集中特征点总数的比值。
  6. 一种平面跟踪方法，其中，所述方法包括：
    获取双目摄像头采集环境场景的连续多帧图像；
    根据当前帧图像的最优有效平面，获取所述当前帧图像的多个有效平 面，所述有效平面的平面帧号与对应的图像的帧号一致；
    将当前帧图像的多个有效平面与上一帧图像的多个有效平面进行融合，确定连续多帧图像的跟踪平面。
  7. 根据权利要求6所述的方法，其中，所述将当前帧图像的多个有效平面与上一帧图像的多个有效平面进行融合，确定连续多帧图像的跟踪平面，包括：
    将当前帧图像的有效平面依次与上一帧图像的多个有效平面进行比较；
    判断所述当前帧图像的多个有效平面中是否存在满足平面融合条件的平面；
    在所述当前帧图像的多个有效平面中存在满足平面融合条件的平面的情况下，将所述当前帧图像中满足平面融合条件的平面与所述上一帧图像的对应平面进行融合，得到第一平面，并将所述第一平面的平面帧号更新为所述当前帧图像的帧号；
    在所述当前帧图像的多个有效平面中不存在满足平面融合条件的平面的情况下，将所述当前帧图像中不满足平面融合条件的平面作为第二平面；
    根据所述第一平面和所述第二平面对所述上一帧图像的有效平面进行更新，确定连续多帧图像的跟踪平面。
  8. 根据权利要求6或7所述的方法，其中，所述平面融合条件包括：
    所述当前帧图像的有效平面与所述上一帧图像的任意一个有效平面的高度差小于第一设定高度；
    所述当前帧图像的有效平面的法向量与所述上一帧图像的任意一个有效平面的法向量之间的夹角小于第一设定角度；以及，
    落入所述上一帧图像的任意一个有效平面内的点占所述当前帧图像的有效平面内所有点的比例大于第一设定比例。
  9. 根据权利要求6-8任一项所述的方法，其中，所述根据所述第一平面和所述第二平面对所述上一帧图像的有效平面进行更新，确定连续多帧图像的跟踪平面之前，所述方法还包括：
    根据所述稀疏点数据集中特征点的三维坐标信息，计算所述第一平面和所述第二平面的平面参数；
    根据所述平面参数，剔除所述第一平面和所述第二平面中的无效平面；其中，所述无效平面为平面长宽比例小于预设的长宽比例阈值的平面；
    根据平面帧号，剔除所述第一平面和所述第二平面中的已丢失平面；其中，所述已丢失平面为所述平面帧号与当前帧图像的帧号的差值大于设定帧数的平面。
  10. 根据权利要求6-9任一项所述的方法，其中，所述根据所述第一平面和所述第二平面对所述上一帧图像的有效平面进行更新，确定连续多帧图像的跟踪平面之后，所述方法还包括：
    判断所述第一平面是否与所述当前帧图像中任意一个未融合的有效平面满足平面融合条件；
    在所述第一平面与所述当前帧图像中任意一个未融合的有效平面满足平面融合条件的情况下，将所述第一平面与所述当前帧图像中对应有效平面进行平面融合。
  11. 根据权利要求6-10任一项所述的方法，其中，所述方法还包括：
    计算所述跟踪平面内的点的数量；
    如果所述跟踪平面内的点的数量超过第一设定数量，剔除所述跟踪平面的中心点，保留所述跟踪平面的边缘点。
PCT/CN2020/139837 2020-01-07 2020-12-28 一种平面检测方法及装置、平面跟踪方法及装置 WO2021139549A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202010014970.2 2020-01-07
CN202010014970.2A CN111242908B (zh) 2020-01-07 2020-01-07 一种平面检测方法及装置、平面跟踪方法及装置

Publications (1)

Publication Number Publication Date
WO2021139549A1 true WO2021139549A1 (zh) 2021-07-15

Family

ID=70866393

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/139837 WO2021139549A1 (zh) 2020-01-07 2020-12-28 一种平面检测方法及装置、平面跟踪方法及装置

Country Status (2)

Country Link
CN (1) CN111242908B (zh)
WO (1) WO2021139549A1 (zh)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113610004A (zh) * 2021-08-09 2021-11-05 上海擎朗智能科技有限公司 一种图像处理方法、机器人及介质
CN113762266A (zh) * 2021-09-01 2021-12-07 北京中星天视科技有限公司 目标检测方法、装置、电子设备和计算机可读介质
CN115810100A (zh) * 2023-02-06 2023-03-17 阿里巴巴（中国）有限公司 确定物体放置平面的方法、设备、存储介质及程序产品

Families Citing this family (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111242908B (zh) * 2020-01-07 2023-09-15 青岛小鸟看看科技有限公司 一种平面检测方法及装置、平面跟踪方法及装置
CN112017300A (zh) * 2020-07-22 2020-12-01 青岛小鸟看看科技有限公司 混合现实图像的处理方法、装置及设备
CN111967342B (zh) * 2020-07-27 2024-04-12 杭州易现先进科技有限公司 平面参数设置的方法、装置、电子装置和存储介质
WO2022202056A1 (ja) * 2021-03-22 2022-09-29 ソニーグループ株式会社 情報処理装置、情報処理方法、およびプログラム
CN113052977A (zh) * 2021-03-30 2021-06-29 联想(北京)有限公司 处理方法及装置
CN113689466B (zh) * 2021-07-30 2022-07-12 稿定（厦门）科技有限公司 一种基于特征点的平面跟踪方法、***

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103106659A (zh) * 2013-01-28 2013-05-15 中国科学院上海微***与信息技术研究所 基于双目视觉稀疏点匹配的空旷区域目标检测与跟踪方法
WO2018112788A1 (zh) * 2016-12-21 2018-06-28 华为技术有限公司 图像处理方法及设备
CN108898661A (zh) * 2018-05-31 2018-11-27 深圳先进技术研究院 三维图像构建的方法、装置及具有存储功能的装置
CN109741240A (zh) * 2018-12-25 2019-05-10 常熟理工学院 一种基于层次聚类的多平面图像拼接方法
CN110120098A (zh) * 2018-02-05 2019-08-13 浙江商汤科技开发有限公司 场景尺度估计及增强现实控制方法、装置和电子设备
CN111242908A (zh) * 2020-01-07 2020-06-05 青岛小鸟看看科技有限公司 一种平面检测方法及装置、平面跟踪方法及装置

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8203582B2 (en) * 2009-08-24 2012-06-19 Samsung Electronics Co., Ltd. Subpixel rendering with color coordinates' weights depending on tests performed on pixels
CN103617624B (zh) * 2013-12-13 2016-05-25 哈尔滨工业大学 用于高速视觉测量的基于合作目标的实时全局搜索方法
US20150227792A1 (en) * 2014-02-10 2015-08-13 Peter Amon Methods and Devices for Object Detection
CN105399020A (zh) * 2015-12-31 2016-03-16 徐州重型机械有限公司 三维空间平面追踪控制方法、***及高空作业设备
CN106023256B (zh) * 2016-05-19 2019-01-18 石家庄铁道大学 面向增强现实辅助维修***平面目标粒子滤波跟踪的状态观测方法

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103106659A (zh) * 2013-01-28 2013-05-15 中国科学院上海微***与信息技术研究所 基于双目视觉稀疏点匹配的空旷区域目标检测与跟踪方法
WO2018112788A1 (zh) * 2016-12-21 2018-06-28 华为技术有限公司 图像处理方法及设备
CN110120098A (zh) * 2018-02-05 2019-08-13 浙江商汤科技开发有限公司 场景尺度估计及增强现实控制方法、装置和电子设备
CN108898661A (zh) * 2018-05-31 2018-11-27 深圳先进技术研究院 三维图像构建的方法、装置及具有存储功能的装置
CN109741240A (zh) * 2018-12-25 2019-05-10 常熟理工学院 一种基于层次聚类的多平面图像拼接方法
CN111242908A (zh) * 2020-01-07 2020-06-05 青岛小鸟看看科技有限公司 一种平面检测方法及装置、平面跟踪方法及装置

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113610004A (zh) * 2021-08-09 2021-11-05 上海擎朗智能科技有限公司 一种图像处理方法、机器人及介质
CN113610004B (zh) * 2021-08-09 2024-04-05 上海擎朗智能科技有限公司 一种图像处理方法、机器人及介质
CN113762266A (zh) * 2021-09-01 2021-12-07 北京中星天视科技有限公司 目标检测方法、装置、电子设备和计算机可读介质
CN113762266B (zh) * 2021-09-01 2024-04-26 北京中星天视科技有限公司 目标检测方法、装置、电子设备和计算机可读介质
CN115810100A (zh) * 2023-02-06 2023-03-17 阿里巴巴（中国）有限公司 确定物体放置平面的方法、设备、存储介质及程序产品

Also Published As

Publication number Publication date
CN111242908B (zh) 2023-09-15
CN111242908A (zh) 2020-06-05

Similar Documents

Publication Publication Date Title
WO2021139549A1 (zh) 一种平面检测方法及装置、平面跟踪方法及装置
US11625896B2 (en) Face modeling method and apparatus, electronic device and computer-readable medium
US11263816B2 (en) Method and device for a placement of a virtual object of an augmented or mixed reality application in a real-world 3D environment
JP2018507476A (ja) コンピュータビジョンに関する遮蔽処理
US10950036B2 (en) Method and apparatus for three-dimensional (3D) rendering
US10032082B2 (en) Method and apparatus for detecting abnormal situation
CN105518712B (zh) 基于字符识别的关键词通知方法及设备
WO2017107537A1 (zh) 虚拟现实设备及避障方法
JP2017529620A (ja) 姿勢推定のシステムおよび方法
US20180165792A1 (en) Scene-based foveated rendering of graphics content
CN112487979B (zh) 目标检测方法和模型训练方法、装置、电子设备和介质
TWI550551B (zh) 景深網格化
US10748000B2 (en) Method, electronic device, and recording medium for notifying of surrounding situation information
US11763479B2 (en) Automatic measurements based on object classification
US10607069B2 (en) Determining a pointing vector for gestures performed before a depth camera
WO2021042638A1 (zh) 投影仪振镜测试目标图的提取方法、装置及电子设备
JP6240706B2 (ja) グラフマッチングおよびサイクル検出による自動モデル初期化を用いた線トラッキング
JP7262530B2 (ja) 位置情報の生成方法、関連装置及びコンピュータプログラム製品
CN114998433A (zh) 位姿计算方法、装置、存储介质以及电子设备
WO2020001016A1 (zh) 运动图像生成方法、装置、电子设备及计算机可读存储介质
CN113129249A (zh) 基于深度视频的空间平面检测方法及其***和电子设备
WO2017112036A2 (en) Detection of shadow regions in image depth data caused by multiple image sensors
US20230260211A1 (en) Three-Dimensional Point Cloud Generation Method, Apparatus and Electronic Device
JP2017016202A (ja) 画像処理装置、画像処理方法
WO2018112875A1 (en) Efficient sub-pixel disparity estimation for all sub-aperture images from densely sampled light field cameras

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20911357

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 03.11.2022)

122 Ep: pct application non-entry in european phase

Ref document number: 20911357

Country of ref document: EP

Kind code of ref document: A1