WO2020114035A1 - Three-dimensional feature extraction method and apparatus based on machine vision - Google Patents

Three-dimensional feature extraction method and apparatus based on machine vision Download PDF

Info

Publication number
WO2020114035A1
WO2020114035A1 PCT/CN2019/105962 CN2019105962W WO2020114035A1 WO 2020114035 A1 WO2020114035 A1 WO 2020114035A1 CN 2019105962 W CN2019105962 W CN 2019105962W WO 2020114035 A1 WO2020114035 A1 WO 2020114035A1
Authority
WO
WIPO (PCT)
Prior art keywords
measured
feature point
feature
position information
machine vision
Prior art date
Application number
PCT/CN2019/105962
Other languages
French (fr)
Chinese (zh)
Inventor
沈震
熊刚
李志帅
彭泓力
郭超
董西松
商秀芹
王飞跃
Original Assignee
中国科学院自动化研究所
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 中国科学院自动化研究所 filed Critical 中国科学院自动化研究所
Publication of WO2020114035A1 publication Critical patent/WO2020114035A1/en

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features

Definitions

  • the invention belongs to the field of machine vision, and in particular relates to a method and device for extracting three-dimensional features based on machine vision.
  • the present invention proposes a method of calculating the foot shape parameters by establishing a model. Since each person’s arch height and angle between the toe and the sole plane are different, if only the two characteristic sizes of the foot length and the foot width are obtained, it is impossible to accurately reflect the difference of different individual foot types belonging to the same model. The foot shape is reconstructed with a three-dimensional model to obtain accurate foot shape parameters.
  • the 3D foot model can be reconstructed by laser 3D scanning and other equipment, but this method is complicated and time-consuming to operate, high in hardware cost, and difficult to popularize. In this way, a simpler three-dimensional model method is needed to accurately obtain the foot shape parameters.
  • the first aspect of the present invention discloses a machine vision-based three-dimensional feature extraction method.
  • the feature extraction method includes the following steps: acquiring a multi-angle image containing a preset feature point to be measured containing a reference object and a target object set relative to the reference object; extracting the position of the feature point to be tested in each of the images Information; acquiring the spatial position information of the feature point to be measured according to the position information of the feature point to be tested in each of the images; calculating a certain to be measured based on the spatial position information and a preset three-dimensional feature category First distance information and/or second distance information corresponding to feature points; wherein, the first distance information is distance information between the certain feature point to be tested and other feature points to be tested, and the second distance The information is vertical distance information between the certain feature point to be measured and a preset plane; the certain feature point to be tested, the other feature points to be tested and the plane all depend on the three-dimensional feature category.
  • the step of “extracting the position information of the feature point to be measured in each of the images” includes: using manual notation to obtain a certain image The pixel position of the feature point to be measured; using the preset feature point matching method and extracting the corresponding pixel position of the feature point to be tested in other images according to the acquired pixel position.
  • the step of “extracting the position information of the feature point to be measured in each of the images” includes: acquiring the feature to be measured in the target The area shape corresponding to the area where the point is located; obtaining the area to be measured corresponding to each image according to the area shape; according to the relative position between the feature point to be measured and the area shape and each of the area to be measured To obtain the position information of the feature point to be measured in each image.
  • the step of “extracting the position information of the feature point to be measured in each of the images” includes: acquiring the to-be-measured using a pre-built neural network Position information of feature points in each of the images; wherein, the neural network is a deep neural network trained based on a preset training set and using deep learning related algorithms.
  • the step of “acquiring the spatial position information of the feature point to be measured according to the position information of the feature point to be measured in each of the images” includes: The triangulation method is used to obtain the Euclidean position of the feature point to be measured according to the position information of the feature point to be measured in each image and the parameters inside and outside the camera.
  • the step of “acquiring the spatial position information of the feature point to be measured according to the position information of the feature point to be measured in each of the images” includes: Build a sparse model using the incremental SFM method and the position information of the feature points to be measured in each of the images, and use the triangulation method to calculate the spatial position information of the feature points to be measured in the world coordinate system;
  • the obtained scale factor restores the spatial position information of the feature point to be measured obtained in the above step in the world coordinate system to obtain the true position of the feature point to be measured.
  • the three-dimensional feature extraction method based on machine vision further includes: using the sparse model and obtaining the reference object vertex in the world coordinate system according to the pixel position of the reference object vertex in the camera coordinate system It should be noted that the coordinates of the vertices in the world coordinate system differ from the real position in space by the scale factor ⁇ ; the scale coefficient is calculated according to the coordinates of the vertex of the reference object and the real position in space of the reference object in the world coordinate system ⁇ .
  • the triangulation method includes: acquiring the information according to the internal and external parameters of the camera and the position information of the feature point to be measured in each image The projective space position of the feature point to be measured, and performing the homogeneous processing on the projective space position to obtain the Euclidean space position of the feature point to be tested.
  • the technical solution of the present invention by acquiring images of different angles of the target object and extracting the positions of the feature points to be measured in the image, and then using the triangulation method or the sparse reconstruction problem solution to calculate the pending Measure the spatial position of the feature point in the world coordinate system, and calculate the first distance information and/or the second distance information between the feature points according to the calculated spatial position information of the feature point to be measured.
  • the three-dimensional feature extraction method of the present invention can quickly determine the three-dimensional feature points of the target object only through the multi-angle image obtained by the photographing device, and then calculate the distance information of the target object without using high-cost and complicated hardware equipment such as laser three-dimensional scanning , Simplifying the 3D reconstruction process.
  • the pixel position of the feature point to be measured in each image is determined by manual marking or an automatic method, wherein the automatic method includes reusing each image according to the shape of the region corresponding to the region where the feature point to be measured is located Or the pre-built neural network to obtain the location information of the feature point to be tested in each image. Then use the reference object to automatically calibrate the camera parameters and then triangulate or solve the sparse reconstruction problem to find the true spatial position of the feature point to be measured. There is no need to rebuild the entire target model, which can reduce the amount of calculation and simplify the model establishment process. Finally, based on the real spatial position of the feature point to be measured and the preset three-dimensional feature category, the distance information corresponding to the feature point to be measured is calculated.
  • a second aspect of the present invention provides a storage device that stores a plurality of programs that are adapted to be loaded by a processor to perform the machine vision-based three-dimensional feature extraction method described in any one of the foregoing.
  • the storage device has all the technical effects of the aforementioned three-dimensional feature extraction method based on machine vision, which will not be repeated here.
  • a third aspect of the present invention also provides a control device, the control device includes a processor and a storage device, the storage and storage device is adapted to store multiple programs, the program is adapted to be loaded by the processor to execute the foregoing Any one of the three-dimensional feature extraction methods based on machine vision.
  • control device has all the technical effects of the aforementioned three-dimensional feature extraction method based on machine vision, which will not be repeated here.
  • FIG. 1 is a flowchart of main steps of a method for extracting a three-dimensional feature of a foot shape based on machine vision in an embodiment of the present invention
  • FIG. 2 is a schematic diagram of a method for extracting feature points using a generalized Hough transform using a circle as a template based on a machine vision-based three-dimensional feature extraction method according to an embodiment of the present invention
  • FIG. 3 is a schematic diagram of a method for detecting feature points using a generalized Hough transform using a circle as a template, based on a machine vision-based three-dimensional feature extraction method according to an embodiment of the present invention
  • FIG. 4 is a schematic diagram of a method for detecting feature points using a generalized Hough transform using a circle as a template, based on a machine vision-based three-dimensional feature extraction method according to an embodiment of the present invention
  • FIG. 5 is a schematic diagram of a method for extracting a reference object using a generalized Hough transform with a straight line as a template, based on a machine vision-based three-dimensional feature extraction method in an embodiment of the present invention
  • FIG. 6 is a schematic diagram of a process of solving feature position space position information in a triangulation process of a foot shape three-dimensional feature extraction method based on machine vision in an embodiment of the present invention
  • FIG. 7 is a schematic diagram of a process of solving feature point spatial position information in a sparse reconstruction process of a foot shape three-dimensional feature extraction method based on machine vision in an embodiment of the present invention.
  • the extraction and calculation process of the three-dimensional feature parameters of the foot shape is converted into determining the spatial position of the corresponding feature point, and then the feature parameters of the foot shape to be measured are calculated by using the Euclidean distance formula.
  • the basic parameters of the foot shape that can be obtained include: foot length, foot circumference, instep height, arch bending point height, foot width, thumb height, heel convexity point height, foot ankle center point height, etc.
  • Required foot parameter information The following takes the three parameters of foot length, foot width, and ankle point height as examples to illustrate a possible implementation manner of the machine vision-based three-dimensional feature extraction method of the present invention.
  • FIG. 1 exemplarily shows the main steps of the machine vision-based foot shape three-dimensional feature extraction method in the embodiment of the present invention.
  • the machine vision-based foot shape three-dimensional feature extraction method in the present invention may include the following steps :
  • Step S100 Acquire a multi-angle image containing preset feature points of the target to be measured.
  • a mobile camera device such as a camera
  • a mobile camera device such as a camera
  • the longest Toe apex and heel bulge to calculate the length of the foot shape to be measured, such as the outer point of the thumb ball and the outer point of the base of the tail to calculate the width of the foot shape to be measured, and the ankle point to calculate the height of the ankle point.
  • the number of foot-shaped images captured should be at least three or more. The more images containing the feature points to be measured, the more accurate the foot-type parameters calculated according to the feature points to be measured.
  • Step S200 Extract the position information of the feature point to be measured in each image.
  • the three-dimensional feature extraction method shown in FIG. 1 can obtain the pixel position (x, y) of the feature point to be measured in each image according to the following steps, specifically:
  • the three-dimensional feature extraction method shown in FIG. 1 can also obtain the pixel position (x, y) of the feature point to be measured in each image according to the following steps, specifically :
  • a feature detection method such as generalized Hough transform
  • a feature detection method is used to detect a specific shape to determine the position information of the feature point to be measured in each image. Specifically, first determine the shape of the area corresponding to the area where the feature point to be measured corresponds, and then automatically find the corresponding area of the feature point to be measured in each image according to the shape of the area and use the generalized Hough transform, and then according to the feature to be measured
  • the relative position between the point and the shape of the area and the area to be measured in each image obtain the position information of the feature point to be measured in each image.
  • the following uses a circle as a template and uses the generalized Hough transform to find feature points as an example to illustrate possible implementations.
  • FIG. 2 is a schematic diagram of a machine vision-based foot shape three-dimensional feature extraction method using a circle as a template to detect feature points using a generalized Hough transform
  • FIG. 3 is A schematic diagram of a machine vision-based foot shape 3D feature extraction method using a circle as a template to detect feature points using a generalized Hough transform in an embodiment of the present invention
  • FIG. 4 is a machine vision-based foot shape in an embodiment of the present invention
  • Figures 2, 3, and 4 respectively show that the area where the ankle point is a circle is used as a template at different angles to use generalization Hough transform to find the specific implementation of feature points.
  • the ankle where the center of the ankle is located is round. It can be seen from the figure that this circular outline is unique in the foot type.
  • the generalized Hough is used When changing, use the circle as the template, and automatically find the position of the circle in the image (the circle template shown by the dotted line in Figure 2-4). This position is the position of the ankle, and the center of the circle position is searched.
  • the point G is the position of the ankle point of the feature point to be measured in the image.
  • the contour of the longest toe can be used as a template for the generalized Hough transform to search in the image, after finding the toe contour, the contour and the longest toe vertex are found Relative position to determine the pixel position of the feature point.
  • the three-dimensional feature extraction method shown in FIG. 1 can also obtain the pixel position (x, y) of the feature point to be measured in each image according to the following steps, specifically :
  • the neural network is then used to obtain the position information of the feature point to be tested in each image.
  • the input is image data containing the feature point to be tested
  • the output is the pixel position (x, y) of the feature point to be tested in the image, where the output includes the real output and the expected output
  • the actual output of the last fully connected layer of the network is the corresponding pixel position (x, y) of the feature point to be measured in the image
  • the expected output of the network is the actual pixel position of the feature point to be marked in the image.
  • the neural network automatically outputs the neural network automatically The pixel position in the image.
  • the neural network select a sufficient amount of image samples that mark the ankle point as the training set, and build a deep neural network, and then use the training set to train the deep neural network. After training, enter a containing ankle point , The neural network automatically outputs the pixel position of the ankle point in the image.
  • the image data samples corresponding to the feature points are used to train the pre-built deep neural network, and then the image to be tested containing the feature points is input to obtain the feature points in the image Pixel location.
  • Step S300 Acquire the spatial position information of the feature point to be measured according to the position information of the feature point to be tested in each image.
  • the three-dimensional feature extraction method shown in FIG. 1 can obtain the spatial location information of the feature point to be measured according to the following steps, specifically:
  • the reference parameters are used to calibrate the camera parameters, and then the triangulation method is used to calculate the spatial position information of the feature points to be measured.
  • the foot shape is placed on the A4 paper, and a plurality of images at different angles are acquired by using an imaging device such as a camera, and the images of these different angles include the outline of the A4 paper.
  • an imaging device such as a camera
  • the images of these different angles include the outline of the A4 paper.
  • FIG. 5 is a schematic diagram of a machine vision-based foot shape three-dimensional feature extraction method using a straight line as a template to detect a reference object using a generalized Hough transform in an embodiment of the present invention.
  • Figure 2 and Figure 4 from the knowledge of spatial geometric transformation, the following relationship between point A in Euclidean space and projective space can be obtained:
  • the parameters K, R and t in formula (1) are the camera's internal parameter matrix, the camera's rotation matrix and translation matrix relative to the world coordinate system ([R
  • " stands for augmented matrix
  • r 1 , r 2 , and r 3 are the expansion forms of the rotation matrix R of the camera relative to the world coordinate system. From matrix multiplication, r 3 and 0 elements are multiplied and eliminated.
  • the parameters K -1 , R, t and H in formula (2) are the inverse matrix of the parameter matrix in the camera, the rotation matrix of the camera relative to the world coordinate system, the translation matrix and the homography matrix, respectively.
  • r 1 and r 2 are the rotation matrix of the camera external parameters relative to the world coordinate system obtained from two images of different angles
  • h 1 , h 2 and h 3 are the relative worlds obtained from three images of different angles, respectively
  • the parameters K -T and K -1 in formula (3) are the orthogonal matrix and inverse matrix of the transposed matrix of the parameter matrix in the camera, and h 1 and h 2 are the relative worlds obtained from two images with different angles
  • the two homography matrices of the coordinate system in the camera, h 1 T and h 2 T are the transpose matrices of the homography matrices h 1 and h 2. From the above, the internal parameters of the two cameras can be obtained for each two images Constraint equation.
  • w is linearly solved by DLT, and then orthogonal Decomposition can be solved to get K.
  • t] K -1 [h 1 h 2 h 3 ], combined with h 1 , h 2 , h 3 and K obtained by the previous solution, r can be obtained 1, r 2, t.
  • t 3 ] of the camera when photographing FIGS. 2, 3, and 4 are obtained.
  • FIG. 6 is a schematic diagram of a process of solving feature position spatial position information in a triangulation process of a machine vision-based three-dimensional feature extraction method according to an embodiment of the present invention.
  • FIG. 6 shows the triangulation process taking the ankle point G in FIGS. 3 and 4 (that is, Image1 and Image2) as an example.
  • the three-dimensional feature extraction method shown in FIG. 1 may also obtain the real spatial position of the feature point to be measured according to the following steps, specifically:
  • the three-dimensional reconstruction problem is converted into the sparse reconstruction problem of the feature points to be measured, such as the incremental SFM method to build a sparse model and the triangulation method to solve the sparse reconstruction problem.
  • the difference from the previous embodiment is to use the incremental SFM method to directly solve the in-camera parameter matrix K,
  • the camera rotation matrix R, the amount of translation t relative to the world coordinates, and the coordinate ⁇ (X, Y, Z) of the feature point to be measured in the world coordinate system omitting the process of calibrating the camera with a reference object, and then using known specifications
  • the scale factor ⁇ is determined by the reference object, and then the real space position coordinates (X, Y, Z) of the feature point are obtained.
  • FIG. 7 is a schematic diagram of a process of solving feature position spatial position information in a sparse reconstruction process of a machine vision-based three-dimensional feature extraction method according to an embodiment of the present invention.
  • the steps of using the incremental SFM method to solve the sparse reconstruction problem specifically include:
  • Step 1 Randomly select two images, Image1 and Image2, from three images at different angles to determine the initial image pair, and use the incremental SFM method to calculate the initial values of the internal and external parameters of the camera that takes the images Image1 and Image2 [R
  • Matrix Use the five pairs of feature points in the images Image1 and Image2 (the longest toe apex and heel bump, the lateral point of the thumb ball, the lateral point of the caudal toe, and the ankle point).
  • the camera rotation matrices R 1 , R 2 and the translation t 1 , t 2 matrices relative to the world coordinates can be decomposed from the essential matrix E. Then, combining the pixel positions of the feature points to be measured in the camera coordinate system obtained in step S200 in the images Image1 and Image2, an initial sparse model is constructed;
  • Step 2 According to the initial sparse model constructed in Step 1, and using the triangulation method to calculate the position coordinates ⁇ (X 1 , Y 1 , Z 1 ) of the feature point to be measured in the world coordinate system of the images Image1 and Image2 and ⁇ (X 2 , Y 2 , Z 2 );
  • Step 3 Enter the pixel position of the feature point to be measured in the camera coordinate system obtained in step S200 of the image Image3 into the initial sparse model obtained in step 2, and the internal and external parameter [R
  • Step 4 According to the sparse model modified in Step 3, and using the triangulation method to calculate the spatial position coordinates ⁇ (X 3 , Y 3 , Z 3 ) of the feature point to be measured in the world coordinate system in the image Image3;
  • Step 5 Use the Bundle Adjustment (BA) method to modify the position coordinates of the feature points obtained in steps 2 and 4 to obtain an optimized sparse model.
  • BA Bundle Adjustment
  • step 5 different coordinate positions of the feature points to be measured in other remaining images are repeatedly bundled and adjusted until the error of the coordinate ⁇ (X, Y, Z) of the feature points to be measured calculated twice before and after is less than or equal to the The set threshold.
  • the present invention only provides a specific embodiment of using incremental SFM method to solve the spatial position information of the feature points to be measured in the three images, those skilled in the art can understand that the incremental method provided by the present invention
  • the SFM method can also be used to solve multiple images at different angles.
  • the pixel position information of the feature point to be measured in the new image under the camera coordinate system is repeatedly substituted, and the camera is reacquired Use the internal and external parameters of the camera to modify the sparse model until all images are added to the sparse model. It can be understood that the more different angles of the acquired image, the more iterative calculations, and the more accurate the internal and external parameters of the camera.
  • the feature points to be measured calculated based on the sparse model constructed by it are in the world coordinate system. The more accurate the spatial location information is.
  • Step 6 Taking the point A in FIG. 4 as the coordinate origin, according to the pixel position information of the vertex D of the A4 paper obtained in step S200 in the camera coordinate system, and then using the sparse model obtained in step 5 to calculate the space of the vertex D
  • the spatial coordinates ⁇ (X, Y, Z) of the feature point to be measured obtained in step 5 in the world coordinate system divided by the scale factor ⁇ to obtain the true spatial position (X, Y, Z) of the feature point to be measured.
  • Step S400 Calculate the first distance information and/or the second distance information corresponding to a certain feature point to be measured based on the spatial position information and the preset three-dimensional feature category.
  • the first distance information is the distance information between a certain feature point to be tested and other feature points to be tested, such as the length
  • the second distance information is the perpendicularity between a certain feature point to be tested and a preset plane Distance information, such as height.
  • the longest toe vertex is (X 1 , Y 1 , Z 1 )
  • the heel convex point is ( X 2 , Y 2 , Z 2 )
  • the outer point of the thumb ball is (X 3 , Y 3 , Z 3 )
  • the outer point of the base of the tail toe is (X 4 , Y 4 , Z 4 )
  • the ankle point is (X 5 , Y 5 , Z 5 )
  • the parameters L, W and H in formula (4) are the foot length, foot width and ankle height, respectively.
  • the three parameters of foot length, foot width and ankle point height can be obtained.
  • the present invention only provides a specific embodiment for calculating three parameters of foot length, foot width and ankle point height by extracting three-dimensional feature points, those skilled in the art can understand that the three-dimensional feature extraction method provided by the present invention You can also calculate other foot shape parameters, such as calculating the instep height. At this time, the images at different angles need to include the feature point instep point, and then sequentially follow the steps of the three-dimensional feature extraction method of the present invention described in the above embodiment. Calculate the instep height.
  • the camera device is used to acquire five features to be measured including the longest toe apex of the foot shape, the heel convex point, the lateral point of the thumb ball, the lateral point of the caudal toe, and the ankle point
  • the Euclidean distance formula is used to calculate the three foot shape parameters of foot length, foot width and ankle point height.
  • the images of different angles of different feature points can be obtained, and the foot parameters corresponding to the feature points can also be calculated.
  • the spatial position information of the instep point can be calculated according to the above steps.
  • the parameter of instep height is calculated.
  • the present invention also provides a storage device, the storage device stores a plurality of programs, the program can be adapted to be loaded by a processor to execute the above-described method embodiment based on machine vision 3D feature extraction method.
  • the present invention also provides a control device, the control device includes a processor and a storage device, wherein the storage device can be adapted to store multiple programs, the program can be applied to the The processor loads to execute the three-dimensional feature extraction method based on machine vision described in the above method embodiment.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Software Systems (AREA)
  • Multimedia (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Biomedical Technology (AREA)
  • Evolutionary Computation (AREA)
  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Computer Graphics (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • Geometry (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Mathematical Physics (AREA)
  • Length Measuring Devices By Optical Means (AREA)
  • Image Processing (AREA)
  • Image Analysis (AREA)

Abstract

Provided are a three-dimensional feature extraction method and apparatus based on machine vision. The invention aims to solve the problems of a complex and time-consuming three-dimensional model reconstruction process, difficulty in popularization, etc. in the prior art. In order to achieve the aim, the three-dimensional feature extraction method based on machine vision comprises the following steps: acquiring multi-angle images of a preset feature point to be detected containing a target object (S100); extracting position information of the feature point to be detected in each of the images (S200); acquiring spatial position information of the feature point to be detected according to the position information of the feature point to be detected in each of the images (S300); and calculating first distance information and/or second distance information corresponding to a certain feature point to be detected based on the spatial position information and a preset three-dimensional feature category (S400). Through machine vision, different angle images containing the feature point to be detected are acquired, and thus, spatial position information of the feature point to be detected is acquired so as to calculate to obtain distance information of the target object.

Description

基于机器视觉的三维特征提取方法及装置Three-dimensional feature extraction method and device based on machine vision 技术领域Technical field
本发明属于机器视觉领域，具体涉及一种基于机器视觉的三维特征提取方法及装置。The invention belongs to the field of machine vision, and in particular relates to a method and device for extracting three-dimensional features based on machine vision.
背景技术Background technique
随着云制造、云计算的发展和“工业4.0”的临近，社会制造模式，即面向顾客定制生产的模式应运而生。社会制造的特点是能够将消费者的需求直接转化为产品，以社会计算理论为基础，基于移动互联网技术、社会媒体与3D打印技术，通过众包等形式让社会民众充分参与产品的全生命制造过程，实现个性化、实时化、经济化的生产和消费模式。也就是说，在社会制造中，每个消费者都可以参与产品生产全生命周期的各个阶段，包括产品的设计、制造和消费。以制鞋为例，社会制造在制鞋过程中的应用体现在用户可以根据自己的需求来进行个性化的定制与选取，这就要求能够简单、快捷、准确地获取用户的脚型三维特征。With the development of cloud manufacturing and cloud computing and the approaching of "Industry 4.0", a social manufacturing model, that is, a customer-oriented customized production model emerged at the historic moment. The characteristic of social manufacturing is that it can directly convert consumer needs into products. Based on social computing theory, based on mobile Internet technology, social media and 3D printing technology, through the form of crowdsourcing, the public can fully participate in the whole life of products. Process to achieve personalized, real-time, and economical production and consumption patterns. That is to say, in social manufacturing, every consumer can participate in all stages of the product life cycle, including product design, manufacturing, and consumption. Taking shoemaking as an example, the application of social manufacturing in the shoemaking process is reflected in the user's personalized customization and selection according to his own needs. This requires simple, fast and accurate access to the three-dimensional characteristics of the user's foot shape.
但是，原始的手工测量能够得到的脚型参数较少，并不能准确描述脚型，只有具有制鞋行业的专业工具才能获得准确的测量结果。为使非专业人士也能获得较为准确的脚型参数以便实现鞋子的个性化定制，本发明提出了采用建立模型计算获得脚型参数的方法。由于每个人的足弓高和脚趾与脚底平面夹角都是不同的，若只获得脚长和脚宽两个特征尺寸是不能准确反映属于同一型号的不同个体脚型的差异，因此就需要对脚型进行三维模型重建来获得准确的脚型参数。目前，可以通过激光三维扫描等设备进行脚型三维模型重建，但是这种方法操作复杂耗时、硬件成本高、普及困难。这样一来，就需要一种更为简便的三维模型方法来准确获取脚型参数。However, the original manual measurement can get less foot shape parameters and can not accurately describe the foot shape. Only with professional tools in the shoe industry can accurate measurement results be obtained. In order to enable non-professionals to obtain more accurate foot shape parameters in order to realize the personalized customization of shoes, the present invention proposes a method of calculating the foot shape parameters by establishing a model. Since each person’s arch height and angle between the toe and the sole plane are different, if only the two characteristic sizes of the foot length and the foot width are obtained, it is impossible to accurately reflect the difference of different individual foot types belonging to the same model. The foot shape is reconstructed with a three-dimensional model to obtain accurate foot shape parameters. At present, the 3D foot model can be reconstructed by laser 3D scanning and other equipment, but this method is complicated and time-consuming to operate, high in hardware cost, and difficult to popularize. In this way, a simpler three-dimensional model method is needed to accurately obtain the foot shape parameters.
相应地，本领域需要一种新的三维模型重建方法来解决上述问题。Accordingly, a new three-dimensional model reconstruction method is needed in the art to solve the above problems.
发明内容Summary of the invention
为了解决现有技术中的上述问题，即为了解决现有的三维模型重建过程复杂耗时、普及困难等问题，本发明第一方面公开了一种基于机器视觉的三维特征提取方法，所述三维特征提取方法包括下列步骤：获取包含参照物及相对于所述参照物设置的目标物的预设待测特征点的多角度图像；提取所述待测特征点在每个所述图像中的位置信息；根据所述待测特征点在每个所述图像中的位置信息获取所述待测特征点的空间位置信息；基于所述空间位置信息和预设的三维特征类别，计算某个待测特征点对应的第一距离信息和/或第二距离信息；其中，所述第一距离信息是所述某个待测特征点与其他待测特征点之间的距离信息，所述第二距离信息是所述某个待测特征点与预设平面之间的垂直距离信息；所述某个待测特征点、所述其他待测特征点与所述平面均取决于所述三维特征类别。In order to solve the above-mentioned problems in the prior art, that is, to solve the problems of complicated and time-consuming and difficult to popularize the existing three-dimensional model reconstruction process, the first aspect of the present invention discloses a machine vision-based three-dimensional feature extraction method. The feature extraction method includes the following steps: acquiring a multi-angle image containing a preset feature point to be measured containing a reference object and a target object set relative to the reference object; extracting the position of the feature point to be tested in each of the images Information; acquiring the spatial position information of the feature point to be measured according to the position information of the feature point to be tested in each of the images; calculating a certain to be measured based on the spatial position information and a preset three-dimensional feature category First distance information and/or second distance information corresponding to feature points; wherein, the first distance information is distance information between the certain feature point to be tested and other feature points to be tested, and the second distance The information is vertical distance information between the certain feature point to be measured and a preset plane; the certain feature point to be tested, the other feature points to be tested and the plane all depend on the three-dimensional feature category.
在上述基于机器视觉的三维特征提取方法的优选技术方案中，“提取所述待测特征点在每个所述图像中的位置信息”的步骤包括：利用手动标记法获取某个所述图像中的所述待测特征点的像素位置；利用预设的特征点匹配法并且根据所获取的像素位置，提取所述待测特征点在其他图像中对应的像素位置。In the above preferred technical solution of the three-dimensional feature extraction method based on machine vision, the step of “extracting the position information of the feature point to be measured in each of the images” includes: using manual notation to obtain a certain image The pixel position of the feature point to be measured; using the preset feature point matching method and extracting the corresponding pixel position of the feature point to be tested in other images according to the acquired pixel position.
在上述基于机器视觉的三维特征提取方法的优选技术方案中，“提取所述待测特征点在每个所述图像中的位置信息”的步骤包括：获取所述目标物中所述待测特征点所在区域对应的区域形状；根据所述区域形状获取所述每个图像对应的待测区域；根据所述待测特征点与所述区域形状之间的相对位置以及每个所述待测区域，获取所述待测特征点在所述每个图像中的位置信息。In the above preferred technical solution of the three-dimensional feature extraction method based on machine vision, the step of “extracting the position information of the feature point to be measured in each of the images” includes: acquiring the feature to be measured in the target The area shape corresponding to the area where the point is located; obtaining the area to be measured corresponding to each image according to the area shape; according to the relative position between the feature point to be measured and the area shape and each of the area to be measured To obtain the position information of the feature point to be measured in each image.
在上述基于机器视觉的三维特征提取方法的优选技术方案中，“提取所述待测特征点在每个所述图像中的位置信息”的步骤包括：利用预先构建的神经网络获取所述待测特征点在每个所述图像中的位置信息；其中，所述神经网络是基于预设的训练集并利用深度学习相关算法训练的深度神经网络。In the above preferred technical solution of the three-dimensional feature extraction method based on machine vision, the step of “extracting the position information of the feature point to be measured in each of the images” includes: acquiring the to-be-measured using a pre-built neural network Position information of feature points in each of the images; wherein, the neural network is a deep neural network trained based on a preset training set and using deep learning related algorithms.
在上述基于机器视觉的三维特征提取方法的优选技术方案中，“根据所述待测特征点在每个所述图像中的位置信息获取所述待测特 征点的空间位置信息”的步骤包括：利用三角化方法并且根据所述待测特征点在所述每个图像中的位置信息与相机内外参数获取所述待测特征点的欧氏位置。In the above preferred technical solution of the three-dimensional feature extraction method based on machine vision, the step of “acquiring the spatial position information of the feature point to be measured according to the position information of the feature point to be measured in each of the images” includes: The triangulation method is used to obtain the Euclidean position of the feature point to be measured according to the position information of the feature point to be measured in each image and the parameters inside and outside the camera.
在上述基于机器视觉的三维特征提取方法的优选技术方案中，“根据所述待测特征点在每个所述图像中的位置信息获取所述待测特征点的空间位置信息”的步骤包括：利用增量式SFM方法和每个所述图像中所述待测特征点的位置信息构建稀疏模型，并利用三角化方法计算所述待测特征点在世界坐标系下的空间位置信息；利用预先获取的尺度系数恢复上述步骤中得到的所述待测特征点在世界坐标系下的空间位置信息，得到所述待测特征点的真实位置。In the above preferred technical solution of the three-dimensional feature extraction method based on machine vision, the step of “acquiring the spatial position information of the feature point to be measured according to the position information of the feature point to be measured in each of the images” includes: Build a sparse model using the incremental SFM method and the position information of the feature points to be measured in each of the images, and use the triangulation method to calculate the spatial position information of the feature points to be measured in the world coordinate system; The obtained scale factor restores the spatial position information of the feature point to be measured obtained in the above step in the world coordinate system to obtain the true position of the feature point to be measured.
在上述基于机器视觉的三维特征提取方法的优选技术方案中，在“利用预先获取的尺度系数恢复上述步骤中得到的所述特征点在世界坐标系下的空间位置信息，得到所述待测特征点的真实位置”之前，所述基于机器视觉的三维特征提取方法还包括：利用所述稀疏模型并根据在相机坐标系下参照物顶点的像素位置，获取在世界坐标系下所述参照物顶点的坐标，需要注意的是，世界坐标系下的顶点坐标与空间真实位置相差尺度系数λ；根据在世界坐标系下所述参照物顶点的坐标以及参照物顶点的空间真实位置，计算该尺度系数λ。In the preferred technical solution of the above-mentioned three-dimensional feature extraction method based on machine vision, in “recovering the spatial position information of the feature point obtained in the above step in the world coordinate system using the scale coefficients obtained in advance to obtain the feature to be measured Before the real position of the point, the three-dimensional feature extraction method based on machine vision further includes: using the sparse model and obtaining the reference object vertex in the world coordinate system according to the pixel position of the reference object vertex in the camera coordinate system It should be noted that the coordinates of the vertices in the world coordinate system differ from the real position in space by the scale factor λ; the scale coefficient is calculated according to the coordinates of the vertex of the reference object and the real position in space of the reference object in the world coordinate system λ.
在上述基于机器视觉的三维特征提取方法的优选技术方案中，所述三角化方法包括：根据所述相机内外参数与所述待测特征点在所述每个图像中的位置信息，获取所述待测特征点的射影空间位置，以及对所述射影空间位置进行齐次化处理得到所述待测特征点的欧氏空间位置。In the above preferred technical solution of the three-dimensional feature extraction method based on machine vision, the triangulation method includes: acquiring the information according to the internal and external parameters of the camera and the position information of the feature point to be measured in each image The projective space position of the feature point to be measured, and performing the homogeneous processing on the projective space position to obtain the Euclidean space position of the feature point to be tested.
本领域技术人员可以理解的是，在本发明的技术方案中，通过获取目标物的不同角度图像并提取待测特征点在图像中的位置，然后利用三角化方法或者稀疏重建问题求解来计算待测特征点在世界坐标系下的空间位置，根据计算得到的待测特征点的空间位置信息计算特征点之间的第一距离信息和/或第二距离信息。本发明的三维特征提取方法仅通过拍照设备获取的多角度图像即可快速确定目标物的三维特征点，进而计算得到目标物的距离信息，无需使用激光三维扫描等高成本、操作复杂的硬件设备，简化了三维重建过程。Those skilled in the art can understand that in the technical solution of the present invention, by acquiring images of different angles of the target object and extracting the positions of the feature points to be measured in the image, and then using the triangulation method or the sparse reconstruction problem solution to calculate the pending Measure the spatial position of the feature point in the world coordinate system, and calculate the first distance information and/or the second distance information between the feature points according to the calculated spatial position information of the feature point to be measured. The three-dimensional feature extraction method of the present invention can quickly determine the three-dimensional feature points of the target object only through the multi-angle image obtained by the photographing device, and then calculate the distance information of the target object without using high-cost and complicated hardware equipment such as laser three-dimensional scanning , Simplifying the 3D reconstruction process.
在本发明的优选技术方案中，通过手动标记或者自动方法确定待测特征点在每一幅图像中的像素位置，其中自动方法包括根据待测特征点所在区域对应的区域形状再利用每个图像的待测区域或者利用预先构建的神经网络来获取待测特征点在每个图像中的位置信息。然后利用参照物自动标定相机参数再三角化或者通过稀疏重建问题求解来求取待测特征点的真实空间位置，不需要对整个目标物的模型重建，能够减少计算量，简化模型建立过程。最后基于待测特征点的真实空间位置以及预设的三维特征类别，计算待测特征点对应的距离信息。In a preferred technical solution of the present invention, the pixel position of the feature point to be measured in each image is determined by manual marking or an automatic method, wherein the automatic method includes reusing each image according to the shape of the region corresponding to the region where the feature point to be measured is located Or the pre-built neural network to obtain the location information of the feature point to be tested in each image. Then use the reference object to automatically calibrate the camera parameters and then triangulate or solve the sparse reconstruction problem to find the true spatial position of the feature point to be measured. There is no need to rebuild the entire target model, which can reduce the amount of calculation and simplify the model establishment process. Finally, based on the real spatial position of the feature point to be measured and the preset three-dimensional feature category, the distance information corresponding to the feature point to be measured is calculated.
本发明第二方面提供了一种存储装置，所述存储装置存储有多条程序，所述程序适于由处理器加载以执行前述任一项所述的基于机器视觉的三维特征提取方法。A second aspect of the present invention provides a storage device that stores a plurality of programs that are adapted to be loaded by a processor to perform the machine vision-based three-dimensional feature extraction method described in any one of the foregoing.
需要说明的是，该存储装置具有前述的基于机器视觉的三维特征提取方法的所有技术效果，在此不再赘述。It should be noted that the storage device has all the technical effects of the aforementioned three-dimensional feature extraction method based on machine vision, which will not be repeated here.
本发明第三方面还提供了一种控制装置，所述控制装置包括处理器和存储设备，所述存执设备适于存储多条程序，所述程序适于由所述处理器加载以执行前述任一项所述的基于机器视觉的三维特征提取方法。A third aspect of the present invention also provides a control device, the control device includes a processor and a storage device, the storage and storage device is adapted to store multiple programs, the program is adapted to be loaded by the processor to execute the foregoing Any one of the three-dimensional feature extraction methods based on machine vision.
需要说明的是，该控制装置具有前述的基于机器视觉的三维特征提取方法的所有技术效果，在此不再赘述。It should be noted that the control device has all the technical effects of the aforementioned three-dimensional feature extraction method based on machine vision, which will not be repeated here.
附图说明BRIEF DESCRIPTION
下面参照附图并结合脚型来描述本发明的基于机器视觉的三维特征提取方法。附图中：The three-dimensional feature extraction method based on machine vision of the present invention will be described below with reference to the drawings and in combination with foot shapes. In the drawings:
图1是本发明实施例中一种基于机器视觉的脚型三维特征提取方法的主要步骤流程图；1 is a flowchart of main steps of a method for extracting a three-dimensional feature of a foot shape based on machine vision in an embodiment of the present invention;
图2是本发明实施例中一种基于机器视觉的脚型三维特征提取方法的以圆形为模板使用广义霍夫变换检测特征点的示意图；2 is a schematic diagram of a method for extracting feature points using a generalized Hough transform using a circle as a template based on a machine vision-based three-dimensional feature extraction method according to an embodiment of the present invention;
图3是本发明实施例中一种基于机器视觉的脚型三维特征提取方法的以圆形为模板使用广义霍夫变换检测特征点的示意图；FIG. 3 is a schematic diagram of a method for detecting feature points using a generalized Hough transform using a circle as a template, based on a machine vision-based three-dimensional feature extraction method according to an embodiment of the present invention;
图4是本发明实施例中一种基于机器视觉的脚型三维特征提取方法的以圆形为模板使用广义霍夫变换检测特征点的示意图；FIG. 4 is a schematic diagram of a method for detecting feature points using a generalized Hough transform using a circle as a template, based on a machine vision-based three-dimensional feature extraction method according to an embodiment of the present invention;
图5是本发明实施例中一种基于机器视觉的脚型三维特征提取方法的以直线为模板使用广义霍夫变换检测参照物的示意图；FIG. 5 is a schematic diagram of a method for extracting a reference object using a generalized Hough transform with a straight line as a template, based on a machine vision-based three-dimensional feature extraction method in an embodiment of the present invention;
图6是本发明实施例中一种基于机器视觉的脚型三维特征提取方法的三角化过程求解特征点空间位置信息的过程示意图；6 is a schematic diagram of a process of solving feature position space position information in a triangulation process of a foot shape three-dimensional feature extraction method based on machine vision in an embodiment of the present invention;
图7是本发明实施例中一种基于机器视觉的脚型三维特征提取方法的稀疏重建过程求解特征点空间位置信息的过程示意图。FIG. 7 is a schematic diagram of a process of solving feature point spatial position information in a sparse reconstruction process of a foot shape three-dimensional feature extraction method based on machine vision in an embodiment of the present invention.
具体实施方式detailed description
下面参照附图来描述本发明的优选实施方式。本领域技术人员应当理解的是，这些实施方式仅仅用于解释本发明的技术原理，并非旨在限制本发明的保护范围。例如，虽然本发明是以脚型为例来描述的，但是还可以是其他可以通过建立模型转化为产品的目标物，如衣服。另外，本发明是以A4纸作为参照物来进行描述的，但是还可以是其他已知尺寸的物体(如地板砖)。本领域技术人员可以根据需要对其作出调整，以便适应具体的应用场合。The preferred embodiments of the present invention are described below with reference to the drawings. Those skilled in the art should understand that these embodiments are only used to explain the technical principles of the present invention, and are not intended to limit the protection scope of the present invention. For example, although the present invention is described with a foot shape as an example, it can also be other objects that can be converted into products by building a model, such as clothes. In addition, the present invention is described with reference to A4 paper, but it can also be objects of other known sizes (such as floor tiles). Those skilled in the art can make adjustments as needed to suit specific applications.
需要说明的是，在本发明的描述中，术语“第一”、“第二”、“第三”仅用于描述目的，而不能理解为指示或暗示相对重要性。It should be noted that, in the description of the present invention, the terms "first", "second", and "third" are used for description purposes only, and cannot be understood as indicating or implying relative importance.
下面结合附图对本发明提供的基于机器视觉的脚型三维特征提取方法进行说明。The method for extracting the three-dimensional feature of the foot shape based on machine vision provided by the present invention will be described below with reference to the drawings.
在本发明的一种具体实施方式中，将脚型的三维特征参数的提取计算过程转化为确定对应特征点的空间位置，然后利用欧氏距离公式计算得到需要测量的脚型的特征参数。其中，可以得到的脚型的基本参数包括：脚长、脚围、脚背围高度、弓上弯点高度、脚宽、拇指高度、足后跟凸度点高度、足外踝骨中心点高度等制鞋所需的脚型参数信息。下面以获取脚长、脚宽以及脚踝点高度三个参数为例来说明了本发明的基于机器视觉的脚型三维特征提取方法的可能的实现方式。In a specific embodiment of the present invention, the extraction and calculation process of the three-dimensional feature parameters of the foot shape is converted into determining the spatial position of the corresponding feature point, and then the feature parameters of the foot shape to be measured are calculated by using the Euclidean distance formula. Among them, the basic parameters of the foot shape that can be obtained include: foot length, foot circumference, instep height, arch bending point height, foot width, thumb height, heel convexity point height, foot ankle center point height, etc. Required foot parameter information. The following takes the three parameters of foot length, foot width, and ankle point height as examples to illustrate a possible implementation manner of the machine vision-based three-dimensional feature extraction method of the present invention.
首先参照图1，图1示例性地示出了本发明实施例中基于机器视觉的脚型三维特征提取方法的主要步骤，本发明中基于机器视觉的脚型三维特征提取方法可以包括下述步骤：Referring first to FIG. 1, FIG. 1 exemplarily shows the main steps of the machine vision-based foot shape three-dimensional feature extraction method in the embodiment of the present invention. The machine vision-based foot shape three-dimensional feature extraction method in the present invention may include the following steps :
步骤S100，获取包含目标物的预设待测特征点的多角度图像。Step S100: Acquire a multi-angle image containing preset feature points of the target to be measured.
具体地，将脚正放在A4纸上，利用移动拍照设备，如照相机，拍摄多个角度的脚型的图像，以便能够充分表现脚型特征，能够获取足够的待测特征点，如最长脚趾顶点和脚后跟凸点以便计算待测脚型的长度，又如指拇指球外侧点和尾趾根部外侧点以便计算待测脚型的宽度，又如脚踝点以便计算脚踝点的高度等。需要说明的是，拍摄的脚型的图像的数量应至少在三张以上，包含待测特征点的图像的数量越多，根据待测特征点计算出来的脚型参数越准确。Specifically, place the foot on the A4 paper and use a mobile camera device, such as a camera, to take images of the foot shape at multiple angles, so as to fully express the characteristics of the foot shape and obtain enough feature points to be tested, such as the longest Toe apex and heel bulge to calculate the length of the foot shape to be measured, such as the outer point of the thumb ball and the outer point of the base of the tail to calculate the width of the foot shape to be measured, and the ankle point to calculate the height of the ankle point. It should be noted that the number of foot-shaped images captured should be at least three or more. The more images containing the feature points to be measured, the more accurate the foot-type parameters calculated according to the feature points to be measured.
步骤S200，提取待测特征点在每个图像中的位置信息。Step S200: Extract the position information of the feature point to be measured in each image.
具体地，在本实施例的一个优选实施方案中图1所示的三维特征提取方法可以按照以下步骤获取待测特征点在每个图像中的像素位置(x，y)，具体为：Specifically, in a preferred embodiment of this embodiment, the three-dimensional feature extraction method shown in FIG. 1 can obtain the pixel position (x, y) of the feature point to be measured in each image according to the following steps, specifically:
首先通过手动标记待测特征点在某个图像中的像素位置，然后利用特征点匹配方法，如尺度不变特征转换(Scale Invariant Feature Transform，SIFT)或者迭代最近点(Iterative Closest Point，ICP)等，找到该待测特征点在其他图像中对应的像素位置。以测量脚踝的高度为例，选取包含脚踝点的一个图像，手动标记脚踝点在该图像中的像素位置，然后使用SIFT或者ICP等特征点匹配方法找到脚踝点在包含脚踝点的其他角度的图像中对应的像素位置。通过这种方法可以快速找到待测特征点在所有图像中对应的像素位置而不需要对每个图像都进行特征点的手动标记，提高了获取特征点的像素位置的效率。First, manually mark the pixel position of the feature point to be measured in an image, and then use feature point matching methods such as Scale Invariant Feature Transformation (SIFT) or Iterative Closest Point (ICP), etc. To find the corresponding pixel position of the feature point to be measured in other images. Taking the measurement of ankle height as an example, select an image containing the ankle point, manually mark the pixel position of the ankle point in the image, and then use SIFT or ICP and other feature point matching methods to find the image of the ankle point at other angles including the ankle point The corresponding pixel position in. This method can quickly find the corresponding pixel position of the feature point to be measured in all images without the need to manually mark the feature point for each image, which improves the efficiency of obtaining the pixel position of the feature point.
可选的，在本实施例的另一个优选实施方案中图1所示的三维特征提取方法还可以按照以下步骤获取待测特征点在每个图像中的像素位置(x，y)，具体为：Optionally, in another preferred embodiment of this embodiment, the three-dimensional feature extraction method shown in FIG. 1 can also obtain the pixel position (x, y) of the feature point to be measured in each image according to the following steps, specifically :
根据待测特征点的所在区域形状的唯一性，利用特征检测的方法，如广义霍夫变换检测特定形状进而确定每个图像中的待测特征点的位置信息。具体而言，首先确定待测特征点所在区域对应的区域形状，然后根据该区域形状并利用广义霍夫变换自动找到待测特征点在每个图像中对应的待测区域，再根据待测特征点与该区域形状之间的相对位置以及在每个图像中的待测区域得到待测特征点在每个图像中的位置信息。下面以圆形为模板并利用广义霍夫变换找到特征点为例来说明的可能的实现方式。According to the uniqueness of the shape of the region where the feature point to be measured is located, a feature detection method, such as generalized Hough transform, is used to detect a specific shape to determine the position information of the feature point to be measured in each image. Specifically, first determine the shape of the area corresponding to the area where the feature point to be measured corresponds, and then automatically find the corresponding area of the feature point to be measured in each image according to the shape of the area and use the generalized Hough transform, and then according to the feature to be measured The relative position between the point and the shape of the area and the area to be measured in each image obtain the position information of the feature point to be measured in each image. The following uses a circle as a template and uses the generalized Hough transform to find feature points as an example to illustrate possible implementations.
参照图2、图3和图4，图2是本发明实施例中一种基于机器视觉的脚型三维特征提取方法的以圆形为模板使用广义霍夫变换检测特征点的示意图；图3是本发明实施例中一种基于机器视觉的脚型三维特征提取方法的以圆形为模板使用广义霍夫变换检测特征点的示意图；图4是本发明实施例中一种基于机器视觉的脚型三维特征提取方法的以圆形为模板使用广义霍夫变换检测特征点的示意图，图2、图3和图4中分别示出了不同角度下以脚踝点所在区域是圆形为模板来利用广义霍夫变换找到特征点的具体实现方式。如图2、图3和图4所示，脚踝中心所在的脚踝是圆形的，从图中可以看出，这个圆形轮廓在脚型中是唯一的，这样一来，在利用广义霍夫变换时，以圆形为模板，在图像中自动找到圆形的位置(如图2-4中虚线所示的圆形模板)，该位置即是脚踝所在的位置，搜索到圆形位置的中心G点即是待测特征点脚踝点在图像中的位置。2, FIG. 3, and FIG. 4, FIG. 2 is a schematic diagram of a machine vision-based foot shape three-dimensional feature extraction method using a circle as a template to detect feature points using a generalized Hough transform; FIG. 3 is A schematic diagram of a machine vision-based foot shape 3D feature extraction method using a circle as a template to detect feature points using a generalized Hough transform in an embodiment of the present invention; FIG. 4 is a machine vision-based foot shape in an embodiment of the present invention A schematic diagram of a three-dimensional feature extraction method using a generalized Hough transform to detect feature points using a circle as a template. Figures 2, 3, and 4 respectively show that the area where the ankle point is a circle is used as a template at different angles to use generalization Hough transform to find the specific implementation of feature points. As shown in Figure 2, Figure 3 and Figure 4, the ankle where the center of the ankle is located is round. It can be seen from the figure that this circular outline is unique in the foot type. In this way, the generalized Hough is used When changing, use the circle as the template, and automatically find the position of the circle in the image (the circle template shown by the dotted line in Figure 2-4). This position is the position of the ankle, and the center of the circle position is searched. The point G is the position of the ankle point of the feature point to be measured in the image.
可以理解的是，在确定最长脚趾顶点的位置信息时，可以以最长脚趾的轮廓作为广义霍夫变换的模板，在图像中进行搜索，找到脚趾轮廓后，通过该轮廓与最长脚趾顶点的相对位置来确定该特征点的像素位置。It can be understood that when determining the position information of the longest toe vertex, the contour of the longest toe can be used as a template for the generalized Hough transform to search in the image, after finding the toe contour, the contour and the longest toe vertex are found Relative position to determine the pixel position of the feature point.
可选的，在本实施例的另一个优选实施方案中图1所示的三维特征提取方法还可以按照以下步骤获取待测特征点在每个图像中的像素位置(x，y)，具体为：Optionally, in another preferred embodiment of this embodiment, the three-dimensional feature extraction method shown in FIG. 1 can also obtain the pixel position (x, y) of the feature point to be measured in each image according to the following steps, specifically :
基于足够量标记好的脚型的特征点的数据样本并利用深度学习算法构建深度神经网络，然后利用该神经网络获取待测特征点在每个图像中的位置信息。具体地，训练该神经网络时，输入的是包含待测特征点的图像数据，输出的是待测特征点在图像中的像素位置(x，y)，其中，输出包括真实输出和期望输出，网络的最后全连接层真实输出的是待测特征点在该图像中对应的像素位置(x，y)，网络的期望输出的是待测特征点在图像中标记好的实际像素位置。然后利用网络的真实输出与期望输出产生的误差反向训练整个网络，迭代训练直到网络收敛，神经网络训练完毕后输入某个包含待测特征点的待测图像，神经网络自动输出神经网络自动输出在该图像中的像素位置。以获取脚踝点的像素位置为例，选择足够量的标记好脚踝点的图像样本作为训练集，并搭建深层神经网络，然后用训练集训练该深度神经网络，训练完毕后，输入一个包含脚踝点的待测图像，神经网络 自动输出脚踝点在该图像中的像素位置。可以理解的是，确定其他特征点的像素位置时，使用与该特征点对应的图像数据样本训练预先搭建的深度神经网络，然后输入包含该特征点的待测图像从而得到该特征点在图像中的像素位置。Based on the data samples of the feature points of the foot shape marked with a sufficient amount and using a deep learning algorithm to construct a deep neural network, the neural network is then used to obtain the position information of the feature point to be tested in each image. Specifically, when training the neural network, the input is image data containing the feature point to be tested, and the output is the pixel position (x, y) of the feature point to be tested in the image, where the output includes the real output and the expected output, The actual output of the last fully connected layer of the network is the corresponding pixel position (x, y) of the feature point to be measured in the image, and the expected output of the network is the actual pixel position of the feature point to be marked in the image. Then use the error generated by the actual output of the network and the expected output to reverse train the entire network, iteratively train until the network converges, and after the neural network training is completed, input a test image containing the feature points to be tested, and the neural network automatically outputs the neural network automatically The pixel position in the image. Taking the pixel position of the ankle point as an example, select a sufficient amount of image samples that mark the ankle point as the training set, and build a deep neural network, and then use the training set to train the deep neural network. After training, enter a containing ankle point , The neural network automatically outputs the pixel position of the ankle point in the image. It is understandable that when determining the pixel positions of other feature points, the image data samples corresponding to the feature points are used to train the pre-built deep neural network, and then the image to be tested containing the feature points is input to obtain the feature points in the image Pixel location.
步骤S300，根据待测特征点在每个图像中的位置信息获取待测特征点的空间位置信息。Step S300: Acquire the spatial position information of the feature point to be measured according to the position information of the feature point to be tested in each image.
具体地，在本实施例的一个优选实施方案中图1所示的三维特征提取方法可以按照以下步骤获取待测特征点的空间位置信息，具体为：Specifically, in a preferred implementation of this embodiment, the three-dimensional feature extraction method shown in FIG. 1 can obtain the spatial location information of the feature point to be measured according to the following steps, specifically:
首先利用参照物标定相机参数，然后利用三角化方法计算待测特征点的空间位置信息。具体而言，以A4纸作为参照物为例，将脚型放置在A4纸上，利用摄像设备，如相机获取多个不同角度的图像，这些不同角度的图像中包含了A4纸的轮廓。利用这些不同角度的图像来标定相机，确定相机内参数矩阵K，外参数相对世界坐标系旋转矩阵R、平移矩阵t。然后根据步骤S200得到的待测特征点在图像中的像素位置(x，y)，并利用三角化方法以及齐次化求解待测特征点在世界坐标系下的空间位置信息(X，Y，Z)。下面结合图5和图6来说明通过三角化方法获取特征点真实空间位置的可能的实现方式。First, the reference parameters are used to calibrate the camera parameters, and then the triangulation method is used to calculate the spatial position information of the feature points to be measured. Specifically, taking A4 paper as a reference object, the foot shape is placed on the A4 paper, and a plurality of images at different angles are acquired by using an imaging device such as a camera, and the images of these different angles include the outline of the A4 paper. Use these images from different angles to calibrate the camera, determine the camera's internal parameter matrix K, external parameters relative to the world coordinate system rotation matrix R, translation matrix t. Then according to the pixel position (x, y) of the feature point to be measured obtained in step S200 in the image, and using the triangulation method and homogeneous method to solve the spatial position information (X, Y, Z). In the following, a possible implementation manner of obtaining the real space position of the feature point by the triangulation method will be described with reference to FIGS. 5 and 6.
参照图5，图5是本发明实施例中一种基于机器视觉的脚型三维特征提取方法的以直线为模板使用广义霍夫变换检测参照物的示意图。如图5所示，利用直线模板并利用随机霍夫变换检测图像中的A4纸的边缘直线。可以看出，检测到四条边缘直线，各直线两两相交，交点即为A4纸的四个顶点(A、B、C、D)的像素位置(x i，y i)，i＝1，2，3，4。继续参照图2、图3和图4，由空间几何变换知识可得A点在欧氏空间和射影空间的如下关系： Referring to FIG. 5, FIG. 5 is a schematic diagram of a machine vision-based foot shape three-dimensional feature extraction method using a straight line as a template to detect a reference object using a generalized Hough transform in an embodiment of the present invention. As shown in FIG. 5, a straight line template and a random Hough transform are used to detect the edge straight line of the A4 paper in the image. It can be seen that four edge straight lines are detected, and each straight line intersects two by two, and the intersection point is the pixel position (x i , y i ) of the four vertices (A, B, C, D) of the A4 paper, i=1, 2 , 3, 4. Continuing to refer to Figure 2, Figure 3 and Figure 4, from the knowledge of spatial geometric transformation, the following relationship between point A in Euclidean space and projective space can be obtained:
Figure PCTCN2019105962-appb-000001
Figure PCTCN2019105962-appb-000001
公式(1)中参数K、R和t分别是相机内参数矩阵、相机相对世界坐标系的旋转矩阵和平移矩阵([R|t]合称为相机外参数矩阵)。 其中，符号“|”代表增广矩阵，r 1、r 2、r 3分别是相机相对世界坐标系的旋转矩阵R的展开形式，由矩阵乘法可知，r 3与0元素相乘消掉。 The parameters K, R and t in formula (1) are the camera's internal parameter matrix, the camera's rotation matrix and translation matrix relative to the world coordinate system ([R|t] collectively referred to as the camera's external parameter matrix). Among them, the symbol "|" stands for augmented matrix, and r 1 , r 2 , and r 3 are the expansion forms of the rotation matrix R of the camera relative to the world coordinate system. From matrix multiplication, r 3 and 0 elements are multiplied and eliminated.
其中，
Figure PCTCN2019105962-appb-000002
是A4纸顶点A的像素位置，(X A，Y A，Z A) T是其在世界坐标系下的真实位置，K[R|t]是相机的内外参数。单应矩阵H＝K[r 1 r 2|t]有8个自由度，将世界坐标系建立在A4纸的顶点A上，则A4纸的四个顶点的世界坐标系为(0，0，0)，(X，0，0)，(0，Y，0)，(X，Y，0)，其中，X＝210mm，Y＝297mm。每个顶点都能写成式(1)形式以构造两组线性方程。因此，四组顶点可以构建8组线性方程，通过直接线性变换(Direct Linear Transform，DLT)方式来求解H。 among them,
Figure PCTCN2019105962-appb-000002
Is the pixel position of the vertex A of the A4 paper, (X A , Y A , Z A ) T is its true position in the world coordinate system, and K[R|t] is the internal and external parameters of the camera. The homography matrix H=K[r 1 r 2 |t] has 8 degrees of freedom, and the world coordinate system is established on the vertex A of the A4 paper, then the world coordinate system of the four vertices of the A4 paper is (0, 0, 0), (X, 0, 0), (0, Y, 0), (X, Y, 0), where X = 210 mm and Y = 297 mm. Each vertex can be written as formula (1) to construct two sets of linear equations. Therefore, four sets of vertices can construct 8 sets of linear equations, and solve H by Direct Linear Transform (DLT).
由于获取三张照片的角度不同，因而三张照片的相机位姿不同，按照上述同样方法可以得到世界坐标系在相机中的三组单应矩阵H 1，H 2，H 3。 Since the angles for acquiring the three photos are different, the camera poses of the three photos are different. According to the same method as described above, three sets of homography matrices H 1 , H 2 , and H 3 in the camera of the world coordinate system can be obtained.
从单应矩阵H中可求得K，由于H＝[h 1 h 2 h 3]＝K[r 1 r 2|t]，因此可得： K can be obtained from the homography matrix H. Since H=[h 1 h 2 h 3 ]=K[r 1 r 2 |t], we can obtain:
K -1[h 1 h 2 h 3]＝[r 1 r 2|t]  (2) K -1 [h 1 h 2 h 3 ]=[r 1 r 2 |t] (2)
公式(2)中参数K -1、R、t和H分别是相机内参数矩阵的逆矩阵、相机相对世界坐标系旋转矩阵、平移矩阵和单应矩阵。其中，r 1、r 2分别是通过两张不同角度的图像得到的相机外参数相对世界坐标系的旋转矩阵，h 1、h 2、h 3分别是通过三张不同角度的图像得到的相对世界坐标系在相机中的三组单应矩阵。 The parameters K -1 , R, t and H in formula (2) are the inverse matrix of the parameter matrix in the camera, the rotation matrix of the camera relative to the world coordinate system, the translation matrix and the homography matrix, respectively. Among them, r 1 and r 2 are the rotation matrix of the camera external parameters relative to the world coordinate system obtained from two images of different angles, h 1 , h 2 and h 3 are the relative worlds obtained from three images of different angles, respectively Three sets of homography matrices of the coordinate system in the camera.
其中，R＝[r 1 r 2 r 3]为旋转矩阵，具有正交性质，即：r 1 Tr 2＝0且‖r 1‖＝‖r 2‖＝1。因此，可以得到：h 1 TK -TK -1h 2＝0，进而可以得到： Where, R=[r 1 r 2 r 3 ] is a rotation matrix, which has orthogonal properties, namely: r 1 T r 2 =0 and ‖r 1 ‖=‖r 2 ‖=1. Therefore, we can get: h 1 T K -T K -1 h 2 =0, and then we can get:
h 1 TK -TK -1h 1＝h 2 TK -TK -1h 2   (3) h 1 T K -T K -1 h 1 = h 2 T K -T K -1 h 2 (3)
公式(3)中参数K -T和K -1分别是相机内参数矩阵的转置矩阵的正交矩阵和逆矩阵，h 1、h 2分别是通过其中两张不同角度的图像得到的相对世界坐标系在相机中的两组单应矩阵，h 1 T、h 2 T是单应矩阵h 1、h 2的转置矩阵，由上述可得，每两张图像可以获得两个相机的内参数的约束方程。 The parameters K -T and K -1 in formula (3) are the orthogonal matrix and inverse matrix of the transposed matrix of the parameter matrix in the camera, and h 1 and h 2 are the relative worlds obtained from two images with different angles The two homography matrices of the coordinate system in the camera, h 1 T and h 2 T are the transpose matrices of the homography matrices h 1 and h 2. From the above, the internal parameters of the two cameras can be obtained for each two images Constraint equation.
相机内参数矩阵K是上三角矩阵，w＝K -TK -1是对称阵，根据图2、图3和图4的三个不同角度的图像并通过DLT线性求解出w，进而通过正交分解可求解得出K。根据公式(1)可知，[r 1 r 2|t]＝K -1[h 1 h 2 h 3]，结合前述求解得出的h 1、h 2、h 3以及K，可以求解得出r 1、r 2、t。由旋转矩 阵的正交性得到r 3＝r 1×r 2，因此R＝[r 1 r 2 r 3]。由该方法得到拍摄图2、图3和图4时的相机的内外参数K[R 1|t 1]、K[R 2|t 2]、K[R 3|t 3]。 The parameter matrix K in the camera is an upper triangular matrix, and w = K -T K -1 is a symmetric matrix. According to the images at three different angles in Figure 2, Figure 3, and Figure 4, w is linearly solved by DLT, and then orthogonal Decomposition can be solved to get K. According to formula (1), [r 1 r 2 |t]=K -1 [h 1 h 2 h 3 ], combined with h 1 , h 2 , h 3 and K obtained by the previous solution, r can be obtained 1, r 2, t. R 3 =r 1 ×r 2 is obtained from the orthogonality of the rotation matrix, so R=[r 1 r 2 r 3 ]. By this method, the internal and external parameters K[R 1 |t 1 ], K[R 2 |t 2 ], and K[R 3 |t 3 ] of the camera when photographing FIGS. 2, 3, and 4 are obtained.
参照图6，图6是本发明实施例中一种基于机器视觉的脚型三维特征提取方法的三角化过程求解特征点空间位置信息的过程示意图。如图6所示，图6中示出了以图3和图4(即Image1和Image2)中的脚踝点G为例的三角化过程，根据步骤S200中得到的脚踝点G在Image1和Image2中的像素位置x 1和x 2，以及上述步骤中求得的相机的内外参数P 1＝K 1[R 1|t 1]、P 2＝K 2[R 2|t 2]，依次进行重投影误差平方和最小化min∑ i‖P iX-x i‖，从而得到待测特征点在射影空间的位置X＝(M，N，O，w)，其中，P 1、P 2分别是根据标定方法得到的相机拍摄Image1和Image2两张图像时的内外参数，K 1、K 2分别是相机拍摄Image1和Image2两张图像时的相机内参数矩阵，R 1、R 2分别是相机拍摄Image1和Image2两张图像时的相对世界坐标系的旋转矩阵，t 1、t 2分别是平移矩阵。最后，通过齐次化射影空间坐标，即可得到待测特征点G的欧氏空间位置X＝(M/w，N/w，O/w)＝(X，Y，Z)，其中，M，N，O，w分别是特征点G在射影空间下的位置坐标。 Referring to FIG. 6, FIG. 6 is a schematic diagram of a process of solving feature position spatial position information in a triangulation process of a machine vision-based three-dimensional feature extraction method according to an embodiment of the present invention. As shown in FIG. 6, FIG. 6 shows the triangulation process taking the ankle point G in FIGS. 3 and 4 (that is, Image1 and Image2) as an example. According to the ankle point G obtained in step S200 in Image1 and Image2 The pixel positions x 1 and x 2 , and the internal and external parameters of the camera P 1 =K 1 [R 1 |t 1 ], P 2 =K 2 [R 2 |t 2 ], which are re-projected in sequence The sum of squared errors minimizes min∑ i ‖P i Xx i ‖, thus obtaining the position of the feature point to be measured in the projective space X = (M, N, O, w), where P 1 and P 2 are respectively based on the calibration method The obtained internal and external parameters of the camera when shooting the two images of Image1 and Image2, K 1 and K 2 are the camera internal parameter matrix when the camera is shooting the two images of Image1 and Image2, and R 1 and R 2 are the two parameters of the camera when shooting the image1 and Image2 respectively The rotation matrix relative to the world coordinate system when images are taken, t 1 and t 2 are translation matrices, respectively. Finally, through homogeneous projective space coordinates, the Euclidean space position of the feature point G to be measured X = (M/w, N/w, O/w) = (X, Y, Z), where, M , N, O, w are the position coordinates of the feature point G in the projective space, respectively.
可选的，在本实施例的另一个优选实施方案中图1所示的三维特征提取方法还可以按照以下步骤获取待测特征点的真实空间位置，具体为：Optionally, in another preferred embodiment of this embodiment, the three-dimensional feature extraction method shown in FIG. 1 may also obtain the real spatial position of the feature point to be measured according to the following steps, specifically:
将三维重建问题转换成待测特征点的稀疏重建问题，如利用增量式SFM方法构建稀疏模型并利用三角化方法来解决稀疏重建问题。具体而言，根据步骤S200得到的待测特征点在多张图像中的像素位置(x，y)，与上一实施方案不同的是利用增量式SFM方法来直接求解相机内参数矩阵K、相机旋转矩阵R、相对与世界坐标的平移量t、待测特征点在世界坐标系下的坐标λ(X，Y，Z)，略去了用参照物标定相机过程，然后使用已知规格的参照物确定尺度系数λ，进而得到特征点的真实空间位置坐标(X，Y，Z)。下面结合附图7来说明利用增量式SFM方法来解决稀疏重建问题的可能的实现方式，以3个不同角度的图像为例。The three-dimensional reconstruction problem is converted into the sparse reconstruction problem of the feature points to be measured, such as the incremental SFM method to build a sparse model and the triangulation method to solve the sparse reconstruction problem. Specifically, according to the pixel position (x, y) of the feature point to be measured obtained in step S200 in multiple images, the difference from the previous embodiment is to use the incremental SFM method to directly solve the in-camera parameter matrix K, The camera rotation matrix R, the amount of translation t relative to the world coordinates, and the coordinate λ (X, Y, Z) of the feature point to be measured in the world coordinate system, omitting the process of calibrating the camera with a reference object, and then using known specifications The scale factor λ is determined by the reference object, and then the real space position coordinates (X, Y, Z) of the feature point are obtained. In the following, a possible implementation method for solving the sparse reconstruction problem using the incremental SFM method will be described with reference to FIG. 7, taking three images at different angles as an example.
参照图7，图7是本发明实施例中一种基于机器视觉的脚型三维特征提取方法的稀疏重建过程求解特征点空间位置信息的过程示意图。如图7所示，利用增量式SFM方法来解决稀疏重建问题的步骤具体包括：Referring to FIG. 7, FIG. 7 is a schematic diagram of a process of solving feature position spatial position information in a sparse reconstruction process of a machine vision-based three-dimensional feature extraction method according to an embodiment of the present invention. As shown in Figure 7, the steps of using the incremental SFM method to solve the sparse reconstruction problem specifically include:
步骤1：在3个不同角度的图像中随机挑选两张图像Image1和Image2以确定初始图像对，利用增量式SFM方法计算拍摄图像Image1和Image2的相机的内外参数的初始值[R|t]矩阵：利用图像Image1和Image2中的5组特征点对(最长脚趾顶点和脚后跟凸点、指拇指球外侧点和尾趾根部外侧点、脚踝点)，利用5点法分别计算图像Image1和Image2对应的本质矩阵E 1和E 2，其中E＝[R|t]，可以从本质矩阵E中分解出相机旋转矩阵R 1、R 2和相对于世界坐标的平移量t 1、t 2矩阵。然后，结合步骤S200中得到的在相机坐标系下的待测特征点在图像Image1和Image2中的像素位置，构建初始稀疏模型； Step 1: Randomly select two images, Image1 and Image2, from three images at different angles to determine the initial image pair, and use the incremental SFM method to calculate the initial values of the internal and external parameters of the camera that takes the images Image1 and Image2 [R|t] Matrix: Use the five pairs of feature points in the images Image1 and Image2 (the longest toe apex and heel bump, the lateral point of the thumb ball, the lateral point of the caudal toe, and the ankle point). Corresponding to the essential matrices E 1 and E 2 , where E=[R|t], the camera rotation matrices R 1 , R 2 and the translation t 1 , t 2 matrices relative to the world coordinates can be decomposed from the essential matrix E. Then, combining the pixel positions of the feature points to be measured in the camera coordinate system obtained in step S200 in the images Image1 and Image2, an initial sparse model is constructed;
步骤2：根据步骤1的中构建的初始稀疏模型，并且利用三角化方法计算待测特征点在图像Image1和Image2中的世界坐标系下的位置坐标λ(X 1，Y 1，Z 1)和λ(X 2，Y 2，Z 2)； Step 2: According to the initial sparse model constructed in Step 1, and using the triangulation method to calculate the position coordinates λ (X 1 , Y 1 , Z 1 ) of the feature point to be measured in the world coordinate system of the images Image1 and Image2 and λ(X 2 , Y 2 , Z 2 );
步骤3：将图像Image3在步骤S200中得到的相机坐标系下待测特征点的像素位置输入步骤2的得到的初始稀疏模型中，可以重新获取相机内外参数[R|t]矩阵，即相机旋转矩阵R 3和相对于世界坐标的平移量t 3，并且利用该相机内外参数修正初始稀疏模型； Step 3: Enter the pixel position of the feature point to be measured in the camera coordinate system obtained in step S200 of the image Image3 into the initial sparse model obtained in step 2, and the internal and external parameter [R|t] matrix of the camera can be obtained again, that is, the camera rotation Matrix R 3 and the translation t 3 relative to the world coordinates, and use the internal and external parameters of the camera to modify the initial sparse model;
步骤4：根据步骤3修正后的稀疏模型，并且利用三角化方法计算待测特征点在图像Image3中的世界坐标系下的空间位置坐标λ(X 3，Y 3，Z 3)； Step 4: According to the sparse model modified in Step 3, and using the triangulation method to calculate the spatial position coordinates λ (X 3 , Y 3 , Z 3 ) of the feature point to be measured in the world coordinate system in the image Image3;
步骤5：利用捆绑调整(Bundle Adjustment,BA)方法对步骤2和4中得到的特征点的位置坐标进行修正，得到优化后的稀疏模型。Step 5: Use the Bundle Adjustment (BA) method to modify the position coordinates of the feature points obtained in steps 2 and 4 to obtain an optimized sparse model.
其中，步骤5中对待测特征点在剩余其他图像中得到不同的坐标位置反复进行捆绑调整，直至前后两次计算得到的待测特征点的坐标λ(X，Y，Z)的误差小于等于预设的阈值。Among them, in step 5, different coordinate positions of the feature points to be measured in other remaining images are repeatedly bundled and adjusted until the error of the coordinate λ (X, Y, Z) of the feature points to be measured calculated twice before and after is less than or equal to the The set threshold.
虽然本发明仅提供了利用增量式SFM方法来求解三个图像中待测特征点的空间位置信息这一种具体实施方案，但是本领域技术人员可以理解的是，本发明提供的增量式SFM方法还可以用来求解多个不同角度的图像，在利用增量式SFM方法构建稀疏模型的过程中，反复代入新图像中待测特征点在相机坐标系下的像素位置信息，重新获取相机内外参数并且利用该相机内外参数修正稀疏模型，直至得到所有的图像都被添加至稀疏模型。可以理解的是，获取的图像的不同角度越多，迭 代计算的次数就越多，得到的相机内外参数也就越精确，根据其构建的稀疏模型计算得到的待测特征点在世界坐标系下的空间位置信息也就越精确。Although the present invention only provides a specific embodiment of using incremental SFM method to solve the spatial position information of the feature points to be measured in the three images, those skilled in the art can understand that the incremental method provided by the present invention The SFM method can also be used to solve multiple images at different angles. In the process of using the incremental SFM method to build a sparse model, the pixel position information of the feature point to be measured in the new image under the camera coordinate system is repeatedly substituted, and the camera is reacquired Use the internal and external parameters of the camera to modify the sparse model until all images are added to the sparse model. It can be understood that the more different angles of the acquired image, the more iterative calculations, and the more accurate the internal and external parameters of the camera. The feature points to be measured calculated based on the sparse model constructed by it are in the world coordinate system. The more accurate the spatial location information is.
步骤6：以图4中的A点为坐标原点，根据步骤S200中得到的A4纸的顶点D在相机坐标系下的像素位置信息，再利用步骤5中得到的稀疏模型计算得到顶点D的空间坐标为(M，N，0)，而顶点D的真实的空间位置为(210mm，297mm，0)，因此，可以得到尺度系数λ＝210mm/M＝297mm/N。再结合步骤5中得到的待测特征点在世界坐标系下的空间坐标λ(X，Y，Z)，除以尺度系数λ得到待测特征点的真实空间位置(X，Y，Z)。Step 6: Taking the point A in FIG. 4 as the coordinate origin, according to the pixel position information of the vertex D of the A4 paper obtained in step S200 in the camera coordinate system, and then using the sparse model obtained in step 5 to calculate the space of the vertex D The coordinates are (M, N, 0), and the true spatial position of the vertex D is (210 mm, 297 mm, 0), therefore, the scale factor λ=210 mm/M=297 mm/N can be obtained. Combined with the spatial coordinates λ(X, Y, Z) of the feature point to be measured obtained in step 5 in the world coordinate system, divided by the scale factor λ to obtain the true spatial position (X, Y, Z) of the feature point to be measured.
步骤S400，基于空间位置信息和预设的三维特征类别，计算某个待测特征点对应的第一距离信息和/或第二距离信息。Step S400: Calculate the first distance information and/or the second distance information corresponding to a certain feature point to be measured based on the spatial position information and the preset three-dimensional feature category.
需要说明的是，第一距离信息是某个待测特征点与其他待测特征点之间的距离信息，如长度，第二距离信息是某个待测特征点与预设平面之间的垂直距离信息，如高度。It should be noted that the first distance information is the distance information between a certain feature point to be tested and other feature points to be tested, such as the length, and the second distance information is the perpendicularity between a certain feature point to be tested and a preset plane Distance information, such as height.
具体地，以脚型为例，根据步骤S300中计算得到的五个待测特征点的空间位置信息，如，最长脚趾顶点为(X 1，Y 1，Z 1)、脚后跟凸点为(X 2，Y 2，Z 2)、指拇指球外侧点为(X 3，Y 3，Z 3)、尾趾根部外侧点为(X 4，Y 4，Z 4)、脚踝点为(X 5，Y 5，Z 5)，利用距离公式，如欧氏距离公式
Figure PCTCN2019105962-appb-000003
可以得到如下的计算公式： Specifically, taking a foot shape as an example, according to the spatial position information of the five feature points to be measured calculated in step S300, for example, the longest toe vertex is (X 1 , Y 1 , Z 1 ), and the heel convex point is ( X 2 , Y 2 , Z 2 ), the outer point of the thumb ball is (X 3 , Y 3 , Z 3 ), the outer point of the base of the tail toe is (X 4 , Y 4 , Z 4 ), and the ankle point is (X 5 , Y 5 , Z 5 ), using the distance formula, such as the Euclidean distance formula
Figure PCTCN2019105962-appb-000003
The following calculation formula can be obtained:
Figure PCTCN2019105962-appb-000004
Figure PCTCN2019105962-appb-000004
公式(4)中参数L、W和H分别是脚长、脚宽和脚踝高度。The parameters L, W and H in formula (4) are the foot length, foot width and ankle height, respectively.
这样一来，脚长、脚宽以及脚踝点高度三个参数即可求出。虽然本发明仅提供了通过提取三维特征点来计算脚长、脚宽以及脚踝点高度三个参数这一种具体实施方案，但是本领域技术人员可以理解的是，本发明提供的三维特征提取方法还可以计算其他的脚型参数，如计算脚背高度，此时，不同角度的图像中均需包含特征点脚背点，然后依次根据上述实施例中所述的本发明的三维特征提取方法的步骤来计算脚背高度。In this way, the three parameters of foot length, foot width and ankle point height can be obtained. Although the present invention only provides a specific embodiment for calculating three parameters of foot length, foot width and ankle point height by extracting three-dimensional feature points, those skilled in the art can understand that the three-dimensional feature extraction method provided by the present invention You can also calculate other foot shape parameters, such as calculating the instep height. At this time, the images at different angles need to include the feature point instep point, and then sequentially follow the steps of the three-dimensional feature extraction method of the present invention described in the above embodiment. Calculate the instep height.
综上所述，在本发明的优选技术方案中，采用摄像设备获取包含脚型的最长脚趾顶点、脚后跟凸点、指拇指球外侧点、尾趾根部外侧点 以及脚踝点五个待测特征点的五个不同角度的图像，通过手动标记或者自动方法确定每个待测特征点在每一幅图像中的像素位置信息，然后利用手动标定相机参数再三角化或者通过稀疏重建问题求解来求取待测特征点的真实空间位置，不需要对整个目标物的模型重建，能够减少计算量，简化模型建立过程。最后基于五个待测特征点的空间位置，利用欧氏距离公式，从而能够计算得到脚长、脚宽以及脚踝点高度三个脚型参数。依次类推，获取不同特征点的不同角度的图像，亦可计算得到该特征点对应的脚型参数，如获取包含脚背点的不同角度的图像，根据上述步骤可以计算得到脚背点的空间位置信息，从而计算得到脚背高度这个参数。In summary, in the preferred technical solution of the present invention, the camera device is used to acquire five features to be measured including the longest toe apex of the foot shape, the heel convex point, the lateral point of the thumb ball, the lateral point of the caudal toe, and the ankle point For images with five different angles of points, determine the pixel position information of each feature point in each image by manual labeling or automatic methods, and then manually calibrate the camera parameters and then triangulate or solve through the sparse reconstruction problem. Taking the real spatial position of the feature points to be measured does not require the reconstruction of the entire target model, which can reduce the amount of calculation and simplify the model building process. Finally, based on the spatial positions of the five feature points to be measured, the Euclidean distance formula is used to calculate the three foot shape parameters of foot length, foot width and ankle point height. By analogy, the images of different angles of different feature points can be obtained, and the foot parameters corresponding to the feature points can also be calculated. For example, if the images of different angles including the instep point are obtained, the spatial position information of the instep point can be calculated according to the above steps. Thus, the parameter of instep height is calculated.
进一步地，基于上述方法实施例，本发明还提供了一种存储装置，该存储装置中存储有多条程序，该程序可以适用于由处理器加载以执行上述方法实施例所述的基于机器视觉的三维特征提取方法。Further, based on the above method embodiment, the present invention also provides a storage device, the storage device stores a plurality of programs, the program can be adapted to be loaded by a processor to execute the above-described method embodiment based on machine vision 3D feature extraction method.
更进一步地，基于上述方法实施例，本发明还提供了一种控制装置，该控制装置包括处理器和存储设备，其中，存储设备可以适用于存储多条程序，该程序能够适用于由所述处理器加载以执行上述方法实施例所述的基于机器视觉的三维特征提取方法。Further, based on the above method embodiment, the present invention also provides a control device, the control device includes a processor and a storage device, wherein the storage device can be adapted to store multiple programs, the program can be applied to the The processor loads to execute the three-dimensional feature extraction method based on machine vision described in the above method embodiment.
至此，已经结合附图所示的优选实施方式描述了本发明的技术方案，但是，本领域技术人员容易理解的是，本发明的保护范围显然不局限于这些具体实施方式。在不偏离本发明的原理的前提下，本领域技术人员可以对相关技术特征作出等同的更改或替换，这些更改或替换之后的技术方案都将落入本发明的保护范围之内。So far, the technical solutions of the present invention have been described in conjunction with the preferred embodiments shown in the accompanying drawings, but it is easily understood by those skilled in the art that the scope of protection of the present invention is obviously not limited to these specific embodiments. Without departing from the principle of the present invention, those skilled in the art can make equivalent changes or replacements to the related technical features, and the technical solutions after these changes or replacements will fall within the protection scope of the present invention.

Claims (10)

  1. 一种基于机器视觉的三维特征提取方法，其特征在于，所述三维特征提取方法包括下列步骤：A three-dimensional feature extraction method based on machine vision, characterized in that the three-dimensional feature extraction method includes the following steps:
    获取包含参照物及相对于所述参照物设置的目标物的预设待测特征点的多角度图像；Acquiring a multi-angle image including a reference object and a preset feature point to be measured of the target object set relative to the reference object;
    提取所述待测特征点在每个所述图像中的位置信息；Extract the position information of the feature point to be measured in each of the images;
    根据所述待测特征点在每个所述图像中的位置信息获取所述待测特征点的空间位置信息；Acquiring the spatial position information of the feature point to be measured according to the position information of the feature point to be tested in each of the images;
    基于所述空间位置信息和预设的三维特征类别，计算某个待测特征点对应的第一距离信息和/或第二距离信息；Calculate first distance information and/or second distance information corresponding to a certain feature point to be measured based on the spatial position information and a preset three-dimensional feature category;
    其中，所述第一距离信息是所述某个待测特征点与其他待测特征点之间的距离信息，所述第二距离信息是所述某个待测特征点与预设平面之间的垂直距离信息；所述某个待测特征点、所述其他待测特征点与所述平面均取决于所述三维特征类别。Wherein, the first distance information is distance information between the certain feature point to be tested and other feature points to be tested, and the second distance information is between the certain feature point to be tested and a preset plane The vertical distance information of; the certain feature point to be tested, the other feature points to be tested and the plane all depend on the three-dimensional feature category.
  2. 根据权利要求1所述的基于机器视觉的三维特征提取方法，其特征在于，“提取所述待测特征点在每个所述图像中的位置信息”的步骤包括：The method for extracting three-dimensional features based on machine vision according to claim 1, wherein the step of "extracting the position information of the feature point to be measured in each of the images" includes:
    利用手动标记法获取某个所述图像中的所述待测特征点的像素位置；Obtaining the pixel position of the feature point to be measured in a certain image by manual marking method;
    利用预设的特征点匹配法并且根据所获取的像素位置，提取所述待测特征点在其他图像中对应的像素位置。Using the preset feature point matching method and extracting the corresponding pixel positions of the feature points to be measured in other images according to the acquired pixel positions.
  3. 根据权利要求1所述的基于机器视觉的三维特征提取方法，其特征在于，“提取所述待测特征点在每个所述图像中的位置信息”的步骤包括：The method for extracting three-dimensional features based on machine vision according to claim 1, wherein the step of "extracting the position information of the feature point to be measured in each of the images" includes:
    获取所述目标物中所述待测特征点所在区域对应的区域形状；Acquiring the shape of the area corresponding to the area where the feature point to be measured in the target is located;
    根据所述区域形状获取所述每个图像对应的待测区域；Acquiring a region to be measured corresponding to each image according to the shape of the region;
    根据所述待测特征点与所述区域形状之间的相对位置以及每个所述待测区域，获取所述待测特征点在所述每个图像中的位置信息。According to the relative position between the feature point to be measured and the shape of the region and each of the regions to be measured, position information of the feature point to be measured in each image is acquired.
  4. 根据权利要求1所述的基于机器视觉的三维特征提取方法，其特征在于，“提取所述待测特征点在每个所述图像中的位置信息”的步骤包括：The method for extracting three-dimensional features based on machine vision according to claim 1, wherein the step of "extracting the position information of the feature point to be measured in each of the images" includes:
    利用预先构建的神经网络获取所述待测特征点在每个所述图像中的位置信息；Use a pre-built neural network to obtain the position information of the feature point to be measured in each of the images;
    其中，所述神经网络是基于预设的训练集并利用深度学习相关算法训练的深度神经网络。Wherein, the neural network is a deep neural network trained based on a preset training set and using deep learning related algorithms.
  5. 根据权利要求1-4中任一项所述的基于机器视觉的三维特征提取方法，其特征在于，“根据所述待测特征点在每个所述图像中的位置信息获取所述待测特征点的空间位置信息”的步骤包括：The three-dimensional feature extraction method based on machine vision according to any one of claims 1 to 4, wherein "the feature to be measured is acquired according to the position information of the feature point to be measured in each of the images" "Spatial location information of points" includes:
    利用三角化方法并且根据所述待测特征点在所述每个图像中的位置信息与相机内外参数获取所述待测特征点的欧氏位置。The triangulation method is used to obtain the Euclidean position of the feature point to be measured according to the position information of the feature point to be measured in each image and the parameters inside and outside the camera.
  6. 根据权利要求1-4中任一项所述的基于机器视觉的三维特征提取方法，其特征在于，“根据所述待测特征点在每个所述图像中的位置信息获取所述待测特征点的空间位置信息”的步骤包括：The three-dimensional feature extraction method based on machine vision according to any one of claims 1 to 4, wherein "the feature to be measured is acquired according to the position information of the feature point to be measured in each of the images" "Spatial location information of points" includes:
    利用增量式SFM方法和每个所述图像中所述待测特征点的位置信息构建稀疏模型，并利用三角化方法计算所述待测特征点在世界坐标系下的空间位置信息；Constructing a sparse model using the incremental SFM method and the position information of the feature points to be measured in each of the images, and using the triangulation method to calculate the spatial position information of the feature points to be measured in the world coordinate system;
    利用预先获取的尺度系数恢复上述步骤中得到的所述待测特征点在世界坐标系下的空间位置信息，得到所述待测特征点的真实位置。The spatial position information of the feature point to be measured obtained in the above step in the world coordinate system is recovered by using the scale coefficients obtained in advance to obtain the true position of the feature point to be measured.
  7. 根据权利要求6所述的基于机器视觉的三维特征提取方法，其特征在于，在“利用预先获取的尺度系数恢复上述步骤中得到的所述特征点在世界坐标系下的空间位置信息，得到所述待测特征点的真实位置”之前，所述基于机器视觉的三维特征提取方法还包括：The method for extracting three-dimensional features based on machine vision according to claim 6, characterized in that "using the pre-acquired scale coefficients to restore the spatial position information of the feature points in the world coordinate system obtained in the above step to obtain all Before describing the true position of the feature point to be measured, the three-dimensional feature extraction method based on machine vision further includes:
    利用所述稀疏模型并根据在相机坐标系下参照物顶点的像素位置，获取在世界坐标系下所述参照物顶点的空间位置；Use the sparse model and obtain the spatial position of the reference object vertex in the world coordinate system according to the pixel position of the reference object vertex in the camera coordinate system;
    根据在世界坐标系下所述参照物顶点的空间位置以及参照物顶点的真实位置，计算尺度系数。The scale factor is calculated according to the spatial position of the vertex of the reference object and the true position of the vertex of the reference object in the world coordinate system.
  8. 根据权利要求7所述的基于机器视觉的三维特征提取方法，其特征在于，所述三角化方法包括：The method for extracting three-dimensional features based on machine vision according to claim 7, wherein the triangulation method includes:
    根据所述相机内外参数与所述待测特征点在所述每个图像中的位置信息，获取所述待测特征点的射影空间位置，以及Acquiring the projected space position of the feature point to be measured according to the internal and external parameters of the camera and the position information of the feature point to be measured in each image, and
    对所述射影空间位置进行齐次化处理得到所述待测特征点的欧氏位置。Perform homogenization processing on the projective space position to obtain the Euclidean position of the feature point to be measured.
  9. 一种存储装置，其中存储有多条程序，其特征在于，所述程序适于由处理器加载以执行权利要求1-8中任一项所述的基于机器视觉的三维特征提取方法。A storage device, in which a plurality of programs are stored, characterized in that the programs are adapted to be loaded by a processor to execute the machine vision-based three-dimensional feature extraction method according to any one of claims 1-8.
  10. 一种控制装置，包括处理器和存储设备，所述存储设备适于存储多条程序，其特征在于，所述程序适于由所述处理器加载以执行权利要求1-8中任一项所述的基于机器视觉的三维特征提取方法。A control device includes a processor and a storage device, the storage device is adapted to store multiple programs, characterized in that the program is adapted to be loaded by the processor to perform any of claims 1-8 The mentioned three-dimensional feature extraction method based on machine vision.
PCT/CN2019/105962 2018-12-04 2019-09-16 Three-dimensional feature extraction method and apparatus based on machine vision WO2020114035A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201811474153.4A CN109816724B (en) 2018-12-04 2018-12-04 Three-dimensional feature extraction method and device based on machine vision
CN201811474153.4 2018-12-04

Publications (1)

Publication Number Publication Date
WO2020114035A1 true WO2020114035A1 (en) 2020-06-11

Family

ID=66601919

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2019/105962 WO2020114035A1 (en) 2018-12-04 2019-09-16 Three-dimensional feature extraction method and apparatus based on machine vision

Country Status (2)

Country Link
CN (1) CN109816724B (en)
WO (1) WO2020114035A1 (en)

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112487979A (en) * 2020-11-30 2021-03-12 北京百度网讯科技有限公司 Target detection method, model training method, device, electronic device and medium
CN112541936A (en) * 2020-12-09 2021-03-23 中国科学院自动化研究所 Method and system for determining visual information of operating space of actuating mechanism
CN114113163A (en) * 2021-12-01 2022-03-01 北京航星机器制造有限公司 Automatic digital ray detection device and method based on intelligent robot
CN115112098A (en) * 2022-08-30 2022-09-27 常州铭赛机器人科技股份有限公司 Monocular vision one-dimensional two-dimensional measurement method
CN116672082A (en) * 2023-07-24 2023-09-01 苏州铸正机器人有限公司 Navigation registration method and device of operation navigation ruler
CN118010751A (en) * 2024-04-08 2024-05-10 杭州汇萃智能科技有限公司 Machine vision detection method and system for workpiece defect detection

Families Citing this family (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109816724B (en) * 2018-12-04 2021-07-23 中国科学院自动化研究所 Three-dimensional feature extraction method and device based on machine vision
CN110133443B (en) * 2019-05-31 2020-06-16 中国科学院自动化研究所 Power transmission line component detection method, system and device based on parallel vision
CN110223383A (en) * 2019-06-17 2019-09-10 重庆大学 A kind of plant three-dimensional reconstruction method and system based on depth map repairing
CN110796705B (en) * 2019-10-23 2022-10-11 北京百度网讯科技有限公司 Model error elimination method, device, equipment and computer readable storage medium
CN112070883A (en) * 2020-08-28 2020-12-11 哈尔滨理工大学 Three-dimensional reconstruction method for 3D printing process based on machine vision
CN114841959B (en) * 2022-05-05 2023-04-04 广州东焊智能装备有限公司 Automatic welding method and system based on computer vision

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20060126895A1 (en) * 2004-12-09 2006-06-15 Sung-Eun Kim Marker-free motion capture apparatus and method for correcting tracking error
CN106204727A (en) * 2016-07-11 2016-12-07 北京大学深圳研究生院 The method and device that a kind of foot 3-D scanning is rebuild
CN108305286A (en) * 2018-01-25 2018-07-20 哈尔滨工业大学深圳研究生院 Multi-view stereo vision foot type method for three-dimensional measurement, system and medium based on color coding
CN109816724A (en) * 2018-12-04 2019-05-28 中国科学院自动化研究所 Three-dimensional feature extracting method and device based on machine vision

Family Cites Families (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH04349583A (en) * 1991-05-27 1992-12-04 Nippon Telegr & Teleph Corp <Ntt> Generalized hough transform circuit
US6795590B1 (en) * 2000-09-22 2004-09-21 Hrl Laboratories, Llc SAR and FLIR image registration method
CN102376089B (en) * 2010-12-09 2014-05-07 深圳大学 Target correction method and system
CN102157013A (en) * 2011-04-09 2011-08-17 温州大学 System for fully automatically reconstructing foot-type three-dimensional surface from a plurality of images captured by a plurality of cameras simultaneously
CN102354457B (en) * 2011-10-24 2013-10-16 复旦大学 General Hough transformation-based method for detecting position of traffic signal lamp
CN105184857B (en) * 2015-09-13 2018-05-25 北京工业大学 Monocular vision based on structure light ranging rebuilds mesoscale factor determination method
JP6291519B2 (en) * 2016-04-14 2018-03-14 有限会社ネットライズ Method of assigning actual dimensions to 3D point cloud data and position measurement of pipes etc. using it
CN106127258B (en) * 2016-07-01 2019-07-23 华中科技大学 A kind of target matching method
CN107767442B (en) * 2017-10-16 2020-12-25 浙江工业大学 Foot type three-dimensional reconstruction and measurement method based on Kinect and binocular vision

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20060126895A1 (en) * 2004-12-09 2006-06-15 Sung-Eun Kim Marker-free motion capture apparatus and method for correcting tracking error
CN106204727A (en) * 2016-07-11 2016-12-07 北京大学深圳研究生院 The method and device that a kind of foot 3-D scanning is rebuild
CN108305286A (en) * 2018-01-25 2018-07-20 哈尔滨工业大学深圳研究生院 Multi-view stereo vision foot type method for three-dimensional measurement, system and medium based on color coding
CN109816724A (en) * 2018-12-04 2019-05-28 中国科学院自动化研究所 Three-dimensional feature extracting method and device based on machine vision

Cited By (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112487979A (en) * 2020-11-30 2021-03-12 北京百度网讯科技有限公司 Target detection method, model training method, device, electronic device and medium
CN112487979B (en) * 2020-11-30 2023-08-04 北京百度网讯科技有限公司 Target detection method, model training method, device, electronic equipment and medium
CN112541936A (en) * 2020-12-09 2021-03-23 中国科学院自动化研究所 Method and system for determining visual information of operating space of actuating mechanism
CN112541936B (en) * 2020-12-09 2022-11-08 中国科学院自动化研究所 Method and system for determining visual information of operating space of actuating mechanism
CN114113163A (en) * 2021-12-01 2022-03-01 北京航星机器制造有限公司 Automatic digital ray detection device and method based on intelligent robot
CN114113163B (en) * 2021-12-01 2023-12-08 北京航星机器制造有限公司 Automatic digital ray detection device and method based on intelligent robot
CN115112098A (en) * 2022-08-30 2022-09-27 常州铭赛机器人科技股份有限公司 Monocular vision one-dimensional two-dimensional measurement method
CN116672082A (en) * 2023-07-24 2023-09-01 苏州铸正机器人有限公司 Navigation registration method and device of operation navigation ruler
CN116672082B (en) * 2023-07-24 2024-03-01 苏州铸正机器人有限公司 Navigation registration method and device of operation navigation ruler
CN118010751A (en) * 2024-04-08 2024-05-10 杭州汇萃智能科技有限公司 Machine vision detection method and system for workpiece defect detection

Also Published As

Publication number Publication date
CN109816724A (en) 2019-05-28
CN109816724B (en) 2021-07-23

Similar Documents

Publication Publication Date Title
WO2020114035A1 (en) Three-dimensional feature extraction method and apparatus based on machine vision
CN107767442B (en) Foot type three-dimensional reconstruction and measurement method based on Kinect and binocular vision
US20220202138A1 (en) Foot Measuring and Sizing Application
JP5671281B2 (en) Position / orientation measuring apparatus, control method and program for position / orientation measuring apparatus
JP6807639B2 (en) How to calibrate the depth camera
US8452081B2 (en) Forming 3D models using multiple images
Läbe et al. Automatic relative orientation of images
US8447099B2 (en) Forming 3D models using two images
JP6740033B2 (en) Information processing device, measurement system, information processing method, and program
US11042973B2 (en) Method and device for three-dimensional reconstruction
US20130259403A1 (en) Flexible easy-to-use system and method of automatically inserting a photorealistic view of a two or three dimensional object into an image using a cd,dvd or blu-ray disc
CN106705849B (en) Calibrating Technique For The Light-strip Sensors
US10977767B2 (en) Propagation of spot healing edits from one image to multiple images
JP2011138490A (en) Method for determining pose of object in scene
JP2011174879A (en) Apparatus and method of estimating position and orientation
JP2011085971A (en) Apparatus, method, and program for processing image, recording medium, and image processing system
CN109613974B (en) AR home experience method in large scene
JP6071522B2 (en) Information processing apparatus and information processing method
JP5976089B2 (en) Position / orientation measuring apparatus, position / orientation measuring method, and program
TWI599987B (en) System and method for combining point clouds
Ye et al. Accurate and dense point cloud generation for industrial Measurement via target-free photogrammetry
US11475629B2 (en) Method for 3D reconstruction of an object
US7046839B1 (en) Techniques for photogrammetric systems
CN109902695B (en) Line feature correction and purification method for image pair linear feature matching
JP4608152B2 (en) Three-dimensional data processing apparatus, three-dimensional data processing method, and program providing medium

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19892883

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 19892883

Country of ref document: EP

Kind code of ref document: A1