CN117036483A - Golf ball three-dimensional positioning method and device - Google Patents
Golf ball three-dimensional positioning method and device Download PDFInfo
- Publication number
- CN117036483A CN117036483A CN202311069862.5A CN202311069862A CN117036483A CN 117036483 A CN117036483 A CN 117036483A CN 202311069862 A CN202311069862 A CN 202311069862A CN 117036483 A CN117036483 A CN 117036483A
- Authority
- CN
- China
- Prior art keywords
- golf ball
- eye image
- binocular camera
- golf
- center
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 43
- 238000012545 processing Methods 0.000 claims abstract description 60
- 238000012937 correction Methods 0.000 claims abstract description 31
- 238000004519 manufacturing process Methods 0.000 claims abstract description 20
- 241001292396 Cirrhitidae Species 0.000 claims description 30
- 230000000007 visual effect Effects 0.000 claims description 5
- 238000004364 calculation method Methods 0.000 claims description 4
- 238000010586 diagram Methods 0.000 description 10
- 238000001514 detection method Methods 0.000 description 7
- 230000008569 process Effects 0.000 description 5
- 238000011161 development Methods 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 3
- 238000012216 screening Methods 0.000 description 3
- 230000000694 effects Effects 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 230000004927 fusion Effects 0.000 description 2
- 238000005286 illumination Methods 0.000 description 2
- 230000001629 suppression Effects 0.000 description 2
- 238000012549 training Methods 0.000 description 2
- 230000009466 transformation Effects 0.000 description 2
- 229920004449 Halon® Polymers 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000013527 convolutional neural network Methods 0.000 description 1
- 230000002708 enhancing effect Effects 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 238000011478 gradient descent method Methods 0.000 description 1
- 238000010191 image analysis Methods 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000005855 radiation Effects 0.000 description 1
- 230000001105 regulatory effect Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
- G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
Landscapes
- Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Length Measuring Devices By Optical Means (AREA)
- Image Processing (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a golf ball three-dimensional positioning method and a golf ball three-dimensional positioning device, wherein the method comprises the following steps: manufacturing a calibration plate; erecting a binocular camera and determining the range of a golf ball positioning space; calibrating the golf course to obtain a virtual three-dimensional coordinate system of the golf course, and storing camera parameters; acquiring a left-eye image and a right-eye image of a golf ball, which are shot by a binocular camera, and processing the left-eye image and the right-eye image to obtain pixel coordinates of a center of the golf ball in the left-eye image of the golf ball and pixel coordinates of the center of the golf ball in the right-eye image of the golf ball; reading binocular camera parameters; processing parameters of the binocular camera to obtain a correction chart; and processing the binocular camera parameters, the correction chart, the pixel coordinates of the golf ball center in the left eye image of the golf ball and the pixel coordinates of the golf ball center in the right eye image of the golf ball to obtain the three-dimensional positioning coordinates of the golf ball. The method can be used in indoor or outdoor golf simulators, and has accurate positioning and high operation efficiency.
Description
Technical Field
The invention relates to the field of computer positioning, in particular to a golf ball three-dimensional positioning method and device.
Background
(1) halcon image recognition technology
Halcon is a set of perfect standard machine vision industrial software, has a wide machine vision integrated development environment, and a flexible development architecture is convenient for quick development of machine vision, medical images and image analysis application and supports and is compatible with various image acquisition devices, including industrial cameras, image acquisition cards and the like.
(2) Binocular stereo recognition technology
Binocular stereoscopic vision is a method for acquiring three-dimensional geometric information of an object from a plurality of images based on the parallax principle. In a machine vision system, binocular vision generally obtains two digital images of a surrounding scene from different angles by a double camera at the same time, or obtains two digital images of the surrounding scene from different angles by a single camera at different times, and can recover three-dimensional geometric information of an object based on a parallax principle, and reconstruct the three-dimensional shape and position of the surrounding scene.
Binocular vision, sometimes referred to as stereovision, is the primary way for humans to obtain three-dimensional information of the environment using both eyes. From the present point of view, with the development of machine vision theory, binocular stereoscopic vision plays an increasingly important role in machine vision research.
Golf has the characteristics of moderate exercise intensity, strong flexibility, strong skill and the like, and is popular with more and more people at present. Many golf courses are built around integrated sports venues and residential areas. With the increase of the number of participants, the requirements on the field service level and the infrastructure are higher and higher, for example, the field area of a common outdoor golf field is 300-10000 square meters, the coverage range is very large, after the golf ball is hit, the accurate position of the golf ball is difficult to find by the staff, and the judgment of the falling point of the golf ball by human eyes is also very inaccurate. If the golf ball can be rapidly, accurately and efficiently identified and positioned, the complex manual labor for picking up is reduced, so that the purpose of reducing labor cost is achieved, and the golf ball picking device has theoretical significance and practical application value.
The invention is mainly used for detecting the real-time position of the ball in high-speed motion by utilizing the binocular three-dimensional positioning technology when the golf is hit, and provides basic algorithm support for the golf simulator. The invention has the characteristics of accurate positioning, quick response, low delay, high data throughput and the like, and can be widely applied to an indoor golf simulator to realize positioning and data acquisition of golf balls.
Disclosure of Invention
The invention aims to solve the technical problem of providing a three-dimensional positioning method and device for golf, which can acquire the position information of the golf in motion and realize the accurate positioning of the golf.
In order to solve the above technical problems, a first aspect of an embodiment of the present invention discloses a three-dimensional positioning method for a golf ball, the method comprising:
s1, manufacturing a calibration plate;
s2, erecting a binocular camera and determining the range of a golf ball positioning space;
s3, calibrating the golf course and storing camera parameters;
s4, acquiring a left-eye image and a right-eye image of the golf ball shot by the binocular camera;
s5, processing the left-eye image and the right-eye image of the golf ball to obtain pixel coordinates of the center of the golf ball in the left-eye image of the golf ball and pixel coordinates of the center of the golf ball in the right-eye image of the golf ball;
s6, reading parameters of the binocular camera;
s7, processing the binocular camera parameters to obtain a correction chart;
and S8, processing the binocular camera parameters, the correction chart, the pixel coordinates of the golf ball center in the golf ball left eye image and the pixel coordinates of the golf ball center in the golf ball right eye image to obtain the three-dimensional positioning coordinates of the golf ball.
As an optional implementation manner, in the first aspect of the embodiment of the present invention, the manufacturing a calibration plate includes:
s11, generating a calibration plate parameter file and an image file according to a golf ball positioning space by using a gen_caltab operator in a halcon;
s12, manufacturing a calibration plate according to the calibration plate parameter file and the image file;
the size of the calibration plate is related to the size of the space to be calibrated, a large space pair is applied with a large calibration plate, and a small space corresponds to a small calibration plate.
As an optional implementation manner, in the first aspect of the embodiment of the present invention, the erecting the binocular camera, determining a range of the golf ball positioning space includes:
s21, fixing the binocular camera;
s22, adjusting the angle of the binocular camera to enable parallax of the binocular camera to exist;
and S23, overlapping visual angles of the binocular cameras are the range of the golf ball positioning space.
In an optional implementation manner, in the first aspect of the embodiment of the present invention, the calibrating the golf course, and saving the camera parameters includes:
s31, placing the calibration plate in the range of the golf ball positioning space;
s32, calibrating the golf course by utilizing a find_caltab operator and a binocular_calibration operator in the halcon to obtain internal parameters and external parameters R1, T1, R2 and T2 of the binocular camera;
r1 and T1 represent the relative positions of the left camera and the world coordinate system, and R2 and T2 represent the relative positions of the right camera and the world coordinate system;
s33, moving the calibration plate to different areas, collecting calibration information again, and repeating for a plurality of times, so that the areas collected by the calibration plate cover all the identifiable areas;
s34, calibrating is completed by using a gen_biological_recovery_map operator in the halcon, and camera parameters are stored.
In a first aspect of the embodiment of the present invention, the processing the left-eye image and the right-eye image of the golf ball to obtain the pixel coordinates of the center of the golf ball in the left-eye image of the golf ball and the pixel coordinates of the center of the golf ball in the right-eye image of the golf ball includes:
and processing the left-eye golf image and the right-eye golf image by using a bipolarity_registration operator in the halcon to obtain the pixel coordinates of the center of the golf ball in the left-eye golf image and the pixel coordinates of the center of the golf ball in the right-eye golf image.
As an optional implementation manner, in the first aspect of the embodiment of the present invention, the reading the binocular camera parameter includes:
and reading the binocular camera parameters by using a read_cam_par operator and a read_phase operator in the halcon.
As an optional implementation manner, in the first aspect of the embodiment of the present invention, the processing the binocular camera parameter to obtain a calibration chart includes:
and processing the binocular camera parameters by using a gen_binocular_recovery_map operator in the halcon to obtain a correction chart. The camera parameter file generated at calibration is read using the read_cam_par (reading the camera's internal parameters from the folder) and the read_post operator (reading a three-dimensional pattern from a text file).
In a first aspect of the embodiment of the present invention, the processing the binocular camera parameter, the calibration chart, the pixel coordinate of the center of the golf ball in the left eye image of the golf ball, and the pixel coordinate of the center of the golf ball in the right eye image of the golf ball to obtain the three-dimensional positioning coordinate of the golf ball includes:
and processing the binocular camera parameters, the correction chart, the pixel coordinates of the golf ball center in the golf ball left eye image and the pixel coordinates of the golf ball center in the golf ball right eye image by using an interject_lines_of_sight operator in the halcon to obtain the three-dimensional positioning coordinates of the golf ball.
In a second aspect, an embodiment of the present invention discloses a golf ball three-dimensional positioning device, the device including:
the calibration plate manufacturing module is used for manufacturing a calibration plate;
the binocular camera setting module is used for erecting a binocular camera and determining the range of a golf ball positioning space;
the golf course calibration module is used for calibrating the golf course and storing camera parameters;
the golf ball image acquisition module is used for acquiring a left eye image and a right eye image of the golf ball shot by the binocular camera;
the golf ball center pixel coordinate calculation module is used for processing the golf ball left eye image and the golf ball right eye image to obtain the pixel coordinate of the golf ball center in the golf ball left eye image and the pixel coordinate of the golf ball center in the golf ball right eye image;
the binocular camera parameter reading module is used for reading the binocular camera parameters;
the correction chart generation module is used for processing the binocular camera parameters to obtain a correction chart;
and the three-dimensional positioning module is used for processing the binocular camera parameters, the correction chart, the pixel coordinates of the golf ball center in the golf ball left-eye image and the pixel coordinates of the golf ball center in the golf ball right-eye image to obtain the three-dimensional positioning coordinates of the golf ball.
In a second aspect of the embodiment of the present invention, the manufacturing the calibration plate includes:
s11, generating a calibration plate parameter file and an image file according to a golf ball positioning space by using a gen_caltab operator in a halcon;
s12, manufacturing a calibration plate according to the calibration plate parameter file and the image file;
the size of the calibration plate is related to the size of the space to be calibrated, a large space pair is applied with a large calibration plate, and a small space corresponds to a small calibration plate.
In a second aspect of the present invention, the erecting binocular camera, determining the range of the golf ball location space, includes:
s21, fixing the binocular camera;
s22, adjusting the angle of the binocular camera to enable parallax of the binocular camera to exist;
and S23, overlapping visual angles of the binocular cameras are the range of the golf ball positioning space.
In a second aspect of the present invention, the calibrating the golf course, storing the camera parameters includes:
s31, placing the calibration plate in the range of the golf ball positioning space;
s32, calibrating the golf course by utilizing a find_caltab operator and a binocular_calibration operator in the halcon to obtain internal parameters and external parameters R1, T1, R2 and T2 of the binocular camera;
r1 and T1 represent the relative positions of the left camera and the world coordinate system, and R2 and T2 represent the relative positions of the right camera and the world coordinate system;
s33, moving the calibration plate to different areas, collecting calibration information again, and repeating for a plurality of times, so that the areas collected by the calibration plate cover all the identifiable areas;
s34, calibrating is completed by using a gen_biological_recovery_map operator in the halcon, and camera parameters are stored.
In a second aspect of the present invention, the processing the left-eye image and the right-eye image of the golf ball to obtain the pixel coordinates of the center of the golf ball in the left-eye image of the golf ball and the pixel coordinates of the center of the golf ball in the right-eye image of the golf ball includes:
and processing the left-eye golf image and the right-eye golf image by using a bipolarity_registration operator in the halcon to obtain the pixel coordinates of the center of the golf ball in the left-eye golf image and the pixel coordinates of the center of the golf ball in the right-eye golf image.
As an optional implementation manner, in the second aspect of the embodiment of the present invention, the reading the binocular camera parameter includes:
and reading the binocular camera parameters by using a read_cam_par operator and a read_phase operator in the halcon.
In a second aspect of the embodiment of the present invention, the processing the binocular camera parameters to obtain a calibration chart includes:
and processing the binocular camera parameters by using a gen_binocular_recovery_map operator in the halcon to obtain a correction chart.
In a second aspect of the present invention, the processing the binocular camera parameter, the calibration chart, the pixel coordinate of the center of the golf ball in the left eye image of the golf ball, and the pixel coordinate of the center of the golf ball in the right eye image of the golf ball to obtain the three-dimensional positioning coordinate of the golf ball includes:
and processing the binocular camera parameters, the correction chart, the pixel coordinates of the golf ball center in the golf ball left eye image and the pixel coordinates of the golf ball center in the golf ball right eye image by using an interject_lines_of_sight operator in the halcon to obtain the three-dimensional positioning coordinates of the golf ball.
In a third aspect, the present invention discloses another golf ball three-dimensional positioning device, the device comprising:
a memory storing executable program code;
a processor coupled to the memory;
the processor invokes the executable program code stored in the memory to perform some or all of the steps in the three-dimensional positioning method for golf balls disclosed in the first aspect of the embodiment of the present invention.
Compared with the prior art, the embodiment of the invention has the following beneficial effects:
(1) In the embodiment of the invention, the golf ball is three-dimensionally positioned based on binocular stereo vision, the position coordinates of the golf ball are calculated through two golf ball photos with different visual angles at the same time, and a virtual three-dimensional coordinate system is established by taking the space calibration origin as the center, so that the method has the characteristics of accurate positioning data, wide positioning space and the like.
(2) The method is developed based on a general algorithm library rich in halons, and has the advantages of simple program algorithm, high operation efficiency and accurate positioning.
(3) The invention can manufacture the calibration plate according to the need, is compatible with different use environments, uses adjustable parameters for coefficients in the algorithm, opens calling authority, comprises parameters such as an effective threshold, average gray scale, sphere identification gray scale range, identifiable range and the like, and can customize and be compatible with different use environments, comprising different light and shadow environments, different space sizes and the like.
(4) The invention has high algorithm compatibility, supports various input modes including modes of industrial cameras, image acquisition cards, image file reading, network streaming and the like, supports various image formats and image types, can be derived into C, C ++, C#, java and other programming languages, and has high algorithm universality and compatibility.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for the description of the embodiments will be briefly described below, and it is apparent that the drawings in the following description are only some embodiments of the present invention, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a schematic flow chart of a three-dimensional golf ball positioning method according to an embodiment of the present invention;
FIG. 2 is a diagram of a calibration plate disclosed in an embodiment of the present invention;
FIG. 3 is a calibration process diagram disclosed in an embodiment of the present invention;
FIG. 4 is a polar diagram after calibration is completed in accordance with an embodiment of the present invention;
FIG. 5 is a binocular artwork disclosed by an embodiment of the present invention;
FIG. 6 is a diagram of a binocular golf ball recognition and coordinates according to an embodiment of the present invention;
FIG. 7 is a schematic view of a golf ball three-dimensional positioning device according to an embodiment of the present invention;
fig. 8 is a schematic view of another golf ball three-dimensional positioning device according to an embodiment of the present invention.
Detailed Description
In order to make the present invention better understood by those skilled in the art, the following description will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings, and it is apparent that the described embodiments are only some embodiments of the present invention, not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
The terms first, second and the like in the description and in the claims and in the above-described figures are used for distinguishing between different objects and not necessarily for describing a sequential or chronological order. Furthermore, the terms "comprise" and "have," as well as any variations thereof, are intended to cover a non-exclusive inclusion. For example, a process, method, apparatus, article, or device that comprises a list of steps or elements is not limited to the list of steps or elements but may, in the alternative, include other steps or elements not expressly listed or inherent to such process, method, article, or device.
Reference herein to "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment may be included in at least one embodiment of the invention. The appearances of such phrases in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments. Those of skill in the art will explicitly and implicitly appreciate that the embodiments described herein may be combined with other embodiments.
The invention discloses a golf ball three-dimensional positioning method and a golf ball three-dimensional positioning device, wherein the method comprises the following steps: manufacturing a calibration plate; erecting a binocular camera and determining the range of a golf ball positioning space; calibrating the golf course to obtain a virtual three-dimensional coordinate system of the golf course, and storing camera parameters; acquiring a left-eye image and a right-eye image of a golf ball, which are shot by a binocular camera, and processing the left-eye image and the right-eye image to obtain pixel coordinates of a center of the golf ball in the left-eye image of the golf ball and pixel coordinates of the center of the golf ball in the right-eye image of the golf ball; reading binocular camera parameters; processing parameters of the binocular camera to obtain a correction chart; and processing the binocular camera parameters, the correction chart, the pixel coordinates of the golf ball center in the left eye image of the golf ball and the pixel coordinates of the golf ball center in the right eye image of the golf ball to obtain the three-dimensional positioning coordinates of the golf ball. The method can be used in indoor or outdoor golf simulators, and has accurate positioning and high operation efficiency.
Example 1
Referring to fig. 1, fig. 1 is a schematic flow chart of a three-dimensional positioning method for golf balls according to an embodiment of the invention. The three-dimensional positioning method of golf ball described in fig. 1 may be used in indoor or outdoor golf simulators, and the embodiments of the present invention are not limited by the application of identifying and detecting three-dimensional position coordinates and trajectories of golf balls at the moment of golf ball striking. As shown in fig. 1, the golf ball three-dimensional positioning method may include the following operations:
s1, manufacturing a calibration plate;
s2, erecting a binocular camera and determining the range of a golf ball positioning space;
s3, calibrating the golf course and storing camera parameters;
s4, acquiring a left-eye image and a right-eye image of the golf ball shot by the binocular camera;
s5, processing the left-eye image and the right-eye image of the golf ball to obtain pixel coordinates of the center of the golf ball in the left-eye image of the golf ball and pixel coordinates of the center of the golf ball in the right-eye image of the golf ball;
s6, reading parameters of the binocular camera;
s7, processing the binocular camera parameters to obtain a correction chart;
and S8, processing the binocular camera parameters, the correction chart, the pixel coordinates of the golf ball center in the golf ball left eye image and the pixel coordinates of the golf ball center in the golf ball right eye image to obtain the three-dimensional positioning coordinates of the golf ball.
Optionally, the manufacturing calibration plate includes:
s11, generating a calibration plate parameter file and an image file according to a golf ball positioning space by using a gen_caltab operator in a halcon;
s12, manufacturing a calibration plate according to the calibration plate parameter file and the image file;
the size of the calibration plate is related to the size of the space to be calibrated, a large space pair is applied with a large calibration plate, and a small space corresponds to a small calibration plate.
Before binocular three-dimensional positioning is used, a virtual three-dimensional space coordinate system is defined through calibration and used for identifying information such as a space origin, XYZ axes, distance units and the like. The first step of calibration is to manufacture a calibration plate, and the calibration plate parameter file and the image file can be quickly produced by using a gen_caltab operator (a calibration plate description file and a corresponding PostScript file are generated for the calibration plate with rectangular arrangement marks), and the image file is printed according to the original size, so that a calibration plate can be obtained, parameters of the operator during generation determine the size of the image file, and a space distance unit calibrated by using the calibration plate. FIG. 2 is a diagram of a calibration plate according to an embodiment of the present invention. FIG. 3 is a diagram of a calibration process disclosed in an embodiment of the present invention.
Optionally, the erecting the binocular camera determines the range of the golf ball positioning space, including:
s21, fixing the binocular camera;
s22, adjusting the angle of the binocular camera to enable parallax of the binocular camera to exist;
and S23, overlapping visual angles of the binocular cameras are the range of the golf ball positioning space.
After the fixing, the movement needs to be calibrated again.
Optionally, the calibrating the golf course, saving the camera parameters, includes:
s31, placing the calibration plate in the range of the golf ball positioning space;
s32, calibrating the golf course by utilizing a find_caltab operator and a binocular_calibration operator in the halcon to obtain internal parameters and external parameters R1, T1, R2 and T2 of the binocular camera;
r1 and T1 represent the relative positions of the left camera and the world coordinate system, and R2 and T2 represent the relative positions of the right camera and the world coordinate system;
s33, moving the calibration plate to different areas, collecting calibration information again, and repeating for a plurality of times, so that the areas collected by the calibration plate cover all the identifiable areas;
s34, calibrating is completed by using a gen_biological_recovery_map operator in the halcon, and camera parameters are stored.
The calibration plate is placed in the identifiable region, the find_caltab operator (dividing the standard calibration plate region in the image, eliminating the hole of the tiny mark with the diameter smaller than the expected size) and the binocularjcalibration operator (placing the calibration plate in a proper position so that it can be fully imaged in both cameras) is operated, the internal parameters of the two cameras and their external parameters (R1, T1 and R2, T2) are determined through calibration, then R1 and T1 represent the relative positions of the left camera and the world coordinate system, R2 and T2 represent the relative positions of the right camera and the world coordinate system, calibration information can be acquired once, the calibration plate is moved to different regions, calibration information is acquired again, and repeated for several times, so that the region acquired by the calibration plate covers substantially all the identifiable regions, and the calibration of the gen_binocularjmap (with camera parameters and gestures as inputs, output as corrected images and corrected parameters and gestures) is completed, and the camera parameter file is saved. And executing limit constraint, producing a polar diagram, and checking the calibrated binocular matching effect and error condition. FIG. 4 is a polar diagram after calibration is completed, as disclosed in an embodiment of the present invention.
Binocular image input: through two cameras that have parallax, gather binocular image, guarantee that binocular image must be the image of same moment, just can guarantee the accuracy of location. Fig. 5 is a binocular artwork as disclosed in an embodiment of the present invention.
Optionally, the processing the left-eye image and the right-eye image of the golf ball to obtain the pixel coordinates of the center of the golf ball in the left-eye image of the golf ball and the pixel coordinates of the center of the golf ball in the right-eye image of the golf ball includes:
and processing the left-eye golf image and the right-eye golf image by using a bipolarity_registration operator in the halcon to obtain the pixel coordinates of the center of the golf ball in the left-eye golf image and the pixel coordinates of the center of the golf ball in the right-eye golf image.
The internal parameters of the two cameras and their external parameters (R1, T1 and R2, T2) are determined by calibration, R1, T1 representing the relative position of the left camera and the world coordinate system, R2, T2 representing the relative position of the right camera and the world coordinate system
And respectively calculating the pixel coordinates of the center of the golf ball in the two images, wherein the pixel coordinates can use the coordinates of the center of the golf ball because the golf ball is a spherical object, the three-dimensional positioning effect is not affected, and if the golf ball is in other special shapes, the positions with the mark features need to be acquired, and the smaller the area is, the more accurate the position is.
Optionally, the reading the binocular camera parameter includes:
and reading the binocular camera parameters by using a read_cam_par operator and a read_phase operator in the halcon. The camera parameter file generated at calibration is read using the read_cam_par (reading the camera's internal parameters from the folder) and the read_post operator (reading a three-dimensional pattern from a text file).
Optionally, the processing the binocular camera parameter to obtain a correction chart includes:
and processing the binocular camera parameters by using a gen_binocular_recovery_map operator in the halcon to obtain a correction chart. Using the gen_biological_reconstruction_map operator (creating a projection map that describes the mapping between the image and its corresponding changing radiation distortion), a correction map based on the camera parameter file is generated.
Optionally, the processing the binocular camera parameter, the correction chart, the pixel coordinate of the golf ball center in the left eye image of the golf ball, and the pixel coordinate of the golf ball center in the right eye image of the golf ball to obtain the three-dimensional positioning coordinate of the golf ball includes:
and processing the binocular camera parameters, the correction chart, the pixel coordinates of the golf ball center in the golf ball left eye image and the pixel coordinates of the golf ball center in the golf ball right eye image by using an interject_lines_of_sight operator in the halcon to obtain the three-dimensional positioning coordinates of the golf ball. The three-dimensional coordinates of the golf ball can be calculated using the inter-lines of sight operator (to obtain the spatial location of the keypoints under the left camera. This operator requires two camera internal parameters and the relative Pose. And the image coordinates of the keypoints in the two images). The camera parameters, the calibration map, and the binocular identified golf ball pixel coordinates are transferred into the operator. Fig. 6 is a diagram of binocular golf ball recognition and coordinates according to an embodiment of the present invention.
Optionally, the method for processing the left-eye image and the right-eye image of the golf ball to obtain the pixel coordinates of the center of the golf ball in the left-eye image of the golf ball and the pixel coordinates of the center of the golf ball in the right-eye image of the golf ball may be:
the method comprises the steps of firstly, processing a golf ball left eye image or a golf ball right eye image to obtain a horizontal gradient and a vertical gradient;
secondly, processing the horizontal gradient and the vertical gradient to obtain a detection model:
T x (i,j)＝sinα×T 1 (i,j)
T x (i,j)＝sinα×T 2 (i,j)
T y (i,j)＝cosα×T 3 (i,j)
T y (i,j)＝cosα×T 4 (i,j)
α＝arctan(|j-b|/|i-a|)
wherein (a, b) is the center coordinates of the center of a circle, alpha is the included angle between the gradient and the horizontal direction, T x (i, j) is a horizontal gradient,T y (i, j) is a vertical gradient, T (i, j) is a detection model comprising a left model T in a horizontal direction 1 (i, j), right model T in horizontal direction 2 (i, j), left model T in vertical direction 3 (i, j), right model T in vertical direction 4 (i, j), (i, j) being coordinate values of pixels;
thirdly, convolving the left eye image or the right eye image of the golf ball by using the detection model to obtain a horizontal convolved left image F xl (i, j), horizontal convolution right image F xr (i, j), vertically convolved left image F yl (i, j), vertical convolution right image F yr (i,j)；
Fourth, the left image F is convolved horizontally xl (i, j), horizontal convolution right image F xr (i, j), vertically convolved left image F yl (i, j), vertical convolution right image F yr Processing (i, j) to obtain a symmetry detection result s (i, j):
s(i,j)＝|log(s l (i,j))/log(s r (i,j))|
fifth, the symmetry detection result s (i, j) is processed to obtain a fusion result G (i, j):
F x (i,j)＝F xl (i,j)+F xr (i,j)
F y (i,j)＝F yl (i,j)+F yr (i,j)
sixth, processing the symmetry detection result s (i, j) and the fusion result G (i, j) to obtain a screening probability p:
seventh, searching a point with the maximum screening probability p value in the left eye image or the right eye image of the golf ball, wherein the point with the maximum screening probability p value is the center position of the left eye image or the right eye image of the golf ball.
Alternatively, the pixel coordinates of the center of the golf ball in the two images calculated respectively according to the embodiment of the present invention may be implemented by the following steps.
(1) Image graying processing
The graying process is to convert a color three-dimensional image into a two-dimensional image. The color image has R, G, B three components, and graying is the conversion of the R, G, B component of color into three equal components. Pixels with large gray values are brighter and pixels with small gray values are darker.
(2) Image enhancement processing
After the image information is grayed, the phenomenon of low contrast of the image is caused, at the moment, edge extraction cannot be directly carried out, otherwise, the extracted limit is blurred, and positioning is inaccurate. Here, let r denote the gray level of the original image, and s denote the gray level after histogram equalization. If the number of pixels of an image is n, there are l gray levels, n k Representative gray level r k The probability of occurrence of the kth gray level can be expressed as:
the transformation function may be rewritten as:
the gray value of each pixel after equalization can be calculated from the histogram of the image.
Alternatively, image enhancement may be performed using the following manner:
optionally, acquiring an image to be enhanced, decomposing the image to be enhanced into a base layer only containing structural information and a detail layer only containing details and noise by using a TV norm model, and respectively enhancing, wherein the regular term coefficients of the TV model are given by the estimated noise level in a self-adaptive manner;
base layer processing: decomposing the base layer into an illumination component and a reflection component by using a Retinex model, processing the illumination component by using nonlinear Sigmoid transformation to stretch contrast, highlighting details, and regulating reflection component equalization by using gamma correction;
detail layer processing: and removing redundant noise of the detail layer by using a Gaussian filter, and improving the quality of the detail layer.
And respectively processing the image base layer and the detail layer to be enhanced to obtain an enhanced image.
The input image I (x, y) is a superposition of two layers:
I(x,y)＝I base (x,y)+I detail (x,y)
wherein the base layer is I base (x, y), detail layer is I detail (x, y) deriving the base layer by minimizing the following objective function
Wherein item 1 is a differential term adapted to texture components for preserving meaningful structure, item 2 is a regularization term based on total variation,represents the gradient operator, λ is 3 times the global noise estimate σ, λ=3σ.
Global noise estimation sigma
Where x represents the convolution operator, W and H are the width and height of the image I (x, y), respectively, and N is the convolution template:
the detail layer is as follows:
I detail (x,y)＝I(x,y)-I base (x,y)
and processing the input image through the base layer and the detail layer to obtain a brightness enhancement image of the base layer and a noise suppression detail image of the detail layer. Finally, according to the processed basic layer and detail layer, the final result can be obtained
I enh (x,y)＝I base_enh (x,y)+θ·I detail_enh (x,y)
Where θ is used for balance detail enhancement and noise suppression, this parameter can be obtained through a number of experiments.
(3) Detection of the golf ball using YOLOv3 and calculation of the pixel coordinates of the center of the golf ball:
step one, carrying out manual marking on an original image to generate a training set with labels.
And step two, inputting the data labels and the images into a Yolov3 convolutional neural network, and training the network.
And thirdly, calculating a YOLOv3 network error, comparing a predicted true value with a label, and calculating the network error, wherein the error mainly comprises a coordinate error, a confidence error and a classification error.
And step four, updating parameters of the YOLOv3 network by using a back propagation method and a gradient descent method, and repeating the step three and the step four for a plurality of times, so that the error finally converges in an acceptable error range, and obtaining a parameter model of the YOLOv3 network.
Inputting the picture to be detected into a trained model, and detecting the golf.
And step six, acquiring the position coordinates of the golf ball by using YOLOv3 and outputting the position coordinates.
Therefore, by implementing the golf ball detecting and positioning method described in the embodiment of the invention, the pixel coordinates of the center of the golf ball in the two images can be obtained.
Example two
Referring to fig. 7, fig. 7 is a schematic structural diagram of a three-dimensional positioning device for golf balls according to an embodiment of the invention. The three-dimensional golf ball positioning device shown in fig. 7 may be used in an indoor or outdoor golf simulator, and the three-dimensional golf ball position coordinates and trajectories at the moment of golf ball striking are identified and detected, which is not limited by the embodiment of the present invention. As shown in fig. 7, the golf ball three-dimensional positioning device may include the following operations:
s301, a calibration plate manufacturing module for manufacturing a calibration plate;
s302, a binocular camera setting module, which is used for erecting a binocular camera and determining the range of a golf ball positioning space;
s303, a golf course calibration module, which is used for calibrating the golf course and storing camera parameters;
s304, a golf ball image acquisition module for acquiring a golf ball left eye image and a golf ball right eye image shot by a binocular camera;
s305, a golf ball center pixel coordinate calculation module, which is used for processing the golf ball left eye image and the golf ball right eye image to obtain the pixel coordinate of the golf ball center in the golf ball left eye image and the pixel coordinate of the golf ball center in the golf ball right eye image;
s306, a binocular camera parameter reading module, which is used for reading the binocular camera parameters;
s307, a correction chart generating module, which is used for processing the binocular camera parameters to obtain a correction chart;
and S308, a three-dimensional positioning module is used for processing the binocular camera parameters, the correction chart, the pixel coordinates of the golf ball center in the golf ball left eye image and the pixel coordinates of the golf ball center in the golf ball right eye image to obtain the three-dimensional positioning coordinates of the golf ball.
Example III
Referring to fig. 8, fig. 8 is a schematic structural view of another golf ball three-dimensional positioning device according to an embodiment of the invention. The three-dimensional golf ball positioning device shown in fig. 8 may be used in an indoor or outdoor golf simulator, and the three-dimensional golf ball positioning device is not limited by the embodiment of the invention. As shown in fig. 8, the golf ball three-dimensional positioning device may include the following operations:
a memory 401 storing executable program codes;
a processor 402 coupled with the memory 401;
processor 402 invokes the executable program code stored in memory 401 to perform the golf ball three-dimensional positioning method as described in embodiment one.
The apparatus embodiments described above are merely illustrative, in which the modules illustrated as separate components may or may not be physically separate, and the components shown as modules may or may not be physical, i.e., may be located in one place, or may be distributed over multiple network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art will understand and implement the present invention without undue burden.
From the above detailed description of the embodiments, it will be apparent to those skilled in the art that the embodiments may be implemented by means of software plus necessary general hardware platforms, or of course by means of hardware. Based on such understanding, the foregoing technical solutions may be embodied essentially or in part in the form of a software product that may be stored in a computer-readable storage medium including Read-Only Memory (ROM), random-access Memory (Random Access Memory, RAM), programmable Read-Only Memory (Programmable Read-Only Memory, PROM), erasable programmable Read-Only Memory (Erasable Programmable Read Only Memory, EPROM), one-time programmable Read-Only Memory (OTPROM), electrically erasable programmable Read-Only Memory (EEPROM), compact disc Read-Only Memory (Compact Disc Read-Only Memory, CD-ROM) or other optical disc Memory, magnetic disc Memory, tape Memory, or any other medium that can be used for computer-readable carrying or storing data.
Finally, it should be noted that: the embodiment of the invention discloses a golf ball three-dimensional positioning method and device, which are disclosed by the embodiment of the invention only for illustrating the technical scheme of the invention, but not limiting the technical scheme; although the invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art will understand that; the technical scheme recorded in the various embodiments can be modified or part of technical features in the technical scheme can be replaced equivalently; such modifications and substitutions do not depart from the spirit and scope of the corresponding technical solutions.
Claims (10)
1. A method of three-dimensional positioning of a golf ball, the method comprising:
s1, manufacturing a calibration plate;
s2, erecting a binocular camera and determining the range of a golf ball positioning space;
s3, calibrating the golf course and storing camera parameters;
s4, acquiring a left-eye image and a right-eye image of the golf ball shot by the binocular camera;
s5, processing the left-eye image and the right-eye image of the golf ball to obtain pixel coordinates of the center of the golf ball in the left-eye image of the golf ball and pixel coordinates of the center of the golf ball in the right-eye image of the golf ball;
s6, reading parameters of the binocular camera;
s7, processing the binocular camera parameters to obtain a correction chart;
and S8, processing the binocular camera parameters, the correction chart, the pixel coordinates of the golf ball center in the golf ball left eye image and the pixel coordinates of the golf ball center in the golf ball right eye image to obtain the three-dimensional positioning coordinates of the golf ball.
2. The method of three-dimensional positioning of a golf ball according to claim 1, wherein said fabricating a calibration plate comprises:
s11, generating a calibration plate parameter file and an image file according to a golf ball positioning space by using a gen_caltab operator in a halcon;
s12, manufacturing a calibration plate according to the calibration plate parameter file and the image file;
the size of the calibration plate is related to the size of the space to be calibrated, a large space pair is applied with a large calibration plate, and a small space corresponds to a small calibration plate.
3. The method of three-dimensional positioning of a golf ball according to claim 1, wherein the erecting a binocular camera, determining the extent of a golf ball positioning space, comprises:
s21, fixing the binocular camera;
s22, adjusting the angle of the binocular camera to enable parallax of the binocular camera to exist;
and S23, overlapping visual angles of the binocular cameras are the range of the golf ball positioning space.
4. The method of three-dimensional positioning of golf balls according to claim 1, wherein calibrating the golf course, storing camera parameters, comprises:
s31, placing the calibration plate in the range of the golf ball positioning space;
s32, calibrating the golf course by utilizing a find_caltab operator and a binocular_calibration operator in the halcon to obtain internal parameters and external parameters R1, T1, R2 and T2 of the binocular camera;
r1 and T1 represent the relative positions of the left camera and the world coordinate system, and R2 and T2 represent the relative positions of the right camera and the world coordinate system;
s33, moving the calibration plate to different areas, collecting calibration information again, and repeating for a plurality of times, so that the areas collected by the calibration plate cover all the identifiable areas;
s34, calibrating is completed by using a gen_biological_recovery_map operator in the halcon, and camera parameters are stored.
5. The method of three-dimensional positioning of a golf ball according to claim 1, wherein processing the left-eye image and the right-eye image of the golf ball to obtain pixel coordinates of a center of the golf ball in the left-eye image and pixel coordinates of a center of the golf ball in the right-eye image of the golf ball comprises:
and processing the left-eye golf image and the right-eye golf image by using a bipolarity_registration operator in the halcon to obtain the pixel coordinates of the center of the golf ball in the left-eye golf image and the pixel coordinates of the center of the golf ball in the right-eye golf image.
6. The golf ball three-dimensional positioning method according to claim 1, wherein the reading the binocular camera parameters comprises:
and reading the binocular camera parameters by using a read_cam_par operator and a read_phase operator in the halcon.
7. The method of three-dimensional positioning of a golf ball according to claim 1, wherein said processing of said binocular camera parameters to obtain a calibration map comprises:
and processing the binocular camera parameters by using a gen_binocular_recovery_map operator in the halcon to obtain a correction chart.
8. The method according to claim 1, wherein the processing the binocular camera parameters, the calibration map, the pixel coordinates of the center of the golf ball in the left eye image of the golf ball, and the pixel coordinates of the center of the golf ball in the right eye image of the golf ball to obtain the three-dimensional positioning coordinates of the golf ball comprises:
and processing the binocular camera parameters, the correction chart, the pixel coordinates of the golf ball center in the golf ball left eye image and the pixel coordinates of the golf ball center in the golf ball right eye image by using an interject_lines_of_sight operator in the halcon to obtain the three-dimensional positioning coordinates of the golf ball.
9. A golf ball three-dimensional positioning device, the device comprising:
the calibration plate manufacturing module is used for manufacturing a calibration plate;
the binocular camera setting module is used for erecting a binocular camera and determining the range of a golf ball positioning space;
the golf course calibration module is used for calibrating the golf course and storing camera parameters;
the golf ball image acquisition module is used for acquiring a left eye image and a right eye image of the golf ball shot by the binocular camera;
the golf ball center pixel coordinate calculation module is used for processing the golf ball left eye image and the golf ball right eye image to obtain the pixel coordinate of the golf ball center in the golf ball left eye image and the pixel coordinate of the golf ball center in the golf ball right eye image;
the binocular camera parameter reading module is used for reading the binocular camera parameters;
the correction chart generation module is used for processing the binocular camera parameters to obtain a correction chart;
and the three-dimensional positioning module is used for processing the binocular camera parameters, the correction chart, the pixel coordinates of the golf ball center in the golf ball left-eye image and the pixel coordinates of the golf ball center in the golf ball right-eye image to obtain the three-dimensional positioning coordinates of the golf ball.
10. A golf ball three-dimensional positioning device, the device comprising:
a memory storing executable program code;
a processor coupled to the memory;
the processor invokes the executable program code stored in the memory for performing the golf ball three-dimensional positioning method according to any one of claims 1-8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311069862.5A CN117036483A (en) | 2023-08-23 | 2023-08-23 | Golf ball three-dimensional positioning method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311069862.5A CN117036483A (en) | 2023-08-23 | 2023-08-23 | Golf ball three-dimensional positioning method and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117036483A true CN117036483A (en) | 2023-11-10 |
Family
ID=88628016
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311069862.5A Pending CN117036483A (en) | 2023-08-23 | 2023-08-23 | Golf ball three-dimensional positioning method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117036483A (en) |
-
2023
- 2023-08-23 CN CN202311069862.5A patent/CN117036483A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111738261B (en) | Single-image robot unordered target grabbing method based on pose estimation and correction | |
| CN107766855B (en) | Chessman positioning method and system based on machine vision, storage medium and robot | |
| CN106503671B (en) | The method and apparatus for determining human face posture | |
| KR101761751B1 (en) | Hmd calibration with direct geometric modeling | |
| CN106651942B (en) | Three-dimensional rotating detection and rotary shaft localization method based on characteristic point | |
| US10217293B2 (en) | Depth camera-based human-body model acquisition method and network virtual fitting system | |
| CN108717531B (en) | Human body posture estimation method based on Faster R-CNN | |
| CN103839277B (en) | A kind of mobile augmented reality register method of outdoor largescale natural scene | |
| CN110544301A (en) | Three-dimensional human body action reconstruction system, method and action training system | |
| CN109247068A (en) | Method and apparatus for rolling shutter compensation | |
| CN112509125A (en) | Three-dimensional reconstruction method based on artificial markers and stereoscopic vision | |
| CN113850865A (en) | Human body posture positioning method and system based on binocular vision and storage medium | |
| CN106464780B (en) | XSLIT camera | |
| CN104424640A (en) | Method and device for carrying out blurring processing on images | |
| CN109752855A (en) | A kind of method of hot spot emitter and detection geometry hot spot | |
| CN112348890B (en) | Space positioning method, device and computer readable storage medium | |
| CN109274883A (en) | Posture antidote, device, terminal and storage medium | |
| CN114022560A (en) | Calibration method and related device and equipment | |
| CN113329179B (en) | Shooting alignment method, device, equipment and storage medium | |
| CN111080776A (en) | Processing method and system for human body action three-dimensional data acquisition and reproduction | |
| CN115035546B (en) | Three-dimensional human body posture detection method and device and electronic equipment | |
| CN111353325A (en) | Key point detection model training method and device | |
| CN112365578A (en) | Three-dimensional human body model reconstruction system and method based on double cameras | |
| CN114882106A (en) | Pose determination method and device, equipment and medium | |
| CN105488780A (en) | Monocular vision ranging tracking device used for industrial production line, and tracking method thereof |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |