CN107481270B - Table tennis target tracking and trajectory prediction method, device, storage medium and computer equipment - Google Patents
Table tennis target tracking and trajectory prediction method, device, storage medium and computer equipment Download PDFInfo
- Publication number
- CN107481270B CN107481270B CN201710682442.2A CN201710682442A CN107481270B CN 107481270 B CN107481270 B CN 107481270B CN 201710682442 A CN201710682442 A CN 201710682442A CN 107481270 B CN107481270 B CN 107481270B
- Authority
- CN
- China
- Prior art keywords
- tracking target
- tracking
- target
- bounding box
- image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
- G06T7/292—Multi-camera tracking
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
- G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
- G06T7/248—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments involving reference images or patches
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Image Analysis (AREA)
Abstract
The invention relates to a table tennis target tracking and trajectory prediction method, a table tennis target tracking and trajectory prediction device, a storage medium and computer equipment. Acquiring a frame of image shot by the two cameras at the same time respectively for the tracking target, and extracting a candidate area corresponding to the tracking target from the image. And inputting the candidate area into a preset tracking model for processing to obtain a bounding box corresponding to the tracking target. And acquiring two-dimensional coordinates of the center of the bounding box corresponding to the tracking target respectively shot by the two cameras at the same moment, and calculating the three-dimensional coordinates of the center of the bounding box according to the projection matrix of the cameras. And acquiring three-dimensional coordinates of the bounding boxes at continuous moments to form a continuous coordinate sequence, inputting the continuous coordinate sequence into a recurrent neural network (LSTM) for calculation to generate a subsequent coordinate sequence, and obtaining the track of the tracking target according to the coordinate sequence. The target position is tracked by adopting a preset tracking model, and the accurate prediction of the motion trail of the tracked target can be realized by combining the advantage that the LSTM can effectively analyze the time sequence characteristics.
Description
Technical Field
The invention relates to the technical field of image processing, in particular to a method, a device, a storage medium and computer equipment for table tennis target tracking and trajectory prediction.
Background
In the design of table tennis robot systems, there are two problems that need to be solved. One is the problem of target tracking, i.e. predicting the position that may occur in the next frame of a tracked target given the position of the last frame of the tracked target. The other is the problem of trajectory prediction, namely, a small section of table tennis coordinate sequence is given, and a subsequent coordinate sequence is automatically generated.
The problem of target tracking, as a classic problem in computer vision, has continued to develop significantly in the last few decades. From the beginning, Lucas-Kanade trackers, mean-shift trackers and the like based on pure computer vision methods, to more complex trackers which integrate detection and learning ideas thereof later, and to the tracking algorithm based on deep learning nowadays. The main deep learning model currently used for tracking is based on CNN, i.e. convolutional neural networks. In a general CNN-based tracking algorithm, a CNN model mainly serves as a feature extractor (feature extractor). The bounding box obtained by the current tracking algorithm is not accurate enough, and the inaccurate bounding box not only means the error of the position information, but also directly causes the whole tracking frame to generate drift and even lose the target. When the target tracking has errors, the error of the track prediction can be directly caused.
Disclosure of Invention
In view of the above, it is necessary to provide a table tennis target tracking and trajectory prediction method, apparatus, storage medium and computer device.
A table tennis target tracking and trajectory prediction method, the method comprising:
acquiring a frame of image shot by two cameras for a tracking target at the same moment;
extracting a candidate region corresponding to the tracking target from the image;
inputting the candidate area into a preset tracking model for processing to obtain a bounding box corresponding to the tracking target;
acquiring two-dimensional coordinates of the bounding box center corresponding to the tracking target respectively shot by the two cameras at the same moment, and calculating the three-dimensional coordinates of the bounding box center of the tracking target corresponding to the moment according to the camera projection matrix;
acquiring three-dimensional coordinates corresponding to bounding boxes at continuous moments to form a continuous coordinate sequence, inputting the continuous coordinate sequence into a recurrent neural network (LSTM) for calculation, and generating a subsequent coordinate sequence;
and obtaining the track of the tracking target according to the continuous coordinate sequence and the subsequent coordinate sequence.
In one embodiment, the method further comprises:
inputting the calculated three-dimensional coordinates of the center of the bounding box of the tracking target corresponding to the moment into an LSTM (local Scale TM) for calculation, and predicting the three-dimensional coordinates of the tracking target in the next frame of image shot by the two cameras;
and taking the area containing the three-dimensional coordinates as a candidate area of a tracking target in the next frame of image.
In one embodiment, the inputting the candidate region into a preset tracking model for processing to obtain a bounding box corresponding to the tracking target includes:
inputting the candidate region into a preset convolutional neural network model, and processing to obtain a bounding box of a tracking target in the image;
inputting the bounding box of the tracking target in the image into a preset regression layer to perform regression processing, and obtaining a regressed bounding box corresponding to the tracking target, wherein the preset regression layer comprises a low-layer convolution layer of the preset convolution neural network model.
In one embodiment, the extracting, from the image, a candidate region corresponding to the tracking target includes:
and extracting a candidate region corresponding to the tracking target from the image by adopting a background subtraction method.
In one embodiment, the process of creating the camera projection matrix includes:
respectively establishing a world coordinate system and a camera coordinate system;
acquiring an internal parameter matrix and an external parameter matrix of a camera;
and establishing a camera projection matrix according to the internal parameter matrix and the external parameter matrix, wherein the camera projection matrix can convert the two-dimensional coordinates of the camera coordinate system into the three-dimensional coordinates of the world coordinate system.
A table tennis target tracking and trajectory prediction apparatus, the apparatus comprising:
the camera shooting module is used for acquiring a frame of image shot by the two cameras on the tracking target at the same moment;
the tracking target candidate region weighting module is used for extracting a candidate region corresponding to the tracking target from the image;
a tracking target bounding box obtaining module, configured to input the candidate region into a preset tracking model for processing to obtain a bounding box corresponding to the tracking target;
the tracking target bounding box three-dimensional coordinate calculation module is used for acquiring two-dimensional coordinates of the bounding box center corresponding to the tracking target respectively shot by the two cameras at the same moment, and then calculating the three-dimensional coordinates of the bounding box center of the tracking target corresponding to the moment according to the camera projection matrix;
the coordinate sequence generation module is used for acquiring three-dimensional coordinates corresponding to bounding boxes at continuous moments to form a continuous coordinate sequence, inputting the continuous coordinate sequence into a recurrent neural network (LSTM) for calculation, and generating a subsequent coordinate sequence;
and the track generation module of the tracking target is used for obtaining the track of the tracking target according to the continuous coordinate sequence and the subsequent coordinate sequence.
In one embodiment, the apparatus further comprises:
the three-dimensional coordinate prediction module of the tracking target is used for inputting the calculated three-dimensional coordinate of the center of the bounding box of the tracking target corresponding to the moment into the LSTM for calculation and predicting the three-dimensional coordinate of the tracking target in the next frame of image shot by the two cameras;
and the candidate region acquisition module of the tracking target is used for taking the region containing the three-dimensional coordinates as a candidate region of the tracking target in the next frame of image.
In one embodiment, the tracked target bounding box obtaining module includes:
the convolutional neural network module is used for inputting the candidate region into a preset convolutional neural network model and processing the candidate region to obtain a bounding box of a tracking target in the image;
and the regression layer module is used for inputting the bounding box of the tracking target in the image into a preset regression layer to perform regression processing to obtain a regressed bounding box corresponding to the tracking target, wherein the preset regression layer comprises a low-layer convolution layer of the preset convolution neural network model.
A computer-readable storage medium, on which a computer program is stored which, when executed by a processor, carries out the steps of:
acquiring a frame of image shot by two cameras for a tracking target at the same moment;
extracting a candidate region corresponding to the tracking target from the image;
inputting the candidate area into a preset tracking model for processing to obtain a bounding box corresponding to the tracking target;
acquiring two-dimensional coordinates of the bounding box center corresponding to the tracking target respectively shot by the two cameras at the same moment, and calculating the three-dimensional coordinates of the bounding box center of the tracking target corresponding to the moment according to the camera projection matrix;
acquiring three-dimensional coordinates corresponding to bounding boxes at continuous moments to form a continuous coordinate sequence, inputting the continuous coordinate sequence into a recurrent neural network (LSTM) for calculation, and generating a subsequent coordinate sequence;
and obtaining the track of the tracking target according to the continuous coordinate sequence and the subsequent coordinate sequence.
A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the following steps when executing the computer program:
acquiring a frame of image shot by two cameras for a tracking target at the same moment;
extracting a candidate region corresponding to the tracking target from the image;
inputting the candidate area into a preset tracking model for processing to obtain a bounding box corresponding to the tracking target;
acquiring two-dimensional coordinates of the bounding box center corresponding to the tracking target respectively shot by the two cameras at the same moment, and calculating the three-dimensional coordinates of the bounding box center of the tracking target corresponding to the moment according to the camera projection matrix;
acquiring three-dimensional coordinates corresponding to bounding boxes at continuous moments to form a continuous coordinate sequence, inputting the continuous coordinate sequence into a recurrent neural network (LSTM) for calculation, and generating a subsequent coordinate sequence;
and obtaining the track of the tracking target according to the continuous coordinate sequence and the subsequent coordinate sequence.
According to the table tennis target tracking and track predicting method, device, storage medium and computer equipment, images shot by the two cameras at the same time for the tracked target are obtained, the candidate area corresponding to the tracked target is extracted from the images, and the candidate area is input into the CNN model to be processed to obtain the bounding box corresponding to the tracked target. Because two cameras shoot simultaneously, two images are obtained at each moment, and then two bounding boxes are obtained, and the two-dimensional coordinates of the centers of the two bounding boxes corresponding to the tracking target in the image at the same moment are combined with the projection matrix of the camera to calculate the three-dimensional coordinates of the bounding box of the tracking target corresponding to the moment. And acquiring three-dimensional coordinates corresponding to the bounding boxes at continuous moments, and forming a continuous coordinate sequence. And inputting the continuous coordinate sequence into the LSTM for calculation to generate a subsequent coordinate sequence. And obtaining a complete track of the tracking target according to the continuous coordinate sequence and the subsequent coordinate sequence. Accurate target position tracking is carried out by adopting the CNN model, and accurate prediction of the motion trail of the tracked target can be realized by combining the advantage that the LSTM can effectively analyze the time sequence characteristics.
Drawings
FIG. 1 is a diagram of an exemplary table tennis target tracking and trajectory prediction method;
FIG. 2 is an internal block diagram of a server in one embodiment;
FIG. 3 is a flow diagram of a table tennis target tracking and trajectory prediction method in one embodiment;
FIG. 4 is a flow diagram of a table tennis target tracking and trajectory prediction method in one embodiment;
FIG. 5 is a flow chart of the method of FIG. 4 for obtaining bounding boxes;
FIG. 6 is a flow diagram of a method for building a camera projection matrix in one embodiment;
FIG. 7 is a schematic diagram of a table tennis target tracking and trajectory prediction device in one embodiment;
FIG. 8 is a schematic diagram of a table tennis target tracking and trajectory prediction device in one embodiment;
FIG. 9 is a schematic diagram of the tracking target bounding box acquisition module of FIG. 7;
fig. 10 is a schematic structural diagram of a table tennis target tracking and trajectory prediction device in one embodiment.
Detailed Description
In order to make the aforementioned objects, features and advantages of the present invention comprehensible, embodiments accompanied with figures are described in detail below. In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present invention. This invention may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein, but rather should be construed as broadly as the present invention is capable of modification in various respects, all without departing from the spirit and scope of the present invention.
Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. The terminology used in the description of the invention herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The technical features of the above embodiments can be arbitrarily combined, and for the sake of brevity, all possible combinations of the technical features in the above embodiments are not described, but should be considered as the scope of the present specification as long as there is no contradiction between the combinations of the technical features.
In recent years, with the development and the gradual maturity of computer vision technology, the specific application of computers in the field of sports is continuously appeared. In the table tennis sport, the table tennis is tracked in each frame of image shot by the camera, so that the position information of the table tennis is recorded, and the motion trail of the table tennis is predicted. Because table tennis balls have the characteristics of small volume, few characteristics and fast movement, tracking and prediction algorithms need to be specially designed to meet the requirements.
The table tennis target tracking and trajectory prediction method provided by the embodiment of the invention needs to be used under a specific actual environment configuration. As shown in fig. 1, assume that a player is playing a ball and a camera is used to photograph a table tennis ball 110 and a table tennis table 120. Specifically, two high-speed cameras are arranged on one side of the table tennis table, are connected in parallel through a hardware trigger, synchronously shoot the table area, and keep not moving in the whole process. In order to ensure that the three-dimensional coordinates can be accurately calculated later, the model specifications of the cameras need to be consistent. A world coordinate system is established with one corner of the table as the origin. Wherein the x-axis is along the table bottom line, the y-axis is along the table side line, and the z-axis is perpendicular to the table. It is necessary to calculate the projection matrix of the two cameras to the table in advance in order to calculate the three-dimensional coordinates from the two-dimensional coordinates obtained by the two cameras or to project the three-dimensional coordinates onto the camera plane.
In one embodiment, as shown in fig. 2, there is also provided a server comprising a processor, a non-volatile storage medium having an operating system stored therein and a table tennis target tracking and trajectory prediction apparatus for performing a table tennis target tracking and trajectory prediction method, connected by a system bus, an internal memory, a network interface. The processor is used for improving the calculation and control capacity and supporting the operation of the whole server. An internal memory for providing an environment for operation of a table tennis target tracking and trajectory prediction device in a non-volatile storage medium may have stored therein computer readable instructions that, when executed by a processor, may cause the processor to perform a table tennis target tracking and trajectory prediction method. The network interface receives a video or the like containing a tracking target.
In one embodiment, as shown in fig. 3, a table tennis target tracking and trajectory prediction method is provided, which is described by taking the application scenario in fig. 1 as an example, and includes:
Two high-speed cameras are arranged on one side of the table tennis table to synchronously shoot the table area. The tracking target is a moving table tennis ball, and at each moment, one frame of image is respectively obtained from the two cameras.
In step 320, a candidate region corresponding to the tracking target is extracted from the image.
Before tracking the tracked target in the image, how to obtain an initial bounding box of the tracked target, namely how to detect the tracked target, namely the table tennis ball, in the image is solved. The framework first looks for a possible area for a table tennis ball. Because the cameras are fixed all the time and the whole scene does not have too many moving objects, foreground regions can be extracted from images shot by the two cameras respectively in a background subtraction mode and the like, the search range is narrowed, and the regions are used as candidate regions corresponding to the tracking target.
And 330, inputting the candidate area into a preset tracking model for processing to obtain a bounding box corresponding to the tracking target.
The preset tracking model comprises: presetting a convolutional neural network model and a regression layer. Convolutional Neural Network (CNN) is one of the Network structures that are very representative in deep learning techniques. The preset convolutional neural network model is a convolutional neural network model obtained by training a convolutional neural network in advance through a group of labeled training sets and comprises a convolutional layer, a pooling layer and a full-connection layer. The preset regression layer includes: a full connection layer, a region of interest pooling layer and a lower convolutional layer in the preset convolutional neural network model. And establishing a preset regression layer, wherein after another group of labeled training sets are processed by the preset convolutional neural network model to be established, the other group of labeled training sets are trained to establish the preset regression layer after the regression processing is performed on the regression layer.
And respectively inputting the candidate regions extracted from the image into a preset tracking model to be processed to obtain a bounding box corresponding to the tracking target. Specifically, the candidate regions are sequentially input into the CNN for object detection, and a probability value is output to indicate whether the candidate regions contain an object. If the input candidate region does not find a target, all candidate regions need to be input to the CNN detection target. If the target is found in the previous frame image, only the candidate area near the position of the target in the previous frame image is input into the CNN, so that unnecessary calculation can be reduced, and the efficiency can be improved.
And 340, acquiring two-dimensional coordinates of the centers of the bounding boxes corresponding to the tracked targets respectively shot by the two cameras at the same moment, and calculating the three-dimensional coordinates of the centers of the bounding boxes of the tracked targets corresponding to the moment according to the camera projection matrix.
After the preset tracking model processing, the bounding boxes corresponding to the tracking targets in the images respectively shot by the two cameras at the same moment are obtained, and then the two-dimensional coordinates of the centers of the bounding boxes are obtained according to the bounding boxes. And then, calculating the three-dimensional coordinate of the center of the bounding box of the tracking target corresponding to the moment in a world coordinate system according to the camera projection matrix. Wherein the camera projection matrix is pre-calculated. Specifically, a world coordinate system and a camera coordinate system are respectively established in advance, and then an internal parameter matrix and an external parameter matrix of the camera are obtained. And establishing a camera projection matrix according to the internal parameter matrix and the external parameter matrix, wherein the camera projection matrix can convert the two-dimensional coordinates of the camera coordinate system into the three-dimensional coordinates of the world coordinate system.
And 350, acquiring three-dimensional coordinates corresponding to the bounding boxes at continuous moments to form a continuous coordinate sequence, inputting the continuous coordinate sequence into a recurrent neural network (LSTM) for calculation, and generating a subsequent coordinate sequence.
And (4) obtaining the bounding boxes corresponding to the tracking targets at the continuous moments by calculating the images shot at each moment, and then obtaining the three-dimensional coordinates corresponding to the centers of the bounding boxes. And sequentially forming a continuous coordinate sequence by the three-dimensional coordinates of the centers of the bounding boxes corresponding to the continuous time instants. Inputting the continuous coordinate sequence into a recurrent neural network LSTM to automatically generate a subsequent coordinate sequence. LSTM (Long Short-Term Memory) refers to a two-way Long and Short Term Memory network model, which is a time recursive neural network. The bidirectional long and short term memory network model comprises a forward long and short term memory network model and a backward long and short term memory network model.
And step 360, obtaining the track of the tracking target according to the continuous coordinate sequence and the subsequent coordinate sequence.
The subsequent coordinate sequence obtained by calculating the continuous coordinate sequence is input into the LSTM, and the continuous coordinate sequence and the subsequent coordinate sequence form the track of the tracking target, so that the track prediction and the drop point prediction can be carried out on the tracking target such as a table tennis.
In this embodiment, images shot by the two cameras at the same time for the tracking target are obtained, a candidate region corresponding to the tracking target is extracted from the images, and the candidate region is input to a preset tracking model and processed to obtain a bounding box corresponding to the tracking target. Because two cameras shoot simultaneously, two images are obtained at each moment, and then two bounding boxes are obtained, and the two-dimensional coordinates of the centers of the two bounding boxes corresponding to the tracking target in the image at the same moment are combined with the projection matrix of the camera to calculate the three-dimensional coordinates of the bounding box of the tracking target corresponding to the moment. And acquiring three-dimensional coordinates corresponding to the bounding boxes at continuous moments, and forming a continuous coordinate sequence. And inputting the continuous coordinate sequence into the LSTM for calculation to generate a subsequent coordinate sequence. And obtaining a complete track of the tracking target according to the continuous coordinate sequence and the subsequent coordinate sequence. The method adopts a preset tracking model to accurately track the position of the target, and combines the advantage that the LSTM can effectively analyze the time sequence characteristics, so that the accurate prediction of the motion trail of the tracked target can be realized.
In one embodiment, as shown in fig. 4, a table tennis target tracking and trajectory prediction method further comprises:
The LSTM may be used as a model for tracking a target, and the coordinates obtained by processing the currently captured image may be input to the LSTM to predict the coordinates of the target bounding box to be tracked in the next frame image. Specifically, the obtained coordinates are input into the LSTM model in each cycle, and the LSTM model is enabled to output only one parameter of the hybrid gaussian model, which is similar to a kalman filter, to predict the position of the tracking target at which the next frame may appear, thereby reducing the search range of tracking.
And step 380, taking the area containing the three-dimensional coordinates as a candidate area of the tracking target in the next frame image.
The region including the three-dimensional coordinates obtained by the LSTM prediction is set as a candidate region for the tracking target in the next frame image. When the shielding occurs or the tracking target moves at an excessively high speed and the preset tracking model may lose the target, the prediction result of the LSTM is used as the tracking result, so that the whole tracking frame can continue to work, and the target is not paralyzed due to the loss of the preset tracking model.
In this embodiment, the continuous coordinate sequence of the tracking target calculated from the previous frames of images is used, and the LSTM model is used to obtain not only the subsequent coordinate sequence of the tracking target, but also the complete track of the tracking target according to the coordinate sequence. The target in the next frame image can also be tracked. Therefore, the searching range of tracking by using the preset tracking model can be reduced, and the defects that the preset tracking model can lose targets when the tracked targets are shielded or the moving speed is too high can be overcome.
In an embodiment, as shown in fig. 5, inputting the candidate region into a preset tracking model to process the candidate region to obtain a bounding box corresponding to the tracking target, includes:
and step 331, inputting the candidate region into a preset convolutional neural network model, and processing to obtain a bounding box of the tracking target in the image.
The preset tracking model comprises: presetting a convolutional neural network model and a regression layer. Specifically, a candidate area with 100 × 100 pixels extracted from the image is input into a preset convolutional neural network model for convolution operation. The convolutional layer in the convolutional neural network model is pre-trained using CaffeNet. The convolution operation may be a plurality of convolution operations to extract a feature map of the candidate region.
And a pooling layer is arranged above the convolution layer, and the feature map of the extracted image is input into the pooling layer to be pooled, namely, the feature is compressed to obtain a compressed feature map. In particular, the pooling layer may be a spatial pyramid pooling layer (spatial pyramid pooling layer) for retaining more location information.
The compressed feature map obtained by the pooling layer is passed through two full-connected layers, and the output 2500-dimensional vector is converted into a 50 × 50 matrix, i.e., a 50 × 50 probability map is output. Each element of the matrix is a probability value representing the probability that a pixel at a corresponding location in the input image belongs to the tracked object. For an image containing a tracked object, a connected region is typically output. The probability values within this region are significantly higher than the probability values outside. A bounding box can be calculated by thresholding the probability values that are within the bounding box above a certain probability value, and this bounding box serves as a prediction of the target location.
The method comprises the following steps of establishing a preset convolutional neural network model: acquiring a convolutional neural network training set for modeling, wherein the convolutional neural network training set comprises an image containing a target and an image not containing the target, and the image is acquired from a video containing the target; labeling the image, setting a value inside an actual bounding box of the target in the image as a first value, and setting a value outside the actual bounding box of the target in the image as a second value; inputting the convolutional neural network training set into a convolutional neural network of the initialized network parameters for training to obtain a bounding box of a target in an image; calculating network parameters of the modeled convolutional neural network according to the bounding box of the target in the image, the marked actual bounding box and the Softmax loss function; and obtaining a preset convolutional neural network model according to the network parameters.
The method comprises the following steps of establishing a preset regression layer: obtaining a regression layer training set for modeling, wherein the regression layer training set comprises images containing targets, and the images are obtained from videos containing the targets; labeling the image, and labeling the size of an actual bounding box of a target in the image; inputting the regression layer training set into a preset convolutional neural network model for training to obtain a bounding box of a target in an image; inputting the bounding box of the target in the image into a regression layer of the initialized network parameters for regression processing to obtain the size of the regressed bounding box corresponding to the target; calculating the network parameters of the regression layer after modeling according to the size of the regression bounding box corresponding to the target, the size of the marked actual bounding box and a smoothL1 loss function; and obtaining a preset regression layer according to the network parameters.
And 333, inputting the bounding box of the tracking target in the image into a preset regression layer to perform regression processing to obtain a regressed bounding box corresponding to the tracking target, wherein the preset regression layer comprises a low-layer convolution layer of a preset convolution neural network model.
The regression layer follows the pre-set convolutional neural network model. The regression layer is a low-layer convolution layer, an interest region pooling layer and a full-link layer in the preset convolution neural network model from bottom to top in sequence. And (3) obtaining a bounding box of the target by passing the candidate region of the target through a preset convolutional neural network model, projecting the bounding box to a low-layer convolutional layer in the preset convolutional neural network model, and performing convolution processing to obtain a characteristic diagram of the target.
Namely, the feature map of the target obtained in the previous step is input to the interest region pooling layer for feature compression, so as to obtain a compressed feature map. Specifically, the bounding box is clipped on the feature map of the lower convolution layer and scaled to a new feature map of 7 × 7.
Inputting the compressed feature map into a full-connection layer for processing to obtain the feature map, and calculating the displacement in the xy direction and the scaling of the length and the width between the bounding box and the regressed bounding box by using the CNN. And then, the regression bounding box is obtained according to the bounding box calculated by the CNN, the displacement in the xy direction and the scaling of the length and the width. Specifically, a full-link layer is added on the characteristic diagram, and considering that the position precision of the convolution layer cannot be too low, the embodiment of the invention chooses to cut on conv-1 (the first convolution layer). The output after the full connection layer processing is 4 real numbers, which represent the displacement and the scaling of the length and the width of the bounding box calculated by the regression bounding box and the CNN in the xy direction. The bounding box calculated by CNN is thus corrected and fine-tuned. And finally obtaining the bounding box corresponding to the tracking target.
In this embodiment, after the bounding box of the target obtained through the processing of the preset convolutional neural network model is input to the preset regression layer for regression processing, because the preset regression layer includes the lower convolutional layer of the preset convolutional neural network model, semantic information (target type, etc.) of the higher convolutional layer and position information of the lower convolutional layer can be considered at the same time, so that the target in the input image can be correctly identified and the bounding box of the target can be accurately given. Finally, the regression layer calculates the displacement and the scaling of the length and the width of the bounding box after regression and the bounding box calculated by CNN in the xy direction. Therefore, the bounding box calculated by the CNN is corrected, the regressed bounding box is more accurate, and the whole tracking frame is effectively prevented from generating drift and even losing the target.
In one embodiment, extracting a candidate region corresponding to a tracking target from an image includes: and extracting a candidate region corresponding to the tracking target from the image by adopting a background subtraction method.
In general, in the task of object tracking, a tracking algorithm will usually assume that an initial bounding box of an object to be tracked is already given in the first frame image. Therefore, when a tracking algorithm is actually used, how to obtain the initial bounding box, that is, how to detect the tracking target in the image, is the first to be solved. The embodiment of the invention firstly searches a possible area of the target table tennis. Because the camera is fixed all the time and the whole scene has no too many moving objects, the foreground region can be extracted by adopting the modes of background subtraction and the like, and the search range is narrowed. These regions are used as target candidate regions, and then are input into a preset tracking model for calculation. In particular, it is noted that this system does not rely heavily on background subtraction. It only needs to find the initial position of the target in the first few frames and then can use e.g. the LSTM model for target tracking. Of course, background subtraction can still be used as an aid in the body part of the algorithm to provide candidate regions for tracking.
In this embodiment, the search range can be quickly narrowed down by using the background subtraction method, so that the bounding box coordinates of the target in the next frame can be predicted according to the bounding box coordinates of the target in the previous frames of images by using the preset tracking model or the preset LSTM model.
In one embodiment, as shown in fig. 6, the process of building a camera projection matrix includes:
and step 610, respectively establishing a world coordinate system and a camera coordinate system.
A world coordinate system is established with one corner of the table as the origin. Wherein the x-axis is along the table bottom line, the y-axis is along the table side line, and the z-axis is perpendicular to the table. And establishing a camera coordinate system by taking the camera as an origin. Of course, the camera coordinate system may be established in other ways.
Step 630, obtain the intrinsic parameter matrix and the extrinsic parameter matrix of the camera.
Firstly, a chessboard calibration method is used, chessboard pictures are shot in multiple angles, and an internal parameter matrix M of a camera is obtained by using a built-in calibration function of OpenCV3×3And a distortion coefficient. The internal parameter matrix is used for converting the three-dimensional coordinates of the camera coordinate system into two-dimensional coordinates of a camera plane:
and then, identifying a table tennis table area in the image through color features, and obtaining a boundary line of the table tennis table area by utilizing Hough transform. Coordinates of four corners of the table are obtained through intersection points of boundary lines, and then an external parameter matrix from the camera to the table is calculated, wherein the external parameter matrix comprises a rotation matrix R3×3And a displacement matrix T3x1. The extrinsic parameter matrix is used for the transformation of the camera coordinate system and the world coordinate system:
and 650, establishing a camera projection matrix according to the internal parameter matrix and the external parameter matrix, wherein the camera projection matrix can convert the two-dimensional coordinates of the camera coordinate system into the three-dimensional coordinates of the world coordinate system.
Finally, by combining the above two equations, the two-dimensional coordinates of a certain point on the two camera planes are known, and the three-dimensional coordinates are calculated by using the following equation.
Wherein Z iscThe Z coordinate of the point in a camera coordinate system is unknown. u, v are the coordinates of the point in the camera plane. R is a rotation matrix and T is a displacement matrix. Xw,Yw,ZwIs the three-dimensional coordinate in the world coordinate system to be solved. For a total of four unknowns, two cameras each provide one of the above equations, and thus can be solved directly using linear algebra.
In the present embodiment, the camera projection matrix is calculated in advance so as to directly convert the two-dimensional coordinates of the camera coordinate system to the three-dimensional coordinates of the world coordinate system when performing the table tennis target tracking and trajectory prediction. The system is unified to a world coordinate system for calculation, so that the system is convenient and quick.
In one embodiment, as shown in FIG. 7, there is provided a table tennis target tracking and trajectory prediction apparatus 700, comprising: the tracking target tracking system comprises a camera shooting module 710, a tracking target candidate region weighting module 720, a tracking target bounding box acquisition module 730, a tracking target bounding box three-dimensional coordinate calculation module 740, a coordinate sequence generation module 750 and a tracking target track generation module 760.
The camera shooting module 710 is configured to obtain one frame of image shot by two cameras at the same time on the tracking target.
And the tracking target candidate region weighting module 720 is configured to extract a candidate region corresponding to the tracking target from the image.
And a tracked target bounding box obtaining module 730, configured to input the candidate region into a preset tracking model to be processed, so as to obtain a bounding box corresponding to the tracked target.
And the tracking target bounding box three-dimensional coordinate calculation module 740 is configured to obtain two-dimensional coordinates of the centers of bounding boxes corresponding to the tracking targets respectively photographed by the two cameras at the same time, and calculate three-dimensional coordinates of the centers of the bounding boxes of the tracking targets corresponding to the time according to the camera projection matrix.
And a coordinate sequence generation module 750, configured to obtain three-dimensional coordinates corresponding to bounding boxes at consecutive times, form a consecutive coordinate sequence, input the consecutive coordinate sequence into the recurrent neural network LSTM, and perform calculation to generate a subsequent coordinate sequence.
And a track generation module 760 for tracking the target, configured to obtain a track of the tracking target according to the continuous coordinate sequence and the subsequent coordinate sequence.
In one embodiment, as shown in fig. 8, a table tennis target tracking and trajectory prediction apparatus 700 further comprises: a three-dimensional coordinate prediction module 770 of the tracked target and a candidate region acquisition module 780 of the tracked target.
And a tracking target three-dimensional coordinate prediction module 770, configured to input the calculated three-dimensional coordinates of the bounding box center of the tracking target corresponding to the time into the LSTM for calculation, and predict the three-dimensional coordinates of the tracking target in the next frame of image captured by the two cameras.
And a candidate region of tracking target obtaining module 780, configured to use a region including three-dimensional coordinates as a candidate region of tracking target in the next frame image.
In one embodiment, as shown in fig. 9, the tracked target bounding box acquisition module 730 includes: a convolutional neural network module 731 and a regression layer module 733.
And the convolutional neural network module 731 is configured to input the candidate region into a preset convolutional neural network model, and process the candidate region to obtain a bounding box of the tracking target in the image.
The regression layer module 733, configured to input the bounding box of the tracking target in the image into a preset regression layer, and perform regression processing on the bounding box to obtain a regressed bounding box corresponding to the tracking target, where the preset regression layer includes a low-layer convolution layer of a preset convolutional neural network model.
In one embodiment, the tracking target candidate region weighting module 720 is further configured to: and extracting a candidate region corresponding to the tracking target from the image by adopting a background subtraction method.
In one embodiment, as shown in fig. 10, a table tennis target tracking and trajectory predicting apparatus 700 further comprises a camera projection matrix establishing module 790, the camera projection matrix establishing module 790 is used for establishing a world coordinate system and a camera coordinate system respectively; acquiring an internal parameter matrix and an external parameter matrix of a camera; and establishing a camera projection matrix according to the internal parameter matrix and the external parameter matrix, wherein the camera projection matrix can convert the two-dimensional coordinates of the camera coordinate system into the three-dimensional coordinates of the world coordinate system.
In one embodiment, there is also provided a computer readable storage medium having stored thereon a computer program which when executed by a processor performs the steps of: acquiring a frame of image shot by two cameras for a tracking target at the same moment; extracting a candidate region corresponding to a tracking target from the image; inputting the candidate area into a preset tracking model for processing to obtain a bounding box corresponding to a tracking target; acquiring two-dimensional coordinates of the centers of bounding boxes corresponding to the tracked targets respectively shot by the two cameras at the same moment, and calculating the three-dimensional coordinates of the centers of the bounding boxes of the tracked targets corresponding to the moment according to the projection matrix of the cameras; acquiring three-dimensional coordinates corresponding to bounding boxes at continuous moments to form a continuous coordinate sequence, inputting the continuous coordinate sequence into a recurrent neural network (LSTM) for calculation, and generating a subsequent coordinate sequence; and obtaining the track of the tracking target according to the continuous coordinate sequence and the subsequent coordinate sequence.
In one embodiment, the program further implements the following steps when executed by the processor: inputting the three-dimensional coordinates of the center of the bounding box of the tracking target corresponding to the calculated moment into an LSTM (local Scale TM) for calculation, and predicting the three-dimensional coordinates of the tracking target in the next frame of image shot by the two cameras; and taking the area containing the three-dimensional coordinates as a candidate area of the tracking target in the next frame of image.
In one embodiment, the program further implements the following steps when executed by the processor: inputting the candidate area into a preset convolutional neural network model, and processing to obtain a bounding box of a tracking target in the image; and inputting the bounding box of the tracking target in the image into a preset regression layer to perform regression processing to obtain a regressed bounding box corresponding to the tracking target, wherein the preset regression layer comprises a low-layer convolution layer of a preset convolution neural network model.
In one embodiment, the program further implements the following steps when executed by the processor: and extracting a candidate region corresponding to the tracking target from the image by adopting a background subtraction method.
In one embodiment, the program further implements the following steps when executed by the processor: respectively establishing a world coordinate system and a camera coordinate system; acquiring an internal parameter matrix and an external parameter matrix of a camera; and establishing a camera projection matrix according to the internal parameter matrix and the external parameter matrix, wherein the camera projection matrix can convert the two-dimensional coordinates of the camera coordinate system into the three-dimensional coordinates of the world coordinate system.
In one embodiment, there is also provided a computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the following steps when executing the computer program:
acquiring a frame of image shot by two cameras for a tracking target at the same moment; extracting a candidate region corresponding to a tracking target from the image; inputting the candidate area into a preset tracking model for processing to obtain a bounding box corresponding to a tracking target; acquiring two-dimensional coordinates of the centers of bounding boxes corresponding to the tracked targets respectively shot by the two cameras at the same moment, and calculating the three-dimensional coordinates of the centers of the bounding boxes of the tracked targets corresponding to the moment according to the projection matrix of the cameras; acquiring three-dimensional coordinates corresponding to bounding boxes at continuous moments to form a continuous coordinate sequence, inputting the continuous coordinate sequence into a recurrent neural network (LSTM) for calculation, and generating a subsequent coordinate sequence; and obtaining the track of the tracking target according to the continuous coordinate sequence and the subsequent coordinate sequence.
In one embodiment, the processor further implements the following steps when executing the computer program: inputting the three-dimensional coordinates of the center of the bounding box of the tracking target corresponding to the calculated moment into an LSTM (local Scale TM) for calculation, and predicting the three-dimensional coordinates of the tracking target in the next frame of image shot by the two cameras; and taking the area containing the three-dimensional coordinates as a candidate area of the tracking target in the next frame of image.
In one embodiment, the processor further implements the following steps when executing the computer program: inputting the candidate area into a preset convolutional neural network model, and processing to obtain a bounding box of a tracking target in the image; and inputting the bounding box of the tracking target in the image into a preset regression layer to perform regression processing to obtain a regressed bounding box corresponding to the tracking target, wherein the preset regression layer comprises a low-layer convolution layer of a preset convolution neural network model.
In one embodiment, the processor further implements the following steps when executing the computer program: and extracting a candidate region corresponding to the tracking target from the image by adopting a background subtraction method.
In one embodiment, the processor further implements the following steps when executing the computer program: respectively establishing a world coordinate system and a camera coordinate system; acquiring an internal parameter matrix and an external parameter matrix of a camera; and establishing a camera projection matrix according to the internal parameter matrix and the external parameter matrix, wherein the camera projection matrix can convert the two-dimensional coordinates of the camera coordinate system into the three-dimensional coordinates of the world coordinate system.
It will be understood by those skilled in the art that all or part of the processes in the methods of the embodiments described above may be implemented by hardware related to instructions of a computer program, and the program may be stored in a non-volatile computer readable storage medium, and in the embodiments of the present invention, the program may be stored in a storage medium of a computer system and executed by at least one processor in the computer system, so as to implement the processes including the embodiments of the methods described above. The storage medium may be a magnetic disk, an optical disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), or the like.
The technical features of the above embodiments can be arbitrarily combined, and for the sake of brevity, all possible combinations of the technical features in the above embodiments are not described, but should be considered as the scope of the present specification as long as there is no contradiction between the combinations of the technical features.
The above-mentioned embodiments only express several embodiments of the present invention, and the description thereof is more specific and detailed, but not construed as limiting the scope of the invention. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the inventive concept, which falls within the scope of the present invention. Therefore, the protection scope of the present patent shall be subject to the appended claims.
Claims (8)
1. A table tennis target tracking and trajectory prediction method, the method comprising:
acquiring a frame of image shot by two cameras for a tracking target at the same moment;
extracting a candidate region corresponding to the tracking target from the image;
inputting the candidate region into a preset convolutional neural network model, and processing to obtain a bounding box of a tracking target in the image;
inputting the bounding box of the tracking target in the image into a preset regression layer to perform regression processing, and obtaining a regressed bounding box corresponding to the tracking target, wherein the preset regression layer comprises a low-layer convolution layer of the preset convolution neural network model;
acquiring two-dimensional coordinates of the bounding box center corresponding to the tracking target respectively shot by the two cameras at the same moment, and calculating the three-dimensional coordinates of the bounding box center of the tracking target corresponding to the moment according to the camera projection matrix;
acquiring three-dimensional coordinates corresponding to bounding boxes at continuous moments to form a continuous coordinate sequence, inputting the continuous coordinate sequence into a recurrent neural network (LSTM) for calculation, and generating a subsequent coordinate sequence;
and obtaining the track of the tracking target according to the continuous coordinate sequence and the subsequent coordinate sequence.
2. The method of claim 1, further comprising:
inputting the calculated three-dimensional coordinates of the center of the bounding box of the tracking target corresponding to the moment into an LSTM (local Scale TM) for calculation, and predicting the three-dimensional coordinates of the tracking target in the next frame of image shot by the two cameras;
and taking the area containing the three-dimensional coordinates as a candidate area of a tracking target in the next frame of image.
3. The method according to claim 1, wherein the extracting the candidate region corresponding to the tracking target from the image comprises:
and extracting a candidate region corresponding to the tracking target from the image by adopting a background subtraction method.
4. The method of claim 1, wherein the process of building the camera projection matrix comprises:
respectively establishing a world coordinate system and a camera coordinate system;
acquiring an internal parameter matrix and an external parameter matrix of a camera;
and establishing a camera projection matrix according to the internal parameter matrix and the external parameter matrix, wherein the camera projection matrix can convert the two-dimensional coordinates of the camera coordinate system into the three-dimensional coordinates of the world coordinate system.
5. A table tennis target tracking and trajectory prediction apparatus, the apparatus comprising:
the camera shooting module is used for acquiring a frame of image shot by the two cameras on the tracking target at the same moment;
the tracking target candidate region weighting module is used for extracting a candidate region corresponding to the tracking target from the image;
the tracking target bounding box acquisition module is used for inputting the candidate region into a preset convolutional neural network model and processing the candidate region to obtain a bounding box of a tracking target in the image; inputting the bounding box of the tracking target in the image into a preset regression layer to perform regression processing, and obtaining a regressed bounding box corresponding to the tracking target, wherein the preset regression layer comprises a low-layer convolution layer of the preset convolution neural network model;
the tracking target bounding box three-dimensional coordinate calculation module is used for acquiring two-dimensional coordinates of the bounding box center corresponding to the tracking target respectively shot by the two cameras at the same moment, and then calculating the three-dimensional coordinates of the bounding box center of the tracking target corresponding to the moment according to the camera projection matrix;
the coordinate sequence generation module is used for acquiring three-dimensional coordinates corresponding to bounding boxes at continuous moments to form a continuous coordinate sequence, inputting the continuous coordinate sequence into a recurrent neural network (LSTM) for calculation, and generating a subsequent coordinate sequence;
and the track generation module of the tracking target is used for obtaining the track of the tracking target according to the continuous coordinate sequence and the subsequent coordinate sequence.
6. The apparatus of claim 5, further comprising:
the three-dimensional coordinate prediction module of the tracking target is used for inputting the calculated three-dimensional coordinate of the center of the bounding box of the tracking target corresponding to the moment into the LSTM for calculation and predicting the three-dimensional coordinate of the tracking target in the next frame of image shot by the two cameras;
and the candidate region acquisition module of the tracking target is used for taking the region containing the three-dimensional coordinates as a candidate region of the tracking target in the next frame of image.
7. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out a table tennis target tracking and trajectory prediction method according to any one of claims 1 to 4.
8. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor when executing the computer program implements a table tennis target tracking and trajectory prediction method according to any one of claims 1 to 4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710682442.2A CN107481270B (en) | 2017-08-10 | 2017-08-10 | Table tennis target tracking and trajectory prediction method, device, storage medium and computer equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710682442.2A CN107481270B (en) | 2017-08-10 | 2017-08-10 | Table tennis target tracking and trajectory prediction method, device, storage medium and computer equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN107481270A CN107481270A (en) | 2017-12-15 |
| CN107481270B true CN107481270B (en) | 2020-05-19 |
Family
ID=60600283
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710682442.2A Active CN107481270B (en) | 2017-08-10 | 2017-08-10 | Table tennis target tracking and trajectory prediction method, device, storage medium and computer equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107481270B (en) |
Families Citing this family (34)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108062763B (en) * | 2017-12-29 | 2020-10-16 | 纳恩博（北京）科技有限公司 | Target tracking method and device and storage medium |
| CN108053653B (en) * | 2018-01-11 | 2021-03-30 | 广东蔚海数问大数据科技有限公司 | Vehicle behavior prediction method and device based on LSTM |
| CN110362098B (en) * | 2018-03-26 | 2022-07-05 | 北京京东尚科信息技术有限公司 | Unmanned aerial vehicle visual servo control method and device and unmanned aerial vehicle |
| CN108465224A (en) * | 2018-04-07 | 2018-08-31 | 华北理工大学 | Table tennis track analysis system |
| CN108540817B (en) * | 2018-05-08 | 2021-04-20 | 成都市喜爱科技有限公司 | Video data processing method, device, server and computer readable storage medium |
| CN108986141A (en) * | 2018-07-03 | 2018-12-11 | 百度在线网络技术（北京）有限公司 | Object of which movement information processing method, device, augmented reality equipment and storage medium |
| CN108876821B (en) * | 2018-07-05 | 2019-06-07 | 北京云视万维科技有限公司 | Across camera lens multi-object tracking method and system |
| CN109241856A (en) * | 2018-08-13 | 2019-01-18 | 浙江零跑科技有限公司 | A kind of vehicle-mounted vision system solid object detection method of monocular |
| CN110956644B (en) * | 2018-09-27 | 2023-10-10 | 杭州海康威视数字技术股份有限公司 | Motion trail determination method and system |
| CN111028287B (en) * | 2018-10-09 | 2023-10-20 | 杭州海康威视数字技术股份有限公司 | Method and device for determining a transformation matrix of radar coordinates and camera coordinates |
| CN109559332B (en) * | 2018-10-31 | 2021-06-18 | 浙江工业大学 | Sight tracking method combining bidirectional LSTM and Itracker |
| CN109711274A (en) * | 2018-12-05 | 2019-05-03 | 斑马网络技术有限公司 | Vehicle checking method, device, equipment and storage medium |
| CN111291585B (en) * | 2018-12-06 | 2023-12-08 | 杭州海康威视数字技术股份有限公司 | GPS-based target tracking system, method and device and ball machine |
| CN110287764B (en) * | 2019-05-06 | 2022-01-11 | 深圳大学 | Gesture prediction method, gesture prediction device, computer equipment and storage medium |
| CN110111358B (en) * | 2019-05-14 | 2022-05-24 | 西南交通大学 | Target tracking method based on multilayer time sequence filtering |
| CN110340901B (en) * | 2019-06-28 | 2022-09-27 | 深圳盈天下视觉科技有限公司 | Control method, control device and terminal equipment |
| CN110458281B (en) * | 2019-08-02 | 2021-09-03 | 中科新松有限公司 | Method and system for predicting deep reinforcement learning rotation speed of table tennis robot |
| CN110517292A (en) | 2019-08-29 | 2019-11-29 | 京东方科技集团股份有限公司 | Method for tracking target, device, system and computer readable storage medium |
| CN110827320B (en) * | 2019-09-17 | 2022-05-20 | 北京邮电大学 | Target tracking method and device based on time sequence prediction |
| CN110796093A (en) * | 2019-10-30 | 2020-02-14 | 上海眼控科技股份有限公司 | Target tracking method and device, computer equipment and storage medium |
| CN111369629B (en) * | 2019-12-27 | 2024-05-24 | 浙江万里学院 | Ball return track prediction method based on binocular vision perception of swing and batting actions |
| CN113128290A (en) * | 2019-12-31 | 2021-07-16 | 炬星科技（深圳）有限公司 | Moving object tracking method, system, device and computer readable storage medium |
| CN111546332A (en) * | 2020-04-23 | 2020-08-18 | 上海电机学院 | Table tennis robot system based on embedded equipment and application |
| CN111939541A (en) * | 2020-06-23 | 2020-11-17 | 北京瑞盖科技股份有限公司 | Evaluation method, device, equipment and system for table tennis training |
| CN113160275B (en) * | 2021-04-21 | 2022-11-08 | 河南大学 | Automatic target tracking and track calculating method based on multiple videos |
| CN113253755A (en) * | 2021-05-08 | 2021-08-13 | 广东白云学院 | Neural network-based rotor unmanned aerial vehicle tracking algorithm |
| CN114373021A (en) * | 2021-12-06 | 2022-04-19 | 晋城鸿智纳米光机电研究院有限公司 | Table tennis identification method and device |
| CN114419098A (en) * | 2022-01-18 | 2022-04-29 | 长沙慧联智能科技有限公司 | Moving target trajectory prediction method and device based on visual transformation |
| CN114589719B (en) * | 2022-04-02 | 2024-03-08 | 中国电子科技集团公司第五十八研究所 | Real-time calibration and calibration system and method for table tennis service robot |
| CN114612522B (en) * | 2022-05-09 | 2023-01-17 | 广东金融学院 | Table tennis sport parameter detection method and device and table tennis training auxiliary system |
| CN115278194B (en) * | 2022-09-22 | 2022-12-23 | 山东省青东智能科技有限公司 | Image data processing method based on 3D industrial camera |
| CN115965658A (en) * | 2023-03-16 | 2023-04-14 | 江西工业贸易职业技术学院 | Ball motion trajectory prediction method and system, electronic device and storage medium |
| CN116504068A (en) * | 2023-06-26 | 2023-07-28 | 创辉达设计股份有限公司江苏分公司 | Statistical method, device, computer equipment and storage medium for lane-level traffic flow |
| CN117237409B (en) * | 2023-09-06 | 2024-06-14 | 广州飞漫思维数码科技有限公司 | Shooting game sight correction method and system based on Internet of things |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101256673A (en) * | 2008-03-18 | 2008-09-03 | 中国计量学院 | Method for tracing arm motion in real time video tracking system |
| CN101282461A (en) * | 2007-04-02 | 2008-10-08 | 财团法人工业技术研究院 | Image processing methods |
| CN101458434A (en) * | 2009-01-08 | 2009-06-17 | 浙江大学 | System for precision measuring and predicting table tennis track and system operation method |
| CN107690840B (en) * | 2009-06-24 | 2013-07-31 | 中国科学院自动化研究所 | Unmanned plane vision auxiliary navigation method and system |
| CN106022527A (en) * | 2016-05-27 | 2016-10-12 | 河南明晰信息科技有限公司 | Trajectory prediction method and device based on map tiling and LSTM cyclic neural network |
| CN106485226A (en) * | 2016-10-14 | 2017-03-08 | 杭州派尼澳电子科技有限公司 | A kind of video pedestrian detection method based on neutral net |
| CN106780620A (en) * | 2016-11-28 | 2017-05-31 | 长安大学 | A kind of table tennis track identification positioning and tracking system and method |
-
2017
- 2017-08-10 CN CN201710682442.2A patent/CN107481270B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101282461A (en) * | 2007-04-02 | 2008-10-08 | 财团法人工业技术研究院 | Image processing methods |
| CN101256673A (en) * | 2008-03-18 | 2008-09-03 | 中国计量学院 | Method for tracing arm motion in real time video tracking system |
| CN101458434A (en) * | 2009-01-08 | 2009-06-17 | 浙江大学 | System for precision measuring and predicting table tennis track and system operation method |
| CN107690840B (en) * | 2009-06-24 | 2013-07-31 | 中国科学院自动化研究所 | Unmanned plane vision auxiliary navigation method and system |
| CN106022527A (en) * | 2016-05-27 | 2016-10-12 | 河南明晰信息科技有限公司 | Trajectory prediction method and device based on map tiling and LSTM cyclic neural network |
| CN106485226A (en) * | 2016-10-14 | 2017-03-08 | 杭州派尼澳电子科技有限公司 | A kind of video pedestrian detection method based on neutral net |
| CN106780620A (en) * | 2016-11-28 | 2017-05-31 | 长安大学 | A kind of table tennis track identification positioning and tracking system and method |
Also Published As
| Publication number | Publication date |
|---|---|
| CN107481270A (en) | 2017-12-15 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107481270B (en) | Table tennis target tracking and trajectory prediction method, device, storage medium and computer equipment | |
| CN106780620B (en) | Table tennis motion trail identification, positioning and tracking system and method | |
| CN107274433B (en) | Target tracking method and device based on deep learning and storage medium | |
| CN109102522B (en) | Target tracking method and device | |
| CN110176032B (en) | Three-dimensional reconstruction method and device | |
| CN109410316B (en) | Method for three-dimensional reconstruction of object, tracking method, related device and storage medium | |
| CN108182695B (en) | Target tracking model training method and device, electronic equipment and storage medium | |
| CN110827321B (en) | Multi-camera collaborative active target tracking method based on three-dimensional information | |
| CN111199556A (en) | Indoor pedestrian detection and tracking method based on camera | |
| CN109902675B (en) | Object pose acquisition method and scene reconstruction method and device | |
| CN108519102A (en) | A kind of binocular vision speedometer calculation method based on reprojection | |
| CN110516639B (en) | Real-time figure three-dimensional position calculation method based on video stream natural scene | |
| CN111899345B (en) | Three-dimensional reconstruction method based on 2D visual image | |
| Chen et al. | Real-time object tracking via CamShift-based robust framework | |
| CN117036404A (en) | Monocular thermal imaging simultaneous positioning and mapping method and system | |
| Sokolova et al. | Human identification by gait from event-based camera | |
| CN116740126A (en) | Target tracking method, high-speed camera, and storage medium | |
| Tian et al. | High-speed tiny tennis ball detection based on deep convolutional neural networks | |
| Zhu et al. | PairCon-SLAM: Distributed, online, and real-time RGBD-SLAM in large scenarios | |
| CN116105721B (en) | Loop optimization method, device and equipment for map construction and storage medium | |
| Chen et al. | Stingray detection of aerial images with region-based convolution neural network | |
| Shere et al. | Temporally consistent 3D human pose estimation using dual 360deg cameras | |
| CN110910489A (en) | Monocular vision based intelligent court sports information acquisition system and method | |
| CN115953471A (en) | Indoor scene multi-scale vector image retrieval and positioning method, system and medium | |
| Wang et al. | Research and implementation of the sports analysis system based on 3D image technology |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |