CN114898304A - Vehicle tracking method and device, road side equipment and network side equipment - Google Patents
Vehicle tracking method and device, road side equipment and network side equipment Download PDFInfo
- Publication number
- CN114898304A CN114898304A CN202111624101.2A CN202111624101A CN114898304A CN 114898304 A CN114898304 A CN 114898304A CN 202111624101 A CN202111624101 A CN 202111624101A CN 114898304 A CN114898304 A CN 114898304A
- Authority
- CN
- China
- Prior art keywords
- vehicle
- license plate
- target
- image data
- structural body
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 73
- 238000012549 training Methods 0.000 claims abstract description 93
- 239000013598 vector Substances 0.000 claims description 107
- 230000006870 function Effects 0.000 claims description 41
- 238000004422 calculation algorithm Methods 0.000 claims description 14
- 238000004590 computer program Methods 0.000 claims description 13
- 238000001914 filtration Methods 0.000 claims description 7
- 238000004364 calculation method Methods 0.000 claims description 5
- 230000007246 mechanism Effects 0.000 claims description 4
- 238000004458 analytical method Methods 0.000 claims description 3
- 238000003709 image segmentation Methods 0.000 claims description 3
- 238000001514 detection method Methods 0.000 abstract description 27
- 230000006855 networking Effects 0.000 abstract description 2
- 230000008569 process Effects 0.000 description 16
- 238000010586 diagram Methods 0.000 description 10
- 238000000605 extraction Methods 0.000 description 10
- 230000011218 segmentation Effects 0.000 description 10
- 238000005070 sampling Methods 0.000 description 5
- 230000000694 effects Effects 0.000 description 3
- 230000004044 response Effects 0.000 description 3
- 230000004913 activation Effects 0.000 description 2
- 238000000354 decomposition reaction Methods 0.000 description 2
- 238000003702 image correction Methods 0.000 description 2
- 239000011159 matrix material Substances 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- 230000006798 recombination Effects 0.000 description 2
- 238000005215 recombination Methods 0.000 description 2
- PXFBZOLANLWPMH-UHFFFAOYSA-N 16-Epiaffinine Natural products C1C(C2=CC=CC=C2N2)=C2C(=O)CC2C(=CC)CN(C)C1C2CO PXFBZOLANLWPMH-UHFFFAOYSA-N 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 230000004927 fusion Effects 0.000 description 1
- 230000010365 information processing Effects 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 238000011176 pooling Methods 0.000 description 1
- 238000007781 pre-processing Methods 0.000 description 1
- 238000004088 simulation Methods 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/044—Recurrent networks, e.g. Hopfield networks
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Computing Systems (AREA)
- Software Systems (AREA)
- Molecular Biology (AREA)
- Computational Linguistics (AREA)
- Biophysics (AREA)
- Biomedical Technology (AREA)
- Mathematical Physics (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Biology (AREA)
- Image Analysis (AREA)
- Traffic Control Systems (AREA)
Abstract
The invention provides vehicle tracking, and belongs to the technical field of vehicle networking. The method comprises the steps of obtaining image data of a target vehicle; acquiring first characteristics of the image data in different proportions, and inputting the first characteristics into a first training model to obtain a first structure body; calculating a vehicle mask of the target vehicle according to the first structural body, and assigning the vehicle mask to the first structural body to obtain a second structural body; and sending the image data and the second structural body to a network side device. According to the vehicle tracking method provided by the embodiment of the invention, the license plate, the vehicle window and the wheels are simultaneously set as the identification features, so that the influence of incapability of acquiring image data corresponding to the license plate due to the erection angle and height of the camera on vehicle tracking is reduced, and when the vehicle detection and tracking are carried out by utilizing various vehicle identification features, the success probability of feature identification or matching and the integrity of the tracking track of the target vehicle are improved.
Description
Technical Field
The invention belongs to the technical field of vehicle networking, and particularly relates to a vehicle tracking method and device, road side equipment and network side equipment.
Background
In the prior art, the cross-scene tracking method has vehicle track tracking of a single camera and vehicle track tracking of a multi-camera full scene, but both methods use a single feature as a unique judgment basis, and when the feature cannot be extracted or the noise of the extracted feature is too large due to the influence of external factors, misjudgment can be directly caused, so that the tracking effect is influenced. For example, when only the license plate information is selected as the criterion, when the distance between the vehicle and the camera is too far or the license plate cannot be clearly imaged due to an angle problem, the accuracy of license plate recognition is affected, and the robustness of the tracking system is reduced. The vehicle weight recognition method is also influenced by factors such as the angle and the height of the camera, and vehicle weight recognition features extracted from different angles are difficult to cluster. Therefore, when the vehicle detection and tracking is performed by using a single vehicle feature, there may be a case where the feature extraction or matching fails, resulting in a failure in vehicle tracking and incomplete track restoration.
Disclosure of Invention
The embodiment of the invention provides a vehicle tracking method, a vehicle tracking device, road side equipment and network side equipment, and solves the problems that in the prior art, when single vehicle features are used for vehicle detection and tracking, feature extraction or matching fails, so that vehicle tracking fails and track restoration is incomplete.
In a first aspect, an embodiment of the present invention provides a vehicle tracking method applied to a roadside apparatus, including:
acquiring image data of a target vehicle;
acquiring first characteristics of the image data in different proportions, and inputting the first characteristics into a first training model to obtain a first structure body;
calculating a vehicle mask of the target vehicle according to the first structural body, and assigning the vehicle mask to the first structural body to obtain a second structural body;
and sending the image data and the second structural body to a network side device.
Further, the categories of the vehicle mask include a license plate, a window, and a wheel.
Further, the acquiring first features of the image data at different scales includes:
reducing the image data in different proportions to obtain a first high-level feature and a first low-level feature in different proportions;
and fusing the first high-level feature and the first low-level feature to obtain the first feature.
Further, the first structure includes a first image corresponding to the target vehicle, size information of the first image, and position information of the first image in an image corresponding to the image data.
Further, the first training model is model-trained by using a Generalized local and GIoU Loss function.
Further, the calculating a vehicle mask of the target vehicle according to the image data includes:
and zooming the first image to a first preset size, and performing image segmentation through a second training model to obtain the vehicle mask.
Further, the second training model adopts a Focal local function to perform model training.
In a second aspect, an embodiment of the present invention provides a vehicle tracking method, applied to a network-side device, including:
receiving image data of a target vehicle and a second structure;
analyzing the second structural body to obtain license plate information and a feature vector of the target vehicle, wherein the feature vector is used for representing vehicle body features of the target vehicle corresponding to the vehicle mask;
matching the license plate information and the feature vector with a target queue, and updating the target queue; the target queue is the license plate information and the feature vector which are stored before the matching.
Further, the vehicle body characteristics corresponding to the vehicle mask include: license plate, door window and wheel.
Further, analyzing the second structural body to obtain license plate information of the target vehicle includes:
analyzing the second structural body to obtain the vehicle mask in the second structural body;
acquiring a first image area corresponding to the license plate area according to the vehicle mask;
and inputting the first image area which is scaled to the second preset size into a third training model to obtain the license plate information of the target vehicle.
Further, the third training model performs model training by using a cross entropy function and a regularization loss function.
Further, analyzing the second structure to obtain a feature vector of the target vehicle includes:
analyzing the second structural body to obtain the vehicle mask in the second structural body;
acquiring a vehicle body image area corresponding to the vehicle body characteristic of the target vehicle according to the vehicle mask;
inputting the vehicle body image area zoomed to a third preset size into a fourth training model to obtain a vehicle body characteristic vector of the target vehicle;
and inputting the vehicle body feature vector into a fifth training model to obtain the feature vector.
Further, the vehicle body image area includes: at least two of a first image area corresponding to the license plate area, a second image area corresponding to the vehicle window area and a third image area corresponding to the wheel area;
the vehicle body feature vector includes: at least two of a first feature vector corresponding to the license plate, a second feature vector corresponding to the vehicle window and a third feature vector corresponding to the wheel.
Further, the fifth training model is a training model including a self-attention mechanism.
Further, the matching the license plate information and the feature vector with a target queue includes:
and matching the license plate information and the feature vectors with a target queue by adopting a Hungarian algorithm and a related filtering algorithm.
In a third aspect, an embodiment of the present invention provides a vehicle tracking device applied to roadside equipment, including:
the first acquisition module is used for acquiring image data of a target vehicle;
the second acquisition module is used for acquiring first characteristics of the image data in different proportions and inputting the first characteristics into a first training model to obtain a first structural body;
the calculation module is used for calculating a vehicle mask of the target vehicle according to the first structural body, and assigning the vehicle mask to the first structural body to obtain a second structural body;
and the sending module is used for sending the image data and the second structural body to network side equipment.
In a fourth aspect, an embodiment of the present invention provides a vehicle tracking apparatus, applied to a network-side device, including:
a receiving module for receiving image data of a target vehicle and a second structural body;
the analysis module is used for analyzing the second structural body to obtain license plate information and a feature vector of the target vehicle, and the feature vector is used for representing the vehicle body feature of the target vehicle corresponding to the vehicle mask;
the matching updating module is used for matching the license plate information and the characteristic vector with a target queue and updating the target queue; the target queue is the license plate information and the feature vector which are stored before the matching.
In a fifth aspect, an embodiment of the present invention provides a roadside apparatus including: a transceiver, a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the steps of the vehicle tracking method as described above when executing the computer program.
In a sixth aspect, an embodiment of the present invention provides a network-side device, including: a transceiver, a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the steps of the vehicle tracking method as described above when executing the computer program.
In a seventh aspect, embodiments of the present invention provide a computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements the steps of the vehicle tracking method as described above.
The technical scheme of the invention has the beneficial effects that:
according to the vehicle tracking method, the first characteristics of the image data under different proportions can be obtained by obtaining the image data of the target vehicle; obtaining a first structure body by inputting the first characteristic into a first training model; calculating a vehicle mask of the target vehicle according to the first structure body; obtaining a second structure by assigning the vehicle mask to the first structure; and finally, the second structural body is sent to network side equipment through the image data, and data support is provided for vehicle tracking image recognition and vehicle feature extraction.
Drawings
FIG. 1 is a flow chart illustrating a vehicle tracking method according to an embodiment of the present invention;
FIG. 2 is a schematic diagram of a network architecture of a vehicle detection module according to an embodiment of the invention;
FIG. 3 is a schematic diagram of a method for stitching deep and shallow features according to an embodiment of the present invention;
FIG. 4 is another schematic flow chart diagram of a vehicle tracking method according to an embodiment of the invention;
FIG. 5 is a schematic diagram illustrating a license plate recognition process according to an embodiment of the invention;
FIG. 6 is a schematic representation of a vehicle weight identification process according to an embodiment of the present invention;
FIG. 7 is a flowchart illustrating matching of license plate information and feature vectors to a target queue according to an embodiment of the present invention;
FIG. 8 is a schematic diagram of a vehicle tracking device according to an embodiment of the present invention;
FIG. 9 is a schematic view of another embodiment of a vehicle tracking device according to the present invention;
FIG. 10 is a schematic structural diagram of a roadside apparatus according to an embodiment of the present invention;
fig. 11 is a schematic structural diagram of a network-side device according to an embodiment of the present invention.
Detailed Description
In order to make the technical problems, technical solutions and advantages of the present invention more apparent, the following detailed description is given with reference to the accompanying drawings and specific embodiments. In the following description, specific details such as specific configurations and components are provided only to help the full understanding of the embodiments of the present invention. Thus, it will be apparent to those skilled in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the invention. In addition, descriptions of well-known functions and constructions are omitted for clarity and conciseness.
It should be appreciated that reference throughout this specification to "one embodiment" or "an embodiment" means that a particular feature, structure or characteristic described in connection with the embodiment is included in at least one embodiment of the present invention. Thus, the appearances of the phrases "in one embodiment" or "in an embodiment" in various places throughout this specification are not necessarily all referring to the same embodiment. Furthermore, the particular features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.
The term "and/or" in the embodiments of the present invention describes an association relationship of associated objects, and indicates that three relationships may exist, for example, a and/or B may indicate: a exists alone, A and B exist simultaneously, and B exists alone. The character "/" generally indicates that the former and latter associated objects are in an "or" relationship.
In various embodiments of the present invention, it should be understood that the sequence numbers of the following processes do not mean the execution sequence, and the execution sequence of each process should be determined by its function and inherent logic, and should not constitute any limitation to the implementation process of the embodiments of the present invention.
In addition, the terms "system" and "network" are often used interchangeably herein.
In the embodiments provided herein, it should be understood that "B corresponding to a" means that B is associated with a from which B can be determined. It should also be understood that determining B from a does not mean determining B from a alone, but may be determined from a and/or other information.
In making the description of the embodiments of the present invention, some concepts used in the following description will first be explained.
First embodiment
As shown in fig. 1, an embodiment of the present invention provides a vehicle tracking method applied to a roadside apparatus, including:
102, acquiring first characteristics of the image data in different proportions, and inputting the first characteristics into a first training model to obtain a first structure body;
103, calculating a vehicle mask of the target vehicle according to the first structural body, and assigning the vehicle mask to the first structural body to obtain a second structural body;
and 104, sending the image data and the second structural body to a network side device.
In an embodiment of the present invention, the roadside apparatus includes: the device comprises a camera, a vehicle detection module and a vehicle body segmentation module.
The vehicle detection module detects a vehicle in the image data by using a deep learning network; in consideration of the influence of power consumption and computational power, in an embodiment of the invention, the vehicle detection module adopts a lightweight first network structure. As shown in fig. 2, the first network structure includes: backbone networks, Feature Pyramid (FPN) layers, and detection heads.
And acquiring image data of a picture shot by a camera in real time through the vehicle detection module, and detecting a target vehicle in the image data. Zooming the image data in different proportions, acquiring first characteristics of the image data in different proportions, and inputting the first characteristics into a first training model for calculation to obtain the first structure body; wherein the first feature comprises image data corresponding to a target vehicle.
According to the vehicle tracking method, the first characteristics of the image data under different proportions can be obtained by obtaining the image data of the target vehicle; obtaining a first structure body by inputting the first characteristic into a first training model; calculating a vehicle mask of the target vehicle according to the first structure body; obtaining a second structure by assigning the vehicle mask to the first structure; and finally, the second structural body is sent to network side equipment through the image data, and data support is provided for vehicle tracking image recognition and vehicle feature extraction.
Optionally, the categories of the vehicle mask include license plate, window, and wheel.
According to the vehicle tracking method provided by the embodiment of the invention, the license plate, the vehicle window and the wheels are simultaneously set as the identification features, so that the influence of incapability of acquiring image data corresponding to the license plate due to the erection angle and height of the camera on vehicle tracking is reduced, and when the vehicle detection and tracking are carried out by utilizing various vehicle identification features, the success probability of feature identification or matching and the integrity of the tracking track of the target vehicle are improved.
Optionally, the acquiring first features of the image data at different scales includes:
reducing the image data in different proportions to obtain a first high-level feature and a first low-level feature in different proportions;
and fusing the first high-level feature and the first low-level feature to obtain the first feature.
And downsampling the image data through the backbone network to obtain high-level features of the image data in different proportions.
And fusing the high-level features and the low-level features in different proportions through the feature pyramid layer. According to the scheme, the high resolution of the low-layer features and the high semantic information of the high-layer features are utilized, and the target detection calling rate is improved.
Optionally, the first structure includes a first image corresponding to the target vehicle, size information of the first image, and position information of the first image in an image corresponding to the image data.
The detection head is designed for a target detection object, and in the embodiment of the invention, the target detection object is a vehicle.
In an embodiment of the present invention, the network structure of the detection head may include two branch structures, and one branch structure outputs coordinates of a center point of a target detection object and distances from the center point to two opposite corners of the target detection object as a target detection frame, for example, a vehicle detection frame is determined according to a position of a first center point of a target vehicle in an image corresponding to the image data and distances from the two opposite corners of the target vehicle in the image data to the first center point; another branch structure outputs target categories such as vehicles, pedestrians, trees, buildings, etc., and the target categories of the embodiment of the present invention relate only to vehicles.
Optionally, the first training model is model-trained using Generalized local and GIoU Loss functions.
In an embodiment of the invention, in the process of establishing the first training model, a one-stage detector is adopted, and meanwhile, in order to solve the problems that positive and negative samples are unbalanced, and the samples are difficult to converge and the like in the model training process, a Generalized local Loss (GFL) function and a GIoU Loss function are adopted as Loss functions.
The Generalized Focal local function is as follows:
wherein L is Q ＝QFL(σ)＝-|y-σ| β ((1-y)log(1-σ)+ylog(σ))
L D ＝DFL(S i ,S i+1 )＝-((y i+1 -y)log(S i )+(y-y i )log(S i+1 ))
N pos Is the number of positive sample data, L B Is the GIoU Loss function, lambda 0 Is L B The weighting parameter of the sum can take the value of 2; lambda [ alpha ] 1 Is L D May take the value of 1/4; cz is an index function, and is used for summing each position point z in the characteristic diagram in the pyramid layer, if the sum is greater than 0, 1 is taken, and if not, 0 is taken; sigma is a prediction result, y is a truth label, and beta is a super parameter; y is a true point, y i And y (i+1) Is a true value point, S, adjacent to the left and right of y i The result is after Softmax.
The GIoU Loss function is as follows:
wherein, A and B respectively represent the areas of two detection frames; c represents the minimum circumscribed rectangular area of the two detection boxes A and B.
It should be noted that, the first training model may use real images acquired by the urban public transportation camera and the highway camera and image data generated by the 3D emulation game engine as sample data of model training.
And inputting the first characteristic into a first training model, and outputting a vehicle detection frame. If the output result exists, the vehicle detection module packages the output result into a first structural body and sends the first structural body to the vehicle body segmentation module and the network side equipment.
According to the vehicle tracking method, the Generalized local Loss function and the GIoU Loss function are used as Loss functions, the problems that positive and negative samples are unbalanced, the samples are difficult to converge and the like in the model training process are solved, and the accuracy of the first training model is improved.
Optionally, the calculating a vehicle mask of the target vehicle according to the image data includes:
and zooming the first image to a first preset size, and performing image segmentation through a second training model to obtain the vehicle mask.
Optionally, the roadside apparatus further includes a body segmentation module.
And after receiving the image data and the first structure sent by the vehicle detection module, the vehicle body segmentation module inputs the image information of the position information and the size information of the target vehicle in the first structure in the image data into a second training model for semantic segmentation, and calculates the vehicle mask. Assigning the vehicle mask to the first structure to obtain a second structure, and sending the second structure to a network side device through a Remote Procedure Call Protocol (RPC).
In an embodiment of the invention, the vehicle body segmentation module also adopts a lightweight network structure. The network of the vehicle body segmentation module comprises an encoder and a decoder, wherein the encoder is responsible for down-sampling the image data, and a convolution layer and a Max Pooling layer are adopted to down-sample to reduce the resolution and improve the channel dimensionality. The decoder is responsible for up-sampling the image data, and up-sampling is carried out by adopting a convolution layer and a deconvolution layer to improve the resolution and reduce the channel dimensionality. And obtaining deep features and shallow features of the image data by down-sampling and up-sampling along with the image data, splicing the deep features and the shallow features in a jump connection mode as shown in fig. 3 to enrich the feature representation capability of a decoder, and finally outputting a vehicle mask by using a convolutional layer. In an embodiment of the present invention, the convolutional layer is a convolutional layer with a convolutional kernel of 1 × 1, and the number of channels is 5.
In the scheme of the embodiment of the invention, the semantic segmentation is carried out on the first structure body through the second training model to obtain the vehicle mask of the target vehicle, the vehicle mask is assigned to the first structure body to obtain the second structure body, and the vehicle mask is divided into three types, namely a license plate, a vehicle window and a wheel, so that the second structure body comprises the vehicle masks of various types, the success rate of vehicle body feature recognition of the network side equipment is improved, and multi-feature data support is provided for the network side equipment to track the vehicle.
Optionally, the second training model is model-trained by using a Focal local function.
In the process of establishing the second training model, the semantic segmentation task has the problem of serious sample imbalance, namely when the number of samples of an individual category and the proportion in an image are far larger than those of other categories, the other categories are difficult to train, and the accuracy of the established training model is insufficient.
In an embodiment of the present invention, the second training model performs model training by using a Focal local as a Loss function, where the Focal local function is:
wherein y' is the output after the activation function, y is the truth label, and alpha and gamma are the super parameters; α is used as a balance factor, typically 0.25; gamma is a weight parameter, typically 2.
The balance factor alpha is used for balancing the problem of the proportion imbalance of the positive sample and the negative sample, and the weight parameter gamma can enable the loss function to be focused on the difficult sample and is used for adjusting the problem of non-convergence. When the weight parameter gamma is 0, the Focal local function is converted into a cross entropy Loss function.
It should be noted that, the second training model may use real images acquired by the urban public transportation camera and the highway camera and image data generated by the 3D emulation game engine as sample data of model training.
According to the scheme, the second training model is trained by taking the Focal local function as the Loss function, so that the problem of low accuracy of the second training model caused by sample unbalance is solved, and the accuracy of the second training model is improved.
As shown in fig. 4, an embodiment of the present invention provides a vehicle tracking method applied to a network-side device, including:
Optionally, the network-side device includes a vehicle tracking module and a license plate recognition module.
In an embodiment of the present invention, after the vehicle tracking module receives the image data and the second structural body sent by the road side device, the vehicle tracking module schedules the idle license plate recognition module to perform information processing, and inputs the second structural body into the license plate recognition module, so as to obtain license plate information of the target vehicle, which includes license plate text information.
According to the vehicle tracking method provided by the embodiment of the invention, the license plate information of the target vehicle and the feature vector used for representing the body feature of the target vehicle are obtained by analyzing the received second structural body sent by the road side equipment, and the tracking data of the vehicle is updated in a mode of updating the target queue by matching the license plate information and the feature vector with the target queue, so that the integrity of the tracking track of the target vehicle is improved.
Optionally, the vehicle body feature corresponding to the vehicle mask includes: license plate, door window and wheel.
According to the vehicle tracking method provided by the embodiment of the invention, the license plate, the vehicle window and the wheels are simultaneously set as the identification features, so that the influence of incapability of acquiring image data corresponding to the license plate due to the erection angle and height of the camera on vehicle tracking is reduced, and when the vehicle detection and tracking are carried out by utilizing various vehicle identification features, the success probability of failure in feature identification or matching and the integrity of the tracking track of the target vehicle are improved.
Optionally, analyzing the second structural body to obtain license plate information of the target vehicle, including:
analyzing the second structural body to obtain the vehicle mask in the second structural body;
acquiring a first image area corresponding to the license plate area according to the vehicle mask;
and inputting the first image area which is scaled to the second preset size into a third training model to obtain the license plate information of the target vehicle.
The license plate recognition module analyzes the vehicle mask in the second structural body and judges whether first image information corresponding to the license plate region exists in the vehicle mask. And when the image information corresponding to the license plate region in the vehicle mask is determined, acquiring first image information corresponding to the license plate region, scaling the first image region corresponding to the license plate region to a second preset size, inputting a third training model, and finally outputting license plate information comprising license plate character information.
Optionally, the determining whether the image information corresponding to the license plate region exists in the vehicle mask includes:
and acquiring a first image area corresponding to the license plate area according to the vehicle mask, and determining that the vehicle mask comprises first image information corresponding to the license plate area when the pixels of the first image area are larger than a preset value. In an embodiment of the invention, it can be considered that the road side device captures the license plate of the target vehicle when the length and the width of the pixel of the license plate region are both greater than 32.
The network structure of the license plate recognition module comprises a backbone network, a characteristic sequence layer, an LSTM layer, an Attention layer and a Softmax layer. And downsampling the second structural body through a backbone network, generating input data for an LSTM layer through affine transformation of a feature sequence layer after deep convolution features are obtained, and outputting a character probability matrix through the LSTM layer, the Attention layer and the Softmax layer. And finally obtaining the license plate information comprising the license plate character information according to the character probability matrix.
The general license plate identification process comprises the steps of license plate detection, local image capturing, image correction, license plate identification and the like. In the vehicle tracking method, the license plate recognition module adopts the step of license plate segmentation to replace license plate detection, omits the step of image correction, and performs license plate recognition through a neural network with an attention mechanism.
According to the vehicle tracking method, the second structural body is analyzed to obtain the first image area corresponding to the license plate area of the target vehicle, and the license plate information of the target vehicle is calculated through the third training model, so that the license plate recognition accuracy is guaranteed, and meanwhile the license plate recognition process is simplified.
The license plate recognition process of one embodiment of the invention is shown in fig. 5:
and after receiving the second structural body, the license plate recognition module analyzes the vehicle body mask in the second structural body and judges whether a first image area corresponding to the license plate area exists in the vehicle body mask. And if the license plate information exists, the first image information is zoomed to a second preset size, and is input into a third training model, and finally, the license plate information comprising the license plate character information is output.
Optionally, the third training model performs model training by using a cross entropy function and a regularization loss function.
The cross entropy function is:
wherein N is the number of samples; m is the number of sample classes; y is ic Take 1 or 0, y when the true class of sample i is c ic Is 1, otherwise the value is 0; p is a radical of ic Is the predicted probability that sample i belongs to class c.
The regularization loss function is:
wherein n is the number of samples; y is i Is the true label of sample i; f (x) i ) Is the predicted value of sample i.
It should be noted that the training samples of the third training model may use real images collected by the urban public transportation camera and the highway camera, and a character recognition data set of a general scene.
According to the vehicle tracking method, model training is carried out on the third training model through the cross entropy function and the regularization loss function, and the precision of the third training model is improved.
Optionally, analyzing the second structural body to obtain a feature vector of the target vehicle includes:
analyzing the second structural body to obtain the vehicle mask in the second structural body;
acquiring a vehicle body image area corresponding to the vehicle body characteristic of the target vehicle according to the vehicle mask;
inputting the vehicle body image area zoomed to a third preset size into a fourth training model to obtain a vehicle body characteristic vector of the target vehicle;
and inputting the vehicle body feature vector into a fifth training model to obtain the feature vector.
It should be noted that the fourth training model includes a license plate training model, a window training model, and a wheel training model, and the license plate training model, the window training model, and the wheel training model are training models having the same structure.
And the model training of the fourth training model and the fifth training model is divided into two stages, in the stage 1, the network is used as a classification task and is used as pre-training, the Loss function uses a cross entropy function, only a part of training sample data is used, and the training is stopped when the Loss does not decrease (before overfitting). And 2, removing the added two-layer network in the training, recovering the original network structure, using triple loss for a loss function, and adopting all sub-training sample data for the training.
Wherein the formula of the triple loss function is defined as follows:
where a represents an Anchor sample, p represents a positive sample, and n represents a negative sample. Sample(s)
And
are in the same class as the main component,
and
and (4) different classes.
Representing the first euclidean distance between the Anchor sample and the positive sample,
representing the second euclidean distance between the Anchor sample and the negative sample. α represents a Margin parameter representing a minimum interval between the first euclidean distance and the second euclidean distance. [] + When the sum in parentheses is greater than 0, the value is regarded as a loss, and in the remaining cases, the loss is 0. So when the first euclidean distance is greater than the second euclidean distance, a loss is generated; otherwise, the loss is 0.
It should be noted that the training samples of the fourth training model may be real images collected by a city bus camera and a highway camera and images generated by a 3D simulation game engine.
According to the scheme of the embodiment of the invention, the second structural body is analyzed to obtain the vehicle body image area corresponding to the vehicle body characteristics of the target vehicle, and the fourth training model is adopted to calculate the vehicle body image area so as to obtain the vehicle body characteristic vector of the target vehicle. And finally obtaining a feature vector for representing the body features of the target vehicle through the fifth training model.
Optionally, the vehicle body image area includes: at least two of a first image area corresponding to the license plate area, a second image area corresponding to the window area and a third image area corresponding to the wheel area;
the vehicle body feature vector includes: at least two of a first feature vector corresponding to the license plate, a second feature vector corresponding to the vehicle window and a third feature vector corresponding to the wheel.
Optionally, a first image region corresponding to the license plate region is scaled to a third preset size and input into the license plate training model to obtain the first feature vector;
zooming a second image area corresponding to the vehicle window area to a third preset size and inputting the second image area into the vehicle window training model to obtain a second feature vector;
and scaling a third image area corresponding to the wheel area to a third preset size and inputting the third image area to the wheel training model to obtain the third feature vector.
According to the vehicle tracking method provided by the embodiment of the invention, at least two of the license plate, the window and the wheel are taken as the feature extraction objects of the target vehicle, so that the success rate of vehicle tracking is improved.
Optionally, the vehicle body image area includes: at least two of a first image area corresponding to the license plate area, a second image area corresponding to the window area and a third image area corresponding to the wheel area;
the vehicle body feature vector includes: at least two of a first feature vector corresponding to the license plate, a second feature vector corresponding to the vehicle window and a third feature vector corresponding to the wheel.
Optionally, the network-side device further includes a vehicle weight identification module, which provides the vehicle tracking module with the weak feature of the vehicle.
After the vehicle tracking module receives the second structural body, the vehicle re-identification module is called to analyze the second structural body, and vehicle masks, such as mask information of a license plate, a vehicle window and a wheel, are obtained.
Optionally, the network structure of the vehicle weight recognition module comprises a vehicle weight recognition network.
The vehicle weight recognition network includes: including a self-attentive backbone network and convolutional layers. The received image data is downsampled through a backbone network, and in one embodiment of the present invention, downsampling may be set until the feature diagram length and width are all 1 and the number of channels is 1024. And the convolution layer of the 1 multiplied by 1 convolution kernel is connected subsequently, the channel number is reduced from 1024 to 512 and 256 step by step, and finally 256-dimensional feature vectors are output. According to the scheme, the self-attention mechanism on the dimension of the characteristic diagram is added into the backbone network, so that the influence of useless areas and noise on the characteristics of the vehicle body is reduced.
As shown in fig. 6, after receiving the second structure, the vehicle weight recognition module analyzes the second structure to obtain a vehicle mask, determines the number of types of the vehicle mask, performs image preprocessing on the vehicle body image area corresponding to the vehicle body mask through a fourth training model to obtain a vehicle body feature vector of the target vehicle if the type of the vehicle mask is greater than 2, performs principal feature extraction on the vehicle body feature vector through a fifth training model to obtain the feature vector, and outputs the result.
And processing the vehicle body image area through a vehicle weight identification network to obtain a vehicle body feature vector of the target vehicle. It should be noted that, due to the angle and height between the camera of the roadside device and the target vehicle, the vehicle license plate or the vehicle wheel may not be captured by the camera of the roadside device, but the vehicle window may be captured, so the vehicle re-identification module may screen the vehicle body mask, and if two or more mask information of the vehicle license plate, the vehicle window, and the vehicle wheel is missing, the vehicle re-identification process is not performed.
When at least two of the vehicle body features are included in the vehicle mask, at least two N-dimensional vehicle body feature vectors are obtained for the license plate, for the vehicle window, and for the wheel.
According to the scheme of the embodiment of the invention, the mask information of various vehicle body characteristics is obtained through analysis, so that a plurality of vehicle body characteristic vectors can be obtained, and the robustness of vehicle weight identification and tracking is improved.
In an embodiment of the invention, if one of the vehicle body features is missing, the vehicle body feature vector is input into the fifth training model in a zero padding mode, and finally a 3N-dimensional feature vector is obtained.
Optionally, the network structure of the vehicle weight recognition module further includes a main feature extraction network.
The master feature extraction network includes: the system comprises a full connection layer, a first Attention module and a second Attention module. In an embodiment of the present invention, the first feature vector, the second feature vector, and the third feature vector are input into the main feature extraction network, and a full link layer and a ReLu activation function, which include 768 units (typical values, which can be adjusted), are first input, and then sent to the first Attention module, and the output dimension is reduced from 768 to 512. Then, the second extension module transmits the result to a 256-unit (typical value, can be adjusted) full connection layer, and outputs the final feature vector.
The Attention module comprises a full connection layer and a Sigmoid layer, wherein the full connection layer is used for compressing an input vector to 1/2 of an original dimension, the input vector is expanded to the original dimension through the full connection layer, then the Attention vector is output by the Sigmoid layer, the Attention vector is multiplied by the original input of the Attention module after being converted, and then the multiplied Attention vector is added with the original input, so that the output result of the Attention module is the output result of the Attention module.
According to the vehicle tracking method provided by the embodiment of the invention, the feature vector combining at least two vehicle body features is obtained by inputting at least two vehicle body features into the fifth training model for calculation, so that the fusion of various vehicle body features is realized, and the robustness of vehicle re-identification in the vehicle tracking process is improved.
Optionally, the network-side device further includes a vehicle tracking module.
Optionally, the target queue includes: tracking a target queue and a lost target queue;
the tracking target queue is license plate information, a feature vector and an ID of a target vehicle which are successfully matched before the matching, and the lost target queue is the ID of the target vehicle which is not successfully matched before the matching, the license plate information or the feature vector.
Optionally, after the second vehicle body feature is obtained again after the matching, the vehicle body feature may be matched with the lost target queue, and if the matching is successful, the tracked target queue is updated according to the license plate information or the vehicle body feature in the lost target queue, which is successfully matched, so as to improve the running track of the target vehicle.
Matching the license plate information and the feature vector with a target queue, and updating the target queue, wherein the steps of:
matching the license plate information and the feature vector with the tracking target queue;
adding the license plate information and the feature vector which are successfully matched into the tracking target queue, and updating the tracking target queue;
and adding the license plate information and the feature vector which are not successfully matched into the lost target queue, and updating the lost target queue.
Optionally, the matching the license plate information and the feature vector with a target queue includes:
and matching the license plate information and the feature vectors with a target queue by adopting a Hungarian algorithm and a related filtering algorithm.
When the network side equipment is set to be high-frequency tracking, the roadside equipment transmits data to the network side equipment at a frame rate of not less than 20 frames per second, and due to the fact that the load is large, a related filtering algorithm needs to be closed, and only a Hungarian algorithm is used for vehicle tracking. At this time, the roadside device is required to send the second structural body together with the image data to the network side device. And the vehicle tracking module adopts Hungarian algorithm to carry out maximum matching on the vehicle body characteristics analyzed from the second structure body and the target queue.
When the network side equipment is set to be low-frequency tracking, the roadside equipment transmits information to the network side equipment at a frame rate of not more than 20 frames and not less than 10 frames per second, the system load is not high, and a related filtering algorithm can be started for vehicle tracking.
According to the vehicle tracking method, the license plate information, the feature vectors and the target queue are matched through the Hungarian algorithm and the related filtering algorithm, different matching algorithms can be selected according to the tracking frequency set by the network side equipment, and therefore high calculation accuracy is guaranteed under the condition that the load of the network side equipment is met.
In an embodiment of the invention, Average Peak Correlation Energy (APCE) can be introduced as a confidence level to reflect the fluctuation degree of a response map and the confidence level of a detected target, so as to solve the problem that a correlation filtering algorithm KCF does not have a good index to reflect the tracking effect when tracking the target.
Wherein the formula of the average peak correlation energy is defined as:
wherein, F max And F min Respectively representing the maximum and minimum values in the response plot. F (w,h) In response to the pixel value at each location in the map. And when the APCE is larger than the preset tracking threshold, the confidence degree of the current tracking result is considered to be high.
As shown in fig. 7, the matching of the license plate information and the feature vector with the target queue includes: the target is lost and the tracking is successful. If the matching is successful, the tracking is successful, and if the matching is failed, the current frame does not acquire the body characteristics of the target vehicle, namely the target is lost.
Optionally, matching the license plate information and the feature vector with a target queue further includes: newly adding a target;
the newly added target represents that the third vehicle body characteristics acquired by the current frame are not matched with the license plate information and the characteristic vectors in the target queue; the third body feature includes: and the third license plate information and/or the third vehicle body feature vector.
Optionally, matching the license plate information and the feature vector with a target queue, and updating the target queue, further comprising:
matching the license plate information and/or the feature vector of the newly added target with the lost target queue;
and adding the license plate information and the feature vector in the lost target queue which are successfully matched into the tracking target queue, and updating the tracking target queue.
In one embodiment of the invention, when a new added target appears, the license plate information with strong characteristics is taken as a main part, the license plate recognition module is firstly called to recognize the license plate information, and then the vehicle re-recognition module is called to obtain the characteristic vector corresponding to the vehicle body characteristics. Matching the third license plate information with data in a lost target queue, and if the matching is successful, restoring the third license plate information to the target queue to update the target queue; and calculating a first Euclidean distance between the third vehicle body characteristic vector and a lost target queue by taking a weak characteristic vehicle body characteristic vector as an auxiliary, and updating the tracking target queue according to the lost target queue if the first Euclidean distance is smaller than a preset threshold value.
As shown in fig. 8, an embodiment of the present invention further provides a vehicle tracking apparatus 800, applied to roadside equipment, including:
a first obtaining module 801, configured to obtain image data of a target vehicle;
a second obtaining module 802, configured to obtain first features of the image data at different proportions, and input the first features into a first training model to obtain a first structural body;
a calculating module 803, configured to calculate a vehicle mask of the target vehicle according to the first structure, and assign the vehicle mask to the first structure to obtain a second structure;
a sending module 804, configured to send the image data and the second structural body to a network side device.
According to the vehicle tracking device provided by the embodiment of the invention, the first characteristics of the image data under different proportions can be obtained by acquiring the image data of the target vehicle; obtaining a first structure body by inputting the first characteristic into a first training model; calculating a vehicle mask of the target vehicle according to the first structure body; obtaining a second structure by assigning the vehicle mask to the first structure; and finally, the second structural body is sent to network side equipment through the image data, and data support is provided for vehicle tracking image identification and vehicle feature extraction.
As shown in fig. 9, an embodiment of the present invention further provides a vehicle tracking apparatus 900, applied to a network-side device, including:
a receiving module 901 configured to receive image data of a target vehicle and a second structural body;
an analyzing module 902, configured to analyze the second structural body to obtain license plate information and a feature vector of the target vehicle, where the feature vector is used to represent a vehicle body feature of the target vehicle corresponding to a vehicle mask;
a matching update module 903, configured to match the license plate information and the feature vector with a target queue, and update the target queue; the target queue is license plate information and feature vectors stored before the matching.
According to the vehicle tracking device provided by the embodiment of the invention, the license plate information of the target vehicle and the feature vector used for representing the body feature of the target vehicle are obtained by analyzing the received second structural body sent by the road side equipment, and the tracking data of the vehicle is updated in a manner of updating the target queue by matching the license plate information and the feature vector with the target queue, so that the integrity of the tracking track of the target vehicle is improved.
In order to better achieve the above object, as shown in fig. 10, the present invention also provides a roadside apparatus including: the transceiver 1010, the processor 1000, and the memory 1020 connected to the processor 1000 through the bus interface, wherein the memory 1020 is used for storing programs and data used by the processor 1000 when executing operations, and the processor 1000 calls and executes the programs and data stored in the memory 1020.
The transceiver 1010 is connected to the bus interface, and is configured to receive and transmit data under the control of the processor 1000;
the transceiver 1010 is configured to:
acquiring image data of a target vehicle;
and acquiring first characteristics of the image data in different proportions, and inputting the first characteristics into a first training model to obtain a first structure body.
The processor 1000 is used for reading the program in the memory 1020 and executing the following steps:
and calculating a vehicle mask of the target vehicle according to the first structural body, and assigning the vehicle mask to the first structural body to obtain a second structural body.
The transceiver 1010 is further configured to:
and sending the image data and the second structural body to a network side device.
Where in fig. 10, the bus architecture may include any number of interconnected buses and bridges, with various circuits being linked together, particularly one or more processors represented by processor 1000 and memory represented by memory 1020. The bus architecture may also link together various other circuits such as peripherals, voltage regulators, power management circuits, and the like, which are well known in the art, and therefore, will not be described any further herein. The bus interface provides an interface. The transceiver 1010 may be a number of elements, including a transmitter and a transceiver, providing a means for communicating with various other apparatus over a transmission medium. For different terminals, the user interface 1030 may also be an interface capable of interfacing with a desired device, including but not limited to a keypad, display, speaker, microphone, joystick, etc. The processor 1000 is responsible for managing the bus architecture and general processing, and the memory 1020 may store data used by the processor 1000 in performing operations.
In order to better achieve the above object, as shown in fig. 11, the present invention further provides a network side device, including: a transceiver 1111, a processor 1100, and a memory 1120 connected with the processor 1100 through a bus interface, the memory 1120 being used for storing programs and data used by the processor 1100 in performing operations, the processor 1100 calling and executing the programs and data stored in the memory 1120.
The transceiver 1111 is connected to the bus interface, and configured to receive and transmit data under the control of the processor 1100;
the transceiver 1111 is configured to:
image data of the target vehicle and the second structural body are received.
The processor 1100 is used for reading the program in the memory 1120 and executing the following steps:
analyzing the second structural body to obtain license plate information and a feature vector of the target vehicle, wherein the feature vector is used for representing vehicle body features of the target vehicle corresponding to the vehicle mask;
and matching the license plate information and the feature vector with a target queue, and updating the target queue.
Those skilled in the art will appreciate that all or part of the steps for implementing the above embodiments may be performed by hardware, or may be instructed to be performed by associated hardware by a computer program that includes instructions for performing some or all of the steps of the above methods; and the computer program may be stored in a readable storage medium, which may be any form of storage medium.
In addition, embodiments of the present invention also provide a computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, implements the steps of the vehicle tracking method described above. And the same technical effect can be achieved, and in order to avoid repetition, the description is omitted.
Furthermore, it is to be noted that in the device and method of the invention, it is obvious that the individual components or steps can be decomposed and/or recombined. These decompositions and/or recombinations are to be regarded as equivalents of the present invention. Also, the steps of performing the series of processes described above may naturally be performed chronologically in the order described, but need not necessarily be performed chronologically, and some steps may be performed in parallel or independently of each other. It will be understood by those skilled in the art that all or any of the steps or elements of the method and apparatus of the present invention may be implemented in any computing device (including processor, storage medium, etc.) or network of computing devices, in hardware, firmware, software, or any combination thereof, which can be implemented by those skilled in the art using their basic programming skills after reading the description of the present invention.
Thus, the objects of the invention may also be achieved by running a program or a set of programs on any computing device. The computing device may be a general purpose device as is well known. The object of the invention is thus also achieved solely by providing a program product comprising program code for implementing the method or the apparatus. That is, such a program product also constitutes the present invention, and a storage medium storing such a program product also constitutes the present invention. It is to be understood that the storage medium may be any known storage medium or any storage medium developed in the future. It is further noted that in the apparatus and method of the present invention, it is apparent that each component or step can be decomposed and/or recombined. These decompositions and/or recombinations are to be regarded as equivalents of the present invention. Also, the steps of executing the series of processes described above may naturally be executed chronologically in the order described, but need not necessarily be executed chronologically. Some steps may be performed in parallel or independently of each other.
While the foregoing is directed to the preferred embodiment of the present invention, it will be understood by those skilled in the art that various changes and modifications may be made without departing from the spirit and scope of the invention as defined in the appended claims.
Claims (20)
1. A vehicle tracking method is applied to road side equipment and is characterized by comprising the following steps:
acquiring image data of a target vehicle;
acquiring first characteristics of the image data in different proportions, and inputting the first characteristics into a first training model to obtain a first structure body;
calculating a vehicle mask of the target vehicle according to the first structural body, and assigning the vehicle mask to the first structural body to obtain a second structural body;
and sending the image data and the second structural body to a network side device.
2. The vehicle tracking method of claim 1, wherein the categories of the vehicle mask include license plates, windows, and wheels.
3. The vehicle tracking method of claim 1, wherein the obtaining first features of the image data at different scales comprises:
reducing the image data in different proportions to obtain a first high-level feature and a first low-level feature in different proportions;
and fusing the first high-level feature and the first low-level feature to obtain the first feature.
4. The vehicle tracking method according to claim 1, wherein the first structure includes a first image corresponding to the target vehicle, size information of the first image, and position information of the first image in an image corresponding to the image data.
5. The vehicle tracking method of claim 1, wherein the first trained model is model trained using Generalized local and GIoU Loss functions.
6. The vehicle tracking method of claim 4, wherein the calculating a vehicle mask for the target vehicle from the image data comprises:
and zooming the first image to a first preset size, and performing image segmentation through a second training model to obtain the vehicle mask.
7. The vehicle tracking method of claim 6, wherein the second training model is model trained using a Focal local function.
8. A vehicle tracking method is applied to network side equipment and is characterized by comprising the following steps:
receiving image data of a target vehicle and a second structure;
analyzing the second structural body to obtain license plate information and a feature vector of the target vehicle, wherein the feature vector is used for representing vehicle body features of the target vehicle corresponding to the vehicle mask;
matching the license plate information and the feature vector with a target queue, and updating the target queue; the target queue is the license plate information and the feature vector which are stored before the matching.
9. The vehicle tracking method of claim 7, wherein the vehicle body feature corresponding to the vehicle mask comprises: license plate, door window and wheel.
10. The vehicle tracking method of claim 8, wherein analyzing the second structure to obtain license plate information of the target vehicle comprises:
analyzing the second structural body to obtain the vehicle mask in the second structural body;
acquiring a first image area corresponding to the license plate area according to the vehicle mask;
and inputting the first image area which is scaled to the second preset size into a third training model to obtain the license plate information of the target vehicle.
11. The vehicle tracking method of claim 10, wherein the third training model is model trained using a cross entropy function and a regularization loss function.
12. The vehicle tracking method according to claim 8, wherein the analyzing the second structure to obtain the feature vector of the target vehicle includes:
analyzing the second structural body to obtain the vehicle mask in the second structural body;
acquiring a vehicle body image area corresponding to the vehicle body characteristic of the target vehicle according to the vehicle mask;
inputting the vehicle body image area zoomed to a third preset size into a fourth training model to obtain a vehicle body characteristic vector of the target vehicle;
and inputting the vehicle body feature vector into a fifth training model to obtain the feature vector.
13. The vehicle tracking method according to claim 12, wherein the vehicle body image area includes: at least two of a first image area corresponding to the license plate area, a second image area corresponding to the vehicle window area and a third image area corresponding to the wheel area;
the vehicle body feature vector includes: at least two of a first feature vector corresponding to the license plate, a second feature vector corresponding to the vehicle window and a third feature vector corresponding to the wheel.
14. The vehicle tracking method of claim 12, wherein the fifth training model is a training model that includes a self-attentiveness mechanism.
15. The vehicle tracking method of claim 8, wherein the matching the license plate information and the feature vector to a target queue comprises:
and matching the license plate information and the feature vectors with a target queue by adopting a Hungarian algorithm and a related filtering algorithm.
16. A vehicle tracking device is applied to roadside equipment, and is characterized by comprising:
the first acquisition module is used for acquiring image data of a target vehicle;
the second acquisition module is used for acquiring first characteristics of the image data in different proportions and inputting the first characteristics into a first training model to obtain a first structural body;
the calculation module is used for calculating a vehicle mask of the target vehicle according to the first structural body, and assigning the vehicle mask to the first structural body to obtain a second structural body;
and the sending module is used for sending the image data and the second structural body to network side equipment.
17. A vehicle tracking device is applied to network side equipment and is characterized by comprising:
a receiving module for receiving image data of a target vehicle and a second structural body;
the analysis module is used for analyzing the second structural body to obtain license plate information and a feature vector of the target vehicle, and the feature vector is used for representing vehicle body features of the target vehicle corresponding to the vehicle mask;
the matching updating module is used for matching the license plate information and the characteristic vector with a target queue and updating the target queue; the target queue is the license plate information and the feature vector which are stored before the matching.
18. A roadside apparatus comprising: transceiver, memory, processor and computer program stored on the memory and executable on the processor, characterized in that the processor realizes the steps of the vehicle tracking method according to any one of claims 1 to 7 when executing the computer program.
19. A network-side device, comprising: transceiver, memory, processor and computer program stored on the memory and executable on the processor, characterized in that the processor realizes the steps of the vehicle tracking method according to any one of claims 8 to 15 when executing the computer program.
20. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the steps of the vehicle tracking method according to one of claims 1 to 7 or the steps of the vehicle tracking method according to one of claims 8 to 15.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111624101.2A CN114898304A (en) | 2021-12-28 | 2021-12-28 | Vehicle tracking method and device, road side equipment and network side equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111624101.2A CN114898304A (en) | 2021-12-28 | 2021-12-28 | Vehicle tracking method and device, road side equipment and network side equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114898304A true CN114898304A (en) | 2022-08-12 |
Family
ID=82714243
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111624101.2A Pending CN114898304A (en) | 2021-12-28 | 2021-12-28 | Vehicle tracking method and device, road side equipment and network side equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114898304A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117710417A (en) * | 2023-12-04 | 2024-03-15 | 成都臻识科技发展有限公司 | Multi-path multi-target tracking method and equipment for single-camera and readable storage medium |
-
2021
- 2021-12-28 CN CN202111624101.2A patent/CN114898304A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117710417A (en) * | 2023-12-04 | 2024-03-15 | 成都臻识科技发展有限公司 | Multi-path multi-target tracking method and equipment for single-camera and readable storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112052886B (en) | Intelligent human body action posture estimation method and device based on convolutional neural network | |
| US20210248378A1 (en) | Spatiotemporal action detection method | |
| CN107529650B (en) | Closed loop detection method and device and computer equipment | |
| CN104268594B (en) | A kind of video accident detection method and device | |
| CN112150493B (en) | Semantic guidance-based screen area detection method in natural scene | |
| CN112288770A (en) | Video real-time multi-target detection and tracking method and device based on deep learning | |
| CN112395951B (en) | Complex scene-oriented domain-adaptive traffic target detection and identification method | |
| WO2021018106A1 (en) | Pedestrian detection method, apparatus, computer-readable storage medium and chip | |
| CN114758288A (en) | Power distribution network engineering safety control detection method and device | |
| CN112001859A (en) | Method and system for repairing face image | |
| CN113128360A (en) | Driver driving behavior detection and identification method based on deep learning | |
| CN110991444A (en) | Complex scene-oriented license plate recognition method and device | |
| CN114677502A (en) | License plate detection method with any inclination angle | |
| CN114783024A (en) | Face recognition system of gauze mask is worn in public place based on YOLOv5 | |
| CN112084952B (en) | Video point location tracking method based on self-supervision training | |
| CN113297959A (en) | Target tracking method and system based on corner attention twin network | |
| CN115661611A (en) | Infrared small target detection method based on improved Yolov5 network | |
| CN114926796A (en) | Bend detection method based on novel mixed attention module | |
| CN114913604A (en) | Attitude identification method based on two-stage pooling S2E module | |
| CN114898304A (en) | Vehicle tracking method and device, road side equipment and network side equipment | |
| CN117115616A (en) | Real-time low-illumination image target detection method based on convolutional neural network | |
| CN115861883A (en) | Multi-target detection tracking method | |
| CN115690934A (en) | Master and student attendance card punching method and device based on batch face recognition | |
| CN113139483B (en) | Human behavior recognition method, device, apparatus, storage medium, and program product | |
| CN112580633A (en) | Public transport passenger flow statistical device and method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| CB02 | Change of applicant information | ||
| CB02 | Change of applicant information |
Address after: 400040 No. 35, Jinghe Road, Huxi street, high tech Zone, Chongqing Applicant after: CITIC Technology Zhilian Technology Co.,Ltd. Address before: 400040 No. 35, Jinghe Road, Huxi street, high tech Zone, Chongqing Applicant before: Datang Gaohong Zhilian Technology (Chongqing) Co.,Ltd. |
|
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |