CN111723672B - Method and device for acquiring video recognition driving track and storage medium - Google Patents

Method and device for acquiring video recognition driving track and storage medium Download PDF

Info

Publication number
CN111723672B
CN111723672B CN202010447021.3A CN202010447021A CN111723672B CN 111723672 B CN111723672 B CN 111723672B CN 202010447021 A CN202010447021 A CN 202010447021A CN 111723672 B CN111723672 B CN 111723672B
Authority
CN
China
Prior art keywords
data
video
model
training
driving track
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202010447021.3A
Other languages
Chinese (zh)
Other versions
CN111723672A (en
Inventor
王晓飞
郭凯
曾彦杰
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
South China University of Technology SCUT
Original Assignee
South China University of Technology SCUT
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by South China University of Technology SCUT filed Critical South China University of Technology SCUT
Priority to CN202010447021.3A priority Critical patent/CN111723672B/en
Publication of CN111723672A publication Critical patent/CN111723672A/en
Application granted granted Critical
Publication of CN111723672B publication Critical patent/CN111723672B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/40—Scenes; Scene-specific elements in video content
    • G06V20/41—Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/40—Scenes; Scene-specific elements in video content
    • G06V20/48—Matching video sequences
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Computational Linguistics (AREA)
  • Multimedia (AREA)
  • Software Systems (AREA)
  • General Health & Medical Sciences (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • Biophysics (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Biomedical Technology (AREA)
  • Mathematical Physics (AREA)
  • Artificial Intelligence (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Image Analysis (AREA)

Abstract

The invention discloses a method, a device and a storage medium for acquiring a video recognition driving track, which comprises a data acquisition step, a data set division step, a data calibration and preprocessing step, a deep learning model training step and a step of determining an optimal convolutional neural network structure model by evaluating the deep learning model by adopting a verification set. The method is different from the method that the error caused by the difference between the driving track of the vehicle obtained by virtual driving and the reality is large, and the accuracy rate is higher.

Description

Method and device for acquiring video recognition driving track and storage medium
Technical Field
The invention relates to the field of traffic information processing, in particular to a method and a device for acquiring a video identification driving track and a storage medium.
Background
The driving track has very important meaning for each driver, the specific route of the vehicle can be known through the driving track, and the driving track can also be used as the basis for judging traffic accidents. In the field of road design and road traffic safety, the driving track can be used as the theoretical basis of route design, and the deviation degree of the driving track is closely related to the road safety.
The current methods for acquiring the driving track of the automobile can be divided into two categories. Firstly, GPS information is acquired through terminal software to perform data storage and GPS data splicing imaging, the processing method is very dependent on the precision of a GPS, and the current GPS centimeter-level high-precision technology is difficult to realize in a vehicle moving at a high speed. And secondly, establishing a road condition three-dimensional modeling for the real road environment, and obtaining a related vehicle running track through a multi-degree-of-freedom driving cabin for simulating driving. However, the simulation driving experiment is difficult to completely restore the real road condition, and the data and the natural driving experiment have certain difference.
Accordingly, the method is provided. There is an urgent need for a new way of acquiring a trajectory so as to overcome the above-mentioned drawbacks.
Disclosure of Invention
In order to overcome the defects and shortcomings in the prior art, the invention provides a method and a device for acquiring a driving track by video identification based on deep learning and a storage medium.
The invention adopts the following technical scheme:
a video identification driving track obtaining method comprises the following steps:
acquiring data, namely acquiring video data of the highway, wherein the video data comprises a monitoring time period, a monitoring scene and a monitoring line shape;
dividing a data set, namely dividing video data into a picture data set according to 20 frames per second, and dividing the picture data set into a training set train for training a model; a validation set is convenient for preliminary evaluation of training results and test sets;
data calibration and pretreatment, wherein the pretreatment comprises the steps of cutting a picture according to 20 frames, using a calibration tool to calibrate the number and the position of a vehicle in the picture, obtaining the number information and the coordinate information of the vehicle, and then pretreating the obtained information;
training a deep learning model to preliminarily obtain a driving track recognition model;
and evaluating the deep learning model by adopting a verification set to determine an optimal convolutional neural network structure model.
Preferably, the deep learning model is a convolutional neural network structure model, and includes a convolutional layer, a pooling layer and a full connection layer, the convolutional layer generates feature data of an image, the pooling layer performs aggregation statistics on the feature data, and the full connection layer combines a plurality of groups of pooled data features into a group of signal data to be output for image category identification.
Preferably, the loss function based on the convolutional neural network structure model is:
Figure BDA0002506249020000021
and taking the mean square error MSE of the accurate values Xi and Yi obtained by calibration of the training set and the predicted values X, i, Y and i obtained by the preliminary model as an evaluation standard.
Preferably, the monitoring time period refers to three time periods, namely early, middle and late, under the condition of different illumination and brightness;
the monitoring scene refers to the track data of vehicle running under the conditions of free flow, stable flow and blocked flow;
the monitoring line shape refers to the shape of a road, specifically a straight line, an ascending slope and a curve section.
Preferably, the data preprocessing comprises feature extraction, feature dimension reduction, feature null value processing, feature conversion and feature normalization.
Preferably, the deep learning model is evaluated by using a verification set, specifically:
preliminarily evaluating the training result of the deep learning model by adopting a verification set, adjusting parameters, and determining an optimal model if over-fitting occurs;
calibrating coordinate data obtained by the picture as an accurate value, taking the coordinate of the training model as a predicted value, calculating a loss function, and analyzing an evaluation result;
and (4) performing parameter adjustment according to the loss function inverse gradient, and determining an optimal convolutional neural network structure model.
Preferably, the data set partitioning is a training set, a validation set and a test set in terms of ratios 0.6, 0.2 and 0.2.
A device for acquiring a driving track by video recognition comprises the following steps:
the video image acquisition module acquires moving vehicle information in the traffic monitoring video through a camera on the highway;
the video image dividing module is used for carrying out classification division on a verification set, a detection set and a training set on the collected video information;
the video image calibration and pretreatment module is used for calibrating and pretreating pictures in the video;
and the model training evaluation module is used for training the deep learning model to preliminarily obtain a driving track recognition model, evaluating the deep learning model by adopting a verification set and determining an optimal convolutional neural network structure model.
A storage medium is used for storing the video identification driving track acquisition method.
The invention has the beneficial effects that:
the method provided by the invention has the advantages that the error caused by the difference between the vehicle running track obtained by virtual driving and the reality is large, the method is more authentic, and the method has consistency with the vehicle running track in a natural driving experiment; according to the method provided by the invention, the acquired vehicle driving track can reach a certain precision by a relatively mature deep learning method, and the scientific research requirement is met; the method provided by the invention is novel and high in practicability, does not depend on excessive external equipment, and can obtain the driving track only by video and computer algorithms.
Drawings
FIG. 1 is a schematic view of a video capture of the present invention;
FIG. 2 is a flow chart of the operation of the present invention;
FIG. 3 is a graph of the recognition effect of the present invention;
Detailed Description
The present invention will be described in further detail with reference to examples and drawings, but the present invention is not limited to these examples.
Example 1
As shown in fig. 1 to 3, a method for acquiring a video identification driving track includes the following steps:
the method comprises the steps of data acquisition, obtaining of video data of the highway, taking the highway monitoring video as a source of the video data, obtaining of enough amount of video from monitoring time periods (morning, noon and evening), monitoring scenes (free flow, stable flow and blocking flow), monitoring line shapes (straight lines, uphill and curve segments) and guaranteeing of diversity of video styles, wherein the video data comprise week data. The expressway video scene is diversified, and diversified training samples are provided for the convolutional neural network model structure.
And (3) dividing and preprocessing a data set, after the video is acquired, intercepting the acquired video according to 20 frames each to prepare the data set of the next stage. The acquired videos are classified to a certain degree, and the data set is divided into a training set, a verification set and a test set which account for 0.6, 0.2 and 0.2. The data sets are respectively calibrated, the vehicle types, serial numbers and positions are extracted from the pictures after calibration, the vehicle types, serial numbers and positions specifically comprise the same serial number of the same vehicle in different pictures and the vehicle position of each picture, and the (serial number and coordinate) data set related to the picture is generated after calibration. In this link, a verification set of parameters of the preliminary convolutional neural network model structure needs to be made for judging whether the convolutional neural network model structure which can reach the accuracy of the data set can reach the accuracy of the training set.
And preprocessing the calibrated information.
And (5) training the deep learning model to preliminarily obtain a driving track recognition model.
The deep learning model is specifically a convolutional neural network structure model and comprises a convolutional layer, a pooling layer and a full-connection layer, wherein the convolutional layer generates feature data of an image, the pooling layer performs aggregation statistics on the feature data, and the full-connection layer combines a plurality of groups of pooled data features into a group of signal data to be output for image category identification.
The method needs to obtain the accuracy of the offset of the driving track, needs to determine an evaluation standard after determining a neural network model as a training model, and uses the obtained approximate value variance as a loss function MSE by a convolutional neural network.
Figure BDA0002506249020000041
And selecting a convolutional neural network structure model, training the model through a data set obtained by the calibrated vehicle position and the sequence number, and preliminarily obtaining a driving track recognition model, namely a preliminary convolutional neural network model.
And evaluating the deep learning model by adopting a verification set to determine an optimal convolutional neural network structure model.
And (4) using coordinate data obtained by calibrating the picture by the training set as an accurate value, using coordinates obtained by the model obtained by training as a predicted value, calculating a loss function, and analyzing an evaluation result. And reversely solving the gradient according to the loss function to adjust the parameters, and comparing the adjustment with the last model result and the target. And (4) taking out error data, analyzing the existing problems, discussing the adjustment direction and recording the experimental result.
And repeating the training and the evaluation steps to finally obtain a structural model based on the convolutional neural network meeting the target accuracy.
After the model training is finished, the step of adopting the convolutional neural network structure model to obtain the driving track is as follows:
and converting video data acquired by using the monitoring video into pictures, inputting the preprocessed pictures into the model for moving vehicle detection, tracking and feature extraction of moving vehicles, and outputting vehicle position coordinates and drawing vehicle tracks when vehicle identification is greater than a confidence threshold. As shown in FIG. 3, the rectangular box is the vehicle location, "car" is the vehicle type, and the percentage is the confidence.
Example 2
A device for acquiring a driving track by video recognition comprises
The video image acquisition module acquires moving vehicle information in the traffic monitoring video through a camera on the highway;
and the video image division module is used for carrying out verification set, detection set and training set classification division on the acquired video information.
The video image calibration and pretreatment module is used for calibrating and pretreating pictures in the video;
and the model training evaluation module is used for training the deep learning model to preliminarily obtain a driving track recognition model, evaluating the deep learning model by adopting a verification set and determining an optimal convolutional neural network structure model.
Example 3
A storage medium is used for storing a video identification driving track acquisition method.
The above embodiments are preferred embodiments of the present invention, but the present invention is not limited to the above embodiments, and any other changes, modifications, substitutions, combinations, and simplifications which do not depart from the spirit and principle of the present invention should be construed as equivalents thereof, and all such changes, modifications, substitutions, combinations, and simplifications are intended to be included in the scope of the present invention.

Claims (6)

1. A method for acquiring a driving track by video identification is characterized by comprising the following steps:
acquiring data, namely acquiring video data of the highway, wherein the video data comprises video data of different monitoring time periods, different monitoring scenes and different monitoring line shapes;
the monitoring time period refers to three time periods of morning, noon and evening under the condition of different illumination and brightness;
the monitoring scene refers to the track data of vehicle running under the conditions of free flow, stable flow and blocked flow;
the monitoring line shape refers to the shape of a road, specifically a straight line, an ascending slope and a curve section;
dividing a data set, namely dividing video data into a picture data set according to 20 frames per second, and dividing the picture data set into a training set train for training a model; a validation set is convenient for preliminary evaluation of training results and test sets;
data calibration and pretreatment, including cutting a picture according to 20 frames, using a calibration tool to calibrate the number and the position of a vehicle in the picture, obtaining the number information and the coordinate information of the vehicle, and then preprocessing the obtained information;
the preprocessing comprises feature extraction, feature dimension reduction, feature null value processing, feature conversion and feature normalization;
training a deep learning model to preliminarily obtain a driving track recognition model;
evaluating the deep learning model by adopting a verification set to determine an optimal convolutional neural network structure model, which specifically comprises the following steps:
preliminarily evaluating the training result of the deep learning model by adopting a verification set, adjusting parameters, and determining an optimal model if over-fitting occurs;
calibrating coordinate data obtained by the picture as an accurate value, taking the coordinate of the training model as a predicted value, calculating a loss function, and analyzing an evaluation result;
and (4) performing parameter adjustment according to the loss function inverse gradient, and determining an optimal convolutional neural network structure model.
2. The video identification driving track acquisition method according to claim 1, wherein the deep learning model is a convolutional neural network structure model, and includes a convolutional layer, a pooling layer and a full connection layer, the convolutional layer generates feature data of an image, the pooling layer performs aggregation statistics on the feature data, and the full connection layer combines a plurality of pooled data features into a set of signal data to be output for image type identification.
3. The video identification driving track acquisition method according to claim 1, wherein the loss function based on the convolutional neural network structure model is:
Figure FDA0002855439050000021
precise value X obtained by calibration of training seti,YiAnd predicted value X 'obtained from preliminary model'i,Y′iMean square error MSE of (a) as an evaluation criterion.
4. The video identification driving track acquisition method of claim 1, wherein the data set partitioning is a training set, a validation set and a test set according to a ratio of 0.6, 0.2 and 0.2.
5. An apparatus for implementing the method for acquiring a driving track for video recognition according to any one of claims 1 to 4, comprising:
the video image acquisition module acquires moving vehicle information in the traffic monitoring video through a camera on the highway;
the video image dividing module is used for carrying out classification division on a verification set, a detection set and a training set on the collected video information;
the video image calibration and pretreatment module is used for calibrating and pretreating pictures in the video;
and the model training evaluation module is used for training the deep learning model to preliminarily obtain a driving track recognition model, evaluating the deep learning model by adopting a verification set and determining an optimal convolutional neural network structure model.
6. A storage medium for storing the video recognition trajectory acquisition method of any one of claims 1 to 4.
CN202010447021.3A 2020-05-25 2020-05-25 Method and device for acquiring video recognition driving track and storage medium Active CN111723672B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010447021.3A CN111723672B (en) 2020-05-25 2020-05-25 Method and device for acquiring video recognition driving track and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010447021.3A CN111723672B (en) 2020-05-25 2020-05-25 Method and device for acquiring video recognition driving track and storage medium

Publications (2)

Publication Number Publication Date
CN111723672A CN111723672A (en) 2020-09-29
CN111723672B true CN111723672B (en) 2021-03-30

Family

ID=72564963

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010447021.3A Active CN111723672B (en) 2020-05-25 2020-05-25 Method and device for acquiring video recognition driving track and storage medium

Country Status (1)

Country Link
CN (1) CN111723672B (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113850237A (en) * 2021-11-29 2021-12-28 华砺智行(武汉)科技有限公司 Internet vehicle target detection and evaluation method and system based on video and track data

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114566052B (en) * 2022-04-27 2022-08-12 华南理工大学 Method for judging rotation of highway traffic flow monitoring equipment based on traffic flow direction

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113850237A (en) * 2021-11-29 2021-12-28 华砺智行(武汉)科技有限公司 Internet vehicle target detection and evaluation method and system based on video and track data

Also Published As

Publication number Publication date
CN111723672A (en) 2020-09-29

Similar Documents

Publication Publication Date Title
CN111145545B (en) Road traffic behavior unmanned aerial vehicle monitoring system and method based on deep learning
CN111814623A (en) Vehicle lane departure visual detection method based on deep neural network
CN112700470B (en) Target detection and track extraction method based on traffic video stream
CN106935035B (en) Parking offense vehicle real-time detection method based on SSD neural network
CN111027430B (en) Traffic scene complexity calculation method for intelligent evaluation of unmanned vehicles
CN111079640B (en) Vehicle type identification method and system based on automatic amplification sample
CN111723672B (en) Method and device for acquiring video recognition driving track and storage medium
CN110427993B (en) High-speed train navigation blind area positioning method based on meteorological parameters
CN110969142B (en) Abnormal driving scene extraction method based on network-connected vehicle natural driving data
CN114170580A (en) Highway-oriented abnormal event detection method
CN111582339B (en) Vehicle detection and recognition method based on deep learning
CN113592905B (en) Vehicle driving track prediction method based on monocular camera
CN112256590B (en) Virtual scene effectiveness judgment method and device and automatic driving system
CN113327248B (en) Tunnel traffic flow statistical method based on video
CN111797738A (en) Multi-target traffic behavior fast extraction method based on video identification
CN115544888A (en) Dynamic scene boundary assessment method based on physical mechanism and machine learning hybrid theory
CN117152513A (en) Vehicle boundary positioning method for night scene
CN115953447A (en) Point cloud consistency constraint monocular depth estimation method for 3D target detection
CN112945244B (en) Rapid navigation system and navigation method suitable for complex overpass
CN117710843A (en) Intersection dynamic signal timing scheme detection method based on unmanned aerial vehicle video
CN113139615A (en) Unmanned environment target detection method based on embedded equipment
CN117198057A (en) Experimental method and system for road side perception track data quality inspection
CN116129327A (en) Infrared vehicle detection method based on improved YOLOv7 algorithm
CN114613144B (en) Method for describing motion evolution law of hybrid vehicle group based on Embedding-CNN
CN116434056A (en) Target identification method and system based on radar fusion and electronic equipment

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant