CN117251722A - Intelligent traffic management system based on big data - Google Patents
Intelligent traffic management system based on big data Download PDFInfo
- Publication number
- CN117251722A CN117251722A CN202311104343.8A CN202311104343A CN117251722A CN 117251722 A CN117251722 A CN 117251722A CN 202311104343 A CN202311104343 A CN 202311104343A CN 117251722 A CN117251722 A CN 117251722A
- Authority
- CN
- China
- Prior art keywords
- data
- traffic
- track
- road
- vehicle
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000007726 management method Methods 0.000 claims abstract description 33
- 230000006872 improvement Effects 0.000 claims abstract description 27
- YHXISWVBGDMDLQ-UHFFFAOYSA-N moclobemide Chemical compound C1=CC(Cl)=CC=C1C(=O)NCCN1CCOCC1 YHXISWVBGDMDLQ-UHFFFAOYSA-N 0.000 claims abstract description 20
- 238000012300 Sequence Analysis Methods 0.000 claims abstract description 14
- 238000004458 analytical method Methods 0.000 claims abstract description 14
- 238000007418 data mining Methods 0.000 claims abstract description 14
- 238000000513 principal component analysis Methods 0.000 claims abstract description 10
- 230000002068 genetic effect Effects 0.000 claims abstract description 8
- 238000012545 processing Methods 0.000 claims description 25
- 238000013500 data storage Methods 0.000 claims description 18
- 238000007405 data analysis Methods 0.000 claims description 15
- 238000005516 engineering process Methods 0.000 claims description 13
- 239000013598 vector Substances 0.000 claims description 9
- 230000035945 sensitivity Effects 0.000 claims description 8
- 238000010586 diagram Methods 0.000 claims description 7
- 238000000034 method Methods 0.000 claims description 6
- 238000007781 pre-processing Methods 0.000 claims description 6
- 238000013139 quantization Methods 0.000 claims description 6
- 230000002159 abnormal effect Effects 0.000 claims description 3
- 238000009499 grossing Methods 0.000 claims description 3
- 238000005259 measurement Methods 0.000 claims description 3
- 230000008569 process Effects 0.000 claims description 3
- 238000005457 optimization Methods 0.000 description 3
- 206010039203 Road traffic accident Diseases 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 230000035772 mutation Effects 0.000 description 2
- 230000004075 alteration Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 230000002427 irreversible effect Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000006641 stabilisation Effects 0.000 description 1
- 238000011105 stabilization Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/213—Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods
- G06F18/2134—Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods based on separation criteria, e.g. independent component analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/10—Pre-processing; Data cleansing
- G06F18/15—Statistical pre-processing, e.g. techniques for normalisation or restoring missing data
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/23—Clustering techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F21/60—Protecting data
- G06F21/62—Protecting access to data via a platform, e.g. using keys or access control rules
- G06F21/6218—Protecting access to data via a platform, e.g. using keys or access control rules to a system of files or objects, e.g. local or distributed file system or database
- G06F21/6245—Protecting personal data, e.g. for financial or medical purposes
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/12—Computing arrangements based on biological models using genetic models
- G06N3/126—Evolutionary algorithms, e.g. genetic algorithms or genetic programming
-
- G—PHYSICS
- G08—SIGNALLING
- G08G—TRAFFIC CONTROL SYSTEMS
- G08G1/00—Traffic control systems for road vehicles
- G08G1/01—Detecting movement of traffic to be counted or controlled
- G08G1/0104—Measuring and analyzing of parameters relative to traffic conditions
- G08G1/0125—Traffic data processing
- G08G1/0129—Traffic data processing for creating historical data or processing based on historical data
-
- G—PHYSICS
- G08—SIGNALLING
- G08G—TRAFFIC CONTROL SYSTEMS
- G08G1/00—Traffic control systems for road vehicles
- G08G1/01—Detecting movement of traffic to be counted or controlled
- G08G1/0104—Measuring and analyzing of parameters relative to traffic conditions
- G08G1/0125—Traffic data processing
- G08G1/0133—Traffic data processing for classifying traffic situation
-
- G—PHYSICS
- G08—SIGNALLING
- G08G—TRAFFIC CONTROL SYSTEMS
- G08G1/00—Traffic control systems for road vehicles
- G08G1/01—Detecting movement of traffic to be counted or controlled
- G08G1/0104—Measuring and analyzing of parameters relative to traffic conditions
- G08G1/0137—Measuring and analyzing of parameters relative to traffic conditions for specific applications
-
- G—PHYSICS
- G08—SIGNALLING
- G08G—TRAFFIC CONTROL SYSTEMS
- G08G1/00—Traffic control systems for road vehicles
- G08G1/01—Detecting movement of traffic to be counted or controlled
- G08G1/042—Detecting movement of traffic to be counted or controlled using inductive or magnetic detectors
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/12—Protocols specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L9/00—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols
- H04L9/06—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols the encryption apparatus using shift registers or memories for block-wise or stream coding, e.g. DES systems or RC4; Hash functions; Pseudorandom sequence generators
- H04L9/0643—Hash functions, e.g. MD5, SHA, HMAC or f9 MAC
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L2209/00—Additional information or applications relating to cryptographic mechanisms or cryptographic arrangements for secret or secure communication H04L9/00
- H04L2209/42—Anonymization, e.g. involving pseudonyms
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Evolutionary Biology (AREA)
- General Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Artificial Intelligence (AREA)
- General Health & Medical Sciences (AREA)
- Evolutionary Computation (AREA)
- Biophysics (AREA)
- Chemical & Material Sciences (AREA)
- Analytical Chemistry (AREA)
- Bioethics (AREA)
- Computer Networks & Wireless Communication (AREA)
- Software Systems (AREA)
- Computer Security & Cryptography (AREA)
- Computing Systems (AREA)
- Medical Informatics (AREA)
- Signal Processing (AREA)
- Biomedical Technology (AREA)
- Mathematical Physics (AREA)
- Databases & Information Systems (AREA)
- Computer Hardware Design (AREA)
- Molecular Biology (AREA)
- Computational Linguistics (AREA)
- Genetics & Genomics (AREA)
- Power Engineering (AREA)
- Physiology (AREA)
- Probability & Statistics with Applications (AREA)
- Traffic Control Systems (AREA)
Abstract
The invention discloses a big data-based intelligent traffic management system, which relates to the technical field of traffic management, and adopts a hash function, in a data acquisition stage, the system acquires personal driving tracks and driver behavior traffic data, the hash function converts original data into hash values with fixed length, so that sensitive information cannot be directly related to specific individuals, a time sequence analysis and ARIMA model is used for predicting future traffic flow, a clustering algorithm and a principal component analysis are used for identifying congestion reasons, a DBSCAN is used for carrying out clustering analysis for identifying congestion road sections, in a road improvement module, time-space data mining and track analysis are used for further identifying the congestion road sections, an optimal road improvement scheme is searched through a genetic algorithm, the traffic fluency is further improved, the congestion phenomenon is reduced, and road planning is further optimized, so that more intelligent and efficient traffic management and service are realized.
Description
Technical Field
The invention relates to the technical field of traffic management, in particular to an intelligent traffic management system based on big data.
Background
Traffic management systems are used to plan, monitor and optimize traffic flows, typically developed and operated by government agencies or traffic departments, with the aim of improving the efficiency, safety and sustainability of traffic transportation.
The big data technology is one of important technical achievements of the rapid development of the current technology, in an intelligent traffic system, the functions of data information acquisition, intelligent service and the like can be integrated into the intelligent traffic service system through the cloud computing and cloud storage technology of big data, the big data technology is an important technology for realizing the accurate analysis of massive traffic data information, and more intelligent and more effective traffic management decisions are made by collecting, processing and analyzing a large amount of traffic data such as traffic flow, vehicle positions, road states, traffic accidents and the like and other related data such as weather information, activity conditions and the like.
However, the conventional traffic management system adopts a centralized data processing and storing manner, which means that a large amount of personal driving tracks and driver behavior data are stored in a database in a centralized manner, so that the potential privacy leakage risk is easily caused, the traffic safety and personal safety are affected, in addition, the data analysis and road improvement in the conventional traffic management system are often limited by the limitation and processing capacity of the data, the future traffic flow is difficult to predict accurately, the congestion cause is difficult to identify, and the accuracy and effect on the road improvement direction are limited, so that a large data-based intelligent traffic management system for effectively protecting the privacy of individuals is needed to solve the problems.
Disclosure of Invention
(one) solving the technical problems
Aiming at the defects of the prior art, the invention provides a large data-based intelligent traffic management system, which solves the problems that the centralized data processing and storage mode in the prior art is easy to be a potential privacy leakage risk, the data analysis and the road improvement are limited by the limitation and the processing capacity of data, and the future traffic flow is difficult to accurately predict and the congestion cause is difficult to identify.
(II) technical scheme
In order to achieve the above object, the present invention provides an intelligent traffic management system based on big data, which is characterized by comprising:
the data acquisition module is used for deploying traffic cameras, sensors and GPS equipment in urban key road sections and traffic junctions and collecting traffic data of traffic flow, speed and road congestion in real time;
the data storage module is used for processing and managing traffic data by adopting a distributed database and a cloud computing technology, anonymizing acquired personal driving track and driver behavior data before data storage, and adding noise into the data by adopting a differential privacy technology;
the data analysis module predicts the future traffic flow by adopting time sequence analysis and ARIMA autoregressive moving average model, clusters by adopting DBSCAN and performs PCA analysis;
and the road improvement module is used for identifying the congested road section by utilizing space-time data mining and track analysis and searching an optimal road improvement scheme by adopting a genetic algorithm.
The invention is further arranged to: the data acquisition module comprises:
the road traffic camera is responsible for collecting traffic flow, vehicle state and running track data;
the traffic sensor is used for detecting the speed of the vehicle, the occupancy of the lane and traffic incidents, and the traffic time comprises traffic incidents and emergencies;
the GPS equipment is used for collecting the position information and the running track of the vehicle;
the invention is further arranged to: the data storage module comprises a distributed database, a data preprocessing module and a cloud computing platform,
a distributed database storing the collected traffic data in a plurality of nodes;
the data preprocessing processing module processes the data, and comprises the following steps:
removing noise data, and processing missing values and abnormal values;
converting data from different sources into a unified format and unit;
carrying out anonymization treatment on personal identity information by adopting a hash function;
the invention is further arranged to: the anonymization processing is carried out by adopting an SHA-256 hash function, and the specific steps are as follows:
let the personal travel track data T rajectory The driver behavior data is D riverBehavior ；
Carrying out SHA-256 hash processing on the personal driving track data to obtain a hash value H (T) rajectory )：H(T rajectory )＝SHA-256(T rajectory )；
Carrying out SHA-256 hash processing on the driver behavior data to obtain a hash value H riverBehavior ：
H(D riverBehavior )＝SHA-256(D riverBehavior )，
Unbinding the obtained hash value with a specific identity, namely not directly relating to a specific individual any more;
storing the anonymized hash value data in a data storage module for subsequent data analysis and mining;
the invention is further arranged to: the step of adding noise to the data by the data storage module is as follows:
determining parameters of differential privacy, including privacy budget ε and sensitivity S ensitivity ；
Noise adding is carried out on the data of each individual;
let the personal travel track data T rajectory The driver behavior data is D riverBehavior 。
Adding noise to the personal travel track data:
adding noise to the driver behavior data:
laplace is Laplacian noise, Δf is sensitivity, and ε is privacy budget;
unbinding the data added with noise with a specific identity;
storing the noise-containing data after noise addition in a data storage module;
the invention is further arranged to: the specific steps of predicting the future traffic flow are as follows:
preparing historical traffic data including traffic flow data over different time periods measured in hours;
performing time sequence analysis on the historical traffic data;
according to the result of time sequence analysis, an ARIMA model is selected, an autocorrelation diagram ACF and a partial autocorrelation diagram PACF are observed to determine the orders of AR and MA, and simultaneously, information criteria AIC and BIC are used for model comparison;
fitting the historical traffic flow data by using the selected ARIMA model to obtain parameters of the model;
predicting the traffic flow of the future time period by using the fitted ARIMA model;
prediction formula of ARIMA model:
F(t)＝μ+∑(Φ i *F(t-i))+∑(θ j *e(t-j))，
wherein F (t) represents a predicted value at a time point t, i.e., future traffic flow, μ represents a constant term, i.e., mean value, Φ i Representing the autoregressive term AR parameter, F (t-i) represents the actual value at the point in time t-i, i.e., the historical traffic flow, θ j Representing MA moving average term, e (t-j) represents fitting residual at time point t-j;
the invention is further arranged to: the clustering is carried out by adopting DBSCAN and PCA analysis is carried out, and the specific steps comprise:
extracting features from traffic data, where the traffic data includes road traffic flow, vehicle speed, vehicle density, weather, events, converting the traffic data into feature vectors;
two features were chosen from the data: vehicle speed S peed And vehicle density D ensity Form feature vector x= [ S ] peed ,D ensity ]；
Adopting DBSCAN cluster to analyze the distance between data points;
calculating the distance between data points by using Euclidean distance measurement, setting a neighborhood radius and a minimum sample number, and clustering the feature vector X according to set parameters;
analyzing different congestion cluster characteristics according to the DBSCAN clustering result, and finding out main factors causing congestion;
the invention is further arranged to: the data mining and track analysis steps include:
denoising and smoothing the track data, and matching the track points of the vehicle with an actual road grid to obtain a running track of the vehicle on a road;
extracting the average speed, the maximum speed, the driving distance and the stay time of each vehicle from the track data;
clustering the track data by using a DBSCAN clustering algorithm, and dividing the track of the vehicle into different clusters, wherein each cluster represents a track set of the vehicle on a congested road section;
calculating a congestion degree index for each track cluster, and quantifying the congestion degree of the road section;
and adopting a congestion delay index for quantization, and adopting a quantization formula:
wherein S is p For average speed, the average speed of all vehicles in the track cluster is represented, S f Is free flow speed, representing roadIs set, the preset speed limit of (a);
and (3) evaluating the congestion degree of the track cluster according to the calculated congestion degree index, and finding out a road section with higher congestion degree, namely a congested road section.
(III) beneficial effects
The invention provides an intelligent traffic management system based on big data. The beneficial effects are as follows:
in the data acquisition stage, the system acquires the personal running track and the driver behavior traffic data, the hash function converts the original data into hash values with fixed length, so that sensitive information cannot be directly related to specific individuals.
By adding certain noise into the original data, the statistical result of the data cannot be affected by individual individuals, so that the privacy of the individuals is protected, the system can add Laplace noise into the personal driving track data and the driver behavior data according to preset privacy budget and sensitivity, the Laplace noise has randomness, meanwhile, the privacy protection intensity of differential privacy is ensured, the data after noise addition still has usability and statistical value, but the original data cannot be accurately restored, and the individual privacy is fully protected.
In the data analysis module, the time sequence analysis and ARIMA model are used for predicting the future traffic flow, the clustering algorithm and the principal component analysis are used for identifying congestion reasons, the DBSCAN is used for carrying out the clustering analysis to identify congestion road sections, then in the road improvement module, the time space data mining and the track analysis are used for further identifying the congestion road sections, finally, the genetic algorithm is used for searching the optimal road improvement scheme, road expansion and intersection optimization can be adopted, the traffic fluency is further improved, the congestion phenomenon is reduced, and the road planning is further optimized, so that more intelligent and efficient traffic management and service are realized.
In conclusion, through anonymization and differential privacy application, the intelligent traffic management system provided by the invention can protect individual privacy information in traffic data, and meanwhile, through a data analysis and road improvement module, a targeted improvement scheme is provided for traffic management departments and decision makers, and the efficiency and the service quality of the traffic system are improved.
The method solves the problems that the centralized data processing and storage mode existing in the prior art center is easy to be a potential privacy leakage risk, the data analysis and the road improvement are limited by the limitation and the processing capacity of the data, and the future traffic flow is difficult to accurately predict and the congestion cause is difficult to identify.
Drawings
FIG. 1 is a block diagram of a big data based intelligent traffic management system according to the present invention.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
Examples
Referring to fig. 1, the present invention provides an intelligent traffic management system based on big data, comprising:
the data acquisition module is used for deploying traffic cameras, sensors and GPS equipment in urban key road sections and traffic junctions and collecting traffic data of traffic flow, speed and road congestion in real time;
the data acquisition module comprises:
the road traffic camera is responsible for collecting traffic flow, vehicle state and running track data;
the traffic sensor is used for detecting the speed of the vehicle, the occupancy of the lane and traffic incidents, and the traffic time comprises traffic incidents and emergencies;
the GPS equipment is used for collecting the position information and the running track of the vehicle;
the data storage module is used for processing and managing traffic data by adopting a distributed database and a cloud computing technology, anonymizing acquired personal driving track and driver behavior data before data storage, and adding noise into the data by adopting a differential privacy technology so that the statistical result of the data cannot be influenced by individual individuals;
the data storage module comprises a distributed database, a data preprocessing module and a cloud computing platform,
a distributed database storing the collected traffic data in a plurality of nodes; disaster tolerance and expandability of data are improved;
by means of cloud computing technology, efficient storage and management of mass data are achieved;
the data preprocessing processing module processes the data, and comprises the following steps:
removing noise data, and processing missing values and abnormal values;
converting data from different sources into a unified format and unit;
carrying out anonymization treatment on personal identity information by adopting a hash function;
selecting SHA-256 hash function to carry out anonymization, and specifically comprises the following steps:
let the personal travel track data T rajectory The driver behavior data is D riverBehavior ；
Carrying out SHA-256 hash processing on the personal driving track data to obtain a hash value H (T) rajectory )：H(T rajectory )＝SHA-256(T rajectory )；
Carrying out SHA-256 hash processing on the driver behavior data to obtain a hash value H riverBehavior ：
H(D riverBehavior )＝SHA-256(D riverBehavior )，
Unbinding the obtained hash value with a specific identity, namely not directly relating to a specific individual any more;
storing the anonymized hash value data in a data storage module for subsequent data analysis and mining, wherein the SHA-256 hash algorithm maps the input data with any length into a fixed-length 256 hash value, the original data cannot be reversely restored from the hash value, meanwhile, the conflict rate of the SHA-256 hash algorithm is very low, different hash values are usually generated by different input data, and sensitive information is converted into irreversible anonymized hash values by carrying out SHA-256 hash processing on personal driving tracks and driver behavior data, so that the privacy of a user is protected;
the step of adding noise to the data by the data storage module is as follows:
determining parameters of differential privacy, including privacy budget ε and sensitivity S ensitivity ；
The privacy budget determines the size of the added noise, smaller privacy budgets increasing the strength of privacy protection;
noise adding is carried out on the data of each individual; the data release result has differential privacy protection, and the personal driving track data is set as T rajectory The driver behavior data is D riverBehavior 。
Adding noise to the personal travel track data:
adding noise to the driver behavior data:
laplace is Laplacian noise, Δf is sensitivity, and ε is privacy budget;
unbinding the data added with noise with a specific identity;
the noise-added noise-containing data are stored in a data storage module and used for subsequent data analysis and mining, and the differential privacy realizes random disturbance of the data by adding noise into the original data, so that the individual privacy is protected;
the data analysis module predicts the future traffic flow by adopting time sequence analysis and ARIMA autoregressive moving average model, clusters by adopting DBSCAN and performs PCA analysis;
the specific steps of predicting the future traffic flow are as follows:
preparing historical traffic data including traffic flow data over different time periods measured in hours;
carrying out time sequence analysis on historical traffic data, wherein whether the data meets the requirement of time sequence or not needs to be checked, and the time sequence data has trend, seasonality and periodicity, and meanwhile, when the time sequence data does not meet the requirement of stationarity, the stabilization treatment needs to be carried out;
according to the result of time sequence analysis, an ARIMA model is selected, an autocorrelation diagram ACF and a partial autocorrelation diagram PACF are observed to determine the orders of AR and MA, and simultaneously, information criteria AIC and BIC are used for model comparison;
fitting the historical traffic flow data by using the selected ARIMA model to obtain parameters of the model;
predicting the traffic flow of the future time period by using the fitted ARIMA model;
prediction formula of ARIMA model:
F(t)＝μ+∑(Φ i *F(t-i))+∑(θ j *e(t-j))，
wherein F (t) represents a predicted value at a time point t, i.e., future traffic flow, μ represents a constant term, i.e., mean value, Φ i Representing the autoregressive term AR parameter, F (t-i) represents the actual value at the point in time t-i, i.e., the historical traffic flow, θ j Representing MA moving average term, e (t-j) represents fitting residual error at time point t-j, wherein average absolute error MAE is adopted to evaluate prediction result to measure prediction accuracy, traffic flow is predicted through time sequence analysis and ARIMA model, future traffic flow expectation is provided for traffic management departments and users, and travel planning and traffic control are convenient to make;
clustering by adopting DBSCAN and PCA analysis, wherein the method comprises the following specific steps:
extracting features from traffic data, wherein the traffic data comprises road traffic flow, vehicle speed, vehicle density, weather and events, and the events comprise holidays, traffic accidents and accidents, and converting the traffic data into feature vectors;
two features were chosen from the data: vehicle speed S peed And vehicle density D ensity Form feature vector x= [ S ] peed ,D ensity ]；
Adopting DBSCAN cluster to analyze the distance between data points;
calculating the distance between data points by using Euclidean distance measurement, setting a neighborhood radius and a minimum sample number, and clustering the feature vector X according to set parameters;
analyzing different congestion cluster characteristics according to the DBSCAN clustering result, and finding out main factors causing congestion;
the DBSCAN clustering algorithm judges whether the data points belong to the same cluster or not by calculating the distance between the data points, and when the congestion cause is identified, the average speed and the average density in the cluster can be calculated to obtain main factors causing congestion by analyzing the characteristics of the congestion cluster;
the road improvement module is used for identifying a congestion road section by utilizing space-time data mining and track analysis and searching an optimal road improvement scheme by adopting a genetic algorithm;
the data mining and track analysis steps include:
denoising and smoothing the track data, and matching the track points of the vehicle with an actual road grid to obtain a running track of the vehicle on a road;
extracting the average speed, the maximum speed, the driving distance and the stay time of each vehicle from the track data;
clustering the track data by using a DBSCAN clustering algorithm, and dividing the track of the vehicle into different clusters, wherein each cluster represents a track set of the vehicle on a congested road section;
calculating a congestion degree index for each track cluster, and quantifying the congestion degree of the road section;
and adopting a congestion delay index for quantization, and adopting a quantization formula:
wherein S is p For average speed, the average speed of all vehicles in the track cluster is represented, S f Is a free flow speed, representing a preset speed limit on the road;
according to the calculated congestion degree index, carrying out congestion degree evaluation on the track cluster, and finding out a road section with higher congestion degree, namely a congested road section;
the specific steps for searching the optimal road improvement scheme by adopting the genetic algorithm comprise:
minimizing the passing time and maximizing the average speed of the vehicle as an optimized objective function;
taking parameters in the road improvement scheme as decision variables, wherein the decision variables comprise the width of a road, the signal period of an intersection and the number of lanes;
initializing a population by using a random method, wherein each individual represents a road improvement scheme which consists of decision variables;
calculating the fitness value of each individual, namely evaluating the quality of each road improvement scheme according to the optimized objective function;
selecting a part of individuals as good individuals by adopting a selection operation according to the fitness value for subsequent crossing and mutation operations;
performing crossover operation on the selected good individuals to generate new individuals;
performing mutation operation on the individuals obtained after crossing, wherein random disturbance can be introduced to increase the diversity of the population;
adding newly generated individuals into the population, and updating the population;
and stopping the condition when the maximum iteration times are reached, and outputting an optimal road improvement scheme obtained by an optimization algorithm.
In combination with the above, in the present application:
the intelligent traffic management system based on big data provided by the invention is characterized in that traffic cameras, sensors and GPS equipment are deployed on key road sections and traffic junctions, real-time traffic data are collected, a distributed database and a cloud computing technology are used for anonymizing and differential privacy protection on the collected traffic data, the collected traffic data are stored in a plurality of nodes, meanwhile, time sequence analysis and ARIMA model are adopted for predicting future traffic flow, DBSCAN is used for clustering and PCA analysis, congestion road sections are identified, finally space-time data mining and track analysis are utilized for identifying the congestion road sections, and then a genetic algorithm is adopted for searching an optimal road improvement scheme.
Wherein anonymization and differential privacy means are employed for individual privacy protection:
the anonymization adopts a hash function, and in a data acquisition stage, the system acquires personal running track and driver behavior traffic data, the hash function converts original data into hash values with fixed length, so that sensitive information cannot be directly related to specific individuals.
The differential privacy is characterized in that certain noise is added to the original data, so that the statistical result of the data cannot be influenced by individual individuals, the individual privacy is protected, the Laplace noise is added to the personal driving track data and the driver behavior data according to preset privacy budget and sensitivity, the Laplace noise has randomness, meanwhile, the privacy protection intensity of the differential privacy is guaranteed, the data after noise addition still has usability and statistical value, but the original data cannot be accurately restored, and therefore the individual privacy is fully protected.
And for road improvement after data analysis:
in the data analysis module, the time sequence analysis and ARIMA model are used for predicting the future traffic flow, the clustering algorithm and the principal component analysis are used for identifying congestion reasons, the DBSCAN is used for carrying out the clustering analysis to identify congestion road sections, then in the road improvement module, the time space data mining and the track analysis are used for further identifying the congestion road sections, finally, the genetic algorithm is used for searching the optimal road improvement scheme, road expansion and intersection optimization can be adopted, the traffic fluency is further improved, the congestion phenomenon is reduced, and the road planning is further optimized, so that more intelligent and efficient traffic management and service are realized.
In conclusion, through anonymization and differential privacy application, the intelligent traffic management system provided by the invention can protect individual privacy information in traffic data, and meanwhile, through a data analysis and road improvement module, a targeted improvement scheme is provided for traffic management departments and decision makers, and the efficiency and the service quality of the traffic system are improved.
In the description of the embodiments of the present invention, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined in the appended claims and their equivalents.
Claims (8)
1. An intelligent traffic management system based on big data, comprising:
the data acquisition module is used for deploying traffic cameras, sensors and GPS equipment in urban key road sections and traffic junctions and collecting traffic data of traffic flow, speed and road congestion in real time;
the data storage module is used for processing and managing traffic data by adopting a distributed database and a cloud computing technology, anonymizing acquired personal driving track and driver behavior data before data storage, and adding noise into the data by adopting a differential privacy technology;
the data analysis module predicts the future traffic flow by adopting time sequence analysis and ARIMA autoregressive moving average model, clusters by adopting DBSCAN and performs PCA analysis;
and the road improvement module is used for identifying the congested road section by utilizing space-time data mining and track analysis and searching an optimal road improvement scheme by adopting a genetic algorithm.
2. The intelligent traffic management system based on big data according to claim 1, wherein the data acquisition module comprises:
the road traffic camera is responsible for collecting traffic flow, vehicle state and running track data;
the traffic sensor is used for detecting the speed of the vehicle, the occupancy of the lane and traffic incidents, and the traffic time comprises traffic incidents and emergencies;
and the GPS equipment is used for collecting the position information and the running track of the vehicle.
3. The intelligent traffic management system based on big data according to claim 1, wherein the data storage module comprises a distributed database, a data preprocessing module and a cloud computing platform,
a distributed database storing the collected traffic data in a plurality of nodes;
the data preprocessing processing module processes the data, and comprises the following steps:
removing noise data, and processing missing values and abnormal values;
converting data from different sources into a unified format and unit;
and anonymizing the personal identity information by adopting a hash function.
4. The intelligent traffic management system based on big data according to claim 3, wherein the anonymizing processing is performed by using SHA-256 hash function, and the specific steps are as follows:
let the personal travel track data T rajectory The driver behavior data is D riverBehavior ；
Carrying out SHA-256 hash processing on the personal driving track data to obtain a hash value H (T) rajectory )：H(T rajectory )＝SHA-256(T rajectory )；
Carrying out SHA-256 hash processing on the driver behavior data to obtain a hash value H riverBehavior ：
H(D riverBehavior )＝SHA-256(D riverBehavior )，
Unbinding the obtained hash value with a specific identity, namely not directly relating to a specific individual any more;
and storing the anonymized hash value data in a data storage module for subsequent data analysis and mining.
5. The intelligent traffic management system according to claim 1, wherein the data storage module adds noise to the data by:
determining parameters of differential privacy, including privacy budget ε and sensitivity S ensitivity ；
Noise adding is carried out on the data of each individual;
let the personal travel track data T rajectory The driver behavior data is D riverBehavior 。
Adding noise to the personal travel track data:
adding noise to the driver behavior data:
laplace is Laplacian noise, Δf is sensitivity, and ε is privacy budget;
unbinding the data added with noise with a specific identity;
and storing the noise-added noise-containing data in a data storage module.
6. The intelligent traffic management system based on big data according to claim 1, wherein the predicting future traffic flow comprises the specific steps of:
preparing historical traffic data including traffic flow data over different time periods measured in hours;
performing time sequence analysis on the historical traffic data;
according to the result of time sequence analysis, an ARIMA model is selected, an autocorrelation diagram ACF and a partial autocorrelation diagram PACF are observed to determine the orders of AR and MA, and simultaneously, information criteria AIC and BIC are used for model comparison;
fitting the historical traffic flow data by using the selected ARIMA model to obtain parameters of the model;
predicting the traffic flow of the future time period by using the fitted ARIMA model;
prediction formula of ARIMA model:
F(t)＝μ+∑(Φ i *F(t-i))+∑(θ j *e(t-j))，
wherein F (t) represents a predicted value at a time point t, i.e., future traffic flow, μ represents a constant term, i.e., mean value, Φ i Representing the autoregressive term AR parameter, F (t-i) represents the actual value at the point in time t-i, i.e., the historical traffic flow, θ j Representing the MA moving average term, e (t-j) represents the fit residual at time point t-j.
7. The intelligent traffic management system based on big data according to claim 1, wherein the clustering using DBSCAN and PCA analysis specifically comprises:
extracting features from traffic data, where the traffic data includes road traffic flow, vehicle speed, vehicle density, weather, events, converting the traffic data into feature vectors;
two features were chosen from the data: vehicle speed S peed And vehicle density D ensity Form feature vector x= [ S ] peed ,D ensity ]；
Adopting DBSCAN cluster to analyze the distance between data points;
calculating the distance between data points by using Euclidean distance measurement, setting a neighborhood radius and a minimum sample number, and clustering the feature vector X according to set parameters;
and analyzing different congestion cluster characteristics according to the DBSCAN clustering result, and finding out main factors causing congestion.
8. The intelligent traffic management system based on big data according to claim 1, wherein the data mining and trajectory analysis step comprises:
denoising and smoothing the track data, and matching the track points of the vehicle with an actual road grid to obtain a running track of the vehicle on a road;
extracting the average speed, the maximum speed, the driving distance and the stay time of each vehicle from the track data;
clustering the track data by using a DBSCAN clustering algorithm, and dividing the track of the vehicle into different clusters, wherein each cluster represents a track set of the vehicle on a congested road section;
calculating a congestion degree index for each track cluster, and quantifying the congestion degree of the road section;
and adopting a congestion delay index for quantization, and adopting a quantization formula:
wherein S is p For average speed, the average speed of all vehicles in the track cluster is represented, S f Is a free flow speed, representing a preset speed limit on the road;
and (3) evaluating the congestion degree of the track cluster according to the calculated congestion degree index, and finding out a road section with higher congestion degree, namely a congested road section.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311104343.8A CN117251722A (en) | 2023-08-30 | 2023-08-30 | Intelligent traffic management system based on big data |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311104343.8A CN117251722A (en) | 2023-08-30 | 2023-08-30 | Intelligent traffic management system based on big data |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117251722A true CN117251722A (en) | 2023-12-19 |
Family
ID=89128540
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311104343.8A Pending CN117251722A (en) | 2023-08-30 | 2023-08-30 | Intelligent traffic management system based on big data |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117251722A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117456737A (en) * | 2023-12-24 | 2024-01-26 | 广东邦盛北斗科技股份公司 | Intelligent traffic big data processing method and system based on 3D visual intelligence |
-
2023
- 2023-08-30 CN CN202311104343.8A patent/CN117251722A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117456737A (en) * | 2023-12-24 | 2024-01-26 | 广东邦盛北斗科技股份公司 | Intelligent traffic big data processing method and system based on 3D visual intelligence |
| CN117456737B (en) * | 2023-12-24 | 2024-03-26 | 广东邦盛北斗科技股份公司 | Intelligent traffic big data processing method and system based on 3D visual intelligence |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Yao et al. | Short‐term traffic speed prediction for an urban corridor | |
| Essien et al. | Improving urban traffic speed prediction using data source fusion and deep learning | |
| CN109145954B (en) | Network taxi appointment travel safety evaluation method and system based on multi-source time-space data | |
| CN103839409B (en) | Based on the traffic flow modes method of discrimination of multibreak facial vision sensing cluster analysis | |
| CN113704956A (en) | Urban road online microscopic simulation method and system based on digital twin technology | |
| Sadek et al. | A prototype case-based reasoning system for real-time freeway traffic routing | |
| US11195412B2 (en) | Predicting short-term traffic flow congestion on urban motorway networks | |
| CN113379099B (en) | Machine learning and copula model-based highway traffic flow self-adaptive prediction method | |
| CN117251722A (en) | Intelligent traffic management system based on big data | |
| Meng et al. | Expressway crash prediction based on traffic big data | |
| CN111815098A (en) | Traffic information processing method and device based on extreme weather, storage medium and electronic equipment | |
| CN116597642A (en) | Traffic jam condition prediction method and system | |
| Zhu et al. | Early identification of recurrent congestion in heterogeneous urban traffic | |
| Li et al. | New model of travel-time prediction considering weather conditions: case study of urban expressway | |
| Thu et al. | Multi-source data analysis for bike sharing systems | |
| Alrassy et al. | Driver behavior indices from large-scale fleet telematics data as surrogate safety measures | |
| CN110264725B (en) | Method and device for determining road section flow | |
| CN114492544B (en) | Model training method and device and traffic incident occurrence probability evaluation method and device | |
| Huang | SVM‐Based Real‐Time Identification Model of Dangerous Traffic Stream State | |
| Kim et al. | A study on travel time estimation of diverging traffic stream on highways based on timestamp data | |
| Rao et al. | Parallel CNN based big data visualization for traffic monitoring | |
| Obayya et al. | Artificial Intelligence for Traffic Prediction and Estimation in Intelligent Cyber-Physical Transportation Systems | |
| Xu et al. | Tcpmodel: A short-term traffic congestion prediction model based on deep learning | |
| Li | A deep learning approach for real-time crash risk prediction at urban arterials | |
| Das et al. | A Comparative Analysis on Traffic Flow Prediction |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |