CN112990002A - Traffic signal lamp identification method and system on downhill road and computer readable medium - Google Patents

Traffic signal lamp identification method and system on downhill road and computer readable medium Download PDF

Info

Publication number
CN112990002A
CN112990002A CN202110267619.9A CN202110267619A CN112990002A CN 112990002 A CN112990002 A CN 112990002A CN 202110267619 A CN202110267619 A CN 202110267619A CN 112990002 A CN112990002 A CN 112990002A
Authority
CN
China
Prior art keywords
training
traffic signal
sample
image
signal lamp
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202110267619.9A
Other languages
Chinese (zh)
Other versions
CN112990002B (en
Inventor
张素民
卢守义
何睿
支永帅
包智鹏
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Jilin University
Original Assignee
Jilin University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Jilin University filed Critical Jilin University
Priority to CN202110267619.9A priority Critical patent/CN112990002B/en
Publication of CN112990002A publication Critical patent/CN112990002A/en
Application granted granted Critical
Publication of CN112990002B publication Critical patent/CN112990002B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
    • G06V20/58—Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
    • G06V20/584—Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads of vehicle lights or traffic lights
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01B—MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
    • G01B7/00—Measuring arrangements characterised by the use of electric or magnetic techniques
    • G01B7/30—Measuring arrangements characterised by the use of electric or magnetic techniques for measuring angles or tapers; for testing the alignment of axes
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G06F18/2411—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on the proximity to a decision surface, e.g. support vector machines
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/50—Extraction of image or video features by performing operations within image blocks; by using histograms, e.g. histogram of oriented gradients [HoG]; by summing image-intensity values; Projection analysis
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/56—Extraction of image or video features relating to colour
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
    • Y02A30/00—Adapting or protecting infrastructure or their operation
    • Y02A30/60—Planning or developing urban green infrastructure

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Multimedia (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Artificial Intelligence (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Evolutionary Biology (AREA)
  • Evolutionary Computation (AREA)
  • General Engineering & Computer Science (AREA)
  • Traffic Control Systems (AREA)
  • Image Analysis (AREA)

Abstract

The invention discloses a traffic signal lamp identification method, a system and a computer readable medium on a downhill road, which adjust the rotation angle of a camera orientation angle according to road gradient information fed back by a gradient sensor; the camera after the steering angle is adjusted shoots the front of the vehicle to form an image, the shot image is transmitted to the detector, the detector detects the image with the traffic signal lamp through the classifier, and further detects the area of the traffic signal lamp in the image; the detector transmits the identified image of the area with the traffic signal lamp to the identifier, and the identifier identifies the state of the traffic signal lamp in the area and outputs the current state of the traffic signal lamp. The traffic signal lamp identification method, the traffic signal lamp identification system and the computer readable medium on the downhill road can quickly, real-timely and accurately identify the traffic signal lamp state on the downhill road without reducing the image processing speed, and are favorable for the safe driving of vehicles.

Description

Traffic signal lamp identification method and system on downhill road and computer readable medium
Technical Field
The invention belongs to the technical field of automobile environment perception, and relates to a traffic signal lamp identification method and system on a downhill road and a computer readable medium.
Background
The mouthfeel of traffic lights on crossroads is known to be a necessary function for complying with traffic regulations and preventing fatal traffic accidents. In the prior art, a vision-based method is mostly adopted, and a camera arranged on a vehicle windshield is used for identifying a traffic signal lamp. These approaches have some limitations in advanced driver assistance systems or autonomous driving systems. First, the camera requires a wide field of view and high resolution, and can photograph and clearly display the traffic signal light when it is far or near the traffic signal light. These requirements increase the amount of information for image processing and require a large amount of computing power. Secondly, as shown in fig. 1, since the installation position of the camera on the vehicle does not change and the field of view of the camera is fixed, when the vehicle is in a state of a downhill slope, it is difficult for the vehicle to recognize a traffic signal on a flat ground near the end of the slope.
In view of the above problems, it is desirable to provide a system, a method and a computer readable medium for rapidly, real-timely and accurately identifying the state of traffic lights on a downhill without reducing the image processing speed.
Disclosure of Invention
In order to achieve the above purpose, the invention provides a traffic signal lamp recognition method, a system and a computer readable medium on a downhill road, which utilize a normal view field camera, a slope sensor and a camera angle regulator to form a traffic signal lamp recognition system on the downhill road, can quickly, real-timely and accurately recognize the traffic signal lamp state on the downhill road without reducing the image processing speed, is beneficial to the safe driving of vehicles, particularly advanced auxiliary driving vehicles or automatic driving vehicles, and solves the problem that the traffic signal lamp on the downhill road is difficult to accurately recognize in the prior art.
The invention adopts the technical scheme that the traffic signal lamp identification method on the downhill comprises the following steps:
s10, adjusting the rotation angle of the orientation angle of the camera according to the road gradient information fed back by the gradient sensor;
s20, shooting the front of the vehicle by the camera after the steering angle is adjusted to form an image, transmitting the shot image to a detector, detecting the image with the traffic signal lamp by the detector through a classifier, and further detecting the area of the traffic signal lamp in the image;
and S30, the detector transmits the image of the area with the traffic signal lamp to the recognizer, and the recognizer recognizes the state of the traffic signal lamp in the area and outputs the current state of the traffic signal lamp.
Further, in S10, adjusting the rotation angle of the camera toward the angle according to the road gradient information fed back by the gradient sensor includes the following steps:
s11, firstly, using the finite impulse response filter to process the low-pass filtering signal of the voltage signal, the gradient sensor outputs the voltage value X according to the current stateoutAnd the voltage value X output when the vehicle is running under the horizontal road condition staterefBy α ═ arcsin [ (X)out-Xref)/0.8]Calculating to obtain a road slope angle alpha when the vehicle is running at the current state;
s12, the microprocessor receives the road slope angle alpha transmitted by the slope sensor when the vehicle is running in the current state, calculates the angle beta of the camera angle adjusting device which should rotate according to the angle beta being 1.2 alpha, and converts the angle beta value which should rotate into a corresponding PWM signal value;
s13, the camera angle adjusting device is composed of a steering engine and a camera fixing device, a camera is fixedly arranged in the camera fixing device, the steering engine receives a PWM signal value transmitted by the micro processor, the rotating angle of the steering engine is adjusted according to the pulse width duration of the PWM signal value, and then the rotating angle of the orientation angle of the camera is adjusted.
Further, in S20, the detector detects the image with the traffic light through the classifier, and further detects the area of the traffic light in the image, specifically: the method comprises the following steps of detecting an image with a traffic signal lamp by adopting an AdaBoost classifier based on Haar-like characteristics, and further detecting the area of the traffic signal lamp in the image, wherein the AdaBoost classifier comprises two processes: a training process and a recognition process;
in the training process, on one hand, Haar-like features which play a key role in classification and identification are selected from a large number of Haar-like features, and on the other hand, an AdaBoost classifier which accurately identifies a traffic signal light region and a non-traffic signal light region is trained to prepare for the identification process;
in the identification process, the key Haar features extracted in the training process and the trained AdaBoost classifier are used for identifying the images transmitted by the camera, and the images with the traffic signal lamps and the areas of the traffic signal lamps in the images are identified;
the identification process specifically comprises the following steps: and (3) classifying and identifying whether an area with a traffic signal lamp exists in the image transmitted by the camera: comprises the steps of image preprocessing; calculating an integral graph; calculating corresponding Haar-like characteristic values by utilizing the Haar-like characteristic information selected in the training process to form characteristic vectors; and detecting whether the area with the traffic signal lamp exists in the image transmitted by the camera by using the AdaBoost classifier trained in the training process and the obtained feature vector, and outputting a final classification recognition result.
Further, the training process includes four training sub-processes: preprocessing an image; calculating an integral graph; extracting Haar-like features; training an AdaBoost classifier; wherein the content of the first and second substances,
image preprocessing: the shot image is divided into two parts: one part is an image with a traffic light and is used as a traffic light training sample; the other part is an image without a traffic light and is used as a non-traffic light training sample; normalizing all traffic signal lamp training samples and non-traffic signal lamp training samples into 128 multiplied by 128 gray level images as normalized traffic signal lamp training samples and normalized non-traffic signal lamp training samples;
and (3) calculating an integral graph: respectively calculating a forward integral image and an oblique integral image of each normalized traffic signal lamp training sample and each normalized non-traffic signal lamp training sample;
extracting Haar-like features: the expanded Haar-like features are divided into 4 categories and 15 categories, the size of each Haar-like feature is set to be 2 multiplied by 1, and according to a forward integral image and an oblique integral image of each normalized traffic signal lamp training sample and each normalized non-traffic signal lamp training sample, 15 Haar-like features with different sizes of each normalized traffic signal lamp training sample and each normalized non-traffic signal lamp training sample image are calculated and extracted and used for describing the edge and structural features of the traffic signal lamp;
training an AdaBoost classifier, which comprises the steps of constructing a weak classifier, constructing a strong classifier and constructing a cascade classifier:
constructing a weak classifier: selecting one type of Haar features from 15 types of Haar features as input, and obtaining a weak classifier h of the Haar features through trainingj(x) Each weak classifier hj(x) Are formed by the selected Haar-like eigenvalues and a threshold value of the form shown below:
Figure BDA0002972567250000031
wherein x represents a sample, fj(x) Represents the corresponding characteristic value, p, of the jth Haar-like characteristic on the sample xjDetermining the positive or negative of the inequality, thetajRepresentation classifier hj(x) A threshold corresponding to the Haar-like eigenvalue in (1);
constructing a strong classifier: training with weak classifiers for many times, continuously increasing the weight of the misclassified samples in each training, continuously training, selecting the optimal weak classifier from each training, and linearly combining into a strong classifier with better classification capability;
constructing a cascade classifier: adopting a cascade method, separating out negative samples by each stage of strong classifier, entering the rest samples into the next stage of strong classifier, and obtaining positive samples through the prediction of the last strong classifier; and training a sample set based on the constructed cascade classifier so as to obtain features which play a key role in classification and identification and a final classifier.
Further, the specific training process for constructing the strong classifier is as follows:
selecting a training set with n samples, wherein a traffic signal training samples and b non-traffic signal training samples are selected, n is a + b, and the sample set is (x)1,y1),(x2,y2),...,(xn,yn)，xiFor the ith sample, yiTo illustrate xiWhether it is a traffic signal light training sample or a non-traffic signal light training sample, when yiWhen 0, then xiNon-traffic light training samples; when y isiWhen 1, then xiIs a traffic signal light training sample;
initializing sample weights: the weights of all traffic light training samples are 1/2a, and the weights of all non-traffic light training samples are 1/2 b;
for training times T equal to 1, …, T, the process of training to obtain the optimal weak classifier is:
step a1, normalizing the weight of all samples in the round:
Figure BDA0002972567250000041
in the formula, Wt,iRepresents the normalized weight of the ith sample in the t training, wt,iRepresents the weight of the ith sample in the t training, wt,jRepresenting the weight of a jth sample in the tth training, wherein j represents the jth sample, and n represents the total number of samples;
step a2, training weak classifiers h according to each Haar-like feature jjClassifying whether the samples in the sample set belong to traffic signal lamp training samples, and calculating the error of the sample set classification:
Figure BDA0002972567250000042
in the formula, epsilonjError, h, representing sample set classificationj(xi) Is a weak classifier hjFor sample xiAnd (5) classifying results.
Step a3, selecting the optimal weak classifier from each training; updating the weight: if the sample is classified correctly, then
Figure BDA0002972567250000043
εtRepresenting the error rate of the t-th training, W if the sample is misclassifiedt+1,i＝Wt,i(ii) a Linearly combining the T optimal weak classifiers into a strong classifier, wherein the formula of the strong classifier is shown as the following formula:
Figure BDA0002972567250000044
in the formula, atRepresents the weight of the weak classifier trained for the t-th time, ht(x) Represents the optimal weak classifier selected in the t training,
Figure BDA0002972567250000045
further, in S30, the identifier identifies the state of the traffic light in the area, specifically:
identifying the state of the image in the area with the traffic light by a classifier, wherein the classifier adopts two characteristics of an orientation gradient histogram HOG and a bulb HSV color histogram, and integrates and trains the two characteristics of the orientation gradient histogram HOG and the bulb HSV color histogram by a support vector machine;
in the training process, firstly, determining the parameter gamma and the penalty factor C of the optimal radial basis kernel function by a grid search method, then, searching a support vector set through the learning of a training set, and preliminarily determining the Lagrangian multiplier aiAnd b, adjusting and optimizing the obtained relevant parameters of the support vector machine model by using a test set, and finally determining the parameters of a decision function of the optimal classification surface of the support vector machine model;
after the training of the support vector machine model is completed, each image transmitted by the detector is brought into a decision function of each support vector machine model, and finally the classification category of the image is determined from the result of the support vector machine model through a voting method, so that the state of the traffic light is determined.
Furthermore, a support vector machine is used for integrating and training two characteristics of the histogram of oriented gradient HOG and the histogram of color of the lamp bulb HSV, and the method specifically comprises the following steps:
step b 1: normalizing the sample image to 48 × 48 pixel size, selecting the histogram of oriented gradient HOG and the histogram of color of lamp bulb HSV of the normalized sample image as the characteristic value of the normalized sample image, and forming a characteristic value vector x ═ p1,p2) X denotes a vector of eigenvalues, p1Representing oriented gradient histogram feature value, p2Representing the characteristic value of the HSV color histogram, and setting the characteristic value vector x as (p)1,p2) Taking the state y of the traffic signal lamp as an input, and forming a traffic signal lamp state sample set;
in the extraction of the directional gradient histogram of the image, selecting cell units with the size of 6 multiplied by 6 pixels, wherein each cell unit is 9 bins, taking the rectangular interval Block as 2 multiplied by 2, extracting the directional gradient histogram of the image, and then performing dimensionality reduction on data by using a principal component analysis method to obtain a final value;
the states of the traffic signal lamp are three types: the method comprises the following steps of extracting 4/5 from 3 samples corresponding to the state y of the traffic signal lamp as a training set and using the rest 1/5 as a test set, wherein the red lamp, the green lamp and the yellow lamp respectively correspond to y being 1, y being 2 and y being 3;
step b 2: determining the number of Support Vector Machines (SVM): 3 classifiers are constructed, and respectively: a red-green support vector machine, wherein the sample set is taken from a sample set of bright red lights and bright green lights, and if the sample set is taken from the bright red light sample, y is 1; taking a sample with a bright green light, and then y is-1; a red-yellow support vector machine, wherein the sample set is taken from a sample set with bright red light and bright yellow light, and if the sample set is taken from a sample with bright red light, y is 1; taking a sample with yellow light, and then y is-1; a green-yellow support vector machine, wherein the sample set is taken from a sample set of green light and yellow light, and if the sample set is taken from a sample of green light, y is 1; taking a sample of the yellow lamp, and training the three support vector machines respectively if y is equal to-1;
step b 3: training a support vector machine: determining initial values of relevant parameters of a decision function of the optimal classification surface of the sample set through a training set; the relevant parameters comprise optimal kernel function parameters and penalty factors; determining the optimal kernel function parameter by adopting a grid search method; selecting a radial basis kernel function K (x)i,x)＝exp(-γ||xi-x||2) The method is used as an optimal kernel function to realize the mapping of a support vector machine from an original space to a feature space, wherein gamma is a kernel parameter and a relaxation factor xi is introducediAnd a penalty factor C, namely solving the problem of the optimal classification surface equivalently to solve a convex quadratic programming problem shown as the following formula:
Figure BDA0002972567250000061
wherein R (w) represents an objective function, w is a normal vector of an optimal classification surface, C represents a penalty factor, and xiiRepresenting the relaxation factor, b is the offset coefficient, N is the number of samples, xiAs sample feature vector, yiThe traffic signal lamp state corresponding to the sample is obtained;
and (3) applying a Lagrange multiplier method and considering that the Karush-Kuhn-Tucker condition is satisfied, and obtaining a decision function of the optimal classification surface as follows:
Figure BDA0002972567250000062
wherein f (x) represents the decision function of the optimal classification surface, Sgn is the sign function, aiAnd (4) corresponding to each sample, wherein x is a target characteristic vector, and SV represents a support vector.
The method further comprises the step of inputting the image determined to have the traffic signal lamp by the step S20 into a pre-trained neural network for monocular depth estimation, wherein the depth corresponding to each pixel in the image is output, the depth information l of the traffic signal lamp at the position of the vehicle at the current moment is obtained by combining the area of the traffic signal lamp in the image output by the detector S20 through a method of averaging the depths of all pixels in the area, and the distance S from the vehicle to the position of the traffic signal lamp at the current moment is obtained according to the road slope angle alpha obtained by the step S10 and the S-l/cos alpha.
Another object of the present invention is to provide a traffic light recognition system on a downhill road, including:
a memory for storing instructions executable by the processor;
a processor for executing the instructions to implement the method as described above.
It is a further object of the invention to provide a computer readable medium having stored computer program code which, when executed by a processor, implements a method as described above.
The invention has the beneficial effects that:
(1) this patent proposes a traffic signal lamp identification system on downhill path that utilizes normal visual field camera and slope sensor and camera angle regulator to constitute. Compared with a wide view field camera, the normal view field camera has small image information amount, the processing speed of the algorithm on the image is increased, and the camera angle adjuster can adjust the angle of the camera according to the road gradient information fed back by the gradient sensor, so that vehicles on an inclined road can detect traffic lights on a flat ground at the end of the inclined road. The traffic signal lamp identification method on the downhill does not reduce the image processing speed, can quickly, real-timely and accurately identify the traffic signal lamp state on the downhill, and is favorable for the safe driving of vehicles, particularly advanced auxiliary driving vehicles or automatic driving vehicles.
(2) The method for adjusting the camera observation angle by utilizing the slope angle enables a vehicle to accurately observe a traffic signal lamp on a downhill road, an AdaBoost classifier based on a Haar-like feature is used for detecting the area of the traffic signal lamp in an image, and a Support Vector Machine (SVM) is used for integrating and training the two features to identify the state of the traffic signal lamp based on the two features of an oriented gradient histogram and a bulb HSV color histogram. The two methods have the advantages of higher processing speed, better reliability and stronger generalization capability.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.
FIG. 1 is a schematic diagram of a scenario in which a prior art vehicle has limitations on signal light identification on a downhill slope.
Fig. 2 is a schematic structural diagram of a traffic light recognition system on a downhill road according to an embodiment of the present invention.
Fig. 3 is a schematic diagram of traffic light region division in the embodiment of the present invention.
Fig. 4 is a schematic structural diagram of an algorithm for training an AdaBoost classifier in the embodiment of the present invention.
FIG. 5 is a schematic illustration of 15 extended Haar-like features employed in embodiments of the present invention.
FIG. 6 is a schematic diagram of a cascade classifier according to an embodiment of the present invention.
Fig. 7 is a flowchart illustrating a process of classifying and identifying whether there is an area with traffic lights in an image transmitted from a camera according to an embodiment of the present invention.
FIG. 8 is a flow chart of a process for an identifier to identify the status of traffic lights in an area in accordance with an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
A traffic light identification method on a downhill road, as shown in fig. 2, includes the steps of:
and S10, adjusting the rotation angle of the orientation angle of the camera according to the road gradient information fed back by the gradient sensor.
And S11, acquiring the road slope angle alpha when the vehicle is driven at the current state through the relation between voltage signals output by the slope sensor when the vehicle is driven at the current state and when the vehicle is driven at the horizontal road condition state.
Since the voltage signal output by the gradient sensor is accompanied by great noise and is difficult to be directly used for gradient acquisition, the voltage signal output by the gradient sensor is firstly subjected to low-pass filtering signal processing before being used for gradient acquisition.
The gradient sensor outputs a voltage value X according to the current state when drivingoutAnd the voltage value X output when the vehicle is running under the horizontal road condition staterefCalculating the road slope angle alpha when the vehicle is running at the current state through a formula (1):
α＝arcsin[(Xout-Xref)/0.8] (1)。
s12, the microprocessor receives the road slope angle alpha transmitted by the slope sensor when driving in the current state, calculates the angle beta of the camera angle adjusting device which should rotate according to the formula (2), and converts the angle beta value which should rotate into a corresponding PWM signal value:
β＝1.2α (2)。
s13, the camera angle adjusting device is composed of a steering engine and a camera fixing device, a camera is fixedly arranged in the camera fixing device, the steering engine receives a PWM signal value transmitted by the micro processor, the rotating angle of the steering engine is adjusted according to the pulse width duration of the PWM signal value, and then the rotating angle of the orientation angle of the camera is adjusted.
Because the relation between the rotation angle of different steering engines and the pulse width and time length of the PWM signal value is different, the PWM signal value obtained by the beta value does not have a fixed formula and is determined according to the model of the steering engine, the content belongs to the common knowledge, and the invention is not repeated.
S20, the camera adjusted to the steering angle shoots the front of the vehicle to form an image, the shot image is transmitted to the detector, the detector detects the image with the traffic signal lamp through the classifier, and further detects the area of the traffic signal lamp in the image.
The specific structure of the classifier is not limited by the invention, as long as the image with the traffic signal lamp can be detected, and the area of the traffic signal lamp in the image can be further detected. In a preferred embodiment, an AdaBoost classifier based on Haar-like features is used to detect images with traffic lights and further detect the area of the traffic lights in the images. The division of the traffic signal in the image is shown in fig. 3, and the area of each traffic signal has four angular distances of L/2 in addition to the traffic signal, where L is the height of the traffic signal box.
The use of an AdaBoost classifier based on Haar-like features to detect images with traffic signals and further detect the area of the traffic signal in the image involves two processes: a training process and a recognition process.
In the training process, on one hand, Haar-like features which play a key role in classification and identification are selected from a large number of Haar-like features, and on the other hand, an AdaBoost classifier which can accurately identify a traffic signal lamp region and a non-traffic signal lamp region is trained to prepare for the next identification process.
In the identification process, the key Haar features extracted in the training process and the trained AdaBoost classifier are used for identifying the images transmitted by the camera, and the images with the traffic lights and the areas of the traffic lights in the images are identified.
As shown in fig. 4, the training process includes the following four training sub-processes, including:
preprocessing an image; calculating an integral graph; extracting Haar-like features; an AdaBoost classifier is trained.
Image preprocessing:
the shot image is divided into two parts: one part is an image with a traffic light and is used as a traffic light training sample; the other part is an image without a traffic light and is used as a non-traffic light training sample;
all traffic signal training samples and non-traffic signal training samples are normalized into 128 x 128 gray-scale images as normalized traffic signal training samples and normalized non-traffic signal training samples.
And (3) calculating an integral graph:
and respectively calculating a forward integral image and an oblique integral image of each normalized traffic signal lamp training sample and each normalized non-traffic signal lamp training sample.
Extracting Haar-like features: as shown in fig. 5, the expanded Haar-like features of the present invention are divided into 4 categories and 15 categories, the size of each Haar-like feature is set to 2 × 1, and according to the calculated forward integral map and the oblique integral map of each normalized traffic signal training sample and normalized non-traffic signal training sample, 15 different sizes of Haar-like features of each normalized traffic signal training sample and normalized non-traffic signal training sample image are calculated and extracted for describing the edge and structural features of the traffic signal.
Training an AdaBoost classifier:
the process of training the AdaBoost classifier includes: constructing a weak classifier; constructing a strong classifier; and constructing a cascade classifier.
Constructing a weak classifier: selecting a certain class of Haar features from the 15 classes of Haar features as input, and obtaining the weak classifier h of the Haar features through trainingj(x) Each weak classifier hj(x) Are formed by the selected Haar-like eigenvalues and a threshold, which is of the form shown in equation (3):
Figure BDA0002972567250000091
wherein x represents a sample, fj(x) Represents the corresponding characteristic value, p, of the jth Haar-like characteristic on the sample xjDetermine the positive of the inequalityNegative, thetajRepresentation classifier hj(x) The Haar-like characteristic value in (1) is obtained.
Constructing a strong classifier: training with weak classifiers for many times, continuously increasing the weight of the misclassified samples in each training, reducing the weight of the correctly classified samples, enabling the misclassified samples in the weak classifiers to be trained in the following training in a repeated manner, improving the training accuracy, continuing the training, selecting the optimal weak classifier (the weak classifier with the minimum error in the training process) from each training, linearly combining into the strong classifier with better classification capability, and specifically training in the following steps:
selecting a training set with n samples, wherein a traffic signal training samples and b non-traffic signal training samples are selected, n is a + b, and the sample set is (x)1,y1),(x2,y2),...,(xn,yn)，xiFor the ith sample, yiTo illustrate xiWhether it is a traffic signal light training sample or a non-traffic signal light training sample, when yiWhen 0, then xiNon-traffic light training samples; when y isiWhen 1, then xiIs a traffic light training sample.
Initializing sample weights: all traffic light training samples were weighted 1/2a and all non-traffic light training samples were weighted 1/2 b.
For training times T equal to 1, …, T, the process of training to obtain the optimal weak classifier is:
step a1, normalizing the weights of all samples in the current round (so that the sum of the weights of all samples in the current round is 1):
Figure BDA0002972567250000101
in the formula, Wt,iRepresents the normalized weight of the ith sample in the t training, wt,iRepresents the weight of the ith sample in the t training, wt,jRepresents the weight of the jth sample in the t training, j represents the jth sample, n represents the sampleIn total, n in the present invention is 15.
Step a2, training weak classifiers h according to each Haar-like feature jjClassifying whether the samples in the sample set belong to traffic signal lamp training samples, and calculating the error of the sample set classification:
Figure BDA0002972567250000102
in the formula, epsilonjError, h, representing sample set classificationj(xi) Is a weak classifier hjFor sample xiAnd (5) classifying results.
A3, selecting the optimal (i.e. the error is minimum) weak classifier from each training; updating the weight: if the sample is classified correctly, then
Figure BDA0002972567250000103
εtRepresenting the error rate of the t-th training, W if the sample is misclassifiedt+1,i＝Wt,i(ii) a Linearly combining the T optimal weak classifiers into a strong classifier, wherein the formula of the strong classifier is shown as the following formula:
Figure BDA0002972567250000104
in the formula, atRepresents the weight of the weak classifier trained for the t-th time, ht(x) Represents the optimal weak classifier selected in the t training,
Figure BDA0002972567250000111
constructing a cascade classifier: the cascade method is a method of connecting a plurality of strong classifiers in series from small to large according to the classification capability, as shown in fig. 6, each stage of strong classifier separates out negative samples, the rest of samples enter the next stage of strong classifier, and positive samples are obtained through the prediction of the last strong classifier.
Based on the cascade classifier constructed above, the sample set is trained, so as to obtain the features which play a key role in classification and identification and the final classifier.
Whether the images transmitted by the camera have the traffic signal lamps or not is classified and identified; the identification process is as follows: as shown in fig. 7, the classifying and identifying whether the area with the traffic signal lamp exists in the image transmitted by the camera includes image preprocessing, integral graph calculation, Haar-like feature extraction and classifying and identifying by applying the trained AdaBoost classifier;
wherein the image preprocessing and the integral graph calculation are similar to the training process; extracting Haar-like features, namely calculating corresponding Haar-like feature values by using Haar-like feature information selected in a training process, including position, structure and type information, to form feature vectors; and detecting whether the image transmitted by the camera has the area with the traffic signal lamp or not by using the AdaBoost classifier trained in the training process and the obtained feature vector, and outputting a final classification and identification result.
And S30, the detector transmits the image of the area with the traffic signal lamp to the recognizer, and the recognizer recognizes the state of the traffic signal lamp in the area and outputs the current state of the traffic signal lamp.
The invention does not limit the specific structure of the recognizer, as long as the state of the traffic signal lamp in the area can be recognized and the current state of the traffic signal lamp is output. In a preferred embodiment, the identifier identifies the state of the image of the traffic light area identified in the previous step by a classifier. The classifier adopts two characteristics of an orientation gradient histogram and a bulb HSV color histogram and integrates and trains the two characteristics by a Support Vector Machine (SVM). As shown in fig. 8, the training steps using the support vector machine are as follows:
step b 1: because the Support Vector Machine (SVM) is a two-class classifier, the traffic signal lamp state classification is a three-class classification problem, and the support vector machine is expanded to solve the three-class classification problem of the traffic signal lamp by adopting a 1-a-1 algorithm.
Normalizing the sample image to 48 x 48 pixel size, optionally normalizingUsing the Histogram of Oriented Gradient (HOG) and the HSV color histogram of the normalized sample image as the characteristic value of the normalized sample image, and forming a characteristic value vector x ═ p1,p2) X denotes a vector of eigenvalues, p1Representing oriented gradient histogram feature value, p2Representing the characteristic value of the HSV color histogram, and setting the characteristic value vector x as (p)1,p2) As an input, the state y of the traffic light is taken as an output, constituting a traffic light state sample set.
In the extraction of the oriented gradient histogram of the image, cell units with the size of 6 x 6 pixels are selected, each cell unit is 9 bins, the rectangular interval Block is 2 x 2, the oriented gradient histogram of the image is extracted, and then the principal component analysis method is used for reducing the dimension of the data to obtain a final value.
The states of the traffic signal lamp are three types: the red light, the green light and the yellow light respectively correspond to y being 1, y being 2 and y being 3, 4/5 are extracted from 3 samples corresponding to the state y of the traffic signal light respectively to be used as a training set, and the rest 1/5 are used as a test set.
Step b 2: determining the number of Support Vector Machines (SVMs): 3 classifiers are constructed, and respectively: a red-green (1-2) support vector machine, the sample sets being taken from the sample sets of red light and green light, if taken from the sample of red light, y being 1; if taken from a sample with a bright green light, y is-1; a red-yellow (1-3) support vector machine, wherein the sample set is taken from a sample set of red light and yellow light, and if the sample set is taken from the sample of red light, y is 1; if the sample is taken from a sample with yellow light on, y is-1; a green-yellow (2-3) support vector machine, the sample set is taken from a sample set of green light and yellow light, if taken from a sample of green light, y is 1; if taken from a sample with a yellow light on, y is-1. The three support vector machines are trained separately.
Step b 3: training a support vector machine: determining initial values of relevant parameters of a decision function of the optimal classification surface of the sample set through a training set; the relevant parameters comprise optimal kernel function parameters and penalty factors; determining the optimal kernel function parameter by adopting a grid search method; the invention selects a radial basis kernel function K (x)i,x)＝exp(-γ||xi-x||2) The method is used as an optimal kernel function to realize the mapping of a support vector machine from an original space to a feature space, wherein gamma is a kernel parameter and a relaxation factor xi is introducediAnd a penalty factor C, namely solving the problem of the optimal classification surface equivalently to solving the convex quadratic programming problem of the following formula:
Figure BDA0002972567250000121
wherein R (w) represents an objective function, w is a normal vector of an optimal classification surface, C represents a penalty factor, and xiiRepresenting the relaxation factor, b is the offset coefficient, N is the number of samples, xiAs sample feature vector, yiAnd the traffic light state corresponding to the sample is obtained.
Applying a Lagrange multiplier method and considering that the Karush-Kuhn-Tucker condition is satisfied, the decision function for obtaining the optimal classification surface is as follows:
Figure BDA0002972567250000122
wherein f (x) represents the decision function of the optimal classification surface, Sgn is the sign function, aiAnd (4) corresponding to each sample, wherein x is a target characteristic vector, and SV represents a support vector.
In the training process, firstly, determining the parameter gamma and the penalty factor C of the optimal radial basis kernel function by a grid search method, then, searching a support vector set through the learning of a training set, and preliminarily determining the Lagrangian multiplier aiAnd adjusting and optimizing the obtained relevant parameters of the support vector machine model by using the test set, and finally determining the parameters of the decision function of the optimal classification surface of the support vector machine model.
After the training of the support vector machine model is completed, the support vector machine model is applied to the model of the invention, each image transmitted by the detector is brought into a decision function of each support vector machine model, and finally, the classification category of the image is determined from the results of the support vector machine models through a voting method, so that the state of the traffic light is determined.
The method can further comprise the step of inputting the image determined to be provided with the traffic signal lamp by the S20 into a pre-trained neural network for monocular depth estimation, the depth corresponding to each pixel in the image is output, the depth information l of the traffic signal lamp at the current vehicle position is obtained by combining the area of the traffic signal lamp in the image output by the S20 detector and the depth information l of the traffic signal lamp at the current vehicle position by a method of averaging the depths of all pixels in the area, and the distance S from the vehicle to the traffic signal lamp position at the current time is obtained by combining the road slope angle alpha obtained by the S10 and according to the S l/cos alpha.
The pre-trained neural network for monocular depth estimation is mature prior art, and can be realized as long as the neural network has the monocular depth estimation function, which is not described in detail in the application.
Traffic signal lamp identification system includes following three modules on the downhill path:
(1) camera angle adjustment module: as shown in fig. 2, the camera angle adjustment module includes a slope sensor, a finite impulse response filter, a microprocessor, and a camera angle adjustment device.
The gradient sensor is arranged between the primary and secondary driver seats (close to the mass center of the automobile), and the positive direction of the X axis of the gradient sensor is consistent with the positive direction of the X axis of the vehicle coordinate system; the camera angle adjusting device comprises a steering engine and a camera fixing device.
The slope sensor transmits the acquired road slope alpha to the microprocessor, the microprocessor receives road slope information in the current driving state, calculates the angle beta of the camera angle adjusting device which should rotate, converts the angle beta value which should rotate into a corresponding PWM signal value, the steering engine receives the PWM signal value transmitted by the microprocessor, adjusts the rotation angle of the steering engine according to the pulse width duration of the PWM signal value, and then drives the adjustment of the rotation angle of the orientation angle of the camera.
(2) A detector module: the system is used for receiving an image formed by shooting the front of the vehicle by the camera after the steering angle of the heading angle is adjusted, detecting the image with the traffic signal lamp through the classifier, and further detecting the area of the traffic signal lamp in the image.
(3) An identifier module: the traffic signal lamp state recognition device is used for receiving the image of the area with the traffic signal lamp, recognized by the detector, recognizing the state of the traffic signal lamp in the area and outputting the current state of the traffic signal lamp.
The traffic light identification system on the downhill road can be implemented as a computer program, stored in a hard disk, and recorded in a processor for execution so as to implement the method of the embodiment of the present invention.
An embodiment of the present invention further provides a computer readable medium storing computer program code, which when executed by a processor implements the traffic light identification method on a downhill road.
The traffic light identification method on a downhill road may also be stored in a computer-readable storage medium as an article of manufacture when implemented as a computer program. For example, computer-readable storage media can include but are not limited to magnetic storage devices (e.g., hard disk, floppy disk, magnetic strips), optical disks (e.g., Compact Disk (CD), Digital Versatile Disk (DVD)), smart cards, and flash memory devices (e.g., electrically Erasable Programmable Read Only Memory (EPROM), card, stick, key drive). In addition, various storage media described herein as embodiments of the invention can represent one or more devices and/or other machine-readable media for storing information. The term "machine-readable medium" can include, without being limited to, wireless channels and various other media (and/or storage media) capable of storing, containing, and/or carrying code and/or instructions and/or data.
It should be understood that the above-described embodiments are illustrative only. The described embodiments of the invention may be implemented in hardware, software, firmware, middleware, microcode, or any combination thereof. For a hardware implementation, the processing units may be implemented within one or more Application Specific Integrated Circuits (ASICs), Digital Signal Processors (DSPs), Digital Signal Processing Devices (DSPDs), Programmable Logic Devices (PLDs), Field Programmable Gate Arrays (FPGAs), processors, controllers, micro-controllers, microprocessors, and/or other electronic units designed to perform the functions described herein, or a combination thereof.
It is noted that, in the present application, relational terms such as first, second, and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.
All the embodiments in the present specification are described in a related manner, and the same and similar parts among the embodiments may be referred to each other, and each embodiment focuses on the differences from the other embodiments.
The above description is only for the preferred embodiment of the present invention, and is not intended to limit the scope of the present invention. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention shall fall within the protection scope of the present invention.

Claims (10)

1. A traffic signal lamp identification method on a downhill road is characterized by comprising the following steps:
s10, adjusting the rotation angle of the orientation angle of the camera according to the road gradient information fed back by the gradient sensor;
s20, shooting the front of the vehicle by the camera after the steering angle is adjusted to form an image, transmitting the shot image to a detector, detecting the image with the traffic signal lamp by the detector through a classifier, and further detecting the area of the traffic signal lamp in the image;
and S30, the detector transmits the image of the area with the traffic signal lamp to the recognizer, and the recognizer recognizes the state of the traffic signal lamp in the area and outputs the current state of the traffic signal lamp.
2. The method for identifying traffic lights on a downhill road according to claim 1, wherein in S10, the adjusting the rotation angle of the camera orientation angle according to the road gradient information fed back by the gradient sensor comprises the following steps:
s11, firstly, using the finite impulse response filter to process the low-pass filtering signal of the voltage signal, the gradient sensor outputs the voltage value X according to the current stateoutAnd the voltage value X output when the vehicle is running under the horizontal road condition staterefBy α ═ arcsin [ (X)out-Xref)/0.8]Calculating to obtain a road slope angle alpha when the vehicle is running at the current state;
s12, the microprocessor receives the road slope angle alpha transmitted by the slope sensor when the vehicle is running in the current state, calculates the angle beta of the camera angle adjusting device which should rotate according to the angle beta being 1.2 alpha, and converts the angle beta value which should rotate into a corresponding PWM signal value;
s13, the camera angle adjusting device is composed of a steering engine and a camera fixing device, a camera is fixedly arranged in the camera fixing device, the steering engine receives a PWM signal value transmitted by the micro processor, the rotating angle of the steering engine is adjusted according to the pulse width duration of the PWM signal value, and then the rotating angle of the orientation angle of the camera is adjusted.
3. The traffic light recognition method according to claim 1, wherein in S20, the detector detects the image with the traffic light through a classifier, and further detects the area of the traffic light in the image, specifically: the method comprises the following steps of detecting an image with a traffic signal lamp by adopting an AdaBoost classifier based on Haar-like characteristics, and further detecting the area of the traffic signal lamp in the image, wherein the AdaBoost classifier comprises two processes: a training process and a recognition process;
in the training process, on one hand, Haar-like features which play a key role in classification and identification are selected from a large number of Haar-like features, and on the other hand, an AdaBoost classifier which accurately identifies a traffic signal light region and a non-traffic signal light region is trained to prepare for the identification process;
in the identification process, the key Haar features extracted in the training process and the trained AdaBoost classifier are used for identifying the images transmitted by the camera, and the images with the traffic signal lamps and the areas of the traffic signal lamps in the images are obtained by identification:
the identification process specifically comprises the following steps: and (3) classifying and identifying whether an area with a traffic signal lamp exists in the image transmitted by the camera: comprises the steps of image preprocessing; calculating an integral graph; calculating corresponding Haar-like characteristic values by utilizing the Haar-like characteristic information selected in the training process to form characteristic vectors; and detecting whether the area with the traffic signal lamp exists in the image transmitted by the camera by using the AdaBoost classifier trained in the training process and the obtained feature vector, and outputting a final classification recognition result.
4. The method of identifying traffic lights on a downhill slope according to claim 3, wherein the training process comprises four training sub-processes: preprocessing an image; calculating an integral graph; extracting Haar-like features; training an AdaBoost classifier; wherein the content of the first and second substances,
image preprocessing: the shot image is divided into two parts: one part is an image with a traffic light and is used as a traffic light training sample; the other part is an image without a traffic light and is used as a non-traffic light training sample; normalizing all traffic signal lamp training samples and non-traffic signal lamp training samples into 128 multiplied by 128 gray level images as normalized traffic signal lamp training samples and normalized non-traffic signal lamp training samples;
and (3) calculating an integral graph: respectively calculating a forward integral image and an oblique integral image of each normalized traffic signal lamp training sample and each normalized non-traffic signal lamp training sample;
extracting Haar-like features: the expanded Haar-like features are divided into 4 categories and 15 categories, the size of each Haar-like feature is set to be 2 multiplied by 1, and according to a forward integral image and an oblique integral image of each normalized traffic signal lamp training sample and each normalized non-traffic signal lamp training sample, 15 Haar-like features with different sizes of each normalized traffic signal lamp training sample and each normalized non-traffic signal lamp training sample image are calculated and extracted and used for describing the edge and structural features of the traffic signal lamp;
training an AdaBoost classifier, which comprises the steps of constructing a weak classifier, constructing a strong classifier and constructing a cascade classifier:
constructing a weak classifier: selecting one type of Haar features from 15 types of Haar features as input, and obtaining a weak classifier h of the Haar features through trainingj(x) Each weak classifier hj(x) Are formed by the selected Haar-like eigenvalues and a threshold value of the form shown below:
Figure FDA0002972567240000021
wherein x represents a sample, fj(x) Represents the corresponding characteristic value, p, of the jth Haar-like characteristic on the sample xjDetermining the positive or negative of the inequality, thetajRepresentation classifier hj(x) A threshold corresponding to the Haar-like eigenvalue in (1);
constructing a strong classifier: training with weak classifiers for many times, continuously increasing the weight of the misclassified samples in each training, continuously training, selecting the optimal weak classifier from each training, and linearly combining into a strong classifier with better classification capability;
constructing a cascade classifier: adopting a cascade method, separating out negative samples by each stage of strong classifier, entering the rest samples into the next stage of strong classifier, and obtaining positive samples through the prediction of the last strong classifier; and training a sample set based on the constructed cascade classifier so as to obtain features which play a key role in classification and identification and a final classifier.
5. The method for identifying traffic lights on a downhill road according to claim 4, wherein the specific training process for constructing the strong classifier is as follows:
selecting a training set with n samples, wherein a traffic signal training samples and b non-traffic signal training samples are selected, n is a + b, and the sample set is (x)1,y1),(x2,y2),...,(xn,yn)，xiFor the ith sample, yiTo illustrate xiWhether it is a traffic signal light training sample or a non-traffic signal light training sample, when yiWhen 0, then xiNon-traffic light training samples; when y isiWhen 1, then xiIs a traffic signal light training sample;
initializing sample weights: the weights of all traffic light training samples are 1/2a, and the weights of all non-traffic light training samples are 1/2 b;
for training times T equal to 1, …, T, the process of training to obtain the optimal weak classifier is:
step a1, normalizing the weight of all samples in the round:
Figure FDA0002972567240000031
in the formula, Wt,iRepresents the normalized weight of the ith sample in the t training, wt,iRepresents the weight of the ith sample in the t training, wt,jRepresenting the weight of a jth sample in the tth training, wherein j represents the jth sample, and n represents the total number of samples;
step a2, training weak classifiers h according to each Haar-like feature jjClassifying whether the samples in the sample set belong to traffic signal lamp training samples, and calculating the error of the sample set classification:
Figure FDA0002972567240000032
in the formula, epsilonjError, h, representing sample set classificationj(xi) Is a weak classifier hjFor sample xiAnd (5) classifying results.
Step a3, selecting the optimal weak classifier from each training; updating the weight: if the sample is classified correctly, then
Figure FDA0002972567240000033
εtRepresenting the error rate of the t-th training, W if the sample is misclassifiedt+1,i＝Wt,i(ii) a Linearly combining the T optimal weak classifiers into a strong classifier, wherein the formula of the strong classifier is shown as the following formula:
Figure FDA0002972567240000041
in the formula, atRepresents the weight of the weak classifier trained for the t-th time, ht(x) Represents the optimal weak classifier selected in the t training,
Figure FDA0002972567240000042
6. the method for identifying traffic lights on a downhill road according to claim 1, wherein in S30, the identifier identifies the state of the traffic lights in the area, specifically:
identifying the state of the image in the area with the traffic light by a classifier, wherein the classifier adopts two characteristics of an orientation gradient histogram HOG and a bulb HSV color histogram, and integrates and trains the two characteristics of the orientation gradient histogram HOG and the bulb HSV color histogram by a support vector machine;
in the training process, firstly, determining the parameter gamma and the penalty factor C of the optimal radial basis kernel function by a grid search method, then, searching a support vector set through the learning of a training set, and preliminarily determining the Lagrangian multiplier aiAnd b, adjusting and optimizing the obtained relevant parameters of the support vector machine model by using the test set, and finally determining the decision function of the optimal classification surface of the support vector machine modelA parameter;
after the training of the support vector machine model is completed, each image transmitted by the detector is brought into a decision function of each support vector machine model, and finally the classification category of the image is determined from the result of the support vector machine model through a voting method, so that the state of the traffic light is determined.
7. The method for identifying traffic lights on a downhill path as claimed in claim 6, wherein said integrating and training two features of Histogram of Oriented Gradient (HOG) and histogram of color of lamp bulb (HSV) with support vector machine comprises the following steps:
step b 1: normalizing the sample image to 48 × 48 pixel size, selecting the histogram of oriented gradient HOG and the histogram of color of lamp bulb HSV of the normalized sample image as the characteristic value of the normalized sample image, and forming a characteristic value vector x ═ p1,p2) X denotes a vector of eigenvalues, p1Representing oriented gradient histogram feature value, p2Representing the characteristic value of the HSV color histogram, and setting the characteristic value vector x as (p)1,p2) Taking the state y of the traffic signal lamp as an input, and forming a traffic signal lamp state sample set;
in the extraction of the directional gradient histogram of the image, selecting cell units with the size of 6 multiplied by 6 pixels, wherein each cell unit is 9 bins, taking the rectangular interval Block as 2 multiplied by 2, extracting the directional gradient histogram of the image, and then performing dimensionality reduction on data by using a principal component analysis method to obtain a final value;
the states of the traffic signal lamp are three types: the method comprises the following steps of extracting 4/5 from 3 samples corresponding to the state y of the traffic signal lamp as a training set and using the rest 1/5 as a test set, wherein the red lamp, the green lamp and the yellow lamp respectively correspond to y being 1, y being 2 and y being 3;
step b 2: determining the number of Support Vector Machines (SVM): 3 classifiers are constructed, and respectively: a red-green support vector machine, wherein the sample set is taken from a sample set of bright red lights and bright green lights, and if the sample set is taken from the bright red light sample, y is 1; taking a sample with a bright green light, and then y is-1; a red-yellow support vector machine, wherein the sample set is taken from a sample set with bright red light and bright yellow light, and if the sample set is taken from a sample with bright red light, y is 1; taking a sample with yellow light, and then y is-1; a green-yellow support vector machine, wherein the sample set is taken from a sample set of green light and yellow light, and if the sample set is taken from a sample of green light, y is 1; taking a sample of the yellow lamp, and training the three support vector machines respectively if y is equal to-1;
step b 3: training a support vector machine: determining initial values of relevant parameters of a decision function of the optimal classification surface of the sample set through a training set; the relevant parameters comprise optimal kernel function parameters and penalty factors; determining the optimal kernel function parameter by adopting a grid search method; selecting a radial basis kernel function K (x)i,x)＝exp(-γ||xi-x||2) The method is used as an optimal kernel function to realize the mapping of a support vector machine from an original space to a feature space, wherein gamma is a kernel parameter and a relaxation factor xi is introducediAnd a penalty factor C, namely solving the problem of the optimal classification surface equivalently to solve a convex quadratic programming problem shown as the following formula:
Figure FDA0002972567240000051
wherein R (w) represents an objective function, w is a normal vector of an optimal classification surface, C represents a penalty factor, and xiiRepresenting the relaxation factor, b is the offset coefficient, N is the number of samples, xiAs sample feature vector, yiThe traffic signal lamp state corresponding to the sample is obtained;
and (3) applying a Lagrange multiplier method and considering that the Karush-Kuhn-Tucker condition is satisfied, and obtaining a decision function of the optimal classification surface as follows:
Figure FDA0002972567240000052
wherein f (x) represents the decision function of the optimal classification surface, Sgn is the sign function, aiAnd (4) corresponding to each sample, wherein x is a target characteristic vector, and SV represents a support vector.
8. The method for identifying traffic lights on a downhill road according to claim 1, further comprising a step of inputting the image determined to have the traffic light at S20 into a pre-trained neural network for monocular depth estimation, wherein the depth corresponding to each pixel in the output image is combined with the area of the traffic light in the image output by the S20 detector, the depth information l of the traffic light at the vehicle position at the current time is obtained by averaging the depths of all pixels in the area, and the distance S from the vehicle to the traffic light position at the current time is obtained according to S l/cos α in combination with the road slope angle α obtained at S10.
9. A traffic signal lamp identification system on a downhill road, characterized by comprising:
a memory for storing instructions executable by the processor;
a processor for executing the instructions to implement the method of any one of claims 1 to 8.
10. A computer-readable medium, characterized in that a computer program code is stored, which, when being executed by a processor, realizes the method according to any one of claims 1 to 8.
CN202110267619.9A 2021-03-12 2021-03-12 Traffic signal lamp identification method and system on downhill road and computer readable medium Active CN112990002B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110267619.9A CN112990002B (en) 2021-03-12 2021-03-12 Traffic signal lamp identification method and system on downhill road and computer readable medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110267619.9A CN112990002B (en) 2021-03-12 2021-03-12 Traffic signal lamp identification method and system on downhill road and computer readable medium

Publications (2)

Publication Number Publication Date
CN112990002A true CN112990002A (en) 2021-06-18
CN112990002B CN112990002B (en) 2023-04-18

Family

ID=76335046

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110267619.9A Active CN112990002B (en) 2021-03-12 2021-03-12 Traffic signal lamp identification method and system on downhill road and computer readable medium

Country Status (1)

Country Link
CN (1) CN112990002B (en)

Citations (17)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
AU2008264146A1 (en) * 2007-12-24 2009-07-09 Haim Gadassi System and method for improved electric cars and/or electric car batteries and/or improved infrastructures for recharging electric cars.
CN102213589A (en) * 2010-04-09 2011-10-12 常州机械电子工程研究所 Digital gradient sensor
CN102968646A (en) * 2012-10-25 2013-03-13 华中科技大学 Plate number detecting method based on machine learning
CN103150547A (en) * 2013-01-21 2013-06-12 信帧电子技术（北京）有限公司 Vehicle tracking method and device
CN105699095A (en) * 2016-01-27 2016-06-22 常州加美科技有限公司 A test method for a remotely piloted vehicle (PRV)
CN105809184A (en) * 2015-10-30 2016-07-27 哈尔滨工程大学 Vehicle real-time identification tracking and parking space occupancy determining method suitable for gas station
US20160362050A1 (en) * 2015-06-09 2016-12-15 Lg Electronics Inc. Driver assistance apparatus and control method for the same
CN106767803A (en) * 2016-12-16 2017-05-31 杭州奥腾电子股份有限公司 The method and its device of road target missing inspection are avoided using gyroscope adjustment camera
CN107097793A (en) * 2016-02-23 2017-08-29 Lg电子株式会社 Driver assistance and the vehicle with the driver assistance
CN107133597A (en) * 2017-05-11 2017-09-05 南宁市正祥科技有限公司 A kind of front vehicles detection method in the daytime
JP2018081405A (en) * 2016-11-15 2018-05-24 本田技研工業株式会社 Traffic light recognition device, vehicle, and traffic light recognition method
CN109835256A (en) * 2017-11-29 2019-06-04 长城汽车股份有限公司 A kind of automobile panoramic round-looking system, method and automobile
CN109947110A (en) * 2019-04-02 2019-06-28 吉林大学 Lane self-checking algorithm assemblage on-orbit control method and system towards automatic Pilot
CN110475702A (en) * 2017-02-22 2019-11-19 加特可株式会社 Controller of vehicle and control method for vehicle
CN110979183A (en) * 2019-12-07 2020-04-10 吉利汽车研究院（宁波）有限公司 Vehicle blind area detection device and method and vehicle
CN111806357A (en) * 2020-07-31 2020-10-23 北京小马慧行科技有限公司 Signal acquisition method and acquisition device of traffic signal lamp
US20200384980A1 (en) * 2019-06-04 2020-12-10 Pony.ai, Inc. Systems and methods for implementing a preemptive suspension control for an autonomous vehicle to improve ride quality

Patent Citations (17)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
AU2008264146A1 (en) * 2007-12-24 2009-07-09 Haim Gadassi System and method for improved electric cars and/or electric car batteries and/or improved infrastructures for recharging electric cars.
CN102213589A (en) * 2010-04-09 2011-10-12 常州机械电子工程研究所 Digital gradient sensor
CN102968646A (en) * 2012-10-25 2013-03-13 华中科技大学 Plate number detecting method based on machine learning
CN103150547A (en) * 2013-01-21 2013-06-12 信帧电子技术（北京）有限公司 Vehicle tracking method and device
US20160362050A1 (en) * 2015-06-09 2016-12-15 Lg Electronics Inc. Driver assistance apparatus and control method for the same
CN105809184A (en) * 2015-10-30 2016-07-27 哈尔滨工程大学 Vehicle real-time identification tracking and parking space occupancy determining method suitable for gas station
CN105699095A (en) * 2016-01-27 2016-06-22 常州加美科技有限公司 A test method for a remotely piloted vehicle (PRV)
CN107097793A (en) * 2016-02-23 2017-08-29 Lg电子株式会社 Driver assistance and the vehicle with the driver assistance
JP2018081405A (en) * 2016-11-15 2018-05-24 本田技研工業株式会社 Traffic light recognition device, vehicle, and traffic light recognition method
CN106767803A (en) * 2016-12-16 2017-05-31 杭州奥腾电子股份有限公司 The method and its device of road target missing inspection are avoided using gyroscope adjustment camera
CN110475702A (en) * 2017-02-22 2019-11-19 加特可株式会社 Controller of vehicle and control method for vehicle
CN107133597A (en) * 2017-05-11 2017-09-05 南宁市正祥科技有限公司 A kind of front vehicles detection method in the daytime
CN109835256A (en) * 2017-11-29 2019-06-04 长城汽车股份有限公司 A kind of automobile panoramic round-looking system, method and automobile
CN109947110A (en) * 2019-04-02 2019-06-28 吉林大学 Lane self-checking algorithm assemblage on-orbit control method and system towards automatic Pilot
US20200384980A1 (en) * 2019-06-04 2020-12-10 Pony.ai, Inc. Systems and methods for implementing a preemptive suspension control for an autonomous vehicle to improve ride quality
CN110979183A (en) * 2019-12-07 2020-04-10 吉利汽车研究院（宁波）有限公司 Vehicle blind area detection device and method and vehicle
CN111806357A (en) * 2020-07-31 2020-10-23 北京小马慧行科技有限公司 Signal acquisition method and acquisition device of traffic signal lamp

Non-Patent Citations (4)

* Cited by examiner, † Cited by third party
Title
JULIAN MULLER 等: "Multi-camera system for traffic light detection: About camera setup and mapping of detections", 《2017 IEEE 20TH INTERNATIONAL CONFERENCE ON INTELLIGENT TRANSPORTATION SYSTEMS (ITSC)》 *
张雪芹 等: "基于类Haar特征和AdaBoost的车辆识别技术", 《华东理工大学学报（自然科学版）》 *
朱梦超: "基于HOG-PCA和SVM的人行横道信号灯识别方法", 《工业控制计算机》 *
谷明琴 等: "应用圆形度和颜色直方图的交通信号灯识别", 《计算机工程与设计》 *

Also Published As

Publication number Publication date
CN112990002B (en) 2023-04-18

Similar Documents

Publication Publication Date Title
EP1782335B1 (en) Method for traffic sign detection
Chang et al. Online boosting for vehicle detection
AU2018288720B2 (en) Rare instance classifiers
Sivaraman et al. Active learning for on-road vehicle detection: A comparative study
US8842883B2 (en) Global classifier with local adaption for objection detection
US7724962B2 (en) Context adaptive approach in vehicle detection under various visibility conditions
US8824742B2 (en) Occupancy detection for managed lane enforcement based on localization and classification of windshield images
CN102076531B (en) Vehicle clear path detection
Shi et al. Real-time traffic light detection with adaptive background suppression filter
KR101848019B1 (en) Method and Apparatus for Detecting Vehicle License Plate by Detecting Vehicle Area
CN101859382A (en) License plate detection and identification method based on maximum stable extremal region
Kashevnik et al. Seat belt fastness detection based on image analysis from vehicle in-abin camera
CN106570439B (en) Vehicle detection method and device
CN103632170A (en) Pedestrian detection method and device based on characteristic combination
CN104615986A (en) Method for utilizing multiple detectors to conduct pedestrian detection on video images of scene change
CN109886086B (en) Pedestrian detection method based on HOG (histogram of oriented gradient) features and linear SVM (support vector machine) cascade classifier
Biswas et al. Detection and classification of speed limit traffic signs
Wang et al. Simultaneous multi-vehicle detection and tracking framework with pavement constraints based on machine learning and particle filter algorithm
CN112990002B (en) Traffic signal lamp identification method and system on downhill road and computer readable medium
US20230245466A1 (en) Vehicle Lidar System and Object Classification Method Therewith
Manisha et al. An online traffic sign recognition system for intelligent driver assistance
Chen et al. Research on vehicle detection and tracking algorithm for intelligent driving
CN104463135A (en) Vehicle logo recognition method and system
Kaja et al. Two stage intelligent automotive system to detect and classify a traffic light
Ortega et al. Perspective multiscale detection of vehicles for real-time forward collision avoidance systems

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant