CN109657622A - A kind of detection of traffic lights and recognition methods, device and equipment - Google Patents
A kind of detection of traffic lights and recognition methods, device and equipment Download PDFInfo
- Publication number
- CN109657622A CN109657622A CN201811574161.6A CN201811574161A CN109657622A CN 109657622 A CN109657622 A CN 109657622A CN 201811574161 A CN201811574161 A CN 201811574161A CN 109657622 A CN109657622 A CN 109657622A
- Authority
- CN
- China
- Prior art keywords
- sample
- detection
- identification
- traffic lights
- tested
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
- G06V20/58—Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
- G06V20/584—Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads of vehicle lights or traffic lights
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- General Engineering & Computer Science (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Biomedical Technology (AREA)
- Life Sciences & Earth Sciences (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Multimedia (AREA)
- Traffic Control Systems (AREA)
- Image Analysis (AREA)
Abstract
The present invention provides a kind of detections of traffic lights and recognition methods, device, equipment and computer readable storage medium, this method comprises: obtaining includes multiple sample sets with the picture samples marked, picture sample includes training sample and verifying sample, is labeled as the information of position coordinates add in corresponding picture sample, including corresponding to the traffic lights for including in picture sample and classification information；Using training sample and the Faster RCNN frame that is pre-created of verifying sample training, detected and identification model；The sample to be tested for not having mark is input to detection and identification model, the position coordinates and classification information of detection with traffic lights in the sample to be tested of identification model output are obtained, to realize the control of vehicle running state.The application improves detection and accuracy of identification, generalization ability and robustness by the use of Faster RCNN frame, can be efficiently applied to the technical fields such as intelligent driving, intelligent transportation.
Description
Technical field
The present invention relates to object detection and recognition technical fields in image, more specifically to a kind of traffic lights
Detection and recognition methods, device, equipment and computer readable storage medium.
Background technique
Intelligent driving refers to that machine helps people to carry out intelligent driving, and replaces people to realize nothing completely under special circumstances
The technology that people drives.Traffic lights are a kind of indicating equipments generally used in the road, are the signals for commanding traffic circulation
Lamp is generally made of red light, green light, amber light, and red light indicates that no through traffic, and green light indicates to permit passing through, and amber light indicates warning.?
It during realizing intelligent driving, needs that the traffic lights on road are detected and identified, based on detection and identification
The driving status of resulting result control vehicle.
Scheme currently used for being detected and being identified to traffic lights is all based on traditional image procossing and training
What classifier was realized；Specifically, it is necessary first to the picture comprising traffic lights of mobile unit shooting is pre-processed,
Color, edge, Texture eigenvalue are generally used, obtains may be traffic lights mesh target area, then uses Scale invariant
The obtained region of the feature extracting methods such as eigentransformation, histograms of oriented gradients, local binary patterns verifying pretreatment whether be
The traffic lights target feature vector of acquisition is finally input in classifier and identifies, usually by traffic lights target
The classifier of use has support vector machines, decision tree, the learning machine that transfinites etc..These schemes are nearly all using artificial design features
Feature extraction is carried out to target, therefore is easy to be influenced by artificial subjective factor, proposes feature by the experience of researcher
Then extracting mode trains classifier, carry out the detection and identification of traffic lights, and detection is often limited to grind with recognition result
The height of the person's of studying carefully level, and the feature generalization ability of engineer is poor, cannot take into account miscellaneous traffic signals in picture
The feature extraction of lamp (shape is different from color) generally causes the feature of the traffic lights extraction to certain shape and color
It is good, and the feature extracted to the traffic lights of other shapes and color is poor.In addition, real-life traffic scene is also very
Complexity, traffic lights are partially blocked sometimes, but also can be by the shadow of the factors such as complicated road environment, Changes in weather
It rings, therefore, the feature of engineer can not efficiently extract the feature of traffic lights under all situations, if traffic signals
The feature extraction of lamp it is bad, just will affect the positioning and subsequent classifier training of traffic lights target area, then
Detecting will be relatively low with the accuracy of identification, and this mode does not have robustness yet.
There is detection in conclusion being detected in the prior art for realizing traffic lights with the technical solution identified and know
Other accuracy is lower, generalization ability is poor, robustness is lower, and then leads to not be efficiently applied to intelligent driving, intelligent transportation
The problem of equal fields.
Summary of the invention
The object of the present invention is to provide a kind of detections of traffic lights and recognition methods, device, equipment and computer can
Storage medium is read, is able to solve and is detected and detection existing for the technical solution of identification for realizing traffic lights in the prior art
With recognition accuracy is lower, generalization ability is poor, robustness is lower, and then lead to not be efficiently applied to intelligent driving, intelligence
The problem of fields such as traffic.
To achieve the goals above, the invention provides the following technical scheme:
A kind of detection and recognition methods of traffic lights, comprising:
Obtaining includes multiple sample sets with the picture samples marked, and the picture sample includes training sample and verifying
Sample, it is described to be labeled as position add in corresponding picture sample, including corresponding to the traffic lights for including in picture sample
Set the information of coordinate and classification information；
The Faster RCNN frame being pre-created using the training sample and the verifying sample training, is detected
With identification model；
When receiving the picture sample without mark is sample to be tested, the sample to be tested is input to the detection
With identification model, the position coordinates and class of the detection with traffic lights in the sample to be tested of identification model output are obtained
Other information, for realizing the control of vehicle running state based on the position coordinates and classification information.
Preferably, Faster RCNN frame is pre-created, comprising:
Determine that VGG16 network is the feature extraction network of Faster RCNN frame, and rear the three of the VGG16 network
Expansion residual error convolution block is separately added into a convolution block.
Preferably, Faster RCNN frame is pre-created, comprising:
It is added after first convolutional layer that RPN has in the Faster RCNN frame and squeezes and motivate constructing module.
Preferably, the picture sample in the sample set further includes test sample；The sample to be tested is input to described
Before detection and identification model, further includes:
Test the detection and the detection of identification model and accuracy of identification using the test sample, if the detection with
Accuracy of identification reaches preset threshold, it is determined that the detection can be detected and be identified to sample to be tested with identification model；It is no
Then, it is determined that the detection can not be detected and be identified to sample to be tested with identification model.
A kind of detection of traffic lights and identification device, comprising:
Obtain module, be used for: obtaining includes multiple sample sets with the picture samples marked, and the picture sample includes
Training sample and verifying sample, it is described be labeled as it is being added in corresponding picture sample, including including in corresponding picture sample
The position coordinates of traffic lights and the information of classification information；
Training module is used for: the Faster RCNN being pre-created using the training sample and the verifying sample training
Frame, is detected and identification model；
Detection and identification module, are used for: when receiving the picture sample without mark is sample to be tested, will it is described to
Test sample is originally input to the detection and identification model, obtains the detection and traffic in the sample to be tested of identification model output
The position coordinates and classification information of signal lamp, for realizing the control of vehicle running state based on the position coordinates and classification information
System.
Preferably, further includes:
First creation module, is used for: determining that VGG16 network is the feature extraction network of Faster RCNN frame, and in institute
It states and is separately added into expansion residual error convolution block in rear three convolution blocks of VGG16 network.
Preferably, further includes:
Second creation module, is used for: being added after first convolutional layer that RPN has in the Faster RCNN frame
Squeeze and motivate constructing module.
Preferably, further includes:
Test module is used for: the sample to be tested being input to the detection with before identification model, utilizes the test
Detection and accuracy of identification of the detection described in test sample with identification model, if the detection reaches default threshold with accuracy of identification
Value, it is determined that the detection can be detected and be identified to sample to be tested with identification model；Otherwise, it is determined that it is described detection with
Identification model can not be detected and be identified to sample to be tested；Wherein, the picture sample in the sample set further includes test specimens
This.A kind of detection of traffic lights and identification equipment, comprising:
Memory, for storing computer program；
Processor realizes the detection and knowledge of the as above any one traffic lights when for executing the computer program
The step of other method.
A kind of computer readable storage medium is stored with computer program on the computer readable storage medium, described
The step of as above detection and recognition methods of any one traffic lights are realized when computer program is executed by processor.
The present invention provides a kind of detections of traffic lights and recognition methods, device, equipment and computer-readable storage
Medium, wherein the picture sample includes instruction this method comprises: obtaining includes multiple sample sets with the picture samples marked
Practice sample and verifying sample, it is described to be labeled as friendship add in corresponding picture sample, including including in corresponding picture sample
The position coordinates of ventilating signal lamp and the information of classification information；It is pre-created using the training sample and the verifying sample training
Faster RCNN frame, detected and identification model；It is sample to be tested when receiving the picture sample without mark
When, the sample to be tested is input to the detection and identification model, obtain it is described detection with identification model output it is described to
The position coordinates and classification information of traffic lights in test sample sheet, for realizing vehicle row based on the position coordinates and classification information
Sail the control of state.The application realizes the detection and identification of traffic lights, and Faster using Faster RCNN frame
RCNN frame can automatically extract the feature of higher level of abstraction using the characteristic of its convolutional coding structure from picture, and then based on extraction
Feature realize the detection and identification function of respective objects, it is easy to operate and have to eliminate the process of artificial design features
Effect, avoids that object detection and recognition precision is low, generalization ability is poor as caused by artificial design features and robustness is lower
The problem of namely the application based on Faster RCNN frame realize traffic lights detection and identification improve detection and know
Other precision, generalization ability and robustness ensure that the control of the information realization vehicle running state obtained based on detection with identification
When accuracy, the technical fields such as intelligent driving, intelligent transportation can be efficiently applied to.
Detailed description of the invention
In order to more clearly explain the embodiment of the invention or the technical proposal in the existing technology, to embodiment or will show below
There is attached drawing needed in technical description to be briefly described, it should be apparent that, the accompanying drawings in the following description is only this
The embodiment of invention for those of ordinary skill in the art without creative efforts, can also basis
The attached drawing of offer obtains other attached drawings.
Fig. 1 is a kind of flow chart of detection and the recognition methods of traffic lights provided in an embodiment of the present invention；
Fig. 2 is the picture in a kind of detection and recognition methods of traffic lights provided in an embodiment of the present invention with mark
The schematic diagram of sample；
Fig. 3 is classification and position refine in a kind of detection and recognition methods of traffic lights provided in an embodiment of the present invention
The schematic diagram of network；
Fig. 4 is that VGG16 network shows in a kind of detection and recognition methods of traffic lights provided in an embodiment of the present invention
It is intended to；
Fig. 5 is VGG16 network convolution in a kind of detection and recognition methods of traffic lights provided in an embodiment of the present invention
Partial schematic diagram；
Fig. 6 is to expand residual error convolution block in a kind of detection and recognition methods of traffic lights provided in an embodiment of the present invention
Schematic diagram；
Fig. 7 is that expansion residual error volume is introduced in a kind of detection and recognition methods of traffic lights provided in an embodiment of the present invention
The schematic diagram of VGG16 network conventional part after block；
Fig. 8 is the schematic diagram of RPN in a kind of detection and recognition methods of traffic lights provided in an embodiment of the present invention；
Fig. 9 is to squeeze and motivate construction in a kind of detection and recognition methods of traffic lights provided in an embodiment of the present invention
The schematic diagram of module；
Figure 10 is RPN has in a kind of detection and recognition methods of traffic lights provided in an embodiment of the present invention first
The schematic diagram for squeezing and motivating constructing module corresponding construction is added after a convolutional layer；
Figure 11 is a kind of detection of traffic lights provided in an embodiment of the present invention and the structural schematic diagram of identification device.
Specific embodiment
Following will be combined with the drawings in the embodiments of the present invention, and technical solution in the embodiment of the present invention carries out clear, complete
Site preparation description, it is clear that described embodiments are only a part of the embodiments of the present invention, instead of all the embodiments.It is based on
Embodiment in the present invention, it is obtained by those of ordinary skill in the art without making creative efforts every other
Embodiment shall fall within the protection scope of the present invention.
Referring to Fig. 1, it illustrates a kind of detection of traffic lights provided in an embodiment of the present invention and recognition methods
Flow chart may include:
S11: obtaining includes multiple sample sets with the picture samples marked, and picture sample includes training sample and verifying
Sample, be labeled as positions being added in corresponding picture sample, including the traffic lights that include in corresponding picture sample and
The information of classification.
The execution subject of detection with the recognition methods of a kind of traffic lights provided in an embodiment of the present invention can be corresponding
Detection and identification device.It should be noted that picture sample can be to advance on vehicle-mounted pick-up equipment shooting road
The picture that traffic lights obtain, using disclosed in the prior art or the annotation tool of self-setting to picture sample carry out
Mark, generates the xml document of corresponding preset format (such as PASCAL VOC format)；The content of mark includes that traffic lights exist
The classification information of position coordinates and traffic lights in corresponding picture, specifically, traffic lights are one on picture
The rectangular block of traffic lights on each picture is framed by a rectangular block using annotation tool, as where traffic lights
Position coordinates；And the classification of traffic lights includes color, the shape of lamp etc. of current light yellow on traffic lights, such as Fig. 2
Shown, the rectangle frame that traffic lights are enclosed on picture indicates the position of corresponding traffic light, and rectangle frame sidenote is released
Green-left (green-left), green-round (green-circle) are the classification of corresponding traffic light, and green-left representation is corresponding to be handed over
Light yellow is the lamp of the green of left-hand rotation arrow (expression allows to turn left) on ventilating signal lamp, and green-circle indicates in corresponding traffic light
Light yellow is the lamp of the green of border circular areas, and in addition the classification of traffic lights can also include red-left (red-left), red-
Right (red-right), red-round (red-circle), red-up (red-on) etc..
S12: the Faster RCNN frame being pre-created using training sample and verifying sample training is obtained detection and known
Other model.
After obtaining sample set, it can use training sample and verify the Faster RCNN frame that sample training is pre-created,
It should be pointed out that needing to utilize training sample training Faster due to during training Faster RCNN frame
RCNN frame is verified with the Faster RCNN frame that verifying sample obtains training, therefore can be by the picture in sample set
Sample is divided into training sample and verifying sample, and training sample and verifying sample ratio shared in sample set can be according to realities
Border needs to preset.Wherein, verifying sample is utilized to verify Faster using training sample training Faster RCNN frame
The process of RCNN frame is consistent with the realization principle for corresponding to technical solution in the prior art, is no longer described in detail herein.
It, then can be with after obtaining detection with identification model using training sample and verifying sample training Faster RCNN frame
Position and classification using detection with identification model detection with traffic lights in identification sample to be tested.
S13: when receiving the picture sample without mark is sample to be tested, sample to be tested is input to detection and is known
Other model obtains the position coordinates and classification information of detection with traffic lights in the sample to be tested of identification model output, for
The control of vehicle running state is realized based on the position coordinates and classification information.
Wherein, sample to be tested can for during realizing intelligent driving utilize vehicle-mounted pick-up equipment captured in real-time,
It include the picture of traffic lights.After training obtains detection and identification model, do not have as long as needing to detect with identification
The traffic light position and classification of the picture sample of mark then need to realize using detection and identification model；I.e. simply by the presence of
Sample to be tested is then input to detection and identification model by sample to be tested, detection and identification model can export to sample to be tested into
Row detection with identification as a result, detection with identification result include the position of traffic lights and classification in sample to be tested, thus
It can be determined and be controlled with the position of traffic lights in the sample to be tested identified and classification based on detection when realizing intelligent driving
How intelligent automobile processed travels.
It is further to note that the information of detection and identification model output can be picture sample as shown in Figure 2,
In not only include traffic lights position coordinates and classification information, can also include traffic lights in corresponding sample to be tested
The consistent probability of information (confidence level) of classification information and detection and identification model output, as shown in Figure 2 0.998,0.997 is
The information of the classification of the traffic lights of above-mentioned maximum probability is exported with identification model for corresponding probability, namely detection.
Technical solution disclosed by the invention, obtaining first includes traffic in multiple picture samples and each picture sample of expression
The sample set of the mark of the position coordinates and classification information of signal lamp, using in the sample set as the picture sample of training sample
And the picture sample training Faster RCNN frame as verifying sample obtains corresponding detection and identification model, and then utilizes
The position and classification of the detection and identification model detection with traffic lights in sample to be tested of the identification without mark, to be based on
The control for the information realization vehicle running state that detection is obtained with identification.The application realizes traffic using Faster RCNN frame
The detection and identification of signal lamp, and Faster RCNN frame can automatically be mentioned from picture using the characteristic of its convolutional coding structure
The feature of higher level of abstraction is taken, and then the feature based on extraction realizes the detection and identification function of respective objects, to eliminate people
The process of work design feature, it is easy to operate and effective, avoid the essence of the object detection and recognition as caused by artificial design features
It spends low, generalization ability is poor and robustness is lower problem namely the application and traffic signals is realized based on Faster RCNN frame
The detection and identification of lamp improve detection and accuracy of identification, generalization ability and robustness, ensure that and are obtained based on detection with identification
Information realization vehicle running state control when accuracy, the technologies such as intelligent driving, intelligent transportation can be efficiently applied to
Field.
Specifically, Faster RCNN frame includes four parts, respectively feature extraction network, RPN in the application
(candidate region extraction network), RoI Pooling (area-of-interest pond) and classification and position refine network.Wherein, feature
Extracting network is an improved VGG16 network, uses one group of Conv (convolution)+ReLU (activation)+Pooling (pond) layer group
At feature extraction network, for extracting characteristic pattern from the picture of input；RPN is joined after one rpn_conv/3x3 layers
The network for squeezing and motivating constructing module, for extracting object candidate area from characteristic pattern, (object candidate area is as preliminary
The position of determining traffic lights), judge that anchors (anchor) belongs to target or background by softmax, it is then sharp again
Amendment anchors, which is returned, with bounding box obtains accurate object candidate area；RoI Pooling is used for raw to feature extraction network
At characteristic pattern and RPN generate object candidate area, extract object candidate area characteristic pattern (the target candidate area of fixed size
Characteristic of field figure is the characteristic pattern for including the traffic light position primarily determined)；Classification (can be such as Fig. 3 with position refine network
It is shown) Classification and Identification of object candidate area characteristic pattern progress target (traffic lights) for generating to RoI Pooling,
Obtain the exact position of target using bounding box recurrence again simultaneously.Since Faster RCNN frame is by feature extraction, candidate
Extracted region, bounding box refine, target identification have all been incorporated into a frame, so that comprehensive performance improves a lot, especially
Detection and recognition speed and precision are substantially increased, it, can be certainly the frame application in the detection and identification of traffic lights
Dynamic detection and recognition result from the acquistion of a large amount of picture sample middle schools to target, does not need artificial design features, does not need to figure
Piece sample carries out a series of image preprocessing, study that can automatically to a large amount of different samples, and Lai Zengjia model believes traffic
The generalization and robustness of signal lamp detection and identification, and cope with complicated road environment and Changes in weather bring shadow
It rings.
The detection and recognition methods of a kind of traffic lights provided in an embodiment of the present invention, are pre-created Faster RCNN
Frame may include:
Determine that VGG16 network is the feature extraction network of Faster RCNN frame, and in rear three volumes of VGG16 network
Expansion residual error convolution block is separately added into block.
It should be noted that being carried out using picture sample of the feature extraction network in Faster RCNN frame to input
To generate characteristic pattern, the basic network in Faster RCNN frame uses VGG16 network, VGG16 network such as Fig. 4 for feature extraction
It is shown, the network parameter in Faster RCNN frame is carried out using training weight of the VGG16 network on ImageNet initial
Change.The conventional part of VGG16 network is as shown in figure 5, since the characteristic pattern extracted from the picture sample of input is to subsequent RPN
It is all critically important with RoI Pooling, therefore in order to improve the ability in feature extraction of feature extraction network, and prevent network in training
In the process gradient disappear, over-fitting the problems such as, using in the conventional part of VGG16 network Conv3 (Conv3_1, Conv3_2,
Conv3_3), Conv4 (Conv4_1, Conv4_2, Conv4_3), Conv5 (Conv5_1, Conv5_2, Conv5_3) three volumes
The structure of block introduces Dilated Residual Convolution Block (expansion residual error convolution on this basis
Block), expansion residual error convolution block was as shown in fig. 6, therefore the feature extraction network of modified Faster RCNN was as shown in fig. 7, should
Feature extraction network is used for subsequent RPN and RoI Pooling for extracting characteristic pattern from the picture sample of input.
The detection and recognition methods of a kind of traffic lights provided in an embodiment of the present invention, are pre-created Faster RCNN
Frame may include:
It is added after first convolutional layer that RPN has in Faster RCNN frame and squeezes and motivate constructing module.
Object candidate area is extracted using characteristic pattern of the RPN in Faster RCNN frame to input, the structure of RPN can
With as shown in figure 8, RPN in Fig. 8 points for 2 tunnels, classified anchors acquisition target and background by softmax all the way above, under
Face, which is used to calculate all the way, returns offset for the bounding box of anchors, to obtain modified object candidate area, and
Proposal layers of responsible integration objective anchors and for anchors bounding box return offset, to obtain mesh to be identified
The accurate candidate region of target, is equivalent to the positioning function for completing target to be identified.Wherein, the spy of feature extraction network output
When sign figure input RPN, rpn_conv/3x3 convolution (first convolutional layer that RPN has) first is carried out, each point is equivalent to and melts again
The spatial information of surrounding 3x3 is closed, the ability in feature extraction of this convolutional layer, which extracts the candidate region of target to be identified below, to be had
It influences, therefore joined Squeeze-and-Excitation Module after rpn_conv/3x3 convolution and (squeeze and motivate structure
Modeling block), it squeezes and excitation constructing module is as shown in figure 9, the rpn_conv/3x3 tool being added after squeezing and motivating constructing module
Body connection type is as shown in Figure 10, and wherein the output of rpn_conv/3x3 is HxWx512 characteristic pattern, and rpn_conv_global_
Pool is that global max pooling (global maximum pond) is carried out to the HxWx512 characteristic pattern of rpn_conv/3x3 output,
Then 1x1x512 characteristic pattern is exported, and rpn_conv_global_pool is the HxWx512 feature to rpn_conv/3x3 output
Figure carries out global max pooling (global maximum pond), then exports 1x1x512 characteristic pattern, rpn_conv_1x1_
Down is to carry out 1x1 convolution operation and output channel number to 1x1x512 characteristic pattern as 16, rpn_conv_1x1_down/relu to be
The operation of ReLU nonlinear activation is carried out to the 1x1x16 characteristic pattern of rpn_conv_1x1_down output, rpn_conv_1x1_up is
1x1 convolution operation is carried out to the output after the operation of ReLU nonlinear activation and output channel number is that 512, rpn_conv_prob is
The operation of Sigmoid nonlinear activation, rpn_conv_scale are carried out to the 1x1x512 characteristic pattern of rpn_conv_1x1_up output
It is to be carried out to the 1x1x512 feature weight of rpn_conv_prob output and the HxWx512 characteristic pattern of rpn_conv/3x3 output
Channel-wise multiplication (multiplication based on channel), so that effective characteristic pattern weight is big, invalid or effect
Small characteristic pattern weight is small, relation of interdependence that in this way can explicitly between Modelling feature channel, by way of study
Automatically it obtains the significance level in each feature channel, then go to be promoted useful feature according to this significance level and inhibits pair
The little feature of current task use, so that the ability in feature extraction that network is promoted in terms of feature channel is enabled the network to,
There is very big promotion for the accurate extraction of object candidate area.
The detection and recognition methods of a kind of traffic lights provided in an embodiment of the present invention, the picture sample in sample set is also
Including test sample；Before sample to be tested is input to detection and identification model, can also include:
Detection and accuracy of identification using test sample test detection with identification model, if detection reaches with accuracy of identification
Preset threshold, it is determined that detection can be detected and be identified to sample to be tested with identification model；Otherwise, it is determined that detection and knowledge
Other model can not be detected and be identified to sample to be tested.
It should be noted that training obtains detection and after identification model, can use test sample to detection and identification mould
The detection of type is determined with accuracy of identification；Wherein, training sample, verifying sample and test sample ratio shared in sample set
Example can be set according to actual needs.Specifically, each test sample is sequentially input into detection and identification model,
Obtain the position coordinates of traffic lights contained in detection and each test sample of identification model output and classification information
Information will test and compare each available classification with the information of the information of identification model output and corresponding test sample
The AP of traffic lights, and then the traffic lights of the available whole classifications of AP of the traffic lights based on each classification
MAP, wherein AP and mAP is identical as the meaning for corresponding to concept in the prior art, thus using mAP as detection with identification essence
Degree determines inspection of the detection with identification model by judging whether it reaches the preset threshold previously according to actual needs setting
Survey with accuracy of identification whether reach requirement, if it is judged that be it is yes, then illustrate detection with identification model detect and accuracy of identification
It is higher, without training again, otherwise then illustrate that detection is lower with accuracy of identification with the detection of identification model, needs to utilize sample set
In training sample and verifying sample or obtain new training sample and verifying sample training detection and identification model again, directly
Until detecting and reaching preset threshold with accuracy of identification, so that the test and circuit training process by detection with identification model are protected
Detection and identification model detection with higher and accuracy of identification are demonstrate,proved.
The present invention test on the sample set of the homemade 3315 picture sample compositions containing traffic lights
Card, wherein traffic lights mark type one share eight classes (red-left, red-up, red-right, red-round,
Green-left, green-up, green-right, green-round), test sample has 1326, and train sample has 1396
, val sample has 593, is trained and tests on this sample set, and AP, mAP value for obtaining test result are as shown in table 1:
AP, mAP value of 1 test result of table
The embodiment of the invention also provides a kind of detection of traffic lights and identification devices, as shown in figure 11, can wrap
It includes:
Obtain module 11, be used for: obtaining includes multiple sample sets with the picture samples marked, and picture sample includes instruction
Practice sample and verifying sample, is labeled as traffic letter add in corresponding picture sample, including including in corresponding picture sample
The position coordinates of signal lamp and the information of classification information；
Training module 12, is used for: the Faster RCNN frame being pre-created using training sample and verifying sample training,
It is detected and identification model；
Detection and identification module 13, are used for:, will be to be measured when receiving the picture sample without mark is sample to be tested
Sample is input to detection and identification model, obtains detection and the position of traffic lights in the sample to be tested of identification model output is sat
Mark and classification information, for realizing the control of vehicle running state based on the position coordinates and classification information.
A kind of detection of traffic lights provided in an embodiment of the present invention and identification device can also include:
First creation module, is used for: determine that VGG16 network is the feature extraction network of Faster RCNN frame, and
Expansion residual error convolution block is separately added into rear three convolution blocks of VGG16 network.
A kind of detection of traffic lights provided in an embodiment of the present invention and identification device can also include:
Second creation module, is used for: being added and squeezes after first convolutional layer that RPN has in Faster RCNN frame
With excitation constructing module.
A kind of detection of traffic lights provided in an embodiment of the present invention and identification device can also include:
Test module is used for: before sample to be tested is input to detection and identification model, being tested and is detected using test sample
Detection and accuracy of identification with identification model, if detection reaches preset threshold with accuracy of identification, it is determined that detection and identification mould
Type can be detected and be identified to sample to be tested；Otherwise, it is determined that detection can not examine sample to be tested with identification model
It surveys and identifies；Wherein, the picture sample in sample set further includes test sample.
The embodiment of the invention also provides a kind of detections of traffic lights and identification equipment, may include:
Memory, for storing computer program；
Processor realizes the as above detection and recognition methods of any one traffic lights when for executing computer program
Step.
The embodiment of the invention also provides a kind of computer readable storage medium, it is stored on computer readable storage medium
The as above detection of any one traffic lights and identification side may be implemented in computer program when computer program is executed by processor
The step of method.
It should be noted that a kind of detection of traffic lights provided in an embodiment of the present invention and identification device, equipment and
The explanation of relevant portion refers to a kind of inspection of traffic lights provided in an embodiment of the present invention in computer readable storage medium
The detailed description with corresponding part in recognition methods is surveyed, details are not described herein.In addition above-mentioned technology provided in an embodiment of the present invention
In scheme with correspond to the consistent part of technical solution realization principle and unspecified in the prior art, in order to avoid excessively repeat.
The foregoing description of the disclosed embodiments can be realized those skilled in the art or using the present invention.To this
A variety of modifications of a little embodiments will be apparent for a person skilled in the art, and the general principles defined herein can
Without departing from the spirit or scope of the present invention, to realize in other embodiments.Therefore, the present invention will not be limited
It is formed on the embodiments shown herein, and is to fit to consistent with the principles and novel features disclosed in this article widest
Range.
Claims (10)
1. the detection and recognition methods of a kind of traffic lights characterized by comprising
Obtaining includes multiple sample sets with the picture samples marked, and the picture sample includes training sample and verifying sample
This, it is described to be labeled as position add in corresponding picture sample, including corresponding to the traffic lights for including in picture sample
The information of coordinate and classification information；
The Faster RCNN frame being pre-created using the training sample and the verifying sample training, is obtained detection and known
Other model；
When receiving the picture sample without mark is sample to be tested, the sample to be tested is input to the detection and is known
Other model, the position coordinates and classification for obtaining the detection and traffic lights in the sample to be tested of identification model output are believed
Breath, for realizing the control of vehicle running state based on the position coordinates and classification information.
2. the method according to claim 1, wherein Faster RCNN frame is pre-created, comprising:
Determine that VGG16 network is the feature extraction network of Faster RCNN frame, and in rear three volumes of the VGG16 network
Expansion residual error convolution block is separately added into block.
3. according to the method described in claim 2, it is characterized in that, Faster RCNN frame is pre-created, comprising:
It is added after first convolutional layer that RPN has in the Faster RCNN frame and squeezes and motivate constructing module.
4. according to the method described in claim 3, it is characterized in that, the picture sample in the sample set further includes test specimens
This；The sample to be tested is input to before the detection and identification model, further includes:
Detection and accuracy of identification of the detection with identification model are tested using the test sample, if the detection and identification
Precision reaches preset threshold, it is determined that the detection can be detected and be identified to sample to be tested with identification model；Otherwise, then
Determine that the detection can not be detected and be identified to sample to be tested with identification model.
5. detection and the identification device of a kind of traffic lights characterized by comprising
Obtain module, be used for: obtaining includes multiple sample sets with the picture samples marked, and the picture sample includes training
Sample and verifying sample, it is described to be labeled as traffic add in corresponding picture sample, including including in corresponding picture sample
The position coordinates of signal lamp and the information of classification information；
Training module is used for: the Faster RCNN frame being pre-created using the training sample and the verifying sample training
Frame, is detected and identification model；
Detection and identification module, are used for:, will be described to test sample when receiving the picture sample without mark is sample to be tested
Originally it is input to the detection and identification model, obtains the detection and traffic signals in the sample to be tested of identification model output
The position coordinates and classification information of lamp, for realizing the control of vehicle running state based on the position coordinates and classification information.
6. device according to claim 5, which is characterized in that further include:
First creation module, is used for: determining that VGG16 network is the feature extraction network of Faster RCNN frame, and described
Expansion residual error convolution block is separately added into rear three convolution blocks of VGG16 network.
7. device according to claim 6, which is characterized in that further include:
Second creation module, is used for: being added and squeezes after first convolutional layer that RPN has in the Faster RCNN frame
With excitation constructing module.
8. device according to claim 7, which is characterized in that further include:
Test module is used for: the sample to be tested being input to the detection with before identification model, utilizes the test sample
Detection and accuracy of identification of the detection with identification model are tested, if the detection reaches preset threshold with accuracy of identification,
Determine that the detection can be detected and be identified to sample to be tested with identification model；Otherwise, it is determined that the detection and identification
Model can not be detected and be identified to sample to be tested；Wherein, the picture sample in the sample set further includes test sample.
9. a kind of detection of traffic lights and identification equipment characterized by comprising
Memory, for storing computer program；
Processor realizes the inspection of the traffic lights as described in any one of Claims 1-4 when for executing the computer program
The step of survey and recognition methods.
10. a kind of computer readable storage medium, which is characterized in that be stored with computer on the computer readable storage medium
Program realizes the detection of the traffic lights as described in any one of Claims 1-4 when the computer program is executed by processor
The step of with recognition methods.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811574161.6A CN109657622A (en) | 2018-12-21 | 2018-12-21 | A kind of detection of traffic lights and recognition methods, device and equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811574161.6A CN109657622A (en) | 2018-12-21 | 2018-12-21 | A kind of detection of traffic lights and recognition methods, device and equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN109657622A true CN109657622A (en) | 2019-04-19 |
Family
ID=66115877
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811574161.6A Pending CN109657622A (en) | 2018-12-21 | 2018-12-21 | A kind of detection of traffic lights and recognition methods, device and equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109657622A (en) |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111079563A (en) * | 2019-11-27 | 2020-04-28 | 北京三快在线科技有限公司 | Traffic signal lamp identification method and device, electronic equipment and storage medium |
| CN111177811A (en) * | 2019-12-24 | 2020-05-19 | 武汉理工光科股份有限公司 | Automatic fire point location layout method applied to cloud platform |
| CN111738212A (en) * | 2020-07-20 | 2020-10-02 | 平安国际智慧城市科技股份有限公司 | Traffic signal lamp identification method, device, equipment and medium based on artificial intelligence |
| CN112289021A (en) * | 2020-09-24 | 2021-01-29 | 深圳一清创新科技有限公司 | Traffic signal lamp detection method and device and automatic driving automobile |
| CN113065466A (en) * | 2021-04-01 | 2021-07-02 | 安徽嘻哈网络技术有限公司 | Traffic light detection system for driving training based on deep learning |
| CN113077630A (en) * | 2021-04-30 | 2021-07-06 | 安徽江淮汽车集团股份有限公司 | Traffic light detection method, device and equipment based on deep learning and storage medium |
| CN114550468A (en) * | 2022-02-17 | 2022-05-27 | 华中科技大学 | Regional traffic signal lamp cooperation method, device, equipment and medium thereof |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106650641A (en) * | 2016-12-05 | 2017-05-10 | 北京文安智能技术股份有限公司 | Traffic light positioning and identification method, device and system |
| CN107408144A (en) * | 2014-11-14 | 2017-11-28 | Zoll医疗公司 | Medical precursor event estimation |
| CN107480730A (en) * | 2017-09-05 | 2017-12-15 | 广州供电局有限公司 | Power equipment identification model construction method and system, the recognition methods of power equipment |
| CN108009509A (en) * | 2017-12-12 | 2018-05-08 | 河南工业大学 | Vehicle target detection method |
| CN108550274A (en) * | 2018-04-04 | 2018-09-18 | 中国计量大学 | A kind of unmanned auxiliary device of traffic lights based on Faster RCNN and method |
| CN108734086A (en) * | 2018-03-27 | 2018-11-02 | 西安科技大学 | The frequency of wink and gaze estimation method of network are generated based on ocular |
| CN109031287A (en) * | 2018-09-21 | 2018-12-18 | 西安交通大学 | ULTRA-WIDEBAND RADAR human body respiration signal detecting method through walls based on Faster-RCNN network |
-
2018
- 2018-12-21 CN CN201811574161.6A patent/CN109657622A/en active Pending
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107408144A (en) * | 2014-11-14 | 2017-11-28 | Zoll医疗公司 | Medical precursor event estimation |
| CN106650641A (en) * | 2016-12-05 | 2017-05-10 | 北京文安智能技术股份有限公司 | Traffic light positioning and identification method, device and system |
| CN107480730A (en) * | 2017-09-05 | 2017-12-15 | 广州供电局有限公司 | Power equipment identification model construction method and system, the recognition methods of power equipment |
| CN108009509A (en) * | 2017-12-12 | 2018-05-08 | 河南工业大学 | Vehicle target detection method |
| CN108734086A (en) * | 2018-03-27 | 2018-11-02 | 西安科技大学 | The frequency of wink and gaze estimation method of network are generated based on ocular |
| CN108550274A (en) * | 2018-04-04 | 2018-09-18 | 中国计量大学 | A kind of unmanned auxiliary device of traffic lights based on Faster RCNN and method |
| CN109031287A (en) * | 2018-09-21 | 2018-12-18 | 西安交通大学 | ULTRA-WIDEBAND RADAR human body respiration signal detecting method through walls based on Faster-RCNN network |
Non-Patent Citations (4)
| Title |
|---|
| 李子康等: "基于渐变上下文特征的交通灯识别方法", 《传感器与微***》 * |
| 李子康等: "基于渐变上下文特征的交通灯识别方法", 《传感器与微***》, no. 10, 26 September 2018 (2018-09-26) * |
| 王莹等: "基于深度学习的交通信号灯检测及分类方法", 《汽车实用技术》 * |
| 王莹等: "基于深度学习的交通信号灯检测及分类方法", 《汽车实用技术》, no. 17, 15 September 2018 (2018-09-15) * |
Cited By (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111079563A (en) * | 2019-11-27 | 2020-04-28 | 北京三快在线科技有限公司 | Traffic signal lamp identification method and device, electronic equipment and storage medium |
| CN111177811A (en) * | 2019-12-24 | 2020-05-19 | 武汉理工光科股份有限公司 | Automatic fire point location layout method applied to cloud platform |
| CN111738212A (en) * | 2020-07-20 | 2020-10-02 | 平安国际智慧城市科技股份有限公司 | Traffic signal lamp identification method, device, equipment and medium based on artificial intelligence |
| CN111738212B (en) * | 2020-07-20 | 2020-11-20 | 平安国际智慧城市科技股份有限公司 | Traffic signal lamp identification method, device, equipment and medium based on artificial intelligence |
| CN112289021A (en) * | 2020-09-24 | 2021-01-29 | 深圳一清创新科技有限公司 | Traffic signal lamp detection method and device and automatic driving automobile |
| CN113065466A (en) * | 2021-04-01 | 2021-07-02 | 安徽嘻哈网络技术有限公司 | Traffic light detection system for driving training based on deep learning |
| CN113065466B (en) * | 2021-04-01 | 2024-06-04 | 安徽嘻哈网络技术有限公司 | Deep learning-based traffic light detection system for driving training |
| CN113077630A (en) * | 2021-04-30 | 2021-07-06 | 安徽江淮汽车集团股份有限公司 | Traffic light detection method, device and equipment based on deep learning and storage medium |
| CN113077630B (en) * | 2021-04-30 | 2022-06-28 | 安徽江淮汽车集团股份有限公司 | Traffic light detection method, device, equipment and storage medium based on deep learning |
| CN114550468A (en) * | 2022-02-17 | 2022-05-27 | 华中科技大学 | Regional traffic signal lamp cooperation method, device, equipment and medium thereof |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109657622A (en) | A kind of detection of traffic lights and recognition methods, device and equipment | |
| CN109214399B (en) | Improved YOLOV3 target identification method embedded in SENET structure | |
| CN105574550A (en) | Vehicle identification method and device | |
| CN103258213B (en) | A kind of for the dynamic vehicle model recognizing method in intelligent transportation system | |
| CN103366602B (en) | Method of determining parking lot occupancy from digital camera images | |
| CN104166841B (en) | The quick detection recognition methods of pedestrian or vehicle is specified in a kind of video surveillance network | |
| CN109800628A (en) | A kind of network structure and detection method for reinforcing SSD Small object pedestrian detection performance | |
| CN110245644A (en) | A kind of unmanned plane image transmission tower lodging knowledge method for distinguishing based on deep learning | |
| CN107368787A (en) | A kind of Traffic Sign Recognition algorithm that application is driven towards depth intelligence | |
| CN106156744A (en) | SAR target detection method based on CFAR detection with degree of depth study | |
| CN105354568A (en) | Convolutional neural network based vehicle logo identification method | |
| CN113822247B (en) | Method and system for identifying illegal building based on aerial image | |
| CN105654066A (en) | Vehicle identification method and device | |
| CN107330453A (en) | The Pornographic image recognizing method of key position detection is recognized and merged based on substep | |
| CN107085696A (en) | A kind of vehicle location and type identifier method based on bayonet socket image | |
| CN108764082A (en) | A kind of Aircraft Targets detection method, electronic equipment, storage medium and system | |
| CN110309768B (en) | Method and equipment for detecting staff at vehicle inspection station | |
| CN109993031A (en) | A kind of animal-drawn vehicle target is driven against traffic regulations behavioral value method, apparatus and camera | |
| CN110226170A (en) | A kind of traffic sign recognition method in rain and snow weather | |
| CN106023622B (en) | A kind of method and apparatus of determining traffic lights identifying system recognition performance | |
| CN109522855A (en) | In conjunction with low resolution pedestrian detection method, system and the storage medium of ResNet and SENet | |
| CN108776777A (en) | The recognition methods of spatial relationship between a kind of remote sensing image object based on Faster RCNN | |
| CN109711407A (en) | A kind of method and relevant apparatus of Car license recognition | |
| CN106503663A (en) | A kind of signal lighties duration detection method based on deep learning | |
| CN110287936A (en) | Image detecting method, device, equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |