CN112580460A - Traffic signal lamp identification method, device, equipment and storage medium - Google Patents
Traffic signal lamp identification method, device, equipment and storage medium Download PDFInfo
- Publication number
- CN112580460A CN112580460A CN202011436885.1A CN202011436885A CN112580460A CN 112580460 A CN112580460 A CN 112580460A CN 202011436885 A CN202011436885 A CN 202011436885A CN 112580460 A CN112580460 A CN 112580460A
- Authority
- CN
- China
- Prior art keywords
- traffic signal
- signal lamp
- image
- preset
- target
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 72
- 238000003062 neural network model Methods 0.000 claims abstract description 16
- 238000012549 training Methods 0.000 claims abstract description 9
- 238000004590 computer program Methods 0.000 claims description 14
- 238000012545 processing Methods 0.000 claims description 14
- 238000005259 measurement Methods 0.000 claims description 5
- 238000010586 diagram Methods 0.000 description 11
- 230000008569 process Effects 0.000 description 7
- 238000004891 communication Methods 0.000 description 6
- 230000000903 blocking effect Effects 0.000 description 4
- 230000007613 environmental effect Effects 0.000 description 4
- 230000003287 optical effect Effects 0.000 description 4
- 230000009471 action Effects 0.000 description 3
- 238000001514 detection method Methods 0.000 description 3
- 230000006870 function Effects 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 238000004364 calculation method Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000006467 substitution reaction Methods 0.000 description 2
- 238000007792 addition Methods 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 230000001149 cognitive effect Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 239000000835 fiber Substances 0.000 description 1
- 238000005286 illumination Methods 0.000 description 1
- 230000005055 memory storage Effects 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 238000011084 recovery Methods 0.000 description 1
- 230000000717 retained effect Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
- G06V20/58—Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
- G06V20/584—Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads of vehicle lights or traffic lights
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/56—Extraction of image or video features relating to colour
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Physics & Mathematics (AREA)
- Multimedia (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Data Mining & Analysis (AREA)
- Bioinformatics & Computational Biology (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Bioinformatics & Cheminformatics (AREA)
- General Engineering & Computer Science (AREA)
- Artificial Intelligence (AREA)
- Life Sciences & Earth Sciences (AREA)
- Traffic Control Systems (AREA)
Abstract
The embodiment of the application provides a method, a device, equipment and a storage medium for identifying a traffic signal lamp. The identification method of the traffic signal lamp comprises the following steps: acquiring a first image of a target traffic signal lamp containing the driving direction of an automatic driving vehicle; and inputting the first image into a first preset identification model to obtain the characteristic information of the target traffic signal lamp, wherein the first preset identification model is a neural network model obtained by training based on a historical image containing the traffic signal lamp and the characteristic information of the traffic signal lamp corresponding to the historical image. According to the traffic signal lamp identification method, the success rate of the traffic signal lamp identification method can be improved.
Description
Technical Field
The present application belongs to the field of data processing, and in particular, to a method, an apparatus, a device, and a storage medium for identifying a traffic signal lamp.
Background
In the automatic driving technology, in order to realize safe driving of an automatic driving vehicle, a traffic light in a driving direction of the automatic driving vehicle needs to be identified.
At this stage, as disclosed in "detection and identification of traffic signal lamp" (information and electric institute of mining university in china, marmann), a template matching method is generally adopted for identification of traffic signal lamps. Specifically, a camera device can be used for acquiring a real-time image of the traffic signal lamp included in the driving direction of the automatic driving vehicle in real time, and then image elements matched with the template are searched in the real-time image, so that the identification of the characteristic information of the traffic signal lamp, such as the color and the shape of the traffic signal lamp, is realized. However, since the template matching can only be performed by parallel movement, once the traffic signal in the image is deformed or partially shielded, the matching may fail, and thus the success rate of the traffic signal recognition method may be low.
Disclosure of Invention
The embodiment of the application provides a traffic signal lamp identification method, a traffic signal lamp identification device, traffic signal lamp identification equipment and a storage medium, and the success rate of the traffic signal lamp identification method can be improved.
In a first aspect, an embodiment of the present application provides a method for identifying a traffic signal lamp, including:
acquiring a first image of a target traffic signal lamp containing the driving direction of an automatic driving vehicle;
and inputting the first image into a first preset identification model to obtain the characteristic information of the target traffic signal lamp, wherein the first preset identification model is a neural network model obtained by training based on the historical image containing the traffic signal lamp and the characteristic information of the traffic signal lamp corresponding to the historical image.
In some embodiments, before acquiring the first image of the target traffic light including the driving direction of the autonomous vehicle, the method further comprises:
acquiring first position information of the automatic driving vehicle, wherein the first position information is determined based on the positioning device, the inertia measurement unit and the collected data of the radar;
determining target position information of the autonomous vehicle based on the first position information and a preset map;
and determining a target traffic signal lamp in the driving direction of the automatic driving vehicle, wherein the distance between the preset map and the target position information is less than or equal to a preset threshold value based on the target position information and the preset map.
In some embodiments, determining that a distance from the target location information in the preset map is less than or equal to a preset threshold value and a target traffic light in a driving direction of the autonomous vehicle based on the target location information and the preset map includes:
acquiring second position information of the first traffic signal lamp in a preset map;
calculating a distance between the target position information and the second position information of the autonomous vehicle;
and determining the first traffic signal lamp as the target traffic signal lamp when the distance is smaller than or equal to the preset threshold value.
In some embodiments, before inputting the first image to the first preset recognition model, the method further includes:
inputting the first image into a second preset identification model, and judging whether the first image contains a target traffic signal lamp;
under the condition that the first image contains the target traffic signal lamp, controlling a second preset recognition model to output a second image, wherein the second image contains the target traffic signal lamp and a boundary frame for identifying the target traffic signal lamp;
inputting a first image to a first preset recognition model, comprising:
and inputting the second image to the first preset recognition model.
In some embodiments, before inputting the first image to the second preset recognition model, the method further includes:
performing image cutting processing on the first image to obtain a cut image;
inputting the first image to a second preset recognition model, comprising:
and inputting the image subjected to the cutting processing into a second preset identification model.
In some embodiments, after inputting the first image into the second preset identification model and determining whether the first image includes the target traffic signal lamp, the method further includes:
under the condition that the first image does not contain the target traffic signal lamp, whether a shelter for sheltering the target traffic signal lamp exists in the driving direction of the automatic driving vehicle or not is identified;
and under the condition that no shielding object exists in the driving direction of the automatic driving vehicle, controlling the automatic driving vehicle to stop and outputting prompt information.
In some embodiments, the characteristic information includes a color, a position, a shape, a number of counts of the target traffic signal.
In a second aspect, an identification apparatus for a traffic signal is provided, including:
the system comprises an acquisition module, a display module and a display module, wherein the acquisition module is used for acquiring a first image of a target traffic signal lamp containing the driving direction of an automatic driving vehicle;
the identification module is used for inputting the first image into a first preset identification model to obtain the characteristic information of the target traffic signal lamp, wherein the first preset identification model is a neural network model obtained based on historical images containing the traffic signal lamp and historical characteristic information training.
In a third aspect, an embodiment of the present invention provides an identification device for a traffic signal lamp, including: a processor and a memory storing computer program instructions;
the processor, when executing the computer program instructions, implements a method of identifying a traffic signal as in the first aspect.
In a fourth aspect, an embodiment of the present invention provides a computer storage medium, on which computer program instructions are stored, and when the computer program instructions are executed by a processor, the method for identifying a traffic signal lamp according to the first aspect is implemented.
According to the traffic signal lamp identification method, the traffic signal lamp identification device, the traffic signal lamp identification equipment and the storage medium, a first image of a target traffic signal lamp containing the driving direction of an automatic driving vehicle is obtained; and inputting the first image into a first preset identification model to obtain the characteristic information of the target traffic signal lamp, wherein the first preset identification model is a neural network model obtained based on historical images containing the target traffic signal lamp and historical characteristic information training. Therefore, the first image containing the target traffic signal lamp is identified based on the neural network model, so that the identification failure caused by deformation or partial shielding of the target traffic signal lamp in the image can be avoided, and the success rate of the identification method of the traffic signal lamp can be effectively improved.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed to be used in the embodiments of the present application will be briefly described below, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.
Fig. 1 is a schematic flowchart of an identification method for a traffic signal lamp according to an embodiment of the present disclosure;
fig. 2 is a schematic flow chart of an identification method of a traffic signal lamp according to an embodiment of the present disclosure;
fig. 3 is a schematic structural diagram of an identification device of a traffic signal lamp according to an embodiment of the present disclosure;
fig. 4 is a schematic structural diagram of an identification device of a traffic signal lamp according to an embodiment of the present application.
Detailed Description
Features and exemplary embodiments of various aspects of the present application will be described in detail below, and in order to make objects, technical solutions and advantages of the present application more apparent, the present application will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application. It will be apparent to one skilled in the art that the present application may be practiced without some of these specific details. The following description of the embodiments is merely intended to provide a better understanding of the present application by illustrating examples thereof.
It is noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.
In the prior art, when the traffic signal lamp is identified by adopting a template matching method, the traffic signal lamp in an image is deformed or partially shielded, so that the matching is failed, and the success rate of the traffic signal lamp identification method is low.
Therefore, based on the above findings, in order to solve the technical problems in the prior art, embodiments of the present application provide a method, an apparatus, a device and a computer storage medium for identifying a traffic signal,
obtaining a first image of a target traffic signal light containing a driving direction of an autonomous vehicle; and inputting the first image into a first preset identification model to obtain the characteristic information of the target traffic signal lamp, wherein the first preset identification model is a neural network model obtained based on historical images containing the target traffic signal lamp and historical characteristic information training. Therefore, the first image containing the target traffic signal lamp is identified based on the neural network model, so that the identification failure caused by deformation or partial shielding of the target traffic signal lamp in the image can be avoided, and the success rate of the identification method of the traffic signal lamp can be effectively improved.
First, a method for identifying a traffic signal provided in an embodiment of the present application is described with reference to fig. 1.
Fig. 1 shows a schematic flow chart of an identification method of a traffic signal lamp according to an embodiment of the present application. As shown in fig. 1, the method may include the steps of:
s110, a first image of a target traffic light containing the driving direction of the automatic driving vehicle is obtained.
As one example, an autonomous vehicle needs characteristic information of a traffic light that identifies a driving direction during driving to assist autonomous driving. When a traffic signal lamp, i.e., a target traffic signal lamp, in the driving direction of the autonomous vehicle is identified, an image of the driving direction of the autonomous vehicle may be acquired by using a camera (e.g., a camera, a video camera, etc.) to obtain an image, i.e., a first image, of the target traffic signal lamp including the driving direction of the autonomous vehicle. Then, a first image collected by the camera device and containing a target traffic signal lamp of the driving direction of the automatic driving vehicle can be obtained.
And S120, inputting the first image into a first preset identification model to obtain the characteristic information of the target traffic signal lamp.
The first preset identification model can be a neural network model obtained by training based on historical images containing traffic lights and characteristic information of the traffic lights corresponding to the historical images.
As one example, after a first image of a target traffic signal light containing a driving direction of an autonomous vehicle is acquired, the first image may be input to a first preset recognition model. And outputting the characteristic information of the target traffic signal lamp based on the first image by using a first preset identification model.
According to the traffic signal lamp identification method, the traffic signal lamp identification device, the traffic signal lamp identification equipment and the storage medium, a first image of a target traffic signal lamp containing the driving direction of an automatic driving vehicle is obtained; and inputting the first image into a first preset identification model to obtain the characteristic information of the target traffic signal lamp, wherein the first preset identification model is a neural network model obtained based on historical images containing the target traffic signal lamp and historical characteristic information training. Therefore, the first image containing the target traffic signal lamp is identified based on the neural network model, so that the identification failure caused by deformation or partial shielding of the target traffic signal lamp in the image can be avoided, and the success rate of the identification method of the traffic signal lamp can be effectively improved.
And the recognition of the target traffic signal lamp is realized based on the neural network model, the accuracy of the recognized characteristic information of the target traffic signal lamp can be further improved, and the safety of the automatic driving vehicle is further improved.
In some embodiments, the target traffic signal may be determined according to a preset map and the position information of the autonomous vehicle, and accordingly, the following steps may be further performed before the step S110:
acquiring first position information of an autonomous vehicle;
determining target position information of the autonomous vehicle based on the first position information and a preset map;
and determining a target traffic signal lamp in the driving direction of the automatic driving vehicle, wherein the distance between the preset map and the target position information is less than or equal to a preset threshold value based on the target position information and the preset map.
The first position information may be determined based on the acquired data of the positioning apparatus, an Inertial Measurement Unit (IMU), and the radar.
The preset map may be a preset map with high accuracy.
The preset threshold is a preset maximum allowable value of the distance between the target traffic signal lamp and the automatic driving vehicle. For the traffic signal lamp with too large distance from the automatic driving vehicle, as the automatic driving vehicle may still need to spend long time to the traffic signal lamp, the characteristic information of the traffic signal lamp does not influence the safe driving of the automatic driving vehicle generally, therefore, only the traffic signal lamp with the distance from the automatic driving vehicle less than or equal to the preset threshold value can be determined as the target traffic signal lamp, thereby reducing the unnecessary calculation amount to a certain extent and further improving the identification efficiency of the traffic signal lamp.
As one example, the target traffic light may be determined prior to acquiring the first image. Specifically, the collected data of the positioning device (e.g., GPS), IMU, and radar (e.g., lidar) may be obtained first to obtain the position information of the autonomous vehicle, i.e., the first position information. And then acquiring a preset map, and comparing the first position information with the high-precision map to determine the position information of the automatic driving vehicle, namely the target position information. Then, it may be determined that a distance from the target position information in the preset map is less than or equal to a preset threshold value and a target traffic light in a traveling direction of the autonomous vehicle based on the target position information of the autonomous vehicle and the preset map. Then, a first image of the target traffic signal is acquired.
It can be understood that the environmental information around the autonomous vehicle can be determined according to the collected data of the positioning device, the IMU and the radar, the first position information and the environmental information of the autonomous vehicle are respectively compared with a preset map, and the target position information of the autonomous vehicle is determined by combining the environmental information.
Therefore, on the basis of determining the first position information of the automatic driving vehicle based on the collected data of the positioning device, the inertia measurement unit and the radar, the target traffic signal lamp is further determined by combining the preset map, the accuracy of the determined target traffic signal lamp can be improved, the efficiency of the identification method of the traffic signal lamp can be further improved, and the accuracy of the identification result of the traffic signal lamp is improved.
And on the basis of the first position information and the preset map, the target traffic signal lamp is determined by combining the environmental information around the automatic driving vehicle, so that the efficiency of the identification method of the traffic signal lamp can be further improved, and the accuracy of the identification result of the traffic signal lamp is improved.
In some embodiments, the specific implementation manner of determining the target traffic signal lamp based on the target position information of the autonomous vehicle and the preset map may be:
acquiring second position information of the first traffic signal lamp in a preset map;
calculating a distance between the target position information and the second position information of the autonomous vehicle;
and determining the first traffic signal lamp as the target traffic signal lamp when the distance is smaller than or equal to the preset threshold value.
The first traffic light is a traffic light in the driving direction of the automatic driving vehicle in a preset map, and the number of the first traffic lights can be one or more. It is understood that the first traffic signal may be selected from a preset map in such a manner that the distance from the autonomous vehicle in the driving direction of the autonomous vehicle is from far to near.
As one example, a first traffic light in the driving direction of the autonomous vehicle in a preset map may be determined, and the position information of the first traffic light, that is, the second position information may be acquired. The distance between the target position information and the second position information of the autonomous vehicle is calculated, and the distance can be compared with a preset threshold value to judge whether the distance is smaller than or equal to the preset threshold value. And determining the first traffic signal lamp as a target traffic signal lamp when the distance is less than or equal to a preset threshold value.
In some embodiments, after determining the target traffic light based on the first position information and the preset map, the second image containing the target traffic light may be input to the first preset recognition model, and accordingly, before the step S110, the following steps may be further performed:
inputting the first image into a second preset identification model, and judging whether the first image contains a target traffic signal lamp;
and controlling a second preset recognition model to output a second image under the condition that the first image contains the target traffic signal lamp.
The second image comprises a target traffic light and a boundary frame for marking the target traffic light.
At this time, the specific implementation manner of the step S110 may be as follows:
and inputting the second image to the first preset recognition model.
As an example, after determining the target traffic signal based on the first position information and the preset map, the camera may be controlled to capture the first image. Since the target traffic light is determined based on the first position information and the preset map, the first image captured at this time should be the one including the target traffic light. However, it is considered that the first image acquired by the image capturing device, which should include the target traffic signal light, may not actually include the target traffic signal light due to weather, object occlusion, and the like.
Therefore, before the first image is input to the first preset recognition model, the first image may be input to the second preset recognition model, and the second preset recognition model may be trained in advance and used for recognizing whether the image includes a traffic signal lamp, and the first image including the target traffic signal lamp may be processed to obtain the neural network model of the second image. And judging whether the first image contains the target traffic signal lamp or not by utilizing a second preset identification model. In the case where the first image includes the target traffic signal, the second preset recognition model may be controlled to output a second image based on the first image, and the second image may include a region of interest (ROI), i.e., the target traffic signal, and a bounding box of the ROI. Then, the second image can be input into the first preset identification model, and the characteristic information of the target traffic signal lamp is output based on the second image by using the first identification model.
In this way, when the target traffic signal is included in the first image, the second image is input to the first preset recognition model. Therefore, the failure of identification caused by that the first image does not actually contain the target traffic signal lamp can be avoided, and the success rate of the traffic signal lamp identification method can be further improved.
In some embodiments, the cropping process may be performed before the first image is input into the second preset recognition model, and the specific implementation manner may be as follows:
performing image cutting processing on the first image to obtain a cut image;
in this case, the inputting the first image into the second preset recognition model in S110 includes:
and inputting the image subjected to the cutting processing into a second preset identification model.
As an example, considering that a traffic light usually exists in an upper middle area of an image, before the first image is input to the second preset recognition model, the first image may be cropped to obtain a cropped image. For example, the upper half of the first image (e.g., 70% of the first image, and the 70% is located at the upper half of the first image) may be retained, and the road, vehicle, building, and other extraneous information under the first image may be removed. Then, the image after the clipping processing may be input into a second preset recognition model, and the second preset recognition model determines whether the first image includes the target traffic signal lamp.
In this way, by removing the irrelevant information in the first image through the cutting processing, the calculation amount of the second preset identification model can be reduced, so that the identification speed can be further improved, and the identification efficiency can be improved.
In some embodiments, in the case that there is no target traffic light and no obstruction in the first image, a prompt signal may also be output, and a specific implementation manner thereof may be as follows:
under the condition that the first image does not contain the target traffic signal lamp, whether a shelter for sheltering the target traffic signal lamp exists in the driving direction of the automatic driving vehicle or not is identified;
and under the condition that no shielding object exists in the driving direction of the automatic driving vehicle, controlling the automatic driving vehicle to stop and outputting prompt information.
As an example, when the second preset recognition model determines that the first image does not include the target traffic light, it may be detected whether there is an obstruction, such as a vehicle, blocking the target traffic light in the driving direction of the autonomous vehicle. Since the target traffic light is determined based on the first position information and the preset map, that is, if no blocking object exists, the first image should include the target traffic light, and at this time, if it is detected that no blocking object for blocking the target traffic light exists in the driving direction of the autonomous vehicle, it indicates that a fault may occur at this time.
Therefore, at the moment, the vehicle can be controlled to stop, and prompt information can be output and used for prompting relevant personnel to take over the treatment. If the situation that the object sheltering the target traffic light exists in the driving direction of the automatic driving vehicle is detected, the automatic driving vehicle can be controlled to continue to drive with the vehicle, and for example, the position information of the automatic driving vehicle in a preset map, the lane information of the automatic driving vehicle and a navigation map can be combined to control the automatic driving vehicle to drive with the vehicle. In this way, the safety of the autonomous vehicle can be further improved.
In some embodiments, the characteristic information may include a color, a position, a shape, a number of hours of the target traffic signal lamp. The shape refers to an indication arrow of the target traffic signal lamp, such as a left-turn arrow, a right-turn arrow, a turning arrow, a straight arrow, a circle and the like, and may be red, a left-turn arrow, 10 meters in front of the automatic driving vehicle, and 15 seconds counted down.
As one example, the first preset recognition model may be employed to recognize characteristic information of the target traffic signal.
Specifically, the output of the first predetermined identification model may include a 4-dimensional vector, each dimension may represent a probability value of each color (black, red, yellow, and green), and the color corresponding to the probability value which is the largest and is greater than the predetermined minimum value may be determined as the color of the target traffic signal lamp. Otherwise, the signal lamp state is judged to be black, namely the color of the target traffic signal lamp is unknown. The first preset identification model can also extract the shape of the image area of the target traffic signal lamp and output the shape of the traffic signal lamp, such as any one of the shapes of a left-turn arrow, a right-turn arrow, a turning arrow, a straight arrow, a circle and the like. The first preset identification model can also identify a digital countdown board of the target traffic signal lamp to obtain the countdown time of the target traffic signal lamp. The first preset identification model can also summarize the color, shape, countdown and other information of the signal lamp and send the summarized information to the control module of the unmanned vehicle.
After the color, shape and timing number of the target traffic signal are obtained, the characteristic information including the color, shape, timing number, position and the like of the target traffic signal can be combined. And may generate a corresponding control instruction based on the characteristic information to be sent to a control unit of the autonomous vehicle.
It is understood that the cognitive information behind the target traffic signal may be analyzed according to the characteristic information. If it is assumed that the characteristic information is red, a left-turn arrow, 10 meters ahead of the autonomous vehicle, and 15 seconds for countdown, the color jump time of the target traffic signal may be analyzed in combination with the countdown time of 15 seconds and the color red of the target traffic signal.
Therefore, according to the characteristic information including the color, the position, the shape and the timing number of the target traffic signal lamp output by the first preset identification model, the obtained characteristic information of the target traffic signal lamp can be more complete, so that the automatic driving vehicle can be better assisted to run, and the safety of the automatic driving vehicle is improved.
Fig. 2 illustrates an identification method of a traffic signal lamp according to an embodiment of the present application, and as shown in fig. 2, the identification method of the traffic signal lamp may include:
s210, a first image of a target traffic light containing the driving direction of the automatic driving vehicle is obtained.
S220, inputting the first image into a second preset identification model, and judging whether the first image contains a target traffic signal lamp.
And S230, whether the first image contains the target traffic signal lamp or not is judged.
If the target traffic signal is included in the first image, S240-S270 are performed.
And S240, outputting a second image.
And S250, inputting the second image into the first preset identification model.
And S260, obtaining the characteristic information of the target traffic signal lamp.
And S270, generating a control command.
If the target traffic signal is not included in the first image, S280 is performed.
And S280, detecting whether a shelter of the target traffic signal lamp exists in the driving direction of the automatic driving vehicle.
If there is no obstruction in the direction of travel of the autonomous vehicle, S290 is executed.
And S290, controlling the automatic driving vehicle to continuously follow the vehicle.
If there is no obstruction in the direction of travel of the autonomous vehicle, S2100 is executed.
And S2100, controlling the automatic driving vehicle to stop and outputting prompt information.
Like this, because traffic signal lamp's appearance is various, the position also is different, receives the influence of various factors such as light, shelter from thing, trouble easily, this application combines to predetermine the map and confirms whether there is target traffic signal lamp in the direction of driving of autopilot vehicle to confirm the position of autopilot vehicle and target traffic signal lamp's position, make target traffic signal lamp's location more accurate. Moreover, the traffic signal lamp is recognized based on the preset recognition model, and the influence of factors such as illumination, partial shielding and the like can be avoided. The condition that the signal lamp is not detected is fully considered, and safe driving of the vehicle is further guaranteed. Meanwhile, the neural network model is adopted, the characteristic information such as the color, the shape, the counting time and the like of the traffic signal lamp is detected, the detection precision is higher, and the detection result is more accurate, so that the safety of automatic driving can be further ensured.
The specific implementation method of the above steps is the same as that of the above method embodiments, and for the sake of brevity, detailed description is omitted here.
Based on the same inventive concept, the traffic signal lamp identification method provided in the embodiment of the present application further provides a traffic signal lamp identification device, and the following detailed description will be made on the traffic signal lamp identification device.
Fig. 3 is a schematic structural diagram of an identification device of a traffic signal lamp according to an embodiment of the present application. As shown in fig. 3, the identification apparatus 300 for a traffic signal may include:
an acquisition module 310 for acquiring a first image of a target traffic signal including a driving direction of an autonomous vehicle;
the identification module 320 is configured to input the first image into a first preset identification model to obtain feature information of the target traffic signal, where the first preset identification model is a neural network model trained based on a historical image and historical feature information that include the traffic signal.
In some embodiments, the traffic signal light recognition apparatus 300 may further include a determination module, and the determination module may include:
the first acquisition unit can be used for acquiring first position information of the automatic driving vehicle, and the first position information is determined based on the positioning device, the inertia measurement unit and the collected data of the radar;
a first determination unit that may be configured to determine target location information of the autonomous vehicle based on the first location information and a preset map;
and the second determination unit can be used for determining a target traffic signal lamp which is in the driving direction of the automatic driving vehicle and has a distance with the target position information smaller than or equal to a preset threshold value in the preset map based on the target position information and the preset map.
In some embodiments, the second determining unit may include:
the first acquiring subunit is configured to acquire second position information of the first traffic signal lamp in a preset map;
a calculating subunit operable to calculate a distance between the target position information and the second position information of the autonomous vehicle;
the determining subunit may be configured to determine the first traffic signal lamp as the target traffic signal lamp if the distance is less than or equal to a preset threshold.
In some embodiments, the identification apparatus 300 for a traffic signal lamp may further include a determination module, and the determination module may include:
the judging unit can be used for inputting the first image into the second preset identification model and judging whether the first image contains the target traffic signal lamp;
the first control unit can be used for controlling the second preset recognition model to output a second image under the condition that the first image contains the target traffic signal lamp, wherein the second image contains the target traffic signal lamp and a boundary frame for marking the target traffic signal lamp;
an identification module 320 comprising:
the first recognition unit may be configured to input the second image to a first preset recognition model.
In some embodiments, the identification apparatus 300 for a traffic signal lamp may further include:
the cutting module can be used for carrying out image cutting processing on the first image to obtain a cut image;
the judging unit may include:
and the input subunit is used for inputting the image after the cutting processing to the second preset identification model.
In some embodiments, the determining module may include:
the second identification unit can be used for identifying whether a shelter for sheltering the target traffic signal lamp exists in the driving direction of the automatic driving vehicle or not under the condition that the target traffic signal lamp is not contained in the first image;
and the second control unit can be used for controlling the automatic driving vehicle to stop and outputting the prompt information under the condition that no shielding object exists in the driving direction of the automatic driving vehicle.
In some embodiments, the characteristic information includes a color, a position, a shape, a number of counts of the target traffic signal.
The identification device of the traffic signal lamp can be used for executing the methods provided by the method embodiments, the implementation principle and the effect are similar, and for the sake of brevity, the description is omitted here.
The traffic signal lamp identification method and device provided by the embodiment of the application are based on the traffic signal lamp identification method and device, and the application also provides traffic signal lamp identification equipment, and the following embodiments are specifically referred to.
Fig. 4 shows a hardware structure diagram of an identification device of a traffic signal lamp provided in an embodiment of the present application.
The identification device of the traffic signal lamp may include a processor 401 and a memory 402 storing computer program instructions.
Specifically, the processor 401 may include a Central Processing Unit (CPU), an Application Specific Integrated Circuit (ASIC), or one or more Integrated circuits configured to implement the embodiments of the present invention.
The memory may include Read Only Memory (ROM), Random Access Memory (RAM), magnetic disk storage media devices, optical storage media devices, flash memory devices, electrical, optical, or other physical/tangible memory storage devices. Thus, in general, the memory includes one or more tangible (non-transitory) computer-readable storage media (e.g., a memory device) encoded with software comprising computer-executable instructions and when the software is executed (e.g., by one or more processors), it is operable to perform the operations described with reference to the method according to the first aspect of the application.
The processor 401 reads and executes the computer program instructions stored in the memory 402 to implement any one of the traffic signal light identification methods in the above embodiments.
In one example, the identification device of the traffic signal light may also include a communication interface 403 and a bus 410. As shown in fig. 4, the processor 401, the memory 402, and the communication interface 403 are connected via a bus 410 to complete communication therebetween.
The communication interface 403 is mainly used for implementing communication between modules, apparatuses, units and/or devices in the embodiments of the present application.
The bus 410 includes hardware, software, or both that couple the components of the identification device of the traffic signal lamp to one another. By way of example, and not limitation, a Bus may include an Accelerated Graphics Port (AGP) or other Graphics Bus, an Enhanced Industry Standard Architecture (EISA) Bus, a Front-Side Bus (Front Side Bus, FSB), a Hyper Transport (HT) interconnect, an Industry Standard Architecture (ISA) Bus, an infiniband interconnect, a Low Pin Count (LPC) Bus, a memory Bus, a Micro Channel Architecture (MCA) Bus, a Peripheral Component Interconnect (PCI) Bus, a PCI-Express (PCI-X) Bus, a Serial Advanced Technology Attachment (SATA) Bus, a video electronics standards association local (VLB) Bus, or other suitable Bus or a combination of two or more of these. Bus 410 may include one or more buses, where appropriate. Although specific buses are described and shown in the embodiments of the application, any suitable buses or interconnects are contemplated by the application.
In addition, in combination with the identification method of the traffic signal lamp in the above embodiments, the embodiments of the present application may be implemented by providing a computer storage medium. The computer storage medium having computer program instructions stored thereon; the computer program instructions, when executed by the processor, implement any of the above-described embodiments of the traffic signal light identification method.
It is to be understood that the present application is not limited to the particular arrangements and instrumentality described above and shown in the attached drawings. A detailed description of known methods is omitted herein for the sake of brevity. In the above embodiments, several specific steps are described and shown as examples. However, the method processes of the present application are not limited to the specific steps described and illustrated, and those skilled in the art can make various changes, modifications, and additions or change the order between the steps after comprehending the spirit of the present application.
The functional blocks shown in the above-described structural block diagrams may be implemented as hardware, software, firmware, or a combination thereof. When implemented in hardware, it may be, for example, an electronic Circuit, an Application Specific Integrated Circuit (ASIC), suitable firmware, plug-in, function card, or the like. When implemented in software, the elements of the present application are the programs or code segments used to perform the required tasks. The program or code segments may be stored in a machine-readable medium or transmitted by a data signal carried in a carrier wave over a transmission medium or a communication link. A "machine-readable medium" may include any medium that can store or transfer information. Examples of a machine-readable medium include electronic circuits, semiconductor memory devices, ROM, flash memory, Erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, fiber optic media, Radio Frequency (RF) links, and so forth. The code segments may be downloaded via computer networks such as the internet, intranet, etc.
It should also be noted that the exemplary embodiments mentioned in this application describe some methods or systems based on a series of steps or devices. However, the present application is not limited to the order of the above-described steps, that is, the steps may be performed in the order mentioned in the embodiments, may be performed in an order different from the order in the embodiments, or may be performed simultaneously.
Aspects of the present disclosure are described above with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the disclosure. It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, enable the implementation of the functions/acts specified in the flowchart and/or block diagram block or blocks. Such a processor may be, but is not limited to, a general purpose processor, a special purpose processor, an application specific processor, or a field programmable logic circuit. It will also be understood that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware for performing the specified functions or acts, or combinations of special purpose hardware and computer instructions.
As described above, only the specific embodiments of the present application are provided, and it can be clearly understood by those skilled in the art that, for convenience and brevity of description, the specific working processes of the system, the module and the unit described above may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again. It should be understood that the scope of the present application is not limited thereto, and any person skilled in the art can easily conceive various equivalent modifications or substitutions within the technical scope of the present application, and these modifications or substitutions should be covered within the scope of the present application.
Claims (10)
1. A method for identifying a traffic signal, comprising:
acquiring a first image of a target traffic signal lamp containing the driving direction of an automatic driving vehicle;
and inputting the first image into a first preset identification model to obtain the characteristic information of the target traffic signal lamp, wherein the first preset identification model is a neural network model obtained by training based on a historical image containing the traffic signal lamp and the characteristic information of the traffic signal lamp corresponding to the historical image.
2. The method of claim 1, wherein prior to obtaining the first image of the target traffic signal light comprising the direction of travel of the autonomous vehicle, further comprising:
acquiring first position information of the automatic driving vehicle, wherein the first position information is determined based on the positioning device, the inertia measurement unit and the collected data of the radar;
determining target location information of the autonomous vehicle based on the first location information and a preset map;
and determining a target traffic signal lamp in the driving direction of the automatic driving vehicle, wherein the distance between the target position information and the preset map is smaller than or equal to a preset threshold value based on the target position information and the preset map.
3. The method of claim 2, wherein the determining that the distance from the target location information in the preset map is less than or equal to a preset threshold and a target traffic light in the driving direction of the autonomous vehicle based on the target location information and the preset map comprises:
acquiring second position information of the first traffic signal lamp in the preset map;
calculating a distance between the target position information of the autonomous vehicle and the second position information;
and determining the first traffic signal lamp as a target traffic signal lamp when the distance is less than or equal to a preset threshold value.
4. The method according to any one of claims 1-3, further comprising, prior to said inputting said first image into a first predetermined recognition model:
inputting the first image into a second preset identification model, and judging whether the first image contains the target traffic signal lamp;
under the condition that the first image contains the target traffic signal lamp, controlling the second preset recognition model to output a second image, wherein the second image contains the target traffic signal lamp and a boundary frame for identifying the target traffic signal lamp;
the inputting the first image to a first preset recognition model comprises:
inputting the second image to the first preset recognition model.
5. The method according to claim 4, wherein before inputting the first image into the second preset recognition model, further comprising:
performing image cutting processing on the first image to obtain a cut image;
the inputting the first image to a second preset recognition model comprises:
and inputting the image after the cutting processing to the second preset identification model.
6. The method of claim 4, wherein after inputting the first image into a second predetermined recognition model and determining whether the target traffic signal is included in the first image, the method further comprises:
under the condition that the target traffic signal lamp is not included in the first image, identifying whether an obstruction for obstructing the target traffic signal lamp exists in the driving direction of the automatic driving vehicle or not;
and under the condition that the sheltering object does not exist in the driving direction of the automatic driving vehicle, controlling the automatic driving vehicle to stop and outputting prompt information.
7. The method of claim 1, wherein the characteristic information includes a color, a position, a shape, a number of hours of counting of the target traffic signal lamp.
8. An identification device for a traffic signal, comprising:
the system comprises an acquisition module, a display module and a display module, wherein the acquisition module is used for acquiring a first image of a target traffic signal lamp containing the driving direction of an automatic driving vehicle;
and the recognition module is used for inputting the first image into a first preset recognition model to obtain the characteristic information of the target traffic signal lamp, wherein the first preset recognition model is a neural network model obtained based on historical images containing the traffic signal lamp and historical characteristic information training.
9. An identification device for a traffic signal, the device comprising: a processor and a memory storing computer program instructions;
the processor, when executing the computer program instructions, implements the method of traffic signal identification as claimed in any of claims 1-7.
10. A computer storage medium, characterized in that the computer storage medium has stored thereon computer program instructions which, when executed by a processor, implement the identification method of a traffic signal lamp according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011436885.1A CN112580460A (en) | 2020-12-11 | 2020-12-11 | Traffic signal lamp identification method, device, equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011436885.1A CN112580460A (en) | 2020-12-11 | 2020-12-11 | Traffic signal lamp identification method, device, equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112580460A true CN112580460A (en) | 2021-03-30 |
Family
ID=75130419
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011436885.1A Pending CN112580460A (en) | 2020-12-11 | 2020-12-11 | Traffic signal lamp identification method, device, equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112580460A (en) |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112799412A (en) * | 2021-04-12 | 2021-05-14 | 北京三快在线科技有限公司 | Control method and control device of unmanned equipment |
| CN114694123A (en) * | 2022-05-30 | 2022-07-01 | 阿里巴巴达摩院(杭州)科技有限公司 | Traffic signal lamp sensing method, device, equipment and storage medium |
| CN115063764A (en) * | 2022-05-26 | 2022-09-16 | 广州文远知行科技有限公司 | Traffic signal lamp state estimation method and device and electronic equipment |
| CN115984823A (en) * | 2023-02-27 | 2023-04-18 | 安徽蔚来智驾科技有限公司 | Traffic signal lamp sensing method, vehicle control method, device, medium and vehicle |
| WO2023116113A1 (en) * | 2021-12-24 | 2023-06-29 | 广州小鹏自动驾驶科技有限公司 | Traffic light state detection method and apparatus, vehicle and storage medium |
| CN116468868A (en) * | 2023-04-27 | 2023-07-21 | 广州小鹏自动驾驶科技有限公司 | Traffic signal lamp graph building method, device, equipment and storage medium |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109544955A (en) * | 2018-12-26 | 2019-03-29 | 广州小鹏汽车科技有限公司 | A kind of state acquiring method and system of traffic lights |
| CN110688992A (en) * | 2019-12-09 | 2020-01-14 | 中智行科技有限公司 | Traffic signal identification method and device, vehicle navigation equipment and unmanned vehicle |
| CN111582189A (en) * | 2020-05-11 | 2020-08-25 | 腾讯科技（深圳）有限公司 | Traffic signal lamp identification method and device, vehicle-mounted control terminal and motor vehicle |
| CN111695546A (en) * | 2020-06-28 | 2020-09-22 | 北京京东乾石科技有限公司 | Traffic signal lamp identification method and device for unmanned vehicle |
-
2020
- 2020-12-11 CN CN202011436885.1A patent/CN112580460A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109544955A (en) * | 2018-12-26 | 2019-03-29 | 广州小鹏汽车科技有限公司 | A kind of state acquiring method and system of traffic lights |
| CN110688992A (en) * | 2019-12-09 | 2020-01-14 | 中智行科技有限公司 | Traffic signal identification method and device, vehicle navigation equipment and unmanned vehicle |
| CN111582189A (en) * | 2020-05-11 | 2020-08-25 | 腾讯科技（深圳）有限公司 | Traffic signal lamp identification method and device, vehicle-mounted control terminal and motor vehicle |
| CN111695546A (en) * | 2020-06-28 | 2020-09-22 | 北京京东乾石科技有限公司 | Traffic signal lamp identification method and device for unmanned vehicle |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112799412A (en) * | 2021-04-12 | 2021-05-14 | 北京三快在线科技有限公司 | Control method and control device of unmanned equipment |
| WO2023116113A1 (en) * | 2021-12-24 | 2023-06-29 | 广州小鹏自动驾驶科技有限公司 | Traffic light state detection method and apparatus, vehicle and storage medium |
| CN115063764A (en) * | 2022-05-26 | 2022-09-16 | 广州文远知行科技有限公司 | Traffic signal lamp state estimation method and device and electronic equipment |
| CN114694123A (en) * | 2022-05-30 | 2022-07-01 | 阿里巴巴达摩院(杭州)科技有限公司 | Traffic signal lamp sensing method, device, equipment and storage medium |
| CN115984823A (en) * | 2023-02-27 | 2023-04-18 | 安徽蔚来智驾科技有限公司 | Traffic signal lamp sensing method, vehicle control method, device, medium and vehicle |
| CN115984823B (en) * | 2023-02-27 | 2023-06-09 | 安徽蔚来智驾科技有限公司 | Traffic signal lamp sensing method, vehicle control method, device, medium and vehicle |
| CN116468868A (en) * | 2023-04-27 | 2023-07-21 | 广州小鹏自动驾驶科技有限公司 | Traffic signal lamp graph building method, device, equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112580460A (en) | Traffic signal lamp identification method, device, equipment and storage medium | |
| US20100110193A1 (en) | Lane recognition device, vehicle, lane recognition method, and lane recognition program | |
| CN112541396A (en) | Lane line detection method, device, equipment and computer storage medium | |
| US20170259814A1 (en) | Method of switching vehicle drive mode from automatic drive mode to manual drive mode depending on accuracy of detecting object | |
| CN108583622B (en) | Method, device, equipment and medium for identifying rail traffic conditions | |
| JP2007179386A (en) | Method and apparatus for recognizing white line | |
| KR101991067B1 (en) | Apparatus and method for controlling autonomous vehicle | |
| US9508000B2 (en) | Object recognition apparatus | |
| CN110751040B (en) | Three-dimensional object detection method and device, electronic equipment and storage medium | |
| US20180005073A1 (en) | Road recognition apparatus | |
| EP3690802A1 (en) | Vehicle exterior recognition device | |
| CN113297881A (en) | Target detection method and related device | |
| EP3557529A1 (en) | Lane marking recognition device | |
| CN113723216A (en) | Lane line detection method and device, vehicle and storage medium | |
| CN111967384A (en) | Vehicle information processing method, device, equipment and computer readable storage medium | |
| CN113569812A (en) | Unknown obstacle identification method and device and electronic equipment | |
| CN114264310B (en) | Positioning and navigation method, device, electronic equipment and computer storage medium | |
| EP3287940A1 (en) | Intersection detection system for a vehicle | |
| CN117523914A (en) | Collision early warning method, device, equipment, readable storage medium and program product | |
| KR101595317B1 (en) | Precise positioning of the vehicle for detecting a road surface display method and system | |
| CN114998861A (en) | Method and device for detecting distance between vehicle and obstacle | |
| CN112572471B (en) | Automatic driving method, device, electronic equipment and computer storage medium | |
| CN115861659A (en) | Object matching method, device, equipment and computer storage medium | |
| CN112307890B (en) | Object identification method and device, object identification equipment and storage medium | |
| CN114529874A (en) | Behavior detection method, device and equipment and readable storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20210330 |