WO2015024180A1 - Traffic light detection - Google Patents
Traffic light detection Download PDFInfo
- Publication number
- WO2015024180A1 WO2015024180A1 PCT/CN2013/081842 CN2013081842W WO2015024180A1 WO 2015024180 A1 WO2015024180 A1 WO 2015024180A1 CN 2013081842 W CN2013081842 W CN 2013081842W WO 2015024180 A1 WO2015024180 A1 WO 2015024180A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- pixel
- pixels
- traffic light
- blobs
- candidate
- Prior art date
Links
Classifications
-
- G—PHYSICS
- G08—SIGNALLING
- G08G—TRAFFIC CONTROL SYSTEMS
- G08G1/00—Traffic control systems for road vehicles
- G08G1/09—Arrangements for giving variable traffic instructions
- G08G1/0962—Arrangements for giving variable traffic instructions having an indicator mounted inside the vehicle, e.g. giving voice messages
- G08G1/09623—Systems involving the acquisition of information from passive traffic signs by means mounted on the vehicle
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/90—Determination of colour characteristics
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
- G06V20/58—Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
- G06V20/582—Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads of traffic signs
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
- G06V20/58—Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
- G06V20/584—Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads of vehicle lights or traffic lights
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10024—Color image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30248—Vehicle exterior or interior
- G06T2207/30252—Vehicle exterior; Vicinity of vehicle
Definitions
- the present disclosure generally relates to traffic light detection.
- Traffic light detection is more and more widely used in driving assistance systems.
- traffic lights are detected based on color segmentation and template matching.
- a method for traffic light detection may include: obtaining a color image; calculating pixel response values for pixels of the color image, respectively, where each of the pixel response values may be calculated using R, G, and B values of a corresponding pixel directly, such that pixel response values of red traffic light pixels are substantially distributed on a first side of a predetermined range and pixel response values of green traffic light pixels are substantially distributed on a second side of the predetermined range which is opposite to the first side; identifying pixels whose pixel response values are distributed on the first side or the second side as candidate pixels; identifying candidate blobs based on the candidate pixels; and verifying whether the candidate blobs are traffic lights.
- pixel response values of pixels other than red traffic light pixels and green traffic light pixels are substantially distributed within the predetermined range.
- each of the pixel response values may be calculated based on multiplication of a first component and a second component of a corresponding pixel, where the first component may indicate a tendency of the pixel response value, and the second component may be able to amply the tendency if the tendency is towards the first side or the second side.
- the first component may be calculated based on summation of a third, a fourth, and a fifth components which are respectively based on R, G and B values of the corresponding pixel, where the third component may have a first sign, and the fourth and fifth components may have a second sign which is opposite to the first sign.
- the third component may be ki * R
- the fourth component may be -k 2 * G
- the fifth component may be -k 3 * B, where k-i, k 2 and k 3 are set such that the tendency is substantially towards the first side if the corresponding pixel is a red traffic light pixel and the tendency is substantially towards the second side if the corresponding pixel is a green traffic light pixel.
- the second component may be calculated based on summation of a sixth and a seventh components which are respectively based on R and G values of the corresponding pixel, where the sixth and the seventh components have the same sign.
- the sixth component may be k 4 * R and the seventh component may be k 5 *G, where k 4 and k 5 are set such that the pixel response value is substantially on the first side if the corresponding pixel is a red traffic light pixel, and the pixel response value is substantially on the second side if the corresponding pixel is a green traffic light pixel.
- the first component may be (R — ) . Therefore, based on whether the first component is positive or negative, whether the pixel is more likely to be a red traffic light pixel or a green traffic light pixel may be conveniently determined. Further, the larger the absolute value of the first component is, the more possible the pixel could be a traffic light pixel.
- identifying the candidate blobs may include: identifying a first set of blobs based on the candidate pixels; and identifying the candidate blobs from the first set of blobs, where each of the first set of blobs may be determined whether or not a member of the candidate blobs based on pixel response values of its pixels and its surrounding pixels, such that false positives may be further reduced.
- the first set of blobs may be identified using connected component labeling.
- the candidate blobs may be identified based on possibility values of the blobs of the first set, where each probability value may be calculated by: defining an ego segment for a corresponding blob of the first set of blobs, which ego segment may substantially encompass the corresponding blob; calculating an ego segment response value based on pixel response values of pixels within the ego segment; defining a predetermined number of ambient segments each of which partly overlaps the ego segment; calculating an ambient segment response value for each of the ambient segments, where each ambient segment response value may be calculated based on pixel response values of pixels within a corresponding ambient segment; and calculating the probability value based on the ego segment response value and the predetermined number of ambient segment response values.
- each of the ambient segments may have the same size and shape as the ego segment.
- the probability value of the corresponding blob may be calculated based on the following Equation:
- k l
- pv stands for the probability value of the corresponding blob
- ⁇ stands for the ego segment response value
- 3 ⁇ 4 stands for the ambient segment response value of a k h ambient segment
- n is the predetermined number.
- identifying the candidate blobs may further include: identifying a second set of blobs from the first set of blobs, where the candidate blobs are identified from the second set of blobs, where each of the first set of blobs may be determined whether or not a member of the second set of blob based on its corresponding ego segment response value and pixel response values of its pixels.
- the blob may be determined a member of the second set of blobs. False positives may be further reduced. Besides, following computations, including defining ambient segments, calculating ambient segment response values, calculating probability values, etc, for these false positives may be avoided, which may improve the efficiency.
- the method may further include: identifying a third set of blobs from the first set of blobs based on at least one geometric parameter, where the second set of blobs may be identified from the third set of blobs.
- the at least one geometric parameter may include a size parameter.
- the at least one geometric parameter may include a shape parameter.
- blobs in the first set of blobs which are not likely to have a round shape may be filtered out.
- verifying whether the candidate blobs are traffic lights may include: verifying the candidate blobs with a cascade detector to obtain first verified regions; and verifying the first verified regions with a support vector machine (SVM). If a candidate blob passes both the verifications of the cascade detector and the SVM, a traffic light may be detected in the region.
- SVM support vector machine
- a system for traffic light detection may include a processing device configured to: obtain a color image; calculate pixel response values for pixels of the color image, respectively, where each of the pixel response values may be calculated using R, G, and B values of a corresponding pixel directly, such that pixel response values of red traffic light pixels are substantially distributed on a first side of a predetermined range and pixel response values of green traffic light pixels are substantially distributed on a second side of the predetermined range which is opposite to the first side; identify pixels whose pixel response values are distributed on the first side or the second side as candidate pixels; identify candidate blobs based on the candidate pixels; and verify whether the candidate blobs are traffic lights.
- pixel response values of pixels other than red traffic light pixels and green traffic light pixels are substantially distributed within the predetermined range based on the calculation.
- the processing device may be configured to calculate each of the pixel response values based on multiplication of a first component and a second component of a corresponding pixel, where the first component may indicate a tendency of the pixel response value, and the second component may be able to amplify the tendency if the tendency is towards the first side or the second side.
- the first component may be calculated based on summation of a third, a fourth, and a fifth components which are respectively based on R, G and B values of the corresponding pixel, where the third component may have a first sign, and the fourth and fifth components may have a second sign which is opposite to the first sign.
- the third component may be ki * R
- the fourth component may be -k 2 * G
- the fifth component may be -k 3 * B, where k-i, k 2 and k 3 are set such that the tendency is substantially towards the first side if the corresponding pixel is a red traffic light pixel and the tendency is substantially towards the second side if the corresponding pixel is a green traffic light pixel.
- the second component may be calculated based on summation of a sixth and a seventh components which are respectively based on R and G values of the corresponding pixel, where the sixth and the seventh components have the same sign.
- the sixth component may be k *R and the seventh component may be k 5 *G, where k 4 and k 5 are set such that the pixel response value is substantially on the first side if the corresponding pixel is a red traffic light pixel, and the pixel response value is substantially on the second side if the corresponding pixel is a green traffic light pixel.
- the first component may be (R — ) .
- the processing device may be configured to: identify a first set of blobs based on the candidate pixels; and identify the candidate blobs from the first set of blobs, where each of the first set of blobs may be determined whether or not a member of the candidate blobs based on pixel response values of its pixels and its surrounding pixels.
- the processing device may be configured to identify the first set of blobs using connected component labeling.
- the processing device may be configured to identify the candidate blobs from the first set of blobs based on possibility values of the blobs of the first set, where each possibility value may be calculated by: defining an ego segment for a corresponding blob of the first set of blobs, which ego segment may substantially encompass the corresponding blob; calculating an ego segment response value based on pixel response values of pixels within the ego segment; defining a predetermined number of ambient segments each of which partly overlaps the ego segment; calculating an ambient segment response value for each of the ambient segments, where each ambient segment response value may be calculated based on pixel response values of pixels within a corresponding ambient segment; and calculating the probability value based on the ego segment response value and the predetermined number of ambient segment response values.
- each of the ambient segments may have the same size and shape as the ego segment.
- the probability value of the corresponding blob may be calculated based on the following Equation: where pv stands for the probability value of the corresponding blob, ⁇ stands for the ego segment response value, r ⁇ k stands for the ambient segment response value of a ⁇ t h ambient segment and n is the predetermined number.
- the processing device may be further configured to: identify a second set of blobs from the first set of blobs, where the candidate blobs are identified from the second set of blobs, where each of the first set of blobs may be determined whether or not a member of the second set of blob based on its corresponding ego segment response value and pixel response values of its pixels. In some embodiments, if the ego segment response value of a blob of the first set is on the same side of the predetermined range as pixel response values of pixels within the blob, the blob may be determined a member of the second set of blobs.
- he processing device may be further configured to: identify a third set of blobs from the first set of blobs based on at least one geometric parameter, where the second set of blobs may be identified from the third set of blobs.
- the at least one geometric parameter may be a size parameter.
- the at least one geometric parameter may be a shape parameter.
- the processing device may be configured to verify whether the candidate blobs are traffic lights by: verifying the candidate blobs with a cascade detector to obtain first verified regions, and verifying the first verified regions with a support vector machine (SVM).
- SVM support vector machine
- a system for traffic light detection may include: means for calculating pixel response values for pixels of a color image, respectively, where each of the pixel response values may be calculated using R, G, and B values of a corresponding pixel directly, such that pixel response values of red traffic light pixels are substantially distributed on a first side of a predetermined range and pixel response values of green traffic light pixels are substantially distributed on a second side of the predetermined range which is opposite to the first side; means for identifying pixels whose pixel response values are distributed on the first side or the second side as candidate pixels; means for identifying candidate blobs based on the candidate pixels; and means for verifying whether the candidate blobs are traffic lights.
- a non-transitory computer readable medium which contains a computer program for traffic light detection.
- the computer program When the computer program is executed by a processor, it will instruct the processor to: calculate pixel response values for pixels of a color image, respectively, where each of the pixel response values may be calculated using R, G, and B values of a corresponding pixel directly, such that pixel response values of red traffic light pixels are substantially distributed on a first side of a predetermined range and pixel response values of green traffic light pixels are substantially distributed on a second side of the predetermined range which is opposite to the first side; identify pixels whose pixel response values are distributed on the first side or the second side as candidate pixels; identify candidate blobs based on the candidate pixels; and verify whether the candidate blobs are traffic lights.
- FIG. 1 schematically illustrates a flow chart of a method for traffic light detection according to one embodiment
- FIG. 2 illustrates an example color image captured by a camera mounted on a vehicle
- FIG. 3 illustrates an upper part of the color image of FIG. 2
- FIG. 4 schematically illustrates pixel response values of pixels in FIG. 3;
- FIG. 5 schematically illustrates candidate pixels in the color image of FIG. 3;
- FIG. 6 schematically illustrates a first example blob of a first set of blobs and its minimum enclosing shape
- FIG. 7 schematically illustrates a process for identifying candidate blobs from a second set of blobs according to one embodiment
- FIG. 8 schematically illustrates a second example blob of a second set of blobs and its ego segment and ambient segments
- FIG. 9 illustrates rectangle regions in the color image of FIG. 2 corresponding to the candidate blobs.
- FIG. 10 illustrates traffic lights detected in the color image of FIG. 2.
- Traffic light detection may be based on image processing technology. Candidates for a traffic light on an image may need to be verified whether or not a true traffic light. Normally, such verification may be implemented using a traffic light classifier, such as a cascade detector or a support vector machine, which may lead a large computation load. Therefore, to improve efficiency and reliability of traffic light detection, before the verification, segments in the color image which are not likely to be a traffic light need to be filtered out as many as possible, meanwhile, it should be ensured that segments in the color image which are likely to be a traffic light may not be filtered out. A balance between efficiency and reliability is a major concern in traffic light detection.
- a traffic light classifier such as a cascade detector or a support vector machine
- FIG. 1 schematically illustrates a flow chart of a method 100 for traffic light detection according to one embodiment.
- the color image may be captured by a camera.
- the camera may be mounted on a vehicle.
- color images may be captured at evenly spaced time points, and may be processed one by one successively.
- FIG. 2 illustrates an example color image captured by a camera mounted on a vehicle. The color image is captured at an intersection. It could be seen from the color image that, many objects may have a color close to a traffic light at the intersection, such as trees, vehicle taillights, billboard lights, red or green logos, which may be false positives after verification by a conventional traffic light classifier. These false positives may need to be filtered out as many as possible.
- a lower part of the color image may be removed to improve the efficiency, since generally a traffic light may not appear in the lower part. As a result, the rest upper part of the color image is remained for subsequent processing.
- FIG. 3 illustrates an upper part of the color image of FIG. 2. It should be noted that removing the low part of the color image may be optional.
- pixel response values of pixels other than the red traffic light pixels and green traffic light pixels may be substantially distributed in a predetermined range
- pixel response values of the red traffic light pixels may be substantially distributed on a first side of the predetermined range
- pixel response values of the green traffic light pixels may be substantially distributed on a second side of the predetermined range, where the second side is opposite to the first side.
- each of the pixel response values may be calculated based on two components of a corresponding pixel, where a first component may indicate a tendency that whether the corresponding pixel is more like a green traffic light pixel, a red traffic light pixel or a pixel other than a traffic light, i.e., a tendency towards the first side, the second side or the predetermined range; and the second component may be able to amplify the tendency if the tendency is substantially towards the first side or the second side.
- the pixel response value may calculated based on multiplication of the first and second components.
- traffic light pixels should be outstanding in certain color channel values, compared with other objects.
- a red traffic light pixel may have a relatively high red channel (R) value, but very low green channel (G) value and blue channel (B) value; while a green traffic light pixel may have relatively high G value and B value, but very low R value.
- Pixels other than a traffic light pixel may have R, G and B values closer with each other. Therefore, R, G and B values may be used to separate red traffic light pixels, green traffic light pixels and pixels of other objects apart, thus can be used to calculate the first component.
- the first component may be calculated based on summation of a third, a fourth, and a fifth components which are respectively based on R, G and B values of the corresponding pixel, where the third component may have a first sign, and the fourth and fifth components may have a second sign which is opposite to the first sign.
- the first components of a red traffic light pixel and a green traffic light pixel may be towards opposite sides.
- the third component may be ki * R
- the fourth component may be -k 2 * G
- the fifth component may be -k 3 * B, where k ; k 2 and k 3 are set such that the tendency is substantially towards the first side if the corresponding pixel is a red traffic light pixel and the tendency is substantially towards the second side if the corresponding pixel is a green traffic light pixel.
- the weight factors ki , k 2 and k 3 may be adjusted based on ambient lights, camera parameters, etc.
- the first component may be (R — ) . Therefore, based on whether the first component is positive or negative, whether the pixel is more likely to be a red traffic light pixel or a green traffic light pixel may be conveniently determined. Further, the larger the absolute value of the first component is, the more possible the pixel could be a traffic light pixel.
- the second component may be used to amplify the disparity between pixel response values of the red traffic light pixels and pixel response values of the green traffic light pixels.
- the method may be more robust if this disparity is enlarged.
- the traffic light pixels either red or green, may not only be pure in a traffic light color, but also have relatively high luminosity. Therefore, in some embodiments, the second component may represent the luminosity of the corresponding pixel, which is a suitable way to amplify the tendency. As a result, objects which are red or green, but dark, such as tree leaves, may be filtered out.
- the luminosity of the pixel may be highly relevant to the R, G and B values. Therefore, the second component may also be represented using the R, G and B values.
- the second component may be calculated based on summation of a sixth and a seventh components which are respectively based on R and G values of the corresponding pixel, where the sixth and the seventh components have the same sign.
- the sixth component may be k 4 *R and the seventh component may be k 5 *G, where k 4 and k 5 are set such that the pixel response value is substantially on the first side if the corresponding pixel is a red traffic light pixel, and the pixel response value is substantially on the second side if the corresponding pixel is a green traffic light pixel.
- the pixel response value of the corresponding pixel was calculated based on the following Equation (1 ) :
- R, G and B respectively represent R, G and B values of the corresponding pixel.
- FIG. 4 schematically illustrates pixel response values of pixels in FIG. 3 calculated based on Equation (1 ).
- the pixel response values are converted into grayscales. It could be seen from FIG. 4 that the red traffic light pixels are relatively light while the green traffic light pixels are relatively dark. It should be noted that FIG. 4 is merely for illustration. In a practical process, there is no need to form an image for illustrating the pixel response values.
- the pixel response values of the red traffic light pixels, the pixel response values of the green traffic light pixels and the pixel response values of other pixels may be distributed into separated ranges. Therefore, candidate pixels for a red traffic light or a green traffic light may be identified based on the pixel response values.
- the pixel response values were calculated based on Equation (1 ) and were normalized to [0, 1 ].
- pixels having a pixel response value within a predetermined range from -0.25 to 0.375 were identified as pixels other than a traffic light
- pixels having a pixel response value at a first side of the predetermined range, i.e., larger than 0.375 were identified as red traffic light candidate pixels
- pixels having a pixel response value at a second side of the predetermined range, i.e., less than -0.25 were identified as green traffic light candidate pixels.
- the specific range is merely an example and can be adjusted based on camera parameters, etc.
- FIG. 5 schematically illustrates candidate pixels in the color image of FIG. 3. As shown in FIG. 5, the red traffic light candidate pixels and the green traffic light candidate pixels are marked. It should be noted that FIG. 5 is given for illustration only. In practical processing, it is not necessary to obtain an image like FIG. 5 to visibly show the candidate pixels.
- the first set of blobs may be identified using connected component labeling.
- the first set of blobs may include blobs identified based on the red traffic light candidate pixels which are to be subject to red traffic light detection, and blobs identified based on the green traffic light candidate pixels which are to be subject to green traffic light detection.
- red traffic light detection and green traffic light detection can be performed either in parallel or in serial.
- the red traffic light detection and green traffic light detection are performed in parallel as an example.
- the first set of blobs may be taken as candidate blobs to be verified using a traffic light detector or classifier.
- false positives in the first set of blobs may be further filtered out, so that the verifications of the candidate blobs, which normally take the most computation resource, may be reduced.
- further filtering out false positives may be further filtered out.
- blob analysis based on at least one geometric parameter may be used to filter out some blobs which are not likely to be a traffic light based on their shapes and/or sizes.
- the remaining blobs may be identified as the second set of blobs.
- Minimum enclosing shapes respectively encompassing blobs of the first set may be defined in the blob analysis for facilitating analyzing.
- FIG. 6 schematically illustrates a first example blob of the first set of blobs and its minimum enclosing shape.
- the minimum enclosing shapes may be rectangles.
- the at least one geometric parameter may be a pixel number. For example, if the first example blob has a pixel number lager than 255, it may be filtered out because a traffic light should not be too large. It should be noted that specific value of the criterion may be altered based on factors like image resolution. When the image resolution is 720 * 480, the above proposed number 255 may be suitable. A suitable value of this criterion may increase or decrease with the increase or decrease of the image resolution. [0082] In some embodiments, the at least one geometric parameter may be the width or height of the blob.
- a width and a height of the minimum enclosing shape may be taken as the width and height of the first example blob, and if the first example blob has a width or height larger than 24, it may be filtered out. It should be noted that the value of the width or height criterion may also change with the image resolution.
- the at least one geometric parameter may be a width to height ratio. Since normally a traffic light should be round, its corresponding blob may have a minimum enclosing rectangle close to a square. For example, if the first example blob has a width to height ratio beyond a range from about 0.5 to 2, it may be filtered out.
- Each of the blobs of the second set may be determined whether or not a member of the candidate blobs base on pixel response values of its pixels and surroundings. False positives may be reduced, because normally a traffic light should not only be bright and pure in the traffic light color itself, but also be prominent among its local surroundings, which means the traffic light blob may have be outstanding in pixel response values.
- FIG. 7 schematically illustrates a flow chart of a process 200 for identifying the candidate blobs from the second set of blobs according to one embodiment.
- the ego segment may substantially encompass the second example blob.
- the ego segment may be a minimum enclosing rectangle of the second example blob.
- the ego segment response value may be calculated based on summation of pixel response values of pixels in the ego segment.
- the ego blob response may be calculated based on Equation (2):
- T J ⁇ ⁇ pr(x, y)
- ⁇ stands for the ego segment response value
- pr(x,y) stands for the pixel response value of a pixel in ego segment having an image coordinate (x,y)
- y m and y n stand for upmost and lowest coordinates of the ego segment on a y axis, respectively
- Xj and x j stand for leftmost and rightmost coordinates of the ego segment on a x axis, respectively.
- whether the second example blob is a false positive may be determined. Specifically, if the ego segment response value and the pixel response values of pixels in the second example blob are not on the same side of the predetermined range, the second example blob may be filtered out. Therefore, false positives may be further filtered out, especially for some objects like red and/or green characters, logos, and the like, which may have extreme pixel response values and hardly be filtered out in the blob analysis. Besides, following computations for these false positives may be avoided.
- each of the ambient segments may have a same shape and size with the ego segment.
- FIG. 8 schematically illustrates the second example blob and an example of its ego segment and ambient segments.
- a minimum enclosing rectangle (windowed with heavy lines) which is defined as the ego segment of the second example blob.
- Eight rectangles having a same shape and size as the minimum enclosing rectangle are defined as the ambient segments.
- c 0 to c 8 are center points of the ego segment and the ambient segments.
- the eight ambient segments are aia 3 c 0 ai 5 , a 2 a c c 8 , a 3 a5a 7 Co, C2a 6 a 8 C6, c 0 a 7 a 9 an , c 8 c a-i 0 a-i 2, a-i 5C 0 ai-ia-i 3 ,
- Each of the ambient segment response values may be calculated based on pixel response values of pixels in a corresponding ambient segment. For example, summation of the pixel response values may be calculated.
- whether the second example blob is a false positive may be determined based on its ego segment response value and ambient segment response value. For example, if the pixel response values were calculated based on the above Equation (1 ), the ego segment response value of the second example blob should be larger than any of its ambient segment response values if its pixels are red traffic light pixels, vice versa, or else, the second example blob may be filtered out.
- the probability value of the second example blob may be calculated based on Equation (3): where pv stands for the probability value of the corresponding blob, ⁇ stands for the ego segment response value,
- n is the predetermined number.
- the blobs for red traffic light and green traffic light are respectively sorted decreasingly based on the probability values, where top 20 blobs for red traffic light and top 30 blobs for green traffic light are selected as the candidate blobs.
- a relatively small number of blobs may be selected as the candidate blobs. False positives may be reduced and processing speed may be increased.
- a traffic light detector or classifier such as template matching, neural network, support vector machine (SVM), cascade detector, and the like, whether a region corresponding to a candidate blob represents a traffic light may be determined. In some embodiments, regions in the color image corresponding to the candidate blobs may be verified. In some embodiments, regions in a grayscale image converted from the color image corresponding to the candidate blobs may be verified. As stated above, based on the above processing, a relatively small number of candidate blobs may be remained, thereby improving both the reliability and efficiency.
- SVM support vector machine
- regions corresponding to the candidate blobs may be segmented in the color image firstly. Segmenting the regions may be implemented based on positions, sizes and types of the candidate blobs, and a traffic light template. In some embodiments, for a vertically arranged traffic light template, the candidate blob for a red traffic light should be located on top of the corresponding region, the candidate blob for a green traffic light should be located on bottom of the corresponding region, and the corresponding region may have a width equaling to or a little larger than that of the blob and a height more than three times of that of the blob.
- the candidate blob for a red traffic light should be located on left of the corresponding region
- the candidate blob for a green traffic light should be located on right of the corresponding region
- the corresponding region may have a width at least three times of that of the blob and a height equaling to or a little larger than that of the blob.
- the corresponding regions may be enlarged to ensure that they can cover the traffic light(s) in the color image.
- FIG. 9 illustrates rectangle regions in the color image of FIG. 2 corresponding to the candidate blobs.
- the rectangle regions corresponding to the candidate blobs for red and green traffic lights are marked. It could be seen from FIG. 9 that false positives may still exist. Thereafter, these rectangle regions may be verified using a traffic light detector or classifier.
- the traffic light detector may include a cascade detector and a SVM.
- the cascade detector may be used prior to the SVM, since the SVM takes more time than the cascade detector.
- the rectangle regions in the color image may be converted into monochrome images before being verified by the traffic light detector.
- a result may be output by framing the identified traffic light(s), such as illustrated in FIG. 10.
- a system for traffic light detection may include a processing device configured to: obtain a color image; calculate pixel response values for pixels of the upper part of the color image using R, G, and B values of the pixels directly; identify candidate pixels based on the calculated pixel response values; identify a first set of blobs based on the candidate pixels; identify a second set of blobs from the first set of blobs based on at least one geometric parameter; identify candidate blobs from the second set of blobs based on pixel response values of pixels in and surrounding the second set of blobs; and verify whether the candidate blobs are traffic lights.
- Specific configurations of the processing device may be found by referring S101 to S113 above, which may not be illustrated in detail here.
- a system for traffic light detection may include: means for calculating pixel response values for pixels of a color image, respectively, where each of the pixel response values may be calculated using R, G, and B values of a corresponding pixel directly, such that pixel response values of red traffic light pixels are substantially distributed on a first side of a predetermined range and pixel response values of green traffic light pixels are substantially distributed on a second side of the predetermined range which is opposite to the first side; means for identifying pixels whose pixel response values are distributed on the first side or the second side as candidate pixels; means for identifying candidate blobs based on the candidate pixels; and means for verifying whether the candidate blobs are traffic lights.
- a non-transitory computer readable medium which contains a computer program for traffic light detection.
- the computer program When the computer program is executed by a processor, it will instruct the processor to: calculate pixel response values for pixels of a color image, respectively, where each of the pixel response values may be calculated using R, G, and B values of a corresponding pixel directly, such that pixel response values of red traffic light pixels are substantially distributed on a first side of a predetermined range and pixel response values of green traffic light pixels are substantially distributed on a second side of the predetermined range which is opposite to the first side; identify pixels whose pixel response values are distributed on the first side or the second side as candidate pixels; identify candidate blobs based on the candidate pixels; and verify whether the candidate blobs are traffic lights.
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Multimedia (AREA)
- Data Mining & Analysis (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Artificial Intelligence (AREA)
- Bioinformatics & Computational Biology (AREA)
- Life Sciences & Earth Sciences (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- General Engineering & Computer Science (AREA)
- Traffic Control Systems (AREA)
- Image Analysis (AREA)
Abstract
A method and a system for traffic light detection are provided. The method may include: obtaining a color image; calculating pixel response values for pixels of the color image, respectively, where each of the pixel response values may be calculated using R, G, and B values of a corresponding pixel directly, such that pixel response values of red traffic light pixels are substantially distributed on a first side of a predetermined range and pixel response values of green traffic light pixels are substantially distributed on a second side of the predetermined range which is opposite to the first side; identifying pixels whose pixel response values are distributed on the first side or the second side as candidate pixels; identifying candidate blobs based on the candidate pixels; and verifying whether the candidate blobs are traffic lights. Efficiency and reliability may be improved.
Description
TRAFFIC LIGHT DETECTION
TECHNICAL FIELD
[0001] The present disclosure generally relates to traffic light detection.
BACKGROUND
[0002] Traffic light detection is more and more widely used in driving assistance systems. In some conventional methods, traffic lights are detected based on color segmentation and template matching.
SUMMARY
[0003] According to one embodiment, a method for traffic light detection is provided. The method may include: obtaining a color image; calculating pixel response values for pixels of the color image, respectively, where each of the pixel response values may be calculated using R, G, and B values of a corresponding pixel directly, such that pixel response values of red traffic light pixels are substantially distributed on a first side of a predetermined range and pixel response values of green traffic light pixels are substantially distributed on a second side of the predetermined range which is opposite to the first side; identifying pixels whose pixel response values are distributed on the first side or the second side as candidate pixels; identifying candidate blobs based on the candidate pixels; and verifying whether the candidate blobs are traffic lights.
[0004] In some embodiments, pixel response values of pixels other than red traffic light
pixels and green traffic light pixels are substantially distributed within the predetermined range.
[0005] In some embodiments, each of the pixel response values may be calculated based on multiplication of a first component and a second component of a corresponding pixel, where the first component may indicate a tendency of the pixel response value, and the second component may be able to amply the tendency if the tendency is towards the first side or the second side.
[0006] In some embodiments, the first component may be calculated based on summation of a third, a fourth, and a fifth components which are respectively based on R, G and B values of the corresponding pixel, where the third component may have a first sign, and the fourth and fifth components may have a second sign which is opposite to the first sign.
[0007] In some embodiments, the third component may be ki*R, the fourth component may be -k2 *G, and the fifth component may be -k3 *B, where k-i, k2 and k3 are set such that the tendency is substantially towards the first side if the corresponding pixel is a red traffic light pixel and the tendency is substantially towards the second side if the corresponding pixel is a green traffic light pixel.
[0008] In some embodiments, the second component may be calculated based on summation of a sixth and a seventh components which are respectively based on R and G values of the corresponding pixel, where the sixth and the seventh components have the same sign.
[0009] In some embodiments, the sixth component may be k4 *R and the seventh
component may be k5*G, where k4 and k5 are set such that the pixel response value is substantially on the first side if the corresponding pixel is a red traffic light pixel, and the pixel response value is substantially on the second side if the corresponding pixel is a green traffic light pixel.
G + B
[0010] In some embodiments, the first component may be (R — ) . Therefore, based on whether the first component is positive or negative, whether the pixel is more likely to be a red traffic light pixel or a green traffic light pixel may be conveniently determined. Further, the larger the absolute value of the first component is, the more possible the pixel could be a traffic light pixel.
[0011] In some embodiments, identifying the candidate blobs may include: identifying a first set of blobs based on the candidate pixels; and identifying the candidate blobs from the first set of blobs, where each of the first set of blobs may be determined whether or not a member of the candidate blobs based on pixel response values of its pixels and its surrounding pixels, such that false positives may be further reduced.
[0012] In some embodiments, the first set of blobs may be identified using connected component labeling.
[0013] In some embodiments, the candidate blobs may be identified based on possibility values of the blobs of the first set, where each probability value may be calculated by: defining an ego segment for a corresponding blob of the first set of blobs, which ego segment may substantially encompass the corresponding blob; calculating an ego segment response value based on pixel response values of pixels within the ego segment; defining a predetermined number of ambient segments each of which partly
overlaps the ego segment; calculating an ambient segment response value for each of the ambient segments, where each ambient segment response value may be calculated based on pixel response values of pixels within a corresponding ambient segment; and calculating the probability value based on the ego segment response value and the predetermined number of ambient segment response values.
[0014] In some embodiments, each of the ambient segments may have the same size and shape as the ego segment.
[0015] In some embodiments, the probability value of the corresponding blob may be calculated based on the following Equation:
n
k=l where pv stands for the probability value of the corresponding blob, η stands for the ego segment response value, ¾ stands for the ambient segment response value of a kh ambient segment and n is the predetermined number.
[0016] In some embodiments, identifying the candidate blobs may further include: identifying a second set of blobs from the first set of blobs, where the candidate blobs are identified from the second set of blobs, where each of the first set of blobs may be determined whether or not a member of the second set of blob based on its corresponding ego segment response value and pixel response values of its pixels. In some embodiments, if the ego segment response value of a blob of the first set is on the same side of the predetermined range as pixel response values of pixels within the blob, the blob may be determined a member of the second set of blobs. False positives may be further reduced. Besides, following computations, including defining ambient
segments, calculating ambient segment response values, calculating probability values, etc, for these false positives may be avoided, which may improve the efficiency.
[0017] In some embodiments, the method may further include: identifying a third set of blobs from the first set of blobs based on at least one geometric parameter, where the second set of blobs may be identified from the third set of blobs.
[0018] In some embodiments, the at least one geometric parameter may include a size parameter. As a result, blobs in the first set of blobs with too large sizes, which are not likely to be a traffic light, may be filtered out.
[0019] In some embodiments, the at least one geometric parameter may include a shape parameter. As a result, blobs in the first set of blobs which are not likely to have a round shape may be filtered out.
[0020] In some embodiments, verifying whether the candidate blobs are traffic lights may include: verifying the candidate blobs with a cascade detector to obtain first verified regions; and verifying the first verified regions with a support vector machine (SVM). If a candidate blob passes both the verifications of the cascade detector and the SVM, a traffic light may be detected in the region.
[0021] According to one embodiment, a system for traffic light detection is provided. The system may include a processing device configured to: obtain a color image; calculate pixel response values for pixels of the color image, respectively, where each of the pixel response values may be calculated using R, G, and B values of a corresponding pixel directly, such that pixel response values of red traffic light pixels are substantially distributed on a first side of a predetermined range and pixel response
values of green traffic light pixels are substantially distributed on a second side of the predetermined range which is opposite to the first side; identify pixels whose pixel response values are distributed on the first side or the second side as candidate pixels; identify candidate blobs based on the candidate pixels; and verify whether the candidate blobs are traffic lights.
[0022] In some embodiments, pixel response values of pixels other than red traffic light pixels and green traffic light pixels are substantially distributed within the predetermined range based on the calculation.
[0023] In some embodiments, the processing device may be configured to calculate each of the pixel response values based on multiplication of a first component and a second component of a corresponding pixel, where the first component may indicate a tendency of the pixel response value, and the second component may be able to amplify the tendency if the tendency is towards the first side or the second side.
[0024] In some embodiments, the first component may be calculated based on summation of a third, a fourth, and a fifth components which are respectively based on R, G and B values of the corresponding pixel, where the third component may have a first sign, and the fourth and fifth components may have a second sign which is opposite to the first sign.
[0025] In some embodiments, the third component may be ki*R, the fourth component may be -k2 *G, and the fifth component may be -k3 *B, where k-i, k2 and k3 are set such that the tendency is substantially towards the first side if the corresponding pixel is a red traffic light pixel and the tendency is substantially towards the second side if the corresponding pixel is a green traffic light pixel.
[0026] In some embodiments, the second component may be calculated based on summation of a sixth and a seventh components which are respectively based on R and G values of the corresponding pixel, where the sixth and the seventh components have the same sign.
[0027] In some embodiments, the sixth component may be k *R and the seventh component may be k5*G, where k4 and k5 are set such that the pixel response value is substantially on the first side if the corresponding pixel is a red traffic light pixel, and the pixel response value is substantially on the second side if the corresponding pixel is a green traffic light pixel.
G + B
[0028] In some embodiments, the first component may be (R — ) .
[0029] In some embodiments, the processing device may be configured to: identify a first set of blobs based on the candidate pixels; and identify the candidate blobs from the first set of blobs, where each of the first set of blobs may be determined whether or not a member of the candidate blobs based on pixel response values of its pixels and its surrounding pixels.
[0030] In some embodiments, the processing device may be configured to identify the first set of blobs using connected component labeling.
[0031] In some embodiments, the processing device may be configured to identify the candidate blobs from the first set of blobs based on possibility values of the blobs of the first set, where each possibility value may be calculated by: defining an ego segment for a corresponding blob of the first set of blobs, which ego segment may substantially encompass the corresponding blob; calculating an ego segment response value based
on pixel response values of pixels within the ego segment; defining a predetermined number of ambient segments each of which partly overlaps the ego segment; calculating an ambient segment response value for each of the ambient segments, where each ambient segment response value may be calculated based on pixel response values of pixels within a corresponding ambient segment; and calculating the probability value based on the ego segment response value and the predetermined number of ambient segment response values.
[0032] In some embodiments, each of the ambient segments may have the same size and shape as the ego segment.
[0033] In some embodiments, the probability value of the corresponding blob may be calculated based on the following Equation:
where pv stands for the probability value of the corresponding blob, η stands for the ego segment response value, r\k stands for the ambient segment response value of a \th ambient segment and n is the predetermined number.
[0034] In some embodiments, the processing device may be further configured to: identify a second set of blobs from the first set of blobs, where the candidate blobs are identified from the second set of blobs, where each of the first set of blobs may be determined whether or not a member of the second set of blob based on its corresponding ego segment response value and pixel response values of its pixels. In some embodiments, if the ego segment response value of a blob of the first set is on the same side of the predetermined range as pixel response values of pixels within the
blob, the blob may be determined a member of the second set of blobs.
[0035] In some embodiments, he processing device may be further configured to: identify a third set of blobs from the first set of blobs based on at least one geometric parameter, where the second set of blobs may be identified from the third set of blobs.
[0036] In some embodiments, the at least one geometric parameter may be a size parameter.
[0037] In some embodiments, the at least one geometric parameter may be a shape parameter.
[0038] In some embodiments, the processing device may be configured to verify whether the candidate blobs are traffic lights by: verifying the candidate blobs with a cascade detector to obtain first verified regions, and verifying the first verified regions with a support vector machine (SVM).
[0039] According to one embodiment, a system for traffic light detection is provided. The system may include: means for calculating pixel response values for pixels of a color image, respectively, where each of the pixel response values may be calculated using R, G, and B values of a corresponding pixel directly, such that pixel response values of red traffic light pixels are substantially distributed on a first side of a predetermined range and pixel response values of green traffic light pixels are substantially distributed on a second side of the predetermined range which is opposite to the first side; means for identifying pixels whose pixel response values are distributed on the first side or the second side as candidate pixels; means for identifying candidate blobs based on the candidate pixels; and means for verifying whether the candidate
blobs are traffic lights.
[0040] According to one embodiment, a non-transitory computer readable medium, which contains a computer program for traffic light detection, is provided. When the computer program is executed by a processor, it will instruct the processor to: calculate pixel response values for pixels of a color image, respectively, where each of the pixel response values may be calculated using R, G, and B values of a corresponding pixel directly, such that pixel response values of red traffic light pixels are substantially distributed on a first side of a predetermined range and pixel response values of green traffic light pixels are substantially distributed on a second side of the predetermined range which is opposite to the first side; identify pixels whose pixel response values are distributed on the first side or the second side as candidate pixels; identify candidate blobs based on the candidate pixels; and verify whether the candidate blobs are traffic lights.
BRIEF DESCRIPTION OF THE DRAWINGS
[0041] The foregoing and other features of the present disclosure will become more fully apparent from the following description and appended claims, taken in conjunction with the accompanying drawings. Understanding that these drawings depict only several embodiments in accordance with the disclosure and are, therefore, not to be considered limiting of its scope, the disclosure will be described with additional specificity and detail through use of the accompanying drawings.
[0042] FIG. 1 schematically illustrates a flow chart of a method for traffic light detection according to one embodiment;
[0043] FIG. 2 illustrates an example color image captured by a camera mounted on a vehicle;
[0044] FIG. 3 illustrates an upper part of the color image of FIG. 2;
[0045] FIG. 4 schematically illustrates pixel response values of pixels in FIG. 3;
[0046] FIG. 5 schematically illustrates candidate pixels in the color image of FIG. 3;
[0047] FIG. 6 schematically illustrates a first example blob of a first set of blobs and its minimum enclosing shape;
[0048] FIG. 7 schematically illustrates a process for identifying candidate blobs from a second set of blobs according to one embodiment;
[0049] FIG. 8 schematically illustrates a second example blob of a second set of blobs and its ego segment and ambient segments;
[0050] FIG. 9 illustrates rectangle regions in the color image of FIG. 2 corresponding to the candidate blobs; and
[0051] FIG. 10 illustrates traffic lights detected in the color image of FIG. 2.
DETAILED DESCRIPTION
[0052] In the following detailed description, reference is made to the accompanying drawings, which form a part hereof. In the drawings, similar symbols typically identify similar components, unless context dictates otherwise. The illustrative embodiments described in the detailed description, drawings, and claims are not meant to be limiting. Other embodiments may be utilized, and other changes may be made, without departing
from the spirit or scope of the subject matter presented here. It will be readily understood that the aspects of the present disclosure, as generally described herein, and illustrated in the Figures, can be arranged, substituted, combined, and designed in a wide variety of different configurations, all of which are explicitly contemplated and make part of this disclosure.
[0053] Traffic light detection may be based on image processing technology. Candidates for a traffic light on an image may need to be verified whether or not a true traffic light. Normally, such verification may be implemented using a traffic light classifier, such as a cascade detector or a support vector machine, which may lead a large computation load. Therefore, to improve efficiency and reliability of traffic light detection, before the verification, segments in the color image which are not likely to be a traffic light need to be filtered out as many as possible, meanwhile, it should be ensured that segments in the color image which are likely to be a traffic light may not be filtered out. A balance between efficiency and reliability is a major concern in traffic light detection.
[0054] FIG. 1 schematically illustrates a flow chart of a method 100 for traffic light detection according to one embodiment.
[0055] Referring to FIG. 1 , in S101 , obtaining a color image.
[0056] In some embodiments, the color image may be captured by a camera. In some embodiments, the camera may be mounted on a vehicle. In some embodiments, color images may be captured at evenly spaced time points, and may be processed one by one successively.
[0057] FIG. 2 illustrates an example color image captured by a camera mounted on a vehicle. The color image is captured at an intersection. It could be seen from the color image that, many objects may have a color close to a traffic light at the intersection, such as trees, vehicle taillights, billboard lights, red or green logos, which may be false positives after verification by a conventional traffic light classifier. These false positives may need to be filtered out as many as possible.
[0058] In some embodiment, a lower part of the color image may be removed to improve the efficiency, since generally a traffic light may not appear in the lower part. As a result, the rest upper part of the color image is remained for subsequent processing. FIG. 3 illustrates an upper part of the color image of FIG. 2. It should be noted that removing the low part of the color image may be optional.
[0059] In S1 03, calculating pixel response values for pixels of the upper part of the color image using R, G, and B values of the pixels directly.
[0060] Based on the calculation, pixel response values of pixels other than the red traffic light pixels and green traffic light pixels may be substantially distributed in a predetermined range, pixel response values of the red traffic light pixels may be substantially distributed on a first side of the predetermined range, and pixel response values of the green traffic light pixels may be substantially distributed on a second side of the predetermined range, where the second side is opposite to the first side.
[0061] In some embodiments, each of the pixel response values may be calculated based on two components of a corresponding pixel, where a first component may indicate a tendency that whether the corresponding pixel is more like a green traffic light pixel, a red traffic light pixel or a pixel other than a traffic light, i.e., a tendency towards
the first side, the second side or the predetermined range; and the second component may be able to amplify the tendency if the tendency is substantially towards the first side or the second side. In some embodiments, the pixel response value may calculated based on multiplication of the first and second components.
[0062] Normally, traffic light pixels, either red or green, should be outstanding in certain color channel values, compared with other objects. For example, a red traffic light pixel may have a relatively high red channel (R) value, but very low green channel (G) value and blue channel (B) value; while a green traffic light pixel may have relatively high G value and B value, but very low R value. Pixels other than a traffic light pixel may have R, G and B values closer with each other. Therefore, R, G and B values may be used to separate red traffic light pixels, green traffic light pixels and pixels of other objects apart, thus can be used to calculate the first component.
[0063] In some embodiments, the first component may be calculated based on summation of a third, a fourth, and a fifth components which are respectively based on R, G and B values of the corresponding pixel, where the third component may have a first sign, and the fourth and fifth components may have a second sign which is opposite to the first sign. As a result, the first components of a red traffic light pixel and a green traffic light pixel may be towards opposite sides.
[0064] In some embodiments, the third component may be ki*R, the fourth component may be -k2 *G, and the fifth component may be -k3 *B, where k ; k2 and k3 are set such that the tendency is substantially towards the first side if the corresponding pixel is a red traffic light pixel and the tendency is substantially towards the second side if the corresponding pixel is a green traffic light pixel. The weight factors ki , k2 and k3 may be
adjusted based on ambient lights, camera parameters, etc.
G + B
[0065] In some embodiments, the first component may be (R — ) . Therefore, based on whether the first component is positive or negative, whether the pixel is more likely to be a red traffic light pixel or a green traffic light pixel may be conveniently determined. Further, the larger the absolute value of the first component is, the more possible the pixel could be a traffic light pixel.
[0066] The second component may be used to amplify the disparity between pixel response values of the red traffic light pixels and pixel response values of the green traffic light pixels. The method may be more robust if this disparity is enlarged. Normally the traffic light pixels, either red or green, may not only be pure in a traffic light color, but also have relatively high luminosity. Therefore, in some embodiments, the second component may represent the luminosity of the corresponding pixel, which is a suitable way to amplify the tendency. As a result, objects which are red or green, but dark, such as tree leaves, may be filtered out. As well known in the art, the luminosity of the pixel may be highly relevant to the R, G and B values. Therefore, the second component may also be represented using the R, G and B values.
[0067] In some embodiments, the second component may be calculated based on summation of a sixth and a seventh components which are respectively based on R and G values of the corresponding pixel, where the sixth and the seventh components have the same sign.
[0068] In some embodiments, the sixth component may be k4*R and the seventh component may be k5*G, where k4 and k5 are set such that the pixel response value is
substantially on the first side if the corresponding pixel is a red traffic light pixel, and the pixel response value is substantially on the second side if the corresponding pixel is a green traffic light pixel.
[0069] In an example, the pixel response value of the corresponding pixel was calculated based on the following Equation (1 ) :
where pr represents the pixel response value, R, G and B respectively represent R, G and B values of the corresponding pixel.
[0070] Based on Equation (1 ), it could be concluded that pixel response values of the red traffic light pixels may be positively larger, pixel response values of the green traffic light pixels may be negatively larger, and pixel response values of objects other than the traffic lights may have relatively small absolute values. FIG. 4 schematically illustrates pixel response values of pixels in FIG. 3 calculated based on Equation (1 ). In FIG. 4, the pixel response values are converted into grayscales. It could be seen from FIG. 4 that the red traffic light pixels are relatively light while the green traffic light pixels are relatively dark. It should be noted that FIG. 4 is merely for illustration. In a practical process, there is no need to form an image for illustrating the pixel response values.
[0071] In S1 05, identifying candidate pixels based on the calculated pixel response values.
[0072] The pixel response values of the red traffic light pixels, the pixel response values of the green traffic light pixels and the pixel response values of other pixels may be distributed into separated ranges. Therefore, candidate pixels for a red traffic light or
a green traffic light may be identified based on the pixel response values.
[0073] In an example, the pixel response values were calculated based on Equation (1 ) and were normalized to [0, 1 ]. In the example, pixels having a pixel response value within a predetermined range from -0.25 to 0.375 were identified as pixels other than a traffic light, pixels having a pixel response value at a first side of the predetermined range, i.e., larger than 0.375 were identified as red traffic light candidate pixels, and pixels having a pixel response value at a second side of the predetermined range, i.e., less than -0.25, were identified as green traffic light candidate pixels. The specific range is merely an example and can be adjusted based on camera parameters, etc.
[0074] FIG. 5 schematically illustrates candidate pixels in the color image of FIG. 3. As shown in FIG. 5, the red traffic light candidate pixels and the green traffic light candidate pixels are marked. It should be noted that FIG. 5 is given for illustration only. In practical processing, it is not necessary to obtain an image like FIG. 5 to visibly show the candidate pixels.
[0075] In S107, identifying a first set of blobs based on the candidate pixels.
[0076] In some embodiments, the first set of blobs may be identified using connected component labeling. The first set of blobs may include blobs identified based on the red traffic light candidate pixels which are to be subject to red traffic light detection, and blobs identified based on the green traffic light candidate pixels which are to be subject to green traffic light detection. Following processes for red traffic light detection and green traffic light detection can be performed either in parallel or in serial. Hereunder, the red traffic light detection and green traffic light detection are performed in parallel as an example.
[0077] It should be noted that the first set of blobs may be taken as candidate blobs to be verified using a traffic light detector or classifier. However, in some embodiments, false positives in the first set of blobs may be further filtered out, so that the verifications of the candidate blobs, which normally take the most computation resource, may be reduced. Hereunder gives an example for further filtering out false positives.
[0078] In S109, identifying a second set of blobs from the first set of blobs based on at least one geometric parameter.
[0079] To reduce computation load, blob analysis based on at least one geometric parameter may be used to filter out some blobs which are not likely to be a traffic light based on their shapes and/or sizes. Correspondingly, the remaining blobs may be identified as the second set of blobs.
[0080] Minimum enclosing shapes respectively encompassing blobs of the first set may be defined in the blob analysis for facilitating analyzing. FIG. 6 schematically illustrates a first example blob of the first set of blobs and its minimum enclosing shape. In some embodiment, the minimum enclosing shapes may be rectangles.
[0081] In some embodiments, the at least one geometric parameter may be a pixel number. For example, if the first example blob has a pixel number lager than 255, it may be filtered out because a traffic light should not be too large. It should be noted that specific value of the criterion may be altered based on factors like image resolution. When the image resolution is 720*480, the above proposed number 255 may be suitable. A suitable value of this criterion may increase or decrease with the increase or decrease of the image resolution.
[0082] In some embodiments, the at least one geometric parameter may be the width or height of the blob. In some embodiments, a width and a height of the minimum enclosing shape may be taken as the width and height of the first example blob, and if the first example blob has a width or height larger than 24, it may be filtered out. It should be noted that the value of the width or height criterion may also change with the image resolution.
[0083] In some embodiments, the at least one geometric parameter may be a width to height ratio. Since normally a traffic light should be round, its corresponding blob may have a minimum enclosing rectangle close to a square. For example, if the first example blob has a width to height ratio beyond a range from about 0.5 to 2, it may be filtered out.
[0084] The above described geometric parameters may be used individually or together as criteria for the blob analysis. It should be noted that, the above described numerical values are merely examples and may be adjusted according to specific scenarios and/or camera parameters.
[0085] In S111 , identifying candidate blobs from the second set of blobs based on pixel response values of pixels in and surrounding the second set of blobs.
[0086] Each of the blobs of the second set may be determined whether or not a member of the candidate blobs base on pixel response values of its pixels and surroundings. False positives may be reduced, because normally a traffic light should not only be bright and pure in the traffic light color itself, but also be prominent among its local surroundings, which means the traffic light blob may have be outstanding in pixel response values.
[0087] FIG. 7 schematically illustrates a flow chart of a process 200 for identifying the candidate blobs from the second set of blobs according to one embodiment.
[0088] Referring to FIG. 7, in S201 , defining an ego segment for a second example blob in the second set of blobs.
[0089] The ego segment may substantially encompass the second example blob. In some embodiments, the ego segment may be a minimum enclosing rectangle of the second example blob.
[0090] In some embodiments, minimum enclosing rectangles were defined during the blob analysis in S109, so S201 may be avoided.
[0091] In S203, calculating an ego segment response value.
[0092] The ego segment response value may be calculated based on summation of pixel response values of pixels in the ego segment. In some embodiments, the ego blob response may be calculated based on Equation (2):
TJ = ∑ ∑ pr(x, y)
ym ≤y<yn ¾≤*<*,■ where η stands for the ego segment response value, pr(x,y) stands for the pixel response value of a pixel in ego segment having an image coordinate (x,y), ym and yn stand for upmost and lowest coordinates of the ego segment on a y axis, respectively, and Xj and xj stand for leftmost and rightmost coordinates of the ego segment on a x axis, respectively.
[0093] In some embodiments, after the calculation of the ego segment response value,
whether the second example blob is a false positive may be determined. Specifically, if the ego segment response value and the pixel response values of pixels in the second example blob are not on the same side of the predetermined range, the second example blob may be filtered out. Therefore, false positives may be further filtered out, especially for some objects like red and/or green characters, logos, and the like, which may have extreme pixel response values and hardly be filtered out in the blob analysis. Besides, following computations for these false positives may be avoided.
[0094] In S205, defining a predetermined number of ambient segments, each of which may overlap with the ego segment.
[0095] There are various ways to define the ambient segments. In some embodiments, each of the ambient segments may have a same shape and size with the ego segment.
[0096] FIG. 8 schematically illustrates the second example blob and an example of its ego segment and ambient segments. As shown in FIG. 8, a minimum enclosing rectangle (windowed with heavy lines) which is defined as the ego segment of the second example blob. Eight rectangles having a same shape and size as the minimum enclosing rectangle are defined as the ambient segments. Specifically, c0 to c8 are center points of the ego segment and the ambient segments. The eight ambient segments are aia3c0ai5, a2a c c8, a3a5a7Co, C2a6a8C6, c0a7a9an , c8c a-i0a-i 2, a-i 5C0ai-ia-i3,
[0097] In S207, calculating ambient segment response values.
[0098] Each of the ambient segment response values may be calculated based on
pixel response values of pixels in a corresponding ambient segment. For example, summation of the pixel response values may be calculated.
[0099] In some embodiments, whether the second example blob is a false positive may be determined based on its ego segment response value and ambient segment response value. For example, if the pixel response values were calculated based on the above Equation (1 ), the ego segment response value of the second example blob should be larger than any of its ambient segment response values if its pixels are red traffic light pixels, vice versa, or else, the second example blob may be filtered out.
[0100] In S209, calculating a probability value for the second example blob based on its ego segment response value and ambient segment response values.
[0101] In some embodiments, the probability value of the second example blob may be calculated based on Equation (3):
where pv stands for the probability value of the corresponding blob, η stands for the ego segment response value,
ambient blob and n is the predetermined number.
[0102] It could be concluded that, the larger the probability value is, the more likely the second example blob represents a traffic light.
[0103] In S211 , calculating probability values for other blobs in the second set of blobs based on S201 to S209.
[0104] In S213, selecting a predetermined number of blobs having greatest probability
values as the candidate blobs.
[0105] In some embodiments, the blobs for red traffic light and green traffic light are respectively sorted decreasingly based on the probability values, where top 20 blobs for red traffic light and top 30 blobs for green traffic light are selected as the candidate blobs.
[0106] In such ways, a relatively small number of blobs may be selected as the candidate blobs. False positives may be reduced and processing speed may be increased.
[0107] Referring back to FIG. 1 , in S113, verifying whether the candidate blobs are traffic lights.
[0108] Using a traffic light detector or classifier such as template matching, neural network, support vector machine (SVM), cascade detector, and the like, whether a region corresponding to a candidate blob represents a traffic light may be determined. In some embodiments, regions in the color image corresponding to the candidate blobs may be verified. In some embodiments, regions in a grayscale image converted from the color image corresponding to the candidate blobs may be verified. As stated above, based on the above processing, a relatively small number of candidate blobs may be remained, thereby improving both the reliability and efficiency.
[0109] Specifically, regions corresponding to the candidate blobs may be segmented in the color image firstly. Segmenting the regions may be implemented based on positions, sizes and types of the candidate blobs, and a traffic light template. In some embodiments, for a vertically arranged traffic light template, the candidate blob for a red traffic light should be located on top of the corresponding region, the candidate blob for a
green traffic light should be located on bottom of the corresponding region, and the corresponding region may have a width equaling to or a little larger than that of the blob and a height more than three times of that of the blob. For a horizontally arranged traffic light template, the candidate blob for a red traffic light should be located on left of the corresponding region, the candidate blob for a green traffic light should be located on right of the corresponding region, the corresponding region may have a width at least three times of that of the blob and a height equaling to or a little larger than that of the blob.
[0110] In some embodiments, the corresponding regions may be enlarged to ensure that they can cover the traffic light(s) in the color image. For example, a rectangle region corresponding to a candidate blob in the third set of blob may be calculated based on Equation (4): r = 1.2 max( , h) + l
where max(w,h) represents a larger value between the candidate blob's width and height measured in pixel number, cx and cy respectively represent x and y coordinates of a center point of the candidate blob, left, right, top and bottom respectively represent four side edges of the rectangle region.
[0111] FIG. 9 illustrates rectangle regions in the color image of FIG. 2 corresponding to the candidate blobs. The rectangle regions corresponding to the candidate blobs for red and green traffic lights are marked. It could be seen from FIG. 9 that false positives
may still exist. Thereafter, these rectangle regions may be verified using a traffic light detector or classifier. In some embodiments, the traffic light detector may include a cascade detector and a SVM. In some embodiments, the cascade detector may be used prior to the SVM, since the SVM takes more time than the cascade detector. In some embodiments, the rectangle regions in the color image may be converted into monochrome images before being verified by the traffic light detector. Based on the verification, whether there is a region in the color image, which region corresponds to one of the candidate blobs, representing a traffic light may be determined and the traffic light may be localized. A result may be output by framing the identified traffic light(s), such as illustrated in FIG. 10.
[0112] According to one embodiment, a system for traffic light detection is provided. The system may include a processing device configured to: obtain a color image; calculate pixel response values for pixels of the upper part of the color image using R, G, and B values of the pixels directly; identify candidate pixels based on the calculated pixel response values; identify a first set of blobs based on the candidate pixels; identify a second set of blobs from the first set of blobs based on at least one geometric parameter; identify candidate blobs from the second set of blobs based on pixel response values of pixels in and surrounding the second set of blobs; and verify whether the candidate blobs are traffic lights. Specific configurations of the processing device may be found by referring S101 to S113 above, which may not be illustrated in detail here.
[0113] According to one embodiment, a system for traffic light detection is provided. The system may include: means for calculating pixel response values for pixels of a color image, respectively, where each of the pixel response values may be calculated
using R, G, and B values of a corresponding pixel directly, such that pixel response values of red traffic light pixels are substantially distributed on a first side of a predetermined range and pixel response values of green traffic light pixels are substantially distributed on a second side of the predetermined range which is opposite to the first side; means for identifying pixels whose pixel response values are distributed on the first side or the second side as candidate pixels; means for identifying candidate blobs based on the candidate pixels; and means for verifying whether the candidate blobs are traffic lights.
[0114] According to one embodiment, a non-transitory computer readable medium, which contains a computer program for traffic light detection, is provided. When the computer program is executed by a processor, it will instruct the processor to: calculate pixel response values for pixels of a color image, respectively, where each of the pixel response values may be calculated using R, G, and B values of a corresponding pixel directly, such that pixel response values of red traffic light pixels are substantially distributed on a first side of a predetermined range and pixel response values of green traffic light pixels are substantially distributed on a second side of the predetermined range which is opposite to the first side; identify pixels whose pixel response values are distributed on the first side or the second side as candidate pixels; identify candidate blobs based on the candidate pixels; and verify whether the candidate blobs are traffic lights.
[0115] There is little distinction left between hardware and software implementations of aspects of systems; the use of hardware or software is generally a design choice representing cost vs. efficiency tradeoffs. For example, if an implementer determines
that speed and accuracy are paramount, the implementer may opt for a mainly hardware and/or firmware vehicle; if flexibility is paramount, the implementer may opt for a mainly software implementation; or, yet again alternatively, the implementer may opt for some combination of hardware, software, and/or firmware.
[0116] While various aspects and embodiments have been disclosed herein, other aspects and embodiments will be apparent to those skilled in the art. The various aspects and embodiments disclosed herein are for purposes of illustration and are not intended to be limiting, with the true scope and spirit being indicated by the following claims.
Claims
1 . A method for traffic light detection, comprising: obtaining a color image; calculating pixel response values for pixels of the color image, respectively, where each of the pixel response values is calculated using R, G, and B values of a corresponding pixel directly, such that pixel response values of red traffic light pixels are substantially distributed on a first side of a predetermined range and pixel response values of green traffic light pixels are substantially distributed on a second side of the predetermined range which is opposite to the first side; identifying pixels whose pixel response values are distributed on the first side or the second side as candidate pixels; identifying candidate blobs based on the candidate pixels; and verifying whether the candidate blobs are traffic lights.
2. The method according to claim 1 , wherein pixel response values of pixels other than red traffic light pixels and green traffic light pixels are substantially distributed within the predetermined range.
3. The method according to claim 1 , wherein each of the pixel response values is calculated based on multiplication of a first component and a second component of a corresponding pixel, where the first component indicates a tendency of the pixel response value, and the second component is able to amply the tendency if the tendency is towards the first side or the second side.
4. The method according to claim 3, wherein the first component is calculated based on summation of a third, a fourth, and a fifth components which are respectively based on R, G and B values of the corresponding pixel, where the third component has a first sign, and the fourth and fifth components have a second sign which is opposite to the first sign.
5. The method according to claim 4, wherein the third component is ki*R, the fourth component is -k2 *G, and the fifth component is -k3 *B, where k-i , k2 and k3 are set such that the tendency is substantially towards the first side if the corresponding pixel is a red traffic light pixel and the tendency is substantially towards the second side if the corresponding pixel is a green traffic light pixel.
6. The method according to claim 3, wherein the second component is calculated based on summation of a sixth and a seventh components which are respectively based on R and G values of the corresponding pixel, where the sixth and seventh components have the same sign.
7. The method according to claim 6, wherein the sixth component is k4 *R and the seventh component is k5 *G, where k4 and k5 are set such that the pixel response value is substantially on the first side if the corresponding pixel is a red traffic light pixel, and the pixel response value is substantially on the second side if the corresponding pixel is a green traffic light pixel.
8. The method according to claim 1 , wherein identifying the candidate blobs based on the candidate pixels comprises: identifying a first set of blobs based on the candidate pixels; and
identifying the candidate blobs from the first set of blobs, where each of the first set of blobs is determined whether or not a member of the candidate blobs based on pixel response values of its pixels and its surrounding pixels.
9. A system for traffic light detection, comprising a processing device configured to: obtain a color image; calculate pixel response values for pixels of the color image, respectively, where each of the pixel response values is calculated using R, G, and B values of a corresponding pixel directly, such that pixel response values of red traffic light pixels are substantially distributed on a first side of a predetermined range and pixel response values of green traffic light pixels are substantially distributed on a second side of the predetermined range which is opposite to the first side; identify pixels whose pixel response values are distributed on the first side or the second side as candidate pixels; identify candidate blobs based on the candidate pixels; and verify whether the candidate blobs are traffic lights.
10. The system according to claim 9, wherein pixel response values of pixels other than red traffic light pixels and green traffic light pixels are substantially distributed within the predetermined range.
1 1 . The system according to claim 9, wherein the processing device is configured to calculate each of the pixel response values based on multiplication of a first component and a second component of a corresponding pixel, where the first component indicates a tendency of the pixel response value, and the second
component is able to amply the tendency if the tendency is towards the first side or the second side.
12. The system according to claim 11 , wherein the first component is calculated based on summation of a third, a fourth, and a fifth components which are respectively based on R, G and B values of the corresponding pixel, where the third component has a first sign, and the fourth and fifth components have a second sign which is opposite to the first sign.
13. The system according to claim 12, wherein the third component is ki*R, the fourth component is -k2 *G, and the fifth component is -k3 *B, where k-i , k2 and k3 are set such that the tendency is substantially towards the first side if the corresponding pixel is a red traffic light pixel and the tendency is substantially towards the second side if the corresponding pixel is a green traffic light pixel.
14. The system according to claim 11 , wherein the second component is calculated based on summation of a sixth and a seventh components which are respectively based on R and G values of the corresponding pixel, where the sixth and seventh components have the same sign.
15. The system according to claim 14, wherein the sixth component is k4 *R and the seventh component is k5 *G, where k4 and k5 are set such that the pixel response value is substantially on the first side if the corresponding pixel is a red traffic light pixel, and the pixel response value is substantially on the second side if the corresponding pixel is a green traffic light pixel.
16. The system according to claim 9, wherein the processing device is configured to identify the candidate blobs by:
identifying a first set of blobs based on the candidate pixels; and identifying the candidate blobs from the first set of blobs, where each of the first set of blobs is determined whether or not a member of the candidate blobs based on pixel response values of its pixels and its surrounding pixels.
17. A system for traffic light detection, comprising: means for calculating pixel response values for pixels of a color image, respectively, where each of the pixel response values is calculated using R, G, and B values of a corresponding pixel directly, such that pixel response values of red traffic light pixels are substantially distributed on a first side of a predetermined range and pixel response values of green traffic light pixels are substantially distributed on a second side of the predetermined range which is opposite to the first side; means for identifying pixels whose pixel response values are distributed on the first side or the second side as candidate pixels; means for identifying candidate blobs based on the candidate pixels; and means for verifying whether the candidate blobs are traffic lights.
Priority Applications (4)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
PCT/CN2013/081842 WO2015024180A1 (en) | 2013-08-20 | 2013-08-20 | Traffic light detection |
EP13891786.9A EP3036730B1 (en) | 2013-08-20 | 2013-08-20 | Traffic light detection |
US14/901,651 US9953229B2 (en) | 2013-08-20 | 2013-08-20 | Traffic light detection |
CN201380078944.4A CN105453153B (en) | 2013-08-20 | 2013-08-20 | Traffic lights detects |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
PCT/CN2013/081842 WO2015024180A1 (en) | 2013-08-20 | 2013-08-20 | Traffic light detection |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2015024180A1 true WO2015024180A1 (en) | 2015-02-26 |
Family
ID=52482928
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/CN2013/081842 WO2015024180A1 (en) | 2013-08-20 | 2013-08-20 | Traffic light detection |
Country Status (4)
Country | Link |
---|---|
US (1) | US9953229B2 (en) |
EP (1) | EP3036730B1 (en) |
CN (1) | CN105453153B (en) |
WO (1) | WO2015024180A1 (en) |
Families Citing this family (23)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US9779314B1 (en) | 2014-08-21 | 2017-10-03 | Waymo Llc | Vision-based detection and classification of traffic lights |
US10309792B2 (en) | 2016-06-14 | 2019-06-04 | nuTonomy Inc. | Route planning for an autonomous vehicle |
US10126136B2 (en) | 2016-06-14 | 2018-11-13 | nuTonomy Inc. | Route planning for an autonomous vehicle |
US11092446B2 (en) | 2016-06-14 | 2021-08-17 | Motional Ad Llc | Route planning for an autonomous vehicle |
US10829116B2 (en) | 2016-07-01 | 2020-11-10 | nuTonomy Inc. | Affecting functions of a vehicle based on function-related information about its environment |
CN106023623A (en) * | 2016-07-28 | 2016-10-12 | 南京理工大学 | Recognition and early warning method of vehicle-borne traffic signal and symbol based on machine vision |
US10473470B2 (en) | 2016-10-20 | 2019-11-12 | nuTonomy Inc. | Identifying a stopping place for an autonomous vehicle |
US10331129B2 (en) | 2016-10-20 | 2019-06-25 | nuTonomy Inc. | Identifying a stopping place for an autonomous vehicle |
US10857994B2 (en) | 2016-10-20 | 2020-12-08 | Motional Ad Llc | Identifying a stopping place for an autonomous vehicle |
US10681513B2 (en) | 2016-10-20 | 2020-06-09 | nuTonomy Inc. | Identifying a stopping place for an autonomous vehicle |
FR3062944B1 (en) * | 2017-02-10 | 2021-05-14 | Continental Automotive France | FALSE-POSITIVE DETECTION PROCEDURE RELATING TO A SIGNALING FIRE |
US10614326B2 (en) * | 2017-03-06 | 2020-04-07 | Honda Motor Co., Ltd. | System and method for vehicle control based on object and color detection |
US10380438B2 (en) * | 2017-03-06 | 2019-08-13 | Honda Motor Co., Ltd. | System and method for vehicle control based on red color and green color detection |
US10650256B2 (en) | 2017-04-18 | 2020-05-12 | nuTonomy Inc. | Automatically perceiving travel signals |
US10643084B2 (en) | 2017-04-18 | 2020-05-05 | nuTonomy Inc. | Automatically perceiving travel signals |
JP2019053619A (en) * | 2017-09-15 | 2019-04-04 | 株式会社東芝 | Signal identification device, signal identification method, and driving support system |
EP3673233A4 (en) | 2018-04-18 | 2021-05-19 | Mobileye Vision Technologies Ltd. | Vehicle environment modeling with a camera |
US11087153B2 (en) * | 2018-10-31 | 2021-08-10 | Cognizant Technology Solutions India Pvt. Ltd. | Traffic light recognition system and method |
WO2020160144A1 (en) | 2019-01-29 | 2020-08-06 | Aptiv Technologies Limited | Traffic light estimation |
CN110716680B (en) * | 2019-10-09 | 2021-05-07 | 广州视源电子科技股份有限公司 | Control method and device of intelligent interactive panel |
CN111553477A (en) * | 2020-04-30 | 2020-08-18 | 深圳市商汤科技有限公司 | Image processing method, device and storage medium |
CN112863194B (en) * | 2021-01-20 | 2022-08-23 | 青岛以萨数据技术有限公司 | Image processing method, device, terminal and medium |
US11858504B2 (en) * | 2021-10-28 | 2024-01-02 | Mobileye Vision Technologies Ltd. | Stereo-assist network for determining an object's location |
Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2007072948A (en) * | 2005-09-09 | 2007-03-22 | Matsushita Electric Ind Co Ltd | Traffic signal lighting state identification method and apparatus |
KR20090055848A (en) * | 2007-11-29 | 2009-06-03 | 한국전자통신연구원 | Apparatus and method of detecting signal light |
CN102176287A (en) * | 2011-02-28 | 2011-09-07 | 无锡中星微电子有限公司 | Traffic signal lamp identifying system and method |
JP2011216051A (en) * | 2010-04-02 | 2011-10-27 | Institute Of National Colleges Of Technology Japan | Program and device for discriminating traffic light |
CN102354457A (en) * | 2011-10-24 | 2012-02-15 | 复旦大学 | General Hough transformation-based method for detecting position of traffic signal lamp |
KR101237374B1 (en) * | 2011-09-28 | 2013-02-26 | 주식회사 피엘케이 테크놀로지 | Image processing method for detecting traffic light |
Family Cites Families (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US8751154B2 (en) * | 2008-04-24 | 2014-06-10 | GM Global Technology Operations LLC | Enhanced clear path detection in the presence of traffic infrastructure indicator |
US20100262336A1 (en) * | 2009-04-09 | 2010-10-14 | Qualcomm Incorporated | System and method for generating and rendering multimedia data including environmental metadata |
JP5386537B2 (en) * | 2011-05-12 | 2014-01-15 | 富士重工業株式会社 | Environment recognition device |
CN102496282B (en) * | 2011-12-16 | 2014-04-16 | 湖南工业大学 | Traffic intersection signal light state identification method based on RGB color transformation |
CN102542260A (en) * | 2011-12-30 | 2012-07-04 | 中南大学 | Method for recognizing road traffic sign for unmanned vehicle |
WO2014078979A1 (en) * | 2012-11-20 | 2014-05-30 | Harman International Industries, Incorporated | Method and system for detecting traffic lights |
-
2013
- 2013-08-20 US US14/901,651 patent/US9953229B2/en active Active
- 2013-08-20 WO PCT/CN2013/081842 patent/WO2015024180A1/en active Application Filing
- 2013-08-20 EP EP13891786.9A patent/EP3036730B1/en active Active
- 2013-08-20 CN CN201380078944.4A patent/CN105453153B/en active Active
Patent Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2007072948A (en) * | 2005-09-09 | 2007-03-22 | Matsushita Electric Ind Co Ltd | Traffic signal lighting state identification method and apparatus |
KR20090055848A (en) * | 2007-11-29 | 2009-06-03 | 한국전자통신연구원 | Apparatus and method of detecting signal light |
JP2011216051A (en) * | 2010-04-02 | 2011-10-27 | Institute Of National Colleges Of Technology Japan | Program and device for discriminating traffic light |
CN102176287A (en) * | 2011-02-28 | 2011-09-07 | 无锡中星微电子有限公司 | Traffic signal lamp identifying system and method |
KR101237374B1 (en) * | 2011-09-28 | 2013-02-26 | 주식회사 피엘케이 테크놀로지 | Image processing method for detecting traffic light |
CN102354457A (en) * | 2011-10-24 | 2012-02-15 | 复旦大学 | General Hough transformation-based method for detecting position of traffic signal lamp |
Non-Patent Citations (4)
Title |
---|
GEORGE SIOGKAS ET AL.: "Traffic Lights Detection In Adverse Conditions Using Color, Symmetry And Spatiotemporal Information", INTERNATIONAL CONFERENCE ON COMPUTER VISION THEORY AND APPLICATIONS, 24 February 2012 (2012-02-24), pages 620 - 627 |
MASAKO OMACHI ET AL.: "Detection of Traffic Light Using Structural Information", ICSP2010 PROCEEDINGS, 31 December 2010 (2010-12-31), pages 809 - 811, XP031816718 * |
MOISES DIAZ-CABRERA ET AL.: "Suspended Traffic Lights Detection and Distance Estimation Using Color Features", 2012 15TH INTERNATIONAL IEEE CONFERENCE ON INTELLIGENT TRANSPORTATION SYSTEMS, 31 December 2012 (2012-12-31), XP032264096 * |
See also references of EP3036730A4 |
Also Published As
Publication number | Publication date |
---|---|
CN105453153A (en) | 2016-03-30 |
CN105453153B (en) | 2018-07-20 |
EP3036730A1 (en) | 2016-06-29 |
EP3036730B1 (en) | 2019-12-18 |
US9953229B2 (en) | 2018-04-24 |
EP3036730A4 (en) | 2017-03-29 |
US20160148063A1 (en) | 2016-05-26 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
EP3036730B1 (en) | Traffic light detection | |
US9811746B2 (en) | Method and system for detecting traffic lights | |
RU2601185C2 (en) | Method, system and computer data medium for face detection | |
CN108509907B (en) | Car light detection method, device, medium and equipment for realizing intelligent driving | |
US10943141B2 (en) | Object detection device and object detection method | |
EP3036714B1 (en) | Unstructured road boundary detection | |
JP2011216051A (en) | Program and device for discriminating traffic light | |
CN111882568B (en) | Fire image edge extraction processing method, terminal and system | |
JP2009048629A (en) | Detecting method | |
EP3171295A1 (en) | Method for processing image and computer-readable non-transitory recording medium storing program | |
CN107644538B (en) | Traffic signal lamp identification method and device | |
KR101246120B1 (en) | A system for recognizing license plate using both images taken from front and back faces of vehicle | |
CN107844761B (en) | Traffic sign detection method and device | |
CN106022268A (en) | Identification method and device of speed limiting sign | |
EP3051461A1 (en) | A computer implemented system and method for extracting and recognizing alphanumeric characters from traffic signs | |
CN112991374A (en) | Canny algorithm-based edge enhancement method, device, equipment and storage medium | |
US10115028B2 (en) | Method and device for classifying an object in an image | |
WO2020001631A1 (en) | Visual camera-based method for identifying edge of self-shadowing object, device, and vehicle | |
CN108734123B (en) | Highway sign recognition method, electronic device, storage medium, and system | |
CN106803064B (en) | Traffic light rapid identification method | |
JP7264428B2 (en) | Road sign recognition device and its program | |
WO2024016632A1 (en) | Bright spot location method, bright spot location apparatus, electronic device and storage medium | |
CN108647713B (en) | Embryo boundary identification and laser track fitting method | |
TW201344630A (en) | Moving subject detecting system and method | |
JP6354316B2 (en) | Image detection apparatus and program |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
WWE | Wipo information: entry into national phase |
Ref document number: 201380078944.4 Country of ref document: CN |
|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 13891786 Country of ref document: EP Kind code of ref document: A1 |
|
WWE | Wipo information: entry into national phase |
Ref document number: 2013891786 Country of ref document: EP |
|
WWE | Wipo information: entry into national phase |
Ref document number: 14901651 Country of ref document: US |
|
NENP | Non-entry into the national phase |
Ref country code: DE |