CN115909731A

CN115909731A - Method and device for predicting switching time of traffic signal lamp, electronic equipment and medium

Info

Publication number: CN115909731A
Application number: CN202211396642.9A
Authority: CN
Inventors: 刘子昊; 宋雨坤; 张岩; 白红霞
Original assignee: Beijing Baidu Netcom Science and Technology Co Ltd
Current assignee: Beijing Baidu Netcom Science and Technology Co Ltd
Priority date: 2022-11-09
Filing date: 2022-11-09
Publication date: 2023-04-04

Abstract

The disclosure provides a method and a device for predicting a switching moment of a traffic signal lamp, electronic equipment and a medium, and relates to the technical field of image processing, in particular to the technical field of intelligent traffic. The specific implementation scheme is as follows: aiming at each target signal lamp needing to be scheduled, sending a collection task to a collection client in a vehicle which can pass through the target signal lamp in a specified time period; determining switching information of a target signal lamp based on an acquisition result fed back by an acquisition client, wherein the switching information comprises switching time and a lamp state before switching; and predicting switching information of each switching of the target signal lamp in a future specified period based on the switching information and the switching rule of the target signal lamp, and issuing the predicted switching information to a navigation client in the vehicle passing through the target signal lamp in the specified period so that the navigation client displays countdown of the target signal lamp based on the received switching information. Therefore, the user can intuitively know when the target signal lamp is switched to the light state.

Description

Method and device for predicting switching time of traffic signal lamp, electronic equipment and medium

Technical Field

The present disclosure relates to the field of image processing technology, and in particular, to the field of intelligent transportation technology.

Background

When a driver drives the vehicle by using vehicle-mounted navigation, the passing opportunity can be judged at the intersection of the traffic lights according to the display state of the traffic lights. For example, for a traffic light capable of displaying countdown, a driver can timely decelerate and brake when knowing that the green light is to be switched into the red light, so that the situation of sudden braking and even rear-end collision can be avoided, and the driver can prepare for starting the vehicle in advance when knowing that the red light is to be switched into the green light.

Disclosure of Invention

The disclosure provides a method and a device for predicting switching time of traffic signal lamps, electronic equipment and a medium.

In a first aspect, the present disclosure provides a method for predicting a switching time of a traffic signal lamp, including:

sending an acquisition task to an acquisition client in a vehicle capable of passing through a target signal lamp within a specified time period aiming at each target signal lamp to be scheduled, wherein the acquisition task is used for indicating the acquisition client to acquire an image of the target signal lamp within the specified time period;

determining switching information of the target signal lamp based on an acquisition result fed back by the acquisition client, wherein the switching information comprises switching time and a lamp state before switching;

and predicting switching information of each switching of the target signal lamp in a future specified period based on the switching information and the switching rule of the target signal lamp, and issuing the predicted switching information to a navigation client in a vehicle passing through the target signal lamp in the specified period so that the navigation client displays the countdown of the target signal lamp based on the received switching information.

In a second aspect, the present disclosure provides a device for predicting a switching time of a traffic signal lamp, including:

the system comprises a sending module, a scheduling module and a scheduling module, wherein the sending module is used for sending a collection task to a collection client in a vehicle which can pass through a target signal lamp in a specified time period aiming at each target signal lamp to be scheduled, and the collection task is used for indicating the collection client to collect an image of the target signal lamp in the specified time period;

the determining module is used for determining switching information of the target signal lamp based on an acquisition result fed back by the acquisition client, wherein the switching information comprises switching time and a lamp state before switching;

the prediction module is used for predicting switching information of each switching of the target signal lamp in a future specified period based on the switching information and the switching rule of the target signal lamp;

the sending module is further configured to send predicted switching information to a navigation client in the vehicle passing through the target signal lamp in the specified period, so that the navigation client displays countdown of the target signal lamp based on the received switching information.

In a third aspect, the present disclosure provides an electronic device comprising:

at least one processor; and

a memory communicatively coupled to the at least one processor; wherein,

the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of the first aspect.

In a fourth aspect, the present disclosure provides a non-transitory computer readable storage medium having stored thereon computer instructions for causing the computer to perform the method according to the first aspect.

In a fifth aspect, the present disclosure provides a computer program product comprising a computer program which, when executed by a processor, implements the method according to the first aspect.

It should be understood that the statements in this section do not necessarily identify key or critical features of the embodiments of the present disclosure, nor do they limit the scope of the present disclosure. Other features of the present disclosure will become apparent from the following description.

Drawings

The drawings are included to provide a better understanding of the present solution and are not to be construed as limiting the present disclosure. Wherein:

fig. 1 is a first flowchart of a method for predicting a switching time of a traffic light according to an embodiment of the present disclosure;

fig. 2 is a second flowchart of a method for predicting a switching time of a traffic light according to an embodiment of the disclosure;

fig. 3 is a third flowchart illustrating a method for predicting a switching time of a traffic signal according to an embodiment of the disclosure;

fig. 4 is a fourth flowchart illustrating a method for predicting a switching time of a traffic light according to an embodiment of the disclosure;

fig. 5 is a fifth flowchart illustrating a method for predicting a switching time of a traffic light according to an embodiment of the disclosure;

fig. 6 is an exemplary schematic diagram of a method for predicting a switching time of a traffic signal according to an embodiment of the present disclosure;

fig. 7 is a schematic structural diagram of a device for predicting a switching time of a traffic signal lamp according to an embodiment of the present disclosure;

fig. 8 is a block diagram of an electronic device for implementing the method for predicting the switching time of traffic signal lights according to the embodiment of the disclosure.

Detailed Description

Exemplary embodiments of the present disclosure are described below with reference to the accompanying drawings, in which various details of the embodiments of the disclosure are included to assist understanding, and which are to be considered as merely exemplary. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the present disclosure. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.

When a user navigates by using the vehicle-mounted navigation client, the user cannot see the traffic light countdown at the traffic light intersection at a high probability, and cannot know the current remaining time of the red light or the green light when approaching the traffic light intersection, so that the psychological uncertainty of the user is caused, and the driving decision of the user is inconvenient. For example, when a vehicle stops at a traffic light intersection and waits for the end of a red light, the remaining time of the red light is not known, and the vehicle may not be started in time after switching a green light due to distraction. For another example, when driving to a traffic light intersection, it is not known that the green light is about to end, and a sudden brake or even a rear-end collision may occur.

In order to enable a user to know the remaining time of the current red light or green light, the method and the device for displaying the traffic light countdown on the navigation client can predict the future traffic light switching time, so that the user can know the traffic light switching time accurately and better experience is brought to the user.

The following describes in detail a method for predicting a switching time of a traffic signal according to an embodiment of the present disclosure.

As shown in fig. 1, an embodiment of the present disclosure provides a method for predicting a switching time of a traffic signal lamp, where the method may be applied to a server, and the method includes:

s101, aiming at each target signal lamp needing to be scheduled, sending a collection task to a collection client in a vehicle which can pass through the target signal lamp in a specified time period, wherein the collection task is used for indicating the collection client to collect the image of the target signal lamp in the specified time period.

The target signal lamps to be scheduled are multiple, and the embodiment of the disclosure can predict the switching time of each target signal lamp to be scheduled.

The server can determine vehicles which can pass through the intersection where the target signal lamp is located within a specified time period based on the real-time collected driving track information, and then sends collection tasks to collection clients installed on the vehicles.

The acquisition client can be a vehicle-mounted terminal. The acquisition client can control the vehicle-mounted camera to acquire the image of the target signal lamp according to the acquisition task and acquire the image acquired by the vehicle-mounted camera; or the acquisition client can extract the image of the target signal lamp from the video shot by the vehicle-mounted camera in real time according to the acquisition task.

S102, switching information of the target signal lamp is determined based on the collection result fed back by the collection client, and the switching information comprises switching time and a lamp state before switching.

Wherein the switching moment may be in the form of a time stamp, the light state refers to the color of a signal light, such as a red light, a green light or a yellow light. If the green light is on before switching, the light state before switching is the green light.

S103, based on the switching information and the switching rule of the target signal lamp, switching information of each switching of the target signal lamp in a future specified period is predicted, and the predicted switching information is issued to a navigation client in a vehicle passing through the target signal lamp in the specified period, so that the navigation client displays countdown of the target signal lamp based on the received switching information.

The server can acquire information such as a running track and a navigation route of a vehicle where the navigation client is located in real time, and can acquire the vehicles which can pass through a target signal lamp in a specified period based on the information of the running track and the navigation route acquired in real time, and then send predicted switching information to the navigation clients in the vehicles. Therefore, when the navigation client passes through or approaches to the intersection where the target signal lamp is located in the designated period, the navigation client can display the countdown of the target signal lamp in the navigation interface.

In the embodiment of the present disclosure, the specified period may be determined based on the switching rule of the target signal lamp and the specified time period, for example, if the switching rule of the target signal lamp from 7 o 'clock to 10 o' clock in the morning of each day is the same, the specified time period may be from 7 o 'clock to 7 o' clock 10 minutes of a day, and the specified period may be from 7 o 'clock to 10 o' clock of each subsequent day.

In order to improve the accuracy of the prediction, the method flow shown in fig. 1 may be executed again for the target signal lamp at regular intervals. For example, the specified period is from 7 to 10 o ' clock per day in one week, then the switching information is again predicted with 7 to 7 o ' clock 10 minutes in the morning of the first day in the second week as the specified time period, and then the navigation client that passes through the target signal displays the countdown of the target signal using the switching information obtained by the prediction again in 7 to 10 o ' clock in the morning of the second week.

The above-mentioned specified time period and the specified period may be set based on actual conditions, for example, if the prediction of the target signal lamp is found to be not accurate enough through testing, the frequency of re-prediction may be increased, and if the prediction of the target signal lamp is found to be accurate through testing, the frequency of re-prediction may be decreased.

By adopting the method, the acquisition task can be sent to the acquisition client side which can pass through the target signal lamp in the appointed time period, so that the acquisition client side acquires the image of the target signal lamp in the appointed time period, and the acquisition result fed back by the acquisition client side is further acquired. The acquisition result is obtained based on the image of the target signal lamp acquired in the specified time period, so that the switching information included in the acquisition result is the switching information of the actual switching of the lamp state, and the switching information of each switching of the target signal lamp in the future specified period can be accurately predicted by using the switching information and the switching rule of the target signal lamp. Furthermore, by sending the switching information to the navigation client, the navigation client can display countdown when passing through the target signal lamp in a future specified period, so that a user can intuitively know when the target signal lamp is switched to be in a light state, driving decision is facilitated, and the experience of the user in using the navigation client can be improved.

In some embodiments, before executing the method flow of fig. 1, the server may further obtain real-time trajectory information returned by the multiple navigation clients, and determine, based on the real-time trajectory information and a pre-collected switching rule of the multiple traffic lights, multiple target traffic lights that need to be scheduled and a specified time period corresponding to each target traffic light.

In the embodiment of the disclosure, a switching rule of the traffic signal lamp can be excavated by using massive historical track information and traffic light images, or the switching rule of the traffic signal lamp provided by a relevant department can be acquired by cooperating with the relevant department.

Optionally, the real-time track information returned by the navigation client may be used to determine a traffic signal lamp with a higher current heat, the traffic signal lamp with the higher heat is used as a target signal lamp, and then a scheduling time interval, that is, an assigned time period, corresponding to the target signal lamp is determined based on a switching rule of the target signal lamp. For example, if the switching law of the target signal lamp includes the switching law in the early peak period, the switching law in the late peak period and the switching laws in the rest periods, it may be determined that the scheduling time interval corresponding to the target signal includes a period of the early peak, a period of the late peak and a period of the rest periods.

For another example, if the switching rule of the target signal lamp is to switch every 1 minute for 1 time all day, any period of time all day may be selected as the designated period of time corresponding to the target signal lamp.

It is understood that, in order to make the acquisition client know the position of the target signal lamp, the geographical location area where the target signal lamp is located may also be specified in the acquisition task.

By adopting the method, the server can determine the target signal lamps to be scheduled and the designated time period corresponding to each target signal lamp based on the real-time track information and the switching rules of the traffic signal lamps collected in advance, so that the collection tasks can be sent underground more accurately, and the collection results fed back by the collection client based on the collection tasks can be predicted more accurately.

In the above embodiments, the target signal lamp is a countdown signal lamp or a non-countdown signal lamp.

The countdown signal lamp means a signal lamp capable of displaying a countdown, and the non-countdown signal lamp means a signal lamp not displaying a countdown.

The acquisition mode of the acquisition task instruction of the non-countdown signal lamp is as follows: images including non-countdown lights are acquired a plurality of times in succession at a designated acquisition frequency. It will be appreciated that for a signal without a countdown, a number of successive images including the signal are required to determine the timing of the signal switching. For example, the specified acquisition frequency may be once per second, or once every 500 milliseconds.

The collection mode of the collection task instruction of the countdown signal lamp is as follows: at a specified time, an image is acquired that includes a countdown signal light.

By adopting the method, the proper collection mode can be selected by utilizing the characteristics of the countdown signal lamp and the non-countdown signal lamp, and for the non-countdown signal lamp, the client can continuously collect images including the non-countdown signal lamp for multiple times, so that the time of switching the lamp state can be determined by utilizing the continuous images subsequently. For the countdown signal lamp, the time when the lamp state switching occurs can be determined through countdown numbers, so that excessive images do not need to be acquired. Therefore, the accuracy of the determined switching information can be ensured on the basis of collecting as few images as possible.

In one implementation manner of the present disclosure, acquiring a first type of acquisition result fed back by a client for a non-countdown signal lamp includes: the acquisition client determines the light state before switching, the light state after switching and the switching time based on the acquired images. For example, if the light states of the traffic lights included in two adjacent images are different in the plurality of images continuously acquired, the light states of the traffic lights included in the two adjacent images may be set as the light state before switching and the light state after switching, respectively, and the smaller time stamp of the acquisition time stamps of the two adjacent images may be set as the switching time.

The second type of acquisition results fed back by the client aiming at the countdown signal lamp comprise: the acquisition client determines a current light state, a countdown number, and an acquisition timestamp based on the acquired image.

In the embodiment of the present disclosure, the first type of acquisition result and the second type of acquisition result may both be in the form of text.

On this basis, as shown in fig. 2, the step S102 of determining the switching information of the target signal lamp based on the collection result fed back by the collection client may be implemented as step S1021 or step S1022.

And S1021, taking the first-type acquisition result as the switching information of the target signal lamp under the condition of receiving the first-type acquisition result.

It can be understood that, in this case, the switching time included in the switching information is the switching time at which the switching of the light state actually occurs, and the server may predict the future switching time of the target signal light by using the switching time and combining the switching rule of the target signal light.

As an example, assuming that the switching time included in the switching information is 07.

And S1022, under the condition that the second type of acquisition result is received, taking the sum of the acquisition timestamp and the countdown number as the switching time of the target signal lamp, and taking the current lamp state as the lamp state before switching to obtain the switching information of the target signal lamp.

It is understood that the switching time included in the switching information in this case is a switching time calculated based on the acquisition time stamp and the count-down number, and it is determined that the target signal lamp will be switched from the current lamp state to the next lamp state at the switching time. For example, assuming that the acquisition timestamp of the image is 07.

By adopting the method, the server can receive the first type acquisition result and the second type acquisition result fed back by the acquisition client based on the acquired image, namely the acquisition client completes the image identification of the acquired image, so that the acquisition client only needs to feed back the acquisition result obtained after the image identification to the server, the acquired image does not need to be transmitted to the server, the transmitted data volume can be reduced, the information transmission overhead is reduced, the transmission efficiency is improved, and the overlarge calculation pressure of the server can be avoided. The server can determine the accurate switching information of the target signal lamp by utilizing different types of acquisition results, and then can more accurately predict the future switching information of the target signal lamp subsequently.

In another implementation manner of the embodiment of the present disclosure, the third type of collection result fed back by the collection client for the non-countdown signal lamp includes a plurality of first images, and the first images are images including the non-countdown signal lamp. The plurality of first images are acquired by the acquisition client according to the received acquisition task.

The fourth type of acquisition result fed back by the acquisition client for the countdown signal lamp comprises a second image, and the second image is an image comprising the countdown signal lamp. And the second image is an image acquired by the acquisition client according to the received acquisition task.

On this basis, as shown in fig. 3, the step S102 determines the switching information of the target signal lamp based on the collection result fed back by the collection client, and may be specifically implemented as S1023 or S1024.

And S1023, under the condition that a third type of acquisition result is received, carrying out image recognition on the plurality of first images, and taking the image recognition to obtain the lamp state before switching, the lamp state after switching and the switching time as switching information of the target signal lamp.

And S1024, under the condition that a fourth type of acquisition result is received, carrying out image identification on the second image to obtain a current headlight state, a count-down number and an acquisition timestamp, taking the sum of the acquisition timestamp and the count-down number as the switching time of the target signal lamp, and taking the current headlight state and the calculated switching time as the switching information of the target signal lamp.

By adopting the method, the server can receive the acquired image fed back by the acquisition client based on the acquisition task, and then the server can obtain the switching information of the target signal lamp through image identification, and the server can reduce the requirement on the acquisition client by carrying out image identification, so that the deployment is simpler. Moreover, the server can identify the image with the countdown signal lamp and the image without the countdown signal lamp in different modes, so that the determined switching information can be more accurate.

In some embodiments of the present disclosure, as shown in fig. 4, the image recognition on the plurality of first images may specifically include the following steps:

s401, acquiring the acquisition time stamp of each first image.

Wherein, the server can obtain the acquisition time stamp of the first image from the detailed information of the first image.

The detailed information of the image includes information such as a captured time stamp and a capturing position of the image, and the server may acquire the captured time stamp of the first image from the detailed information of the first image as the capture time stamp.

S402, aiming at each first image, identifying the first image by using a target detection model to obtain the target position of a non-countdown signal lamp in the first image, wherein the non-countdown signal lamp is included in the first image.

The server may input each first image into the target detection model, and obtain a target position output by the target detection model for each first image.

The target detection model is a pre-trained neural network model capable of detecting the traffic signal lamps in the images.

And S403, identifying the non-countdown signal lamp at the target position in the first image by using the lamp state identification model to obtain the lamp state of the non-countdown signal lamp.

Optionally, the server may intercept an image of the non-countdown signal from the target position in the first image, input the image of the non-countdown signal into the light state identification model, and obtain the light state output by the light state identification model.

Or, the server may input the first image and the target position into the light state identification model, and then the light state identification model may identify the non-countdown signal lamp included in the first image based on the target position and output the light state, and the server may acquire the light state output by the light state identification model.

S404, determining two adjacent first images with the lamp state switching based on the lamp states of the non-countdown signal lamps included in the first images.

For example, if the server acquires 20 continuously acquired first images in which the light states of the traffic lights included in the first 10 first images are all red lights and the light states of the traffic lights included in the 11 th first image are green lights, it may be determined that a light state switching has occurred between the 10 th and 11 th first images.

It should be noted that the embodiment of the present disclosure does not limit the execution sequence between S401 and S402-S404, and S401 may be executed before S405, and in fig. 4, S401 is executed first, and then S402-S404 are executed as an example.

S405, the smaller collecting time stamp in the collecting time stamps of the two adjacent first images is used as the switching time, and the lamp state of the signal lamp included in the first image with the smaller collecting time stamp in the two adjacent first images is used as the lamp state before switching.

Continuing with the example in S404, two adjacent first images are the 10 th and 11 th first images, and the capture timestamp of the 10 th first image is smaller, the capture timestamp of the first image may be used as the switching time, and the light state of the signal light included in the first image may be used as the light state before switching, that is, the light state before switching is the red light.

By adopting the method, the server can identify the target position of the non-countdown signal lamp in the first image by using the target detection model, and further identify the lamp state of the non-countdown signal lamp in the first image by using the lamp state identification model, so that the switching time when the lamp state is switched and the lamp state before switching can be accurately determined according to the identification result. Compared with a mode of acquiring switching information provided by a relevant department in cooperation with the relevant department managing the traffic signal lamp or a mode of determining the switching information by using time information such as acceleration, deceleration, parking starting and the like in a driving track in the related technology, the embodiment of the disclosure can more accurately determine the switching information of the target signal lamp with lower cost.

In some embodiments of the present disclosure, as shown in fig. 5, the image recognition on the second image may specifically include the following steps:

and S501, acquiring a collecting time stamp of the second image.

The manner of acquiring the acquisition time stamp of the second image is the same as the manner of acquiring the acquisition time stamp of the first image, and reference may be made to the related description in S401, which is not repeated herein.

S502, identifying the second image by using the target detection model to obtain the target position of the non-countdown signal lamp in the second image.

The server may input the second image into the target detection model and obtain a target position output by the target detection model.

S503, recognizing the countdown signal lamp of the target position in the second image by using the lamp state recognition model, and obtaining the current lamp state.

Optionally, the server may intercept the image of the countdown signal from the target position in the second image, input the image of the countdown signal into the light state identification model, and obtain the light state output by the light state identification model.

Or, the server may input the second image and the target position into the light state identification model, and then the light state identification model may identify the countdown signal lamp included in the second image based on the target position and output the light state, and the server may acquire the light state output by the light state identification model.

S504, recognizing the countdown signal lamp at the target position in the second image by utilizing an Optical Character Recognition (OCR) model to obtain a countdown number.

Alternatively, the server may intercept the image of the countdown signal from the target location in the second image, input the image of the countdown signal into the OCR model, and obtain the countdown number output by the OCR model.

Alternatively, the server may input the second image and the target position into the OCR model, and the OCR model may recognize the countdown lamps included in the second image based on the target position and output the countdown number, and the server may acquire the countdown number output by the OCR model.

It should be noted that, in the embodiment of the present disclosure, the execution sequence between S503 and S504 is not limited, and may be executed in parallel, or may be executed according to a sequential order, where S503 is executed first in fig. 5 as an example.

In addition, the disclosed embodiment also does not limit the execution sequence between S501 and S502-S504, and S501 is executed first and then S502-S504 are executed in fig. 5 as an example.

By adopting the method, the server can firstly detect the target position of the countdown signal lamp included in the second image by utilizing the target detection model, then respectively utilize the lamp state recognition model to recognize the lamp state of the countdown signal lamp, and utilize the OCR recognition model to recognize the countdown number in the countdown signal lamp, so that the switching information of the countdown signal lamp can be accurately obtained. Compared with a mode of acquiring switching information provided by a relevant department in cooperation with the relevant department managing the traffic signal lamp or a mode of determining the switching information by using time information such as acceleration, deceleration, parking starting and the like in a driving track in the related technology, the embodiment of the disclosure can more accurately determine the switching information of the target signal lamp with lower cost.

It should be noted that, as an alternative embodiment, the server may send the same collection task to multiple collection clients. Accordingly, the server may receive the collection results fed back by the plurality of collection clients. In this case, the server may perform the timestamp calibration and the abnormal value filtering process on the acquisition results fed back by the plurality of acquisition clients.

The clocks of the cameras on different vehicles may not be synchronized, and a clock error may exist, so that an error may also exist in acquiring the timestamp of the image in the image fed back by the client. In the embodiment of the disclosure, the timestamp calibration may be performed on images fed back by multiple acquisition clients based on the same acquisition task, and the image with an error timestamp may be used as an abnormal image with a too low confidence level to perform deletion processing.

For example, the specified time period specified by the acquisition task is 7 to 10 minutes, the server receives the acquisition results fed back by 3 acquisition clients for the same acquisition task, the timestamps of the images fed back by the acquisition clients 1 and 2 are both 7 to 10 minutes, and the images fed back by the acquisition clients 3 can be deleted after the timestamps of the images fed back by the acquisition clients 3 are 7 to 10 minutes.

For the situation that the acquisition client side feeds back multiple first images, if the server receives the first images fed back by the acquisition client sides for the same acquisition task, after the abnormal value filtering is completed, the first images fed back by the acquisition client sides are sorted in the order from small to large according to the acquisition timestamps, and then the sorted multiple first images are subjected to image recognition according to the method described in the embodiment. This may further improve the accuracy of the determined handover information.

It should be noted that, in the scenario of performing image recognition by the capture client described in the foregoing embodiment, the capture client may also perform image recognition according to the method in fig. 4 and fig. 5.

The target detection model, the light state recognition model and the OCR model used in the embodiments corresponding to fig. 4 and 5 are models trained based on a signal lamp image sample set. Alternatively, the target detection model may be implemented by using YOLO (english full name: you Only Look one) or other target detection algorithms, and YOLO is a target detection algorithm.

The signal lamp image sample set comprises images of high-heat signal lamps, images of signal lamps with countdown and images of signal lamps with appointed switching rules, wherein the high-heat signal lamps are signal lamps of intersections with the number of passing vehicles larger than a preset number threshold value in a preset time period.

In the embodiment of the disclosure, the server can dig a set of valuable traffic signal light images by using massive user tracks and offline-scheduled traffic signal light images in an offline calculation stage, and the set of valuable traffic signal light images is used as an image in a signal light image sample set.

The signal lamp with high heat can be determined based on the user track, for example, if the number of passing vehicles at a certain intersection is greater than a preset number threshold in a preset time period, it can be determined that the signal lamp image including the intersection is the signal lamp image with high heat.

The preset time period may be a week, a day, or several hours, which is not specifically limited by the embodiments of the present disclosure.

The signal lamp with the appointed switching rule can be determined through user track information, and can also cooperate with related departments for managing the signal lamp, so that part of the signal lamp with the appointed switching rule is obtained.

Alternatively, the screening of images of valuable traffic lights may also be performed based on the acquisition timestamps of the images of the traffic lights and the proportion of the acquisition timestamps at day and night.

For example, traffic light images that can cover all day's time periods can be screened out, and the proportion of traffic lights collected daytime and the proportion of traffic lights collected night can be preset, the preset proportion being in accordance with the real proportion of vehicles passing through the traffic lights daytime and night.

In the embodiment of the disclosure, the excavated traffic signal lamp image can be labeled in a manual labeling mode. And can cooperate with related departments to acquire real data of the lamps of the traffic signals.

For example, to train the target detection model, the position of the traffic light in each of the traffic light images as a sample may be manually marked.

In order to train the light state recognition model, the light state in the traffic signal lamp image can be manually marked, and the image of the traffic signal lamp and the corresponding light state provided by the relevant department can also be directly obtained.

In order to train the OCR model, the countdown number in the countdown signal lamp image may be manually marked, or the countdown signal lamp image and the corresponding countdown number provided by the relevant department may be directly obtained.

After the signal lamp image sample set is obtained, supervised training can be performed on the target detection model, the lamp state recognition model and the OCR model by using the signal lamp image sample set, so that the target detection model, the lamp state recognition model and the OCR model with high recognition accuracy are obtained.

Optionally, in the process of training the model, the overfitting phenomenon can be reduced by utilizing a regularization coefficient or a focal-loss function in the related technology, the false recall of non-signal lamp images can be reduced by setting reasonable sample distribution, and the generalization capability of the trained model to images acquired at night and images acquired at a long distance can be improved in a data enhancement mode.

With this method, the signal lamp image sample set includes an image of a high-heat signal lamp, an image of a signal lamp with countdown, and an image of a signal lamp with a specified switching law. Wherein, by means of the image of the high-heat signal lamp, the method can be made to cover as many users as possible with fewer signal lamp images. The utilization rate of the acquired images can be improved through the images of the signal lamps with countdown. The stability and accuracy of the switching time of the subsequent prediction can be higher through the image of the signal lamp with the specified switching rule. The signal lamp image sample set covers more situations which may occur in an actual scene, so that the trained model can accurately identify the traffic signal lamp image, and the accuracy of subsequent predicted switching information is improved.

As shown in fig. 6, fig. 6 is an exemplary schematic diagram of a method for predicting a traffic signal light switching time provided by the embodiment of the present disclosure.

The server may mine the more hot lamps, the lamps with countdown information, and the lamps with specific rules from the full set of lamps based on the user trajectory information, combine the images of the lamps into a sample set of signal lamp images.

And then a scheduling control module in the server can generate a real-time acquisition task based on the signal lamp image sample set and real-time track information returned by the navigation client, wherein the real-time acquisition task comprises an appointed time, an appointed scheduling time, an appointed position area, an appointed acquisition frequency and an appointed acquisition mode for acquiring a target signal lamp.

The scheduling control module can perform collection subtask distribution to a plurality of vehicles based on the real-time track returned by the navigation client.

For example, if the scheduling control module determines that the vehicle a and the vehicle B will pass through the target signal lamp 1 in the specified time period, the acquisition subtasks corresponding to the target signal lamp 1 may be issued to the acquisition clients in the vehicle a and the vehicle B, respectively. If the vehicle C is determined to pass through the target signal lamp 2 in the specified time period, the acquisition subtask corresponding to the target signal lamp 2 can be issued to the vehicle C.

Then, an image recovery module in the server can recover the traffic signal lamp images collected by the collecting client based on the collecting subtasks.

Then an image identification module in the server can identify the images of the traffic lights, the positions and states of the traffic lights can be identified for the images of the non-countdown lights, and the positions, states and numbers of the countdown lights can be identified for the images of the countdown lights. For a specific identification method, reference may be made to the related description in the above embodiments, and details are not repeated here.

The server can further calculate the traffic light switching point by utilizing the identification result, namely for the image with the countdown number, the acquisition timestamp and the countdown number can be added to obtain the switching time. For images without countdown, the timing of the light state switching can be determined using the recognition results for a plurality of consecutive images.

And the server may also perform time stamp correction based on the recognition results of the plurality of images to delete the recognition result of the image for which the time stamp is erroneous.

The traffic light switching point prediction module of the server can predict the switching information of the traffic light in a future specified period based on the switching information calculated by the traffic light switching point calculation module, and further send the switching information to the navigation client, so that a user can see the form of a countdown product, namely the countdown of the traffic light can be displayed by the navigation client. Moreover, the switching point calculation module and the switching point prediction module of the server can process the batch image recognition results to predict the switching information of the plurality of traffic signal lamps.

For a specific prediction method, reference may be made to the related description in the above embodiments, which is not repeated herein.

By adopting the embodiment of the disclosure, more than half map navigation users can meet traffic lights in the navigation process, and on average, each user can pass through the traffic lights more than 20 times per day, and map navigation users in the whole country can pass through the traffic lights more than 5 hundred million times per day. Therefore, in the embodiment of the disclosure, the countdown function is displayed through the navigation client, millions of users in a single day and tens of millions of Page Views (PV) can be touched, more comfortable user experience can be brought to the users, the occurrence of traffic accidents is reduced, and the overall public praise of the users on map navigation is improved.

In the technical scheme of the disclosure, the collection, storage, use, processing, transmission, provision, disclosure and other processing of the personal information of the related user are all in accordance with the regulations of related laws and regulations and do not violate the good customs of the public order.

Corresponding to the above method embodiment, an embodiment of the present disclosure further provides a device for predicting a switching time of a traffic signal, as shown in fig. 7, the device includes:

the sending module 701 is used for sending an acquisition task to an acquisition client in a vehicle which can pass through a target signal lamp within a specified time period aiming at each target signal lamp to be scheduled, wherein the acquisition task is used for indicating the acquisition client to acquire an image of the target signal lamp within the specified time period;

the determining module 702 is configured to determine switching information of the target signal lamp based on an acquisition result fed back by the acquisition client, where the switching information includes a switching time and a lamp state before switching;

the prediction module 703 is configured to predict, based on the switching information and the switching rule of the target signal lamp, switching information of each switching of the target signal lamp in a future specified period;

the sending module 701 is further configured to issue the predicted switching information to a navigation client in a vehicle that passes through the target signal lamp within a specified period, so that the navigation client displays countdown of the target signal lamp based on the received switching information.

Optionally, the target signal lamp is a countdown signal lamp or a non-countdown signal lamp;

the acquisition mode of the acquisition task instruction of the non-countdown signal lamp is as follows: continuously and repeatedly acquiring images including a non-countdown signal lamp according to a specified acquisition frequency;

Optionally, the collecting the first type of collection result fed back by the client for the non-countdown signal lamp includes: the method comprises the steps that a lamp state before switching, a lamp state after switching and switching time are determined by a collecting client based on collected images; the second type of acquisition results fed back by the client aiming at the countdown signal lamp comprise: the acquisition client determines a current light state, a countdown number and an acquisition timestamp based on the acquired image;

the determining module 702 is specifically configured to:

under the condition that a first-type acquisition result is received, taking the first-type acquisition result as switching information of a target signal lamp; or,

and under the condition of receiving the second type of acquisition result, taking the sum of the acquisition timestamp and the countdown number as the switching time of the target signal lamp, and taking the current lamp state as the lamp state before switching to obtain the switching information of the target signal lamp.

Optionally, the third type of acquisition result fed back by the acquisition client for the non-countdown signal lamp includes a plurality of first images, and the first images are images including the non-countdown signal lamp; the fourth type of acquisition result fed back by the acquisition client aiming at the countdown signal lamp comprises a second image, and the second image is an image comprising the countdown signal lamp;

the determining module 702 is specifically configured to:

under the condition that a third type of acquisition result is received, carrying out image recognition on the plurality of first images, and taking the image recognition to obtain a light state before switching, a light state after switching and switching time as switching information of a target signal lamp; or,

and under the condition that a fourth type of acquisition result is received, performing image recognition on the second image to obtain a current headlight state, a count-down number and an acquisition timestamp, taking the sum of the acquisition timestamp and the count-down number as the switching time of the target signal lamp, and taking the current headlight state and the switching time obtained by calculation as the switching information of the target signal lamp.

Optionally, the determining module 702 is specifically configured to:

acquiring an acquisition time stamp of each first image;

for each first image, identifying the first image by using a target detection model to obtain a target position of a non-countdown signal lamp in the first image, wherein the non-countdown signal lamp is included in the first image;

identifying the non-countdown signal lamp at the target position in the first image by using a lamp state identification model to obtain the lamp state of the non-countdown signal lamp;

determining two adjacent first images with lamp state switching based on the lamp states of the non-countdown signal lamps in the plurality of first images;

taking the smaller acquisition timestamp of the acquisition timestamps of the two adjacent first images as the switching moment, and taking the light state of the signal lamp included in the first image with the smaller acquisition timestamp of the two adjacent first images as the light state before switching;

the determining module 702 is further specifically configured to:

acquiring an acquisition time stamp of the second image;

identifying the second image by using a target detection model to obtain a target position of the non-countdown signal lamp in the second image;

recognizing the countdown signal lamp of the target position in the second image by using a lamp state recognition model to obtain a current lamp state;

recognizing a countdown signal lamp at the target position in the second image by using an Optical Character Recognition (OCR) model to obtain a countdown number;

the target detection model, the lamp state recognition model and the OCR model are all models obtained based on signal lamp image sample set training.

Optionally, the signal lamp image sample set includes an image of a high-heat signal lamp, an image of a signal lamp with countdown and an image of a signal lamp with a designated switching rule, and the high-heat signal lamp is a signal lamp of an intersection where the number of passing vehicles is greater than a preset number threshold in a preset time period.

Optionally, the apparatus further comprises:

the acquisition module is used for acquiring real-time track information returned by the plurality of navigation clients;

the determining module 702 is further configured to determine, based on the real-time trajectory information and a pre-collected switching rule of the plurality of traffic signal lamps, a plurality of target signal lamps to be scheduled and a specified time period corresponding to each target signal lamp.

The present disclosure also provides an electronic device, a readable storage medium, and a computer program product according to embodiments of the present disclosure.

FIG. 8 illustrates a schematic block diagram of an example electronic device 800 that can be used to implement embodiments of the present disclosure. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be examples only, and are not meant to limit implementations of the disclosure described and/or claimed herein.

As shown in fig. 8, the apparatus 800 includes a computing unit 801 that can perform various appropriate actions and processes according to a computer program stored in a Read Only Memory (ROM) 802 or a computer program loaded from a storage unit 808 into a Random Access Memory (RAM) 803. In the RAM 803, various programs and data required for the operation of the device 800 can also be stored. The calculation unit 801, the ROM 802, and the RAM 803 are connected to each other by a bus 804. An input/output (I/O) interface 805 is also connected to bus 804.

A number of components in the device 800 are connected to the I/O interface 805, including: an input unit 806, such as a keyboard, a mouse, or the like; an output unit 807 such as various types of displays, speakers, and the like; a storage unit 808, such as a magnetic disk, optical disk, or the like; and a communication unit 809 such as a network card, modem, wireless communication transceiver, etc. The communication unit 809 allows the device 800 to exchange information/data with other devices via a computer network such as the internet and/or various telecommunication networks.

Computing unit 801 may be a variety of general and/or special purpose processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), various dedicated Artificial Intelligence (AI) computing chips, various computing units running machine learning model algorithms, a Digital Signal Processor (DSP), and any suitable processor, controller, microcontroller, and the like. The calculation unit 801 executes the respective methods and processes described above, such as a prediction method of traffic signal switching timing. For example, in some embodiments, the method of predicting the time of a traffic light switch may be implemented as a computer software program tangibly embodied in a machine-readable medium, such as the storage unit 808. In some embodiments, part or all of a computer program may be loaded onto and/or installed onto device 800 via ROM 802 and/or communications unit 809. When the computer program is loaded into the RAM 803 and executed by the computing unit 801, one or more steps of the above-described method of predicting traffic signal switching timing may be performed. Alternatively, in other embodiments, the calculation unit 801 may be configured by any other suitable means (e.g., by means of firmware) to perform the prediction method of the traffic light switching instant.

Various implementations of the systems and techniques described here above may be implemented in digital electronic circuitry, integrated circuitry, field Programmable Gate Arrays (FPGAs), application Specific Integrated Circuits (ASICs), application Specific Standard Products (ASSPs), system on a chip (SOCs), complex Programmable Logic Devices (CPLDs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs that are executable and/or interpretable on a programmable system including at least one programmable processor, which may be special or general purpose, receiving data and instructions from, and transmitting data and instructions to, a storage system, at least one input device, and at least one output device.

Program code for implementing the methods of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the program codes, when executed by the processor or controller, cause the functions/operations specified in the flowchart and/or block diagram to be performed. The program code may execute entirely on the machine, partly on the machine, as a stand-alone software package partly on the machine and partly on a remote machine or entirely on the remote machine or server.

In the context of this disclosure, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which a user may provide input to the computer. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic, speech, or tactile input.

The systems and techniques described here can be implemented in a computing system that includes a back-end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back-end, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), wide Area Networks (WANs), and the Internet.

The computer system may include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server may be a cloud server, a server of a distributed system, or a server combining a blockchain.

It should be understood that various forms of the flows shown above may be used, with steps reordered, added, or deleted. For example, the steps described in the present disclosure may be executed in parallel or sequentially or in different orders, and are not limited herein as long as the desired results of the technical solutions disclosed in the present disclosure can be achieved.

The above detailed description should not be construed as limiting the scope of the disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may be made in accordance with design requirements and other factors. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present disclosure should be included in the scope of protection of the present disclosure.

Claims

1. A method for predicting the switching time of a traffic signal lamp comprises the following steps:

sending a collection task to a collection client in a vehicle capable of passing through a target signal lamp in a specified time period for each target signal lamp to be scheduled, wherein the collection task is used for indicating the collection client to collect an image of the target signal lamp in the specified time period;

2. The method of claim 1, wherein the target signal light is a countdown signal light or a non-countdown signal light;

the collection mode of the collection task indication of the non-countdown signal lamp is as follows: continuously and repeatedly acquiring images including a non-countdown signal lamp according to a specified acquisition frequency;

3. The method of claim 2, wherein collecting the first type of collection results fed back by the client for the non-countdown lights comprises: the acquisition client determines a pre-switching light state, a post-switching light state and a switching moment based on the acquired image; the second type of acquisition results fed back by the acquisition client aiming at the countdown signal lamp comprise: the acquisition client determines a current light state, a countdown number and an acquisition timestamp based on the acquired image;

the determining the switching information of the target signal lamp based on the collection result fed back by the collection client comprises the following steps:

under the condition that the first type of acquisition result is received, taking the first type of acquisition result as switching information of the target signal lamp; or,

and under the condition that the second type of acquisition result is received, taking the sum of the acquisition timestamp and the countdown digit as the switching time of the target signal lamp, and taking the current lamp state as the lamp state before switching to obtain the switching information of the target signal lamp.

4. The method of claim 2, wherein the third type of acquisition results fed back by the acquisition client for the non-countdown signal lamp comprises a plurality of first images, and the first images are images including the non-countdown signal lamp; the acquisition client side comprises a first image and a second image, wherein the first image comprises a countdown signal lamp;

under the condition that the third type of acquisition result is received, carrying out image recognition on the plurality of first images, and taking the image recognition to obtain the lamp state before switching, the lamp state after switching and the switching time as the switching information of the target signal lamp; or,

under the condition that the fourth type of acquisition result is received, image recognition is carried out on the second image to obtain a current light state, a countdown number and an acquisition time stamp, the sum of the acquisition time stamp and the countdown number is used as the switching time of the target signal lamp, and the current light state and the switching time obtained through calculation are used as the switching information of the target signal lamp.

5. The method of claim 4, wherein the image recognizing the plurality of first images comprises:

acquiring an acquisition time stamp of each first image;

determining two adjacent first images with lamp state switching based on the lamp states of the non-countdown signal lamps in the first images;

taking the smaller acquisition timestamp of the acquisition timestamps of the two adjacent first images as a switching moment, and taking the light state of a signal lamp included in the first image with the smaller acquisition timestamp of the two adjacent first images as the light state before switching;

performing image recognition on the second image, including:

acquiring an acquisition time stamp of the second image;

identifying the second image by using a target detection model to obtain a target position of a non-countdown signal lamp in the second image;

recognizing the countdown signal lamp of the target position in the second image by using a lamp state recognition model to obtain the current lamp state;

recognizing the countdown signal lamp of the target position in the second image by using an Optical Character Recognition (OCR) model to obtain the countdown number;

the target detection model, the light state recognition model and the OCR model are all models obtained based on signal lamp image sample set training.

6. The method of claim 5, wherein the sample set of signal light images includes an image of a high heat signal light, a signal light with a countdown, and a signal light with a specified switching law, the high heat signal light being a signal light of an intersection where a number of vehicles passing through is greater than a preset number threshold for a preset period of time.

7. The method of claim 1, prior to the sending, for each target signal light that needs to be scheduled, an acquisition task to an acquisition client in a vehicle that can pass the target signal light for a specified period of time, the method further comprising:

acquiring real-time track information returned by a plurality of navigation clients;

and determining a plurality of target signal lamps needing to be scheduled and a designated time period corresponding to each target signal lamp based on the real-time track information and the pre-collected switching rules of the plurality of traffic signal lamps.

8. A device for predicting the switching time of a traffic signal lamp comprises:

9. The apparatus of claim 8, wherein the target signal light is a countdown signal light or a non-countdown signal light;

the collection mode of the collection task instruction of the non-countdown signal lamp is as follows: continuously and repeatedly acquiring images including a non-countdown signal lamp according to a specified acquisition frequency;

10. The apparatus of claim 9, wherein the collecting of the first type of collection results of the client for the non-countdown signal light feedback comprises: the acquisition client determines a pre-switching light state, a post-switching light state and a switching moment based on the acquired image; the second type of acquisition results fed back by the acquisition client aiming at the countdown signal lamp comprise: the acquisition client determines a current light state, a countdown number and an acquisition timestamp based on the acquired image;

the determining module is specifically configured to:

and under the condition that the second type of acquisition result is received, taking the sum of the acquisition timestamp and the countdown number as the switching time of the target signal lamp, and taking the current lamp state as the lamp state before switching to obtain the switching information of the target signal lamp.

11. The apparatus according to claim 9, wherein the third type of acquisition result fed back by the acquisition client for the non-countdown signal lamp includes a plurality of first images, and the first images are images including the non-countdown signal lamp; the acquisition client side comprises a first image and a second image, wherein the first image comprises a countdown signal lamp;

the determining module is specifically configured to:

under the condition that the third type of acquisition results are received, performing image recognition on the plurality of first images, and taking the image recognition to obtain a light state before switching, a light state after switching and a switching time as switching information of the target signal lamp; or,

12. The apparatus of claim 11, wherein the determining module is specifically configured to:

acquiring an acquisition time stamp of each first image;

the determining module is specifically further configured to:

acquiring an acquisition time stamp of the second image;

recognizing the countdown signal lamp of the target position in the second image by using an Optical Character Recognition (OCR) model to obtain a countdown number;

13. The apparatus of claim 12, wherein the sample set of signal light images includes an image of a high heat signal light, a signal light with a countdown, and a signal light with a specified switching law, the high heat signal light being a signal light of an intersection where a number of vehicles passing through is greater than a preset number threshold for a preset period of time.

14. The apparatus of claim 8, the apparatus further comprising:

the determining module is further configured to determine a plurality of target signal lamps to be scheduled and a specified time period corresponding to each target signal lamp based on the real-time trajectory information and a pre-collected switching rule of the plurality of traffic signal lamps.

15. An electronic device, comprising:

at least one processor; and

a memory communicatively coupled to the at least one processor; wherein,

the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-7.

16. A non-transitory computer readable storage medium having stored thereon computer instructions for causing the computer to perform the method of any one of claims 1-7.

17. A computer program product comprising a computer program which, when executed by a processor, implements the method according to any one of claims 1-7.