CN117475389B - Pedestrian crossing signal lamp control method, system, equipment and storage medium - Google Patents
Pedestrian crossing signal lamp control method, system, equipment and storage medium Download PDFInfo
- Publication number
- CN117475389B CN117475389B CN202311809930.7A CN202311809930A CN117475389B CN 117475389 B CN117475389 B CN 117475389B CN 202311809930 A CN202311809930 A CN 202311809930A CN 117475389 B CN117475389 B CN 117475389B
- Authority
- CN
- China
- Prior art keywords
- detection frame
- interest
- region
- feature
- pedestrian
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 29
- 238000003860 storage Methods 0.000 title claims abstract description 8
- 238000001514 detection method Methods 0.000 claims description 153
- 238000004220 aggregation Methods 0.000 claims description 61
- 230000002776 aggregation Effects 0.000 claims description 61
- 241000283070 Equus zebra Species 0.000 claims description 48
- 230000004927 fusion Effects 0.000 claims description 39
- 238000000605 extraction Methods 0.000 claims description 27
- 238000010586 diagram Methods 0.000 claims description 19
- 238000007499 fusion processing Methods 0.000 claims description 13
- 238000011176 pooling Methods 0.000 claims description 13
- 238000004590 computer program Methods 0.000 claims description 9
- 238000005457 optimization Methods 0.000 claims description 8
- 238000010606 normalization Methods 0.000 claims description 6
- 238000004364 calculation method Methods 0.000 description 3
- 230000004913 activation Effects 0.000 description 2
- 238000006243 chemical reaction Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000009826 distribution Methods 0.000 description 1
- 238000005096 rolling process Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects
- G06V20/54—Surveillance or monitoring of activities, e.g. for recognising suspicious objects of traffic, e.g. cars on the road, trains or boats
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/80—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level
- G06V10/806—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level of extracted features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
- G06V20/58—Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
- G06V20/584—Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads of vehicle lights or traffic lights
-
- G—PHYSICS
- G08—SIGNALLING
- G08G—TRAFFIC CONTROL SYSTEMS
- G08G1/00—Traffic control systems for road vehicles
- G08G1/005—Traffic control systems for road vehicles including pedestrian guidance indicator
-
- G—PHYSICS
- G08—SIGNALLING
- G08G—TRAFFIC CONTROL SYSTEMS
- G08G1/00—Traffic control systems for road vehicles
- G08G1/01—Detecting movement of traffic to be counted or controlled
-
- G—PHYSICS
- G08—SIGNALLING
- G08G—TRAFFIC CONTROL SYSTEMS
- G08G1/00—Traffic control systems for road vehicles
- G08G1/07—Controlling traffic signals
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02B—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO BUILDINGS, e.g. HOUSING, HOUSE APPLIANCES OR RELATED END-USER APPLICATIONS
- Y02B20/00—Energy efficient lighting technologies, e.g. halogen lamps or gas discharge lamps
- Y02B20/40—Control techniques providing energy savings, e.g. smart controller or presence detection
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Multimedia (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Artificial Intelligence (AREA)
- Health & Medical Sciences (AREA)
- Computing Systems (AREA)
- Databases & Information Systems (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- Software Systems (AREA)
- Image Analysis (AREA)
Abstract
The invention relates to the technical field of image data processing, in particular to a control method, a system, equipment and a storage medium of a crosswalk signal lamp.
Description
Technical Field
The invention relates to the technical field of image data processing, in particular to a control method, a system, equipment and a storage medium of a crosswalk signal lamp.
Background
In the period of the morning and evening peak, more pedestrians and vehicles come and go, if the pedestrian crossing signal lamp is unscientific in display, pedestrian crossing crowding is easily caused, pedestrian collision events occur, subsequent vehicle passing can be influenced, and traffic jam is aggravated.
The existing crosswalk signal lamp mainly controls the passing time by controlling the fixed time length of the signal lamp. However, when the number of pedestrians is large, the switching time of the signal lamps is relatively short, so that the traffic channel is easy to be blocked; when the number of pedestrians is small, the switching time of the signal lamps is relatively long, the phenomenon that pedestrians break the red light is easy to occur, and traffic safety accidents are easy to occur.
Disclosure of Invention
The invention aims to provide a pedestrian crosswalk signal lamp control method, a pedestrian recognition result-based pedestrian crosswalk signal lamp control system, pedestrian recognition result-based pedestrian crosswalk signal lamp control equipment and a storage medium.
The technical scheme of the invention is as follows:
a control method of a crosswalk signal lamp comprises the following operations:
s1, acquiring a crosswalk image, wherein the crosswalk image is subjected to convolution processing to obtain a first feature map; the crosswalk image sequentially passes through first semantic feature extraction and second semantic feature extraction to obtain a second feature map; the first feature map and the second feature map are subjected to feature fusion processing to obtain a fusion feature map;
s2, the fusion feature map is subjected to detection frame prediction processing and classification processing to obtain a detection frame set of the region of interest;
acquiring a corresponding prediction frame of which the intersection ratio with a real frame is larger than a first threshold value in the same region of interest in the region of interest detection frame set to obtain a region of interest optimization detection frame set;
acquiring a detection frame corresponding to the maximum value of the classification probability in the same region of interest in the optimized detection frame set of the region of interest to obtain a classification frame set of the region of interest;
s3, acquiring a classification frame set of the region of interest, wherein the classification result is a pedestrian detection frame, and acquiring a pedestrian detection frame set;
the pedestrian detection frame set is subjected to position feature matching processing to obtain the number of people travelling in the zebra stripes and the number of people travelling in the waiting areas;
and S4, obtaining a crosswalk signal lamp result based on the zebra crossing regional pedestrian number and the waiting regional pedestrian number.
The operation of extracting the first semantic features in the step S1 specifically comprises the following steps: the crosswalk image is subjected to feature convolution and local feature aggregation in sequence to obtain a local aggregation feature image which is used for executing the second semantic feature extraction operation; the local feature aggregation operation specifically comprises the following steps: the convolution input subjected to characteristic convolution processing is subjected to characteristic convolution processing for a plurality of times to obtain a convolution aggregation diagram; and the convolution aggregation graph and the convolution input are subjected to feature fusion processing to obtain the local aggregation feature graph.
The second semantic feature extraction operation specifically comprises the following steps: the local aggregation feature map is subjected to convolution treatment to obtain an optimized local aggregation map; the optimized local aggregation diagram is subjected to characteristic calibration treatment to obtain a calibration aggregation characteristic diagram; and performing feature fusion processing on the calibration aggregation feature map and the optimized local aggregation map to obtain the second feature map.
The operation of the characteristic calibration process is specifically as follows: the optimized local aggregation map is subjected to average pooling treatment to obtain a pooled local aggregation map; the optimized local aggregation map sequentially undergoes average pooling treatment and standard pooling treatment to obtain a standard local aggregation map; the pooled local aggregation map and the standard local aggregation map are subjected to feature fusion to obtain an initial calibration aggregation map; and the initial calibration aggregation diagram is subjected to linear processing, batch normalization processing and nonlinear processing in sequence to obtain the calibration aggregation characteristic diagram.
The operation of obtaining the detection frame set of the region of interest in the step S3 specifically comprises the following steps: the fusion feature map is subjected to bounding box regression processing to obtain a detection frame set formed by a plurality of detection frames in each region of interest; after the characteristics of each detection frame are extracted, the detection frames are matched with a standard database, and the obtained maximum matching degree and the obtained corresponding category are respectively used as a classification probability value and a classification result of the corresponding detection frame; each detection frame, and the corresponding classification probability value and classification result, form the region of interest detection frame set.
The operation of the position feature matching processing in the S3 is specifically as follows: the pedestrian detection frames are concentrated, and the bottom center position point of each pedestrian detection frame is matched with the zebra crossing area position range and the waiting area position range respectively; if the bottom center position point of the pedestrian detection frame is in the position range of the zebra crossing area, the corresponding pedestrian detection frame is the zebra crossing area pedestrian detection frame, and the number of the pedestrian detection frames in all the zebra crossing areas is counted to obtain the number of people travelling in the zebra crossing areas; if the bottom center position point of the pedestrian detection frame is in the waiting area position range, counting the number of the pedestrian detection frames in all the waiting areas to obtain the number of the pedestrians in the waiting areas, wherein the corresponding pedestrian detection frame is the pedestrian detection frame in the waiting area.
The operation of S4 specifically includes: when the number of pedestrians in the zebra crossing area is not 0, the pedestrian crosswalk signal lamp result is passing; when the number of pedestrians in the zebra crossing area is 0, the number of pedestrians in the waiting area exceeds a second threshold, or the waiting time corresponding to the number of pedestrians in the waiting area exceeds a third threshold, and the pedestrian crossing signal lamp result is passing.
A crosswalk signal lamp control system comprising:
the fusion feature map generation module is used for acquiring a crosswalk image, and the crosswalk image is subjected to convolution processing to obtain a first feature map; the crosswalk image sequentially passes through first semantic feature extraction and second semantic feature extraction to obtain a second feature map; the first feature map and the second feature map are subjected to feature fusion processing to obtain a fusion feature map;
the region-of-interest classification frame set generation module is used for obtaining a region-of-interest detection frame set through detection frame prediction processing and classification processing of the fusion feature map; acquiring a corresponding prediction frame of which the intersection ratio with a real frame is larger than a first threshold value in the same region of interest in the region of interest detection frame set to obtain a region of interest optimization detection frame set; acquiring a detection frame corresponding to the maximum value of the classification probability in the same region of interest in the optimized detection frame set of the region of interest to obtain a classification frame set of the region of interest;
the zebra crossing region pedestrian number and waiting region pedestrian number generation module is used for acquiring the classification frame set of the region of interest, wherein the classification result is a pedestrian detection frame, and a pedestrian detection frame set is obtained; the pedestrian detection frame set is subjected to position feature matching processing to obtain the number of people travelling in the zebra stripes and the number of people travelling in the waiting areas;
and the crosswalk signal lamp result generation module is used for obtaining crosswalk signal lamp results based on the number of people travelling in the zebra crossing area and the number of people travelling in the waiting area.
The control equipment of the crosswalk signal lamp comprises a processor and a memory, wherein the control method of the crosswalk signal lamp is realized when the processor executes a computer program stored in the memory.
A computer readable storage medium storing a computer program, wherein the computer program when executed by a processor implements the crosswalk signal lamp control method described above.
The invention has the beneficial effects that:
according to the control method of the pedestrian crossing signal lamp, semantic feature extraction and feature fusion are carried out on the pedestrian crossing images, the obtained fusion feature images are predicted and classified through the detection frames, the detection frames are screened according to the intersection ratio and the classification probability, the detection frames with high reliability are reserved for counting the number of people in the zebra stripes and the number of people in the waiting areas, finally the display condition of the pedestrian crossing signal lamp is flexibly controlled according to the number of people in the zebra stripes and the number of people in the waiting areas, pedestrians are reasonably arranged to cross roads, and traffic safety accidents are avoided.
Detailed Description
The embodiment provides a control method of a crosswalk signal lamp, which comprises the following operations:
s1, acquiring a crosswalk image, wherein the crosswalk image is subjected to convolution processing to obtain a first feature map; the crosswalk image sequentially passes through first semantic feature extraction and second semantic feature extraction to obtain a second feature map; the first feature map and the second feature map are subjected to feature fusion processing to obtain a fusion feature map;
s2, the fusion feature map is subjected to detection frame prediction processing and classification processing to obtain a detection frame set of the region of interest; acquiring a corresponding prediction frame of which the intersection ratio with a real frame is larger than a first threshold value in the same region of interest in the region of interest detection frame set to obtain a region of interest optimization detection frame set; acquiring a detection frame corresponding to the maximum value of the classification probability in the same region of interest in the optimized detection frame set of the region of interest to obtain a classification frame set of the region of interest;
s3, acquiring a classification frame set of the region of interest, wherein the classification result is a pedestrian detection frame, and acquiring a pedestrian detection frame set; the pedestrian detection frame set is subjected to position feature matching processing to obtain the number of people travelling in the zebra stripes and the number of people travelling in the waiting areas;
and S4, obtaining a crosswalk signal lamp result based on the zebra crossing regional pedestrian number and the waiting regional pedestrian number.
S1, acquiring a crosswalk image, and performing convolution processing on the crosswalk image to obtain a first feature map; the crosswalk image sequentially passes through the first semantic feature extraction and the second semantic feature extraction to obtain a second feature map; and carrying out feature fusion processing on the first feature map and the second feature map to obtain a fusion feature map.
And acquiring a crosswalk image. And connecting a camera and an industrial personal computer which are opposite to the crosswalk by using a network cable, and drawing out an RTSP video stream, and then carrying out frame drawing by using an opencv-python library to obtain a crosswalk image to be detected.
The first semantic feature extraction operation is as follows: the crosswalk image is subjected to feature convolution and local feature aggregation in sequence, and the obtained local aggregation feature image is used for executing the second semantic feature extraction operation; the local feature aggregation operation specifically comprises the following steps: the convolution input subjected to characteristic convolution processing is subjected to characteristic convolution processing for a plurality of times to obtain a convolution aggregation diagram; and carrying out feature fusion processing on the convolution aggregation graph and the convolution input to obtain a local aggregation feature graph. The feature convolution includes: 3 x 3 normal convolution, batch normalization, and linear processing (which can be implemented by a leak relu activation function).
The second semantic feature extraction operation specifically comprises the following steps: the local aggregation feature map is subjected to convolution treatment to obtain an optimized local aggregation map; optimizing the local aggregation map, and performing feature calibration treatment to obtain a calibration aggregation feature map; and performing feature fusion processing on the calibration aggregation feature map and the optimized local aggregation map to obtain a second feature map.
The operation of the feature calibration process is: carrying out average pooling treatment on the optimized local aggregation map to obtain a pooled local aggregation map; the optimized local aggregation map is subjected to average pooling treatment and standard pooling treatment in sequence to obtain a standard local aggregation map; the pooling local aggregation diagram and the standard local aggregation diagram are subjected to feature fusion to obtain an initial calibration aggregation diagram; the initial calibration aggregate graph is subjected to linear processing, batch normalization processing and nonlinear processing (which can be realized through a sigmoid activation function) in sequence, so that a calibration aggregate feature graph is obtained.
The operation of the average pooling process can be realized by the following formula:
，
μ nc in order to pool the local aggregate map,x nc in order to optimize the local aggregate map,Hin order to optimize the high of the local aggregate map,Wto optimize the breadth of the local aggregate map.
The operation of the standard pooling process may be achieved by the following formula:
，
σ nc in the case of a standard local aggregate map,x nc in order to optimize the local aggregate map,μ nc in order to pool the local aggregate map,Hin order to optimize the high of the local aggregate map,Wto optimize the breadth of the local aggregate map.
In order to improve the space semantic expression capability of the fusion feature map and improve the accuracy of a detection result, the fusion feature map is subjected to multi-space scale feature processing to obtain an optimized fusion feature map; the fusion profile is optimized for performing the operation in S2.
The operation of the multi-spatial scale feature processing is as follows: the fusion feature map is subjected to image multi-space conversion treatment to obtain a plurality of space feature maps with different space sizes; all the space feature images are respectively subjected to multi-channel feature extraction processing, and all the obtained channel feature images are subjected to weighting processing to obtain an optimized fusion feature image.
The operation of the multichannel characteristic extraction processing is as follows: carrying out image multichannel conversion treatment on the space feature map to obtain a plurality of initial channel feature maps with different channel sizes; all the initial channel feature images are subjected to convolution processing, normalization processing and nonlinear processing in sequence, and all the obtained optimized channel feature images are multiplied by the respective corresponding learning parameters and then subjected to summation processing to obtain the optimized fusion feature images. Taking a single initial channel characteristic diagram as an example, the initial channel characteristic diagram is subjected to convolution processing, normalization processing and nonlinear processing in sequence to obtain optimized channel characteristic diagram characteristics, and the like, all the obtained optimized channel characteristic diagrams are multiplied by respective corresponding learning parameters, and then summation processing is performed to obtain an optimized fusion characteristic diagram.
S2, carrying out prediction processing and classification processing on the fusion feature map or the optimized fusion feature map through a detection frame to obtain a detection frame set of the region of interest; acquiring a corresponding prediction frame with the intersection ratio with a real frame being larger than a first threshold value in the same region of interest in a region of interest detection frame set to obtain a region of interest optimization detection frame set; and acquiring a detection frame corresponding to the maximum value of the classification probability in the same region of interest in the optimized detection frame set of the region of interest to obtain a classification frame set of the region of interest.
The operation of obtaining the detection frame set of the region of interest is as follows: carrying out bounding box regression processing on the fusion feature map or the optimized fusion feature map to obtain a detection frame set formed by a plurality of detection frames in each region of interest; in the detection frame set, each detection frame is matched with a standard database after feature extraction, and the obtained maximum matching degree and the corresponding category are respectively used as a classification probability value and a classification result of the corresponding detection frame; each detection frame, and the corresponding classification probability value and classification result, form a region of interest detection frame set.
Specifically, according to the pixel distribution characteristics of the fusion feature map, performing bounding box regression processing (the bounding box regression processing is in the prior art, so as to save space and avoid excessive description), so that a plurality of detection boxes are distributed in the same interested area on the fusion feature map; then, in the same interested area, each detection frame is subjected to rolling and pooling processing to realize the feature extraction, semantic feature matching is carried out on the feature semantic information in the image after feature extraction and the standard classified image in the standard database, the category of the corresponding standard classified image with the matching degree as the maximum value is output and is used as the classification result of the detection frame, and the maximum value of the matching degree is used as the classification probability value of the corresponding detection frame, namely the similarity between the classification result and the real result, so that the classification probability value and the classification result of the detection frame are obtained; all the detection frames, and the classification probability values and classification results corresponding to the detection frames respectively form a detection frame set of the region of interest.
In order to further reduce the number of detection frames and improve the calculation efficiency, the above-mentioned detection frames of the region of interest are concentrated, corresponding prediction frames with the intersection ratio with the real frames being larger than a first threshold value in the same region of interest are reserved as optimized detection frames closer to the real result, and other detection frames are deleted, so that the obtained optimized detection frame set of the region of interest formed by all the optimized detection frames of different regions of interest is obtained. Wherein, the calculation formula of the cross ratio is: cross ratio (Intersection over Union, IOU) =cross area of detection frame and real frame/cross area of detection frame and real frame.
And finally, extracting a detection frame corresponding to the maximum value of the classification probability in the same region of interest in the region of interest optimization detection frame set, and obtaining the region of interest classification frame set as a final classification result.
In order to improve accuracy of the region-of-interest classification frame set, when feature loss entropy of the region-of-interest detection frame set and the real frame set is smaller than a loss threshold value, outputting a current region-of-interest detection frame as a detection frame.
The feature loss entropy is obtained by summing the classification loss entropy, the confidence loss entropy and the cross-over ratio loss entropy.
The class loss entropy can be obtained by the following formula:
，
L cla in order to classify the loss of entropy,Nin order to predict the total number of frames,y i is the firstiThe classified true values of the individual prediction frames,p(y i ) Is the firstiClassification predictors for the respective prediction frames.
Confidence loss entropy can be obtained by the following formula:
，
L con for the loss of entropy of the confidence,Nin order to predict the total number of frames,x i is the firstiThe true similarity of the individual predicted frames to the true frames,p(y i ) Is the firstiPrediction similarity of each prediction frame to the real frame.
The cross-ratio loss entropy can be achieved by the following formula:
，
L CIoU for the cross-over ratio to lose entropy,IoUto predict the degree of overlap of the frame with the real frame,ρto predict the center point distance of the frame from the real frame,cin order to predict the diagonal length of the frame,vfor prediction of frame to real frame aspect ratio similarity, a first correlation coefficient.
Wherein,vthe calculation formula of (2) is as follows:
，
w A in order to predict the width of the frame,h A in order to predict the height of the frame,w B for the width of the real frame,h B is the high of the real frame and,，the aspect ratios of the prediction and real frames, respectively.
S3, acquiring a classification frame set of the region of interest, wherein the classification result is a pedestrian detection frame, and acquiring a pedestrian detection frame set; and the pedestrian detection frame set is subjected to position feature matching processing to obtain the number of people travelling in the zebra crossing area and the number of people travelling in the waiting area.
The operation of the position feature matching process specifically includes: the pedestrian detection frames are concentrated, and the bottom center position point of each pedestrian detection frame is matched with the zebra crossing area position range and the waiting area position range respectively; if the bottom center position point of the pedestrian detection frame is in the position range of the zebra crossing area, counting the number of the pedestrian detection frames in all the zebra crossing areas to obtain the number of people traveling in the zebra crossing areas, wherein the corresponding pedestrian detection frame is the zebra crossing area pedestrian detection frame; if the bottom center position point of the pedestrian detection frame is in the waiting area position range, counting the number of the pedestrian detection frames in all the waiting areas to obtain the number of pedestrians in the waiting areas, wherein the corresponding pedestrian detection frame is the pedestrian detection frame in the waiting area.
S4, obtaining crosswalk signal lamp results based on the number of people travelling in the zebra stripes and the number of people travelling in the waiting areas.
When the number of pedestrians in the zebra crossing area is not 0, the pedestrian crosswalk signal lamp result is passing; when the number of pedestrians in the zebra crossing area is 0, the number of pedestrians in the waiting area exceeds a second threshold (which can be set to 5), or the waiting time corresponding to the number of pedestrians in the waiting area exceeds a third threshold (which can be set to 60 s), and the traffic light result of the pedestrian crossing is traffic.
Specifically, when people exist in the zebra crossing area, a pedestrian crosswalk signal lamp lights a green light to indicate that pedestrians pass; when no person exists in the zebra crossing area, but the number of pedestrians in the waiting area exceeds 5, the pedestrian crosswalk signal lamp lights a green light to indicate the pedestrian to pass; or the number of pedestrians in the waiting area is not more than 5, but the waiting time of the pedestrians is more than 60s, and the pedestrian crosswalk signal lamp lights a green light to indicate the pedestrians to pass. The waiting time of the pedestrian can be obtained by making a difference between the time when the target pedestrian first enters the waiting area and the time when the current crosswalk image.
In this embodiment, the industrial personal computer is used to transmit the number of pedestrians in the zebra crossing area and the number of pedestrians in the waiting area to the signal lamp system in real time through the serial port, and the serial port of the single chip microcomputer can judge whether to switch the signal lamp after receiving the data. In order to avoid data interference, header data and tail are sent, for example, sent data such as 'aabb 0100 ccdd', where 'aabb' is the frame header 'ccdd' is the frame tail, and the number of pedestrians in the zebra crossing area and the number of pedestrians in the waiting and waiting areas respectively correspond to two 16-system digits of 01 and 00, so that the single chip microcomputer can be ensured to accept signals stably and reliably.
The embodiment also provides a control system of a crosswalk signal lamp, which comprises:
the fusion feature map generation module is used for acquiring a crosswalk image, and the crosswalk image is subjected to convolution processing to obtain a first feature map; the crosswalk image sequentially passes through the first semantic feature extraction and the second semantic feature extraction to obtain a second feature map; the first feature map and the second feature map are subjected to feature fusion processing to obtain a fusion feature map;
the region-of-interest classification frame set generation module is used for fusing the feature images, and obtaining a region-of-interest detection frame set through detection frame prediction processing and classification processing; acquiring a corresponding prediction frame with the intersection ratio with a real frame being larger than a first threshold value in the same region of interest in a region of interest detection frame set to obtain a region of interest optimization detection frame set; acquiring a detection frame corresponding to the maximum value of the classification probability in the same region of interest in the optimized detection frame set of the region of interest to obtain a classification frame set of the region of interest;
the zebra crossing region pedestrian number and waiting region pedestrian number generation module is used for acquiring a classification frame set of the region of interest, wherein the classification result is a detection frame of pedestrians, and a pedestrian detection frame set is obtained; the pedestrian detection frame set is subjected to position feature matching processing to obtain the number of people travelling in the zebra crossing area and the number of people travelling in the waiting area;
and the crosswalk signal lamp result generation module is used for obtaining crosswalk signal lamp results based on the number of people travelling in the zebra crossing area and the number of people travelling in the waiting area.
The embodiment also provides a control device of the crosswalk signal lamp, which comprises a processor and a memory, wherein the control method of the crosswalk signal lamp is realized when the processor executes a computer program stored in the memory.
The embodiment also provides a computer readable storage medium for storing a computer program, wherein the computer program is executed by a processor to implement the method for controlling the crosswalk signal lamp.
According to the pedestrian crossing signal lamp control method, semantic feature extraction and feature fusion are carried out on the pedestrian crossing images, the obtained fusion feature images are predicted and classified through the detection frames, the detection frames are screened according to the intersection ratio and the classification probability, the detection frames with high reliability are reserved for counting the number of people in the zebra crossing area and the number of people in the waiting area, finally the display condition of the pedestrian crossing signal lamp is flexibly controlled according to the number of people in the zebra crossing area and the number of people in the waiting area, pedestrians are reasonably arranged to cross roads, and traffic safety accidents are avoided.
Claims (8)
1. The control method of the crosswalk signal lamp is characterized by comprising the following operations:
s1, acquiring a crosswalk image, wherein the crosswalk image is subjected to convolution processing to obtain a first feature map; the crosswalk image sequentially passes through first semantic feature extraction and second semantic feature extraction to obtain a second feature map; the first feature map and the second feature map are subjected to feature fusion processing to obtain a fusion feature map;
s2, the fusion feature map is subjected to detection frame prediction processing and classification processing to obtain a detection frame set of the region of interest; the operation of obtaining the detection frame set of the region of interest specifically comprises the following steps: the fusion feature map is subjected to bounding box regression processing to obtain a detection frame set formed by a plurality of detection frames in each region of interest; after the characteristics of each detection frame are extracted, the detection frames are matched with a standard database, and the obtained maximum matching degree and the obtained corresponding category are respectively used as a classification probability value and a classification result of the corresponding detection frame; each detection frame, and the corresponding classification probability value and classification result form the detection frame set of the region of interest;
acquiring a corresponding prediction frame of which the intersection ratio with a real frame is larger than a first threshold value in the same region of interest in the region of interest detection frame set to obtain a region of interest optimization detection frame set;
acquiring a detection frame corresponding to the maximum value of the classification probability in the same region of interest in the optimized detection frame set of the region of interest to obtain a classification frame set of the region of interest;
s3, acquiring a classification frame set of the region of interest, wherein the classification result is a pedestrian detection frame, and acquiring a pedestrian detection frame set;
the pedestrian detection frame set is subjected to position feature matching processing to obtain the number of people travelling in the zebra stripes and the number of people travelling in the waiting areas;
s4, obtaining a crosswalk signal lamp result based on the zebra crossing regional pedestrian number and the waiting regional pedestrian number; the method comprises the following steps: when the number of pedestrians in the zebra crossing area is not 0, the pedestrian crosswalk signal lamp result is passing; when the number of pedestrians in the zebra crossing area is 0, the number of pedestrians in the waiting area exceeds a second threshold, or the waiting time corresponding to the number of pedestrians in the waiting area exceeds a third threshold, and the pedestrian crossing signal lamp result is passing.
2. The method for controlling a crosswalk signal lamp according to claim 1, wherein the operation of extracting the first semantic feature in S1 specifically comprises:
the crosswalk image is subjected to feature convolution and local feature aggregation in sequence to obtain a local aggregation feature image which is used for executing the second semantic feature extraction operation;
the local feature aggregation operation specifically comprises the following steps: the convolution input subjected to characteristic convolution processing is subjected to characteristic convolution processing for a plurality of times to obtain a convolution aggregation diagram; and the convolution aggregation graph and the convolution input are subjected to feature fusion processing to obtain the local aggregation feature graph.
3. The method for controlling a crosswalk signal lamp according to claim 2, wherein the operation of extracting the second semantic features is specifically:
the local aggregation feature map is subjected to convolution treatment to obtain an optimized local aggregation map;
the optimized local aggregation diagram is subjected to characteristic calibration treatment to obtain a calibration aggregation characteristic diagram;
and performing feature fusion processing on the calibration aggregation feature map and the optimized local aggregation map to obtain the second feature map.
4. The method for controlling a crosswalk signal lamp according to claim 3, wherein the characteristic calibration process comprises the following steps:
the optimized local aggregation map is subjected to average pooling treatment to obtain a pooled local aggregation map;
the optimized local aggregation map sequentially undergoes average pooling treatment and standard pooling treatment to obtain a standard local aggregation map;
the pooled local aggregation map and the standard local aggregation map are subjected to feature fusion to obtain an initial calibration aggregation map;
and the initial calibration aggregation diagram is subjected to linear processing, batch normalization processing and nonlinear processing in sequence to obtain the calibration aggregation characteristic diagram.
5. The method for controlling a crosswalk signal lamp according to claim 1, wherein the operation of the positional characteristic matching process in S3 is specifically:
the pedestrian detection frames are concentrated, and the bottom center position point of each pedestrian detection frame is matched with the zebra crossing area position range and the waiting area position range respectively;
if the bottom center position point of the pedestrian detection frame is in the position range of the zebra crossing area, the corresponding pedestrian detection frame is the zebra crossing area pedestrian detection frame, and the number of the pedestrian detection frames in all the zebra crossing areas is counted to obtain the number of people travelling in the zebra crossing areas;
if the bottom center position point of the pedestrian detection frame is in the waiting area position range, counting the number of the pedestrian detection frames in all the waiting areas to obtain the number of the pedestrians in the waiting areas, wherein the corresponding pedestrian detection frame is the pedestrian detection frame in the waiting area.
6. A crosswalk signal lamp control system, comprising:
the fusion feature map generation module is used for acquiring a crosswalk image, and the crosswalk image is subjected to convolution processing to obtain a first feature map; the crosswalk image sequentially passes through first semantic feature extraction and second semantic feature extraction to obtain a second feature map; the first feature map and the second feature map are subjected to feature fusion processing to obtain a fusion feature map;
the region-of-interest classification frame set generation module is used for obtaining a region-of-interest detection frame set through detection frame prediction processing and classification processing of the fusion feature map; the operation of obtaining the detection frame set of the region of interest specifically comprises the following steps: the fusion feature map is subjected to bounding box regression processing to obtain a detection frame set formed by a plurality of detection frames in each region of interest; after the characteristics of each detection frame are extracted, the detection frames are matched with a standard database, and the obtained maximum matching degree and the obtained corresponding category are respectively used as a classification probability value and a classification result of the corresponding detection frame; each detection frame, and the corresponding classification probability value and classification result form the detection frame set of the region of interest; acquiring a corresponding prediction frame of which the intersection ratio with a real frame is larger than a first threshold value in the same region of interest in the region of interest detection frame set to obtain a region of interest optimization detection frame set; acquiring a detection frame corresponding to the maximum value of the classification probability in the same region of interest in the optimized detection frame set of the region of interest to obtain a classification frame set of the region of interest;
the zebra crossing region pedestrian number and waiting region pedestrian number generation module is used for acquiring the classification frame set of the region of interest, wherein the classification result is a pedestrian detection frame, and a pedestrian detection frame set is obtained; the pedestrian detection frame set is subjected to position feature matching processing to obtain the number of people travelling in the zebra stripes and the number of people travelling in the waiting areas;
the crosswalk signal lamp result generation module is used for obtaining crosswalk signal lamp results based on the number of people travelling in the zebra crossing area and the number of people travelling in the waiting area; the method comprises the following steps: when the number of pedestrians in the zebra crossing area is not 0, the pedestrian crosswalk signal lamp result is passing; when the number of pedestrians in the zebra crossing area is 0, the number of pedestrians in the waiting area exceeds a second threshold, or the waiting time corresponding to the number of pedestrians in the waiting area exceeds a third threshold, and the pedestrian crossing signal lamp result is passing.
7. A crosswalk signal lamp control apparatus comprising a processor and a memory, wherein the processor implements the crosswalk signal lamp control method according to any one of claims 1-5 when executing a computer program stored in the memory.
8. A computer-readable storage medium for storing a computer program, wherein the computer program, when executed by a processor, implements a method for controlling a crosswalk signal as claimed in any one of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311809930.7A CN117475389B (en) | 2023-12-27 | 2023-12-27 | Pedestrian crossing signal lamp control method, system, equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311809930.7A CN117475389B (en) | 2023-12-27 | 2023-12-27 | Pedestrian crossing signal lamp control method, system, equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN117475389A CN117475389A (en) | 2024-01-30 |
| CN117475389B true CN117475389B (en) | 2024-03-15 |
Family
ID=89627777
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311809930.7A Active CN117475389B (en) | 2023-12-27 | 2023-12-27 | Pedestrian crossing signal lamp control method, system, equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117475389B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117893895A (en) * | 2024-03-15 | 2024-04-16 | 山东省海洋资源与环境研究院(山东省海洋环境监测中心、山东省水产品质量检验中心) | Method, system, equipment and storage medium for identifying portunus trituberculatus |
Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104966401A (en) * | 2015-06-30 | 2015-10-07 | 广州市高科通信技术股份有限公司 | Road traffic signal control method and system using video to detect pedestrian crossing |
| CN110781744A (en) * | 2019-09-23 | 2020-02-11 | 杭州电子科技大学 | Small-scale pedestrian detection method based on multi-level feature fusion |
| CN111027493A (en) * | 2019-12-13 | 2020-04-17 | 电子科技大学 | Pedestrian detection method based on deep learning multi-network soft fusion |
| CN112052787A (en) * | 2020-09-03 | 2020-12-08 | 腾讯科技（深圳）有限公司 | Target detection method and device based on artificial intelligence and electronic equipment |
| CN112446231A (en) * | 2019-08-27 | 2021-03-05 | 丰图科技(深圳)有限公司 | Pedestrian crossing detection method and device, computer equipment and storage medium |
| CN112674998A (en) * | 2020-12-23 | 2021-04-20 | 北京工业大学 | Blind person traffic intersection assisting method based on rapid deep neural network and mobile intelligent device |
| CN113313943A (en) * | 2021-05-27 | 2021-08-27 | 中国科学院合肥物质科学研究院 | Road side perception-based intersection traffic real-time scheduling method and system |
| WO2022104774A1 (en) * | 2020-11-23 | 2022-05-27 | 华为技术有限公司 | Target detection method and apparatus |
| KR102453096B1 (en) * | 2022-06-24 | 2022-10-11 | 유에프엠시스템즈 주식회사 | System for controlling signal at smart cross walk |
| WO2023005258A1 (en) * | 2021-07-26 | 2023-02-02 | 上海商汤智能科技有限公司 | Traffic intersection detection method and apparatus, electronic device, and storage medium |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108830205B (en) * | 2018-06-04 | 2019-06-14 | 江南大学 | Based on the multiple dimensioned perception pedestrian detection method for improving full convolutional network |
| CN110472627B (en) * | 2019-07-02 | 2022-11-08 | 五邑大学 | End-to-end SAR image recognition method, device and storage medium |
-
2023
- 2023-12-27 CN CN202311809930.7A patent/CN117475389B/en active Active
Patent Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104966401A (en) * | 2015-06-30 | 2015-10-07 | 广州市高科通信技术股份有限公司 | Road traffic signal control method and system using video to detect pedestrian crossing |
| CN112446231A (en) * | 2019-08-27 | 2021-03-05 | 丰图科技(深圳)有限公司 | Pedestrian crossing detection method and device, computer equipment and storage medium |
| CN110781744A (en) * | 2019-09-23 | 2020-02-11 | 杭州电子科技大学 | Small-scale pedestrian detection method based on multi-level feature fusion |
| CN111027493A (en) * | 2019-12-13 | 2020-04-17 | 电子科技大学 | Pedestrian detection method based on deep learning multi-network soft fusion |
| CN112052787A (en) * | 2020-09-03 | 2020-12-08 | 腾讯科技（深圳）有限公司 | Target detection method and device based on artificial intelligence and electronic equipment |
| WO2022104774A1 (en) * | 2020-11-23 | 2022-05-27 | 华为技术有限公司 | Target detection method and apparatus |
| CN112674998A (en) * | 2020-12-23 | 2021-04-20 | 北京工业大学 | Blind person traffic intersection assisting method based on rapid deep neural network and mobile intelligent device |
| CN113313943A (en) * | 2021-05-27 | 2021-08-27 | 中国科学院合肥物质科学研究院 | Road side perception-based intersection traffic real-time scheduling method and system |
| WO2023005258A1 (en) * | 2021-07-26 | 2023-02-02 | 上海商汤智能科技有限公司 | Traffic intersection detection method and apparatus, electronic device, and storage medium |
| KR102453096B1 (en) * | 2022-06-24 | 2022-10-11 | 유에프엠시스템즈 주식회사 | System for controlling signal at smart cross walk |
Non-Patent Citations (4)
| Title |
|---|
| "礼让斑马线"对无信号控制人行横道处通行能力影响研究;周云月;李红伟;刘海平;钟馨;;贵州大学学报(自然科学版);20200315(02);118-123 * |
| 《基于深度学习的行人检测与重识别算法研究与应用》;孙鹏炎;中国矿业大学;20221013;20-39 * |
| 基于NVIDIA Jetson TX2的实时交通信号灯检测算法;刘影;姚振鑫;;农业装备与车辆工程;20200710(07);53-57+69 * |
| 基于特征通道和空间联合注意机制的遮挡行人检测方法;陈勇;刘曦;刘焕淋;;电子与信息学报;20200615(06);187-194 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN117475389A (en) | 2024-01-30 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN117475389B (en) | Pedestrian crossing signal lamp control method, system, equipment and storage medium | |
| CN111310773B (en) | Efficient license plate positioning method of convolutional neural network | |
| CN110119726B (en) | Vehicle brand multi-angle identification method based on YOLOv3 model | |
| CN112507844B (en) | Traffic jam detection method based on video analysis | |
| US11024042B2 (en) | Moving object detection apparatus and moving object detection method | |
| KR102122850B1 (en) | Solution for analysis road and recognition vehicle license plate employing deep-learning | |
| CN107689157B (en) | Traffic intersection passable road planning method based on deep learning | |
| CN111291826A (en) | Multi-source remote sensing image pixel-by-pixel classification method based on correlation fusion network | |
| Wang et al. | A hybrid method of vehicle detection based on computer vision for intelligent transportation system | |
| CN114580541A (en) | Fire disaster video smoke identification method based on time-space domain double channels | |
| KR100413382B1 (en) | System and Method for Integrated Remote Control of Watching Camera using Communication Network and Method for Processing Image Data Using the Same | |
| CN112818853A (en) | Traffic element identification method, device, equipment and storage medium | |
| CN103927875A (en) | Traffic overflowing state recognition method based on video | |
| Shankaran et al. | Real-time adaptive traffic control system for smart cities | |
| CN112907992A (en) | Traffic signal lamp control method, device, equipment and storage medium | |
| CN113920585A (en) | Behavior recognition method and device, equipment and storage medium | |
| CN109816003A (en) | A kind of intelligent vehicle front Multi-Target Classification Method based on improvement HOG-LBP feature | |
| CN111339834B (en) | Method for identifying vehicle driving direction, computer device and storage medium | |
| CN111062311B (en) | Pedestrian gesture recognition and interaction method based on depth-level separable convolution network | |
| CN112215188A (en) | Traffic police gesture recognition method, device, equipment and storage medium | |
| KR102656881B1 (en) | System and for combination management of traffic safety based on AIoT | |
| Anagnostopoulos et al. | Intelligent traffic management through MPEG-7 vehicle flow surveillance | |
| CN114882205A (en) | Target detection method based on attention mechanism | |
| bin Che Mansor et al. | Emergency vehicle type classification using convolutional neural network | |
| JP2011081614A (en) | Recognition system, recognition method, and program |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |