CN112861677A

CN112861677A - Method and device for detecting actions of rail transit driver, equipment, medium and tool

Info

Publication number: CN112861677A
Application number: CN202110120985.1A
Authority: CN
Inventors: 何裕康; 梁冠华; 毛宁元; 许亮
Original assignee: Shanghai Sensetime Lingang Intelligent Technology Co Ltd
Current assignee: Shanghai Sensetime Lingang Intelligent Technology Co Ltd
Priority date: 2021-01-28
Filing date: 2021-01-28
Publication date: 2021-05-28
Also published as: WO2022160678A1

Abstract

The disclosure relates to a method and device for detecting the actions of rail transit drivers, equipment, a medium and a tool. The method comprises the following steps: acquiring a first video stream of a driving area of a rail vehicle; detecting motion information of a driver of the rail vehicle according to the first video stream; matching the action information of the driver with preset standard action information; and responding to the fact that the action information of the driver is successfully matched with the preset standard action information, and sending the action information of the driver to preset driver management equipment.

Description

Method and device for detecting actions of rail transit driver, equipment, medium and tool

Technical Field

The present disclosure relates to the field of rail transit technologies, and in particular, to a method and an apparatus for detecting a motion of a rail transit driver, an electronic device, a storage medium, and a rail transit vehicle.

Background

In the rail transit industry, when a driver sees a specific traffic signal lamp or drives to a specific road section in the driving process, the driver needs to perform corresponding actions specified in the 'finger mouth calling' specification. The correct execution of the 'finger mouth call' specification has important significance on the running safety of rail transit.

Disclosure of Invention

The disclosure provides a technical scheme for detecting the actions of rail transit drivers.

According to an aspect of the present disclosure, there is provided a method for detecting a motion of a rail transit driver, including:

acquiring a first video stream of a driving area of a rail vehicle;

detecting motion information of a driver of the rail vehicle according to the first video stream;

matching the action information of the driver with preset standard action information;

and responding to the fact that the action information of the driver is successfully matched with the preset standard action information, and sending the action information of the driver to preset driver management equipment.

In one possible implementation form of the method,

the method further comprises the following steps: acquiring traffic signal information in front of the rail vehicle;

and determining the preset standard action information according to the traffic signal information in front of the rail vehicle before matching the action information of the driver with the preset standard action information.

In one possible implementation, the determining predetermined standard action information according to the traffic signal information in front of the rail vehicle includes:

and under the condition that the traffic signal information in front of the rail vehicle is acquired within a preset time range, determining preset standard action information according to the acquired traffic signal information in front of the rail vehicle.

In one possible implementation, the acquiring traffic signal information in front of the rail vehicle includes: and analyzing the traffic signal of the second video stream in front of the rail vehicle to obtain the traffic signal information in front of the rail vehicle.

In one possible implementation, the traffic signal information ahead of the rail vehicle includes at least one of: the information of the traffic signal lamp in front of the rail vehicle, the information of the gesture traffic guidance signal in front of the rail vehicle, and the information of the protection signal machine in front of the rail vehicle.

In one possible implementation, the acquiring traffic signal information in front of the rail vehicle includes:

and acquiring information of the speed limiting signal in front of the rail vehicle, which is acquired by the train control vehicle-mounted equipment.

In one possible implementation, the matching the action information of the driver with the predetermined standard action information includes:

matching the action information of the driver with preset standard action information corresponding to a preset non-signal triggering operation; or

And matching the action information of the driver with preset standard action information corresponding to a preset non-signal triggering operation under the condition that the traffic signal information in front of the rail vehicle is not acquired within a preset time range.

In one possible implementation, the method further includes:

in response to the action information of the driver being successfully matched with the preset standard action information, sending relevant data of the action information of the driver to the preset driver management equipment, wherein the relevant data comprises:

a video clip corresponding to the action information of the driver in the first video stream,

alternatively, the first and second electrodes may be,

the video clip corresponding to the action information of the driver in the first video stream and the video clip corresponding to the action information of the driver in the second video stream in front of the rail vehicle.

In one possible implementation, the data related to the driver's action information further includes at least one of: the preset standard action information, the traffic signal information in front of the rail vehicle corresponding to the action information of the driver, and the position information of the rail vehicle corresponding to the action information of the driver.

In one possible implementation, the method further includes:

in response to a failure to match the driver's action information with the predetermined standard action information, performing at least one of:

sending first prompt information to the preset driver management equipment;

controlling information output equipment arranged in the rail vehicle to send out second prompt information;

wherein the first prompt message and the second prompt message are used for prompting that the action information of the driver is not matched with the preset standard action information.

According to an aspect of the present disclosure, there is provided a rail transit driver's motion detection apparatus including:

the system comprises a first acquisition module, a second acquisition module and a third acquisition module, wherein the first acquisition module is used for acquiring a first video stream of a driving area of the rail vehicle;

the detection module is used for detecting the action information of a driver of the rail vehicle according to the first video stream;

the matching module is used for matching the action information of the driver with preset standard action information;

and the first sending module is used for responding to the condition that the action information of the driver is successfully matched with the preset standard action information and sending the action information of the driver to preset driver management equipment.

In one possible implementation, the apparatus further includes:

the second acquisition module is used for acquiring traffic signal information in front of the rail vehicle;

and the determining module is used for determining the preset standard action information according to the traffic signal information in front of the rail vehicle before the action information of the driver is matched with the preset standard action information.

In one possible implementation, the determining module is configured to:

In a possible implementation manner, the second obtaining module is configured to: and analyzing the traffic signal of the second video stream in front of the rail vehicle to obtain the traffic signal information in front of the rail vehicle.

In a possible implementation manner, the second obtaining module is configured to:

In one possible implementation, the matching module is configured to:

In one possible implementation, the apparatus further includes:

a second sending module, configured to send, to the preset driver management device, relevant data of the action information of the driver in response to a successful matching between the action information of the driver and the predetermined standard action information, where the relevant data includes:

alternatively, the first and second electrodes may be,

In one possible implementation, the apparatus further includes:

a matching failure operation module for responding the failure of the matching of the action information of the driver and the preset standard action information, and executing at least one of the following:

sending first prompt information to the preset driver management equipment;

According to an aspect of the present disclosure, there is provided a rail vehicle comprising:

the system comprises a first camera, a second camera and a control unit, wherein the first camera is used for collecting a first video stream of a driving area of the rail vehicle;

and the intelligent analysis equipment is used for acquiring the first video stream from the first camera, detecting the action information of the driver of the rail vehicle according to the first video stream, matching the action information of the driver with preset standard action information, and sending the action information of the driver to preset driver management equipment in response to the successful matching of the action information of the driver and the preset standard action information.

In one possible implementation form of the method,

the rail vehicle further comprises: the second camera is used for collecting a second video stream in front of the rail vehicle;

the intelligent analysis device is further configured to: and acquiring a second video stream in front of the rail vehicle from the second camera, carrying out traffic signal analysis on the second video stream to obtain traffic signal information in front of the rail vehicle, and determining preset standard action information according to the traffic signal information in front of the rail vehicle.

In one possible implementation form of the method,

the rail vehicle further comprises: the train control vehicle-mounted equipment is used for acquiring information of a speed limiting signal in front of the rail vehicle;

the intelligent analysis device is further configured to: and acquiring the information of the speed-limiting signal in front of the rail vehicle from the train control vehicle-mounted equipment, and determining preset standard action information according to the information of the speed-limiting signal in front of the rail vehicle.

In one possible implementation, the intelligent analysis device is further configured to:

alternatively, the first and second electrodes may be,

In one possible implementation form of the method,

the intelligent analysis device is further configured to: and when the action information of the driver is failed to be matched with the preset standard action information, executing at least one of the following steps: sending first prompt information to the preset driver management equipment; controlling information output equipment arranged in the rail vehicle to send out second prompt information; wherein the first prompt message and the second prompt message are used for prompting that the action information of the driver is not matched with the preset standard action information;

the rail vehicle further comprises: and the information output equipment is used for sending the second prompt information.

According to an aspect of the present disclosure, there is provided an electronic device including: one or more processors; a memory for storing executable instructions; wherein the one or more processors are configured to invoke the memory-stored executable instructions to perform the above-described method.

According to an aspect of the present disclosure, there is provided a computer readable storage medium having stored thereon computer program instructions which, when executed by a processor, implement the above-described method.

In the embodiment of the disclosure, by acquiring a first video stream of a driving area of a rail vehicle, detecting motion information of a driver of the rail vehicle according to the first video stream, matching the motion information of the driver with predetermined standard motion information, and sending the motion information of the driver to a preset driver management device in response to successful matching of the motion information of the driver with the predetermined standard motion information, the motion of the rail vehicle driver can be analyzed with high precision, low cost and no perception based on the video stream of the driving area of the rail vehicle, so that the driving behavior of the rail vehicle driver can be managed, and the safety of the rail vehicle can be improved.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.

Other features and aspects of the present disclosure will become apparent from the following detailed description of exemplary embodiments, which proceeds with reference to the accompanying drawings.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the disclosure.

Fig. 1 shows a flowchart of a method for detecting a track traffic driver's motion according to an embodiment of the present disclosure.

Fig. 2 is a schematic diagram showing predetermined standard motion information "the right hand extends out of the index finger and the middle finger to close together, the fist is left, and points to a confirmation object" corresponding to the traffic signal information in front of the rail vehicle.

Fig. 3 shows a schematic diagram of predetermined standard action information "right hand is extended horizontally, fist is extended vertically, and fist is left" corresponding to traffic signal information in front of the rail vehicle.

Fig. 4 shows a schematic diagram of predetermined standard action information "right hand is stretched flat, a fist is stretched with thumb and little finger, and a fist is left" corresponding to traffic signal information in front of the rail vehicle.

Fig. 5 shows a schematic diagram of predetermined standard action information "fist is gathered in right arm, lifted and eyebrow, fist is left, and forearm is shaken up and down 3 times" corresponding to traffic signal information in front of the rail vehicle.

Fig. 6 shows a schematic diagram of predetermined standard action information "fist drawn on right arm, large arm and small arm forming 90 °, fist drawn to be flush with eyebrow, and fist heart drawn to left" corresponding to a preset non-signal triggering operation.

Fig. 7 is a schematic diagram illustrating predetermined standard motion information "five fingers of right hand are closed and extended, and point to the relevant confirmation device from left to right" corresponding to the preset non-signal triggering operation.

Fig. 8 shows a block diagram of a motion detection device for rail transit drivers provided by an embodiment of the present disclosure.

Fig. 9 illustrates an exemplary block diagram of a rail vehicle provided by an embodiment of the present disclosure.

Fig. 10 illustrates another exemplary block diagram of a rail vehicle provided by an embodiment of the present disclosure.

Fig. 11 illustrates a block diagram of an electronic device 800 provided by an embodiment of the disclosure.

Fig. 12 shows a block diagram of an electronic device 1900 provided by an embodiment of the disclosure.

Detailed Description

Various exemplary embodiments, features and aspects of the present disclosure will be described in detail below with reference to the accompanying drawings. In the drawings, like reference numbers can indicate functionally identical or similar elements. While the various aspects of the embodiments are presented in drawings, the drawings are not necessarily drawn to scale unless specifically indicated.

The word "exemplary" is used exclusively herein to mean "serving as an example, embodiment, or illustration. Any embodiment described herein as "exemplary" is not necessarily to be construed as preferred or advantageous over other embodiments.

The term "and/or" herein is merely an association describing an associated object, meaning that three relationships may exist, e.g., a and/or B, may mean: a exists alone, A and B exist simultaneously, and B exists alone. In addition, the term "at least one" herein means any one of a plurality or any combination of at least two of a plurality, for example, including at least one of A, B, C, and may mean including any one or more elements selected from the group consisting of A, B and C.

Furthermore, in the following detailed description, numerous specific details are set forth in order to provide a better understanding of the present disclosure. It will be understood by those skilled in the art that the present disclosure may be practiced without some of these specific details. In some instances, methods, means, elements and circuits that are well known to those skilled in the art have not been described in detail so as not to obscure the present disclosure.

Fig. 1 shows a flowchart of a method for detecting a track traffic driver's motion according to an embodiment of the present disclosure. The track traffic driver motion detection method can be executed by a terminal device or a server or other processing devices. Wherein the terminal device can be mounted on the rail vehicle. The terminal device may be a smart analysis device (e.g., a smart analysis box), an in-vehicle device, a User Equipment (UE), a mobile device, a User terminal, a Personal Digital Assistant (PDA), a handheld device, a computing device, or a wearable device, etc. Wherein the intelligent analysis device can be installed at any position of the rail vehicle. For example, the intelligent analysis device may be mounted within a cockpit of the rail vehicle. As another example, the intelligent analysis device may be mounted within a chassis of the rail vehicle. In some possible implementations, the track traffic driver motion detection method may be implemented by a processor calling computer readable instructions stored in a memory. The rail transit driver action detection method can be applied to rail transit vehicles. As shown in fig. 1, the method for detecting the motion of the rail transit driver includes steps S11 to S14.

In step S11, a first video stream of a driving area of a rail vehicle is acquired.

In step S12, motion information of a driver of the rail vehicle is detected from the first video stream.

In step S13, the action information of the driver is matched with predetermined standard action information.

In step S14, in response to the action information of the driver being successfully matched with the predetermined standard action information, the action information of the driver is sent to a preset driver management device.

In the disclosed embodiments, a rail vehicle may represent a vehicle that needs to travel on a particular rail. For example, the rail vehicle may be a train, a subway, a tram, a maglev train, or the like. The first video stream represents a video stream of a driving area of the rail vehicle. The first video stream may include image information of a driver of the rail vehicle. In one possible implementation, the first video stream may further include image information of a driving console of the rail vehicle. Wherein, the driving operation platform can comprise a display screen and/or a meter and the like. In a possible implementation manner, at least one first camera may be installed on the rail vehicle, and the first camera may be used to capture a first video stream. Wherein the first camera may be mounted within a cockpit of the rail vehicle and may be mounted towards a driver.

In the embodiment of the disclosure, at least one type of motion detection may be performed on a driver of the rail vehicle according to the first video stream, so as to obtain motion information of the driver, where the motion information of the driver may include at least one type of motion information. For example, at least one of gesture detection, arm motion detection, head motion detection, mouth motion detection, line of sight detection, and the like may be performed on the driver from the first video stream. Accordingly, the motion information of the driver may include at least one of gesture information, arm motion information, head motion information, mouth motion information, line of sight information, and the like. Accordingly, the predetermined standard motion information may include at least one of predetermined standard gesture information, predetermined arm standard motion information, predetermined head standard motion information, predetermined mouth standard motion information, predetermined sight line standard information, and the like. This makes it possible to detect and manage at least one of the gesture, the arm movement, the head movement, the mouth movement, the line of sight, and the like of the driver of the rail vehicle.

In a possible implementation manner, a first neural network for detecting a motion of a driver may be trained in advance, a first video stream of a driving area of the rail vehicle is input into the first neural network, and the motion of the driver of the rail vehicle is detected through the first neural network according to the detection of the first video stream, so as to obtain motion information of the driver. The different types of motion detection may be performed by different neural networks, or may be performed by the same neural network, which is not limited herein. Of course, in other implementation manners, a function for detecting the motion of the image and/or the video may be designed in advance, and the motion information of the driver may be obtained by detecting the motion of the first video stream in the driving area of the rail vehicle through the first function.

In the disclosed embodiment, the predetermined standard action information may represent information of a predetermined standard action. That is, the predetermined standard action information may embody a predetermined standard action. For example, the predetermined standard action information may include at least one of a type, an order, a number, a duration, and the like of the predetermined standard action. For example, the predetermined standard action information may be determined according to an action specification and/or a mouth call specification in a "finger mouth call" specification. In one example, if the driver's motion information matches the predetermined standard motion information successfully, it may be determined that the driver is in compliance with the "finger-mouth-call" specification. In the embodiment of the present disclosure, the successful matching of the action information of the driver and the predetermined standard action information may indicate that a difference between the action information of the driver and the predetermined standard action information is within a preset allowable range; a failure in matching the driver's motion information with the predetermined standard motion information may indicate that a difference between the driver's motion information and the predetermined standard motion information is outside a preset allowable range.

In one possible implementation, the predetermined standard action information may include predetermined standard gesture information. In this implementation, the predetermined standard gesture information may represent information of a predetermined standard gesture. That is, the predetermined standard gesture information may embody a predetermined standard gesture. For example, the predetermined standard gesture information may include at least one of a type, an order, a number of times, etc. of the predetermined standard gesture.

In one possible implementation, the predetermined standard motion information may include predetermined arm standard motion information. In this implementation, the predetermined arm standard motion information may represent information of a predetermined arm standard motion. That is, the predetermined arm standard motion information may embody a predetermined arm standard motion. Wherein the standard arm movement represents a standard arm movement. For example, the predetermined arm standard motion information may include at least one of a type, an order, a number of times, and the like of the predetermined arm standard motion.

In one possible implementation, the predetermined standard action information may include predetermined header standard action information. In this implementation, the predetermined header standard action information may represent information of a predetermined header standard action. That is, the predetermined header standard action information may embody a predetermined header standard action. Wherein the head standard action represents a standard head action. For example, the predetermined header standard action information may include at least one of a type, an order, a number of times, and the like of the predetermined header standard action.

In one possible implementation, the predetermined standard motion information may include predetermined mouth standard motion information. In this implementation, the predetermined mouth standard action information may represent information of a predetermined mouth standard action. That is, the predetermined mouth standard motion information may embody a predetermined mouth standard motion. Wherein the standard mouth motion means a standard mouth motion. For example, the predetermined mouth standard action information may include at least one of a type, an order, a number, a duration, and the like of the predetermined mouth standard action. For example, the type of predetermined standard mouth movement includes mouth opening.

In one possible implementation, the predetermined standard action information may include predetermined sight line standard information. In this implementation, the predetermined standard sight line information may represent information of a predetermined sight line standard. That is, the predetermined standard sight line information may embody a predetermined sight line standard. For example, the predetermined standard gaze information may include at least one of a direction, an order, a number of times, a duration of stay or gaze, and the like of the predetermined standard gaze.

In the embodiment of the present disclosure, the predetermined standard action information may include predetermined standard action information corresponding to a signal trigger operation and/or predetermined standard action information corresponding to a non-signal trigger operation. The predetermined standard action information corresponding to the signal triggering operation may represent predetermined standard action information determined according to triggering of a traffic signal or other signals (e.g., signals from a cloud device, etc.) in front of the rail vehicle. The predetermined standard motion information corresponding to the non-signal-triggered operation may represent predetermined standard motion information that does not require triggering according to a traffic signal or other signal in front of the rail vehicle.

In the embodiment of the present disclosure, the preset driver management device may represent a device for driver management that is set in advance. Wherein the number of the preset driver management devices may be one or more. In one possible implementation, the preset driver management device may include a cloud device (e.g., a server of a cloud platform). In another possible implementation, the preset driver management device may include a terminal device of an operator of the rail vehicle.

In one possible implementation, the method further includes: acquiring traffic signal information in front of the rail vehicle; and determining the preset standard action information according to the traffic signal information in front of the rail vehicle before matching the action information of the driver with the preset standard action information. In this implementation, the traffic signal information in front of the rail vehicle may represent information of a traffic signal in front of the rail vehicle. That is, the traffic signal information in front of the rail vehicle may embody a traffic signal in front of the rail vehicle. In this implementation, the correspondence between the traffic signal information and the predetermined standard action information may be determined according to the "finger-mouth call" specification. Of course, the corresponding relationship between the traffic signal information and the predetermined standard action information can also be flexibly set according to the requirements of the actual application scene. The relationship between the traffic signal information and the predetermined standard action information may be one-to-one, one-to-many or many-to-one, and is not limited herein. According to the correspondence between the traffic signal information and the predetermined standard action information and the traffic signal information in front of the rail vehicle, the predetermined standard action information corresponding to the traffic signal information in front of the rail vehicle can be determined. In this implementation, by acquiring the traffic signal information in front of the rail vehicle and determining the predetermined standard action information according to the traffic signal information in front of the rail vehicle, the corresponding predetermined standard action information can be accurately determined based on the traffic signal information in front of the rail vehicle during the running of the rail vehicle, thereby facilitating more accurate driver behavior management.

As an example of this implementation, the determining predetermined standard action information from traffic signal information in front of the rail vehicle includes: and under the condition that the traffic signal information in front of the rail vehicle is acquired within a preset time range, determining preset standard action information according to the acquired traffic signal information in front of the rail vehicle. In this example, the predetermined time range may be 5 seconds, 10 seconds, etc., and those skilled in the art may flexibly set the predetermined time range according to an actual application scenario (e.g., a traveling speed of the rail vehicle, etc.). If the traffic signal information in front of the rail vehicle is acquired within the preset time range, the preset standard action information currently used for matching with the action information of the driver can be determined according to the corresponding relation between the traffic signal information and the preset standard action information. In a case where traffic signal information in front of the rail vehicle is not acquired within a predetermined time range, it may be determined that predetermined standard action information corresponding to the traffic signal information does not currently exist. In this example, when the traffic signal information in front of the rail vehicle is acquired within a predetermined time range, the predetermined standard operation information is determined from the acquired traffic signal information in front of the rail vehicle, whereby the predetermined standard operation information can be determined based on the real-time traffic signal information in front of the rail vehicle, which contributes to more accurate driver behavior management.

In one example, the acquiring traffic signal information ahead of the rail vehicle includes: and analyzing the traffic signal of the second video stream in front of the rail vehicle to obtain the traffic signal information in front of the rail vehicle. In this example, the second video stream represents a video stream in front of the rail vehicle. In one example, at least one second camera is further mounted on the rail vehicle, and the second camera is used for collecting a second video stream in front of the rail vehicle. In this example, the second camera may be mounted towards the front of the rail vehicle, e.g. the second camera may be mounted towards a rail to capture a second video stream in front of the rail vehicle. The second camera can be installed in a cockpit of the rail vehicle and can also be installed outside the cockpit of the rail vehicle. The second video stream may include imagery information of a track in front of the rail vehicle and/or the second video stream may include imagery information above a track in front of the rail vehicle. In this example, traffic signal information in front of the rail vehicle is obtained by performing traffic signal analysis on the second video stream in front of the rail vehicle, so that the traffic signal information in front of the rail vehicle can be accurately and timely determined according to the visual information in front of the rail vehicle.

In one example, a second neural network for analyzing traffic signals may be pre-trained, a second video stream in front of the rail vehicle may be input to the second neural network, and traffic signal information in front of the rail vehicle may be output via the second neural network. Of course, in other examples, a second function for performing traffic signal analysis on the image and/or the video may be designed in advance, and traffic signal information in front of the rail vehicle may be obtained by performing traffic signal analysis on a second video stream in front of the rail vehicle through the second function.

As one example of this implementation, the traffic signal information ahead of the rail vehicle includes at least one of: the information of the traffic signal lamp in front of the rail vehicle, the information of the gesture traffic guidance signal in front of the rail vehicle, and the information of the protection signal machine in front of the rail vehicle.

In this example, the information of the traffic light in front of the rail vehicle may indicate a state of the traffic light in front of the rail vehicle. For example, the information of the traffic light in front of the rail vehicle may include: and a traffic signal lamp appears in front of the rail vehicle, and the traffic signal lamp displays green light, yellow light, red light, green-yellow light, double-yellow light or yellow flashing yellow and the like. For example, the information of the traffic light in front of the rail vehicle may be any one of: a traffic signal lamp appears in front of the rail vehicle, and the traffic signal lamp displays a green light or a green-yellow light; a traffic signal lamp appears in front of the rail vehicle, and the traffic signal lamp displays a yellow light; a traffic signal lamp appears in front of the rail vehicle, and the traffic signal lamp displays double yellow lamps or yellow flashing yellow lamps; a traffic signal lamp appears in front of the rail vehicle, and the traffic signal lamp displays a red light; and so on.

In this example, the gesture traffic guidance signal refers to a railway movement signal, and is various commands issued by a person directly waving a signal flag and a signal light. In some application scenarios, the gestural traffic direction signal is also referred to as a hand signal. The information of the gestured traffic guidance signal in front of the rail vehicle may include a category of the gestured traffic guidance signal in front of the rail vehicle. For example, the information of the gesture traffic guidance signal in front of the rail vehicle may include that the gesture traffic guidance signal in front of the rail vehicle is a parking signal, a deceleration signal, a departure signal, a passing signal or a guidance signal, etc.

In this example, the protection traffic signal is a traffic signal which guarantees the traffic safety at the intersection of the railway line plane in the section. The information of the guard signal in front of the rail vehicle may indicate a state of the guard signal in front of the rail vehicle.

This example facilitates accurate driver action analysis by acquiring at least one of information of a traffic signal in front of the rail vehicle, information of a gesture traffic guidance signal in front of the rail vehicle, and information of a guard signal in front of the rail vehicle, thereby determining predetermined standard action information based on at least one of the information of the traffic signal in front of the rail vehicle, the information of the gesture traffic guidance signal in front of the rail vehicle, and the information of the guard signal in front of the rail vehicle.

For example, the traffic signal information in front of the rail vehicle includes "traffic signal lamp appears, and the traffic signal lamp is displayed as a green light or a green-yellow light", and the traffic signal information in front of the rail vehicle "traffic signal lamp appears, and the traffic signal lamp is displayed as a green light or a green-yellow light", and the corresponding predetermined standard action information includes "the right hand stretches out the index finger and the middle finger to be close together, the fist is left, and points to the traffic signal lamp". Wherein "traffic light appears, and traffic light is displayed as green light or green-yellow light" may indicate that the traffic light display in front of the rail vehicle requires passing. Fig. 2 is a schematic diagram showing predetermined standard motion information "the right hand extends out of the index finger and the middle finger to close together, the fist is left, and points to a confirmation object" corresponding to the traffic signal information in front of the rail vehicle.

For another example, the traffic signal information in front of the rail vehicle includes information of the gesture traffic guidance signal, and the predetermined standard motion information corresponding to the information of the gesture traffic guidance signal includes "right hand stretches out the index finger and the middle finger to be close together, fist to the left, and points to the gesture traffic guidance signal" or "left hand stretches out the index finger and the middle finger to be close together, fist to the right, and points to the gesture traffic guidance signal".

For another example, the traffic signal information in front of the rail vehicle includes information of a protection signal (e.g., a derailer), and the predetermined standard action information corresponding to the information of the protection signal includes "the right hand stretches out the index finger and the middle finger to be closed, the fist is leftward, the pointing protection signal" or "the left hand stretches out the index finger and the middle finger to be closed, the fist is rightward, and the pointing protection signal".

If again, the traffic signal information in track vehicle the place ahead includes "traffic signal lamp appears, and traffic signal lamp shows to be the yellow light", traffic signal information in track vehicle the place ahead "traffic signal lamp appears, and traffic signal lamp shows to be the yellow light" predetermined standard action information that corresponds includes "the right hand is flat stretched, and it is upright to draw together the fist, and the fist heart is left" or "the left hand is flat stretched, draws together the fist and stretches the thumb upright, and the fist heart is right". Wherein "traffic light present, and traffic light displayed as yellow light" may indicate that the traffic light display in front of the rail vehicle requires a forward path to be ready for parking. Fig. 3 shows a schematic diagram of predetermined standard action information "right hand is extended horizontally, fist is extended vertically, and fist is left" corresponding to traffic signal information in front of the rail vehicle.

As another example, the traffic signal information in front of the rail vehicle includes "traffic signal lamp appears, and the traffic signal lamp displays as double yellow lamps or yellow flashing yellow", the traffic signal information in front of the rail vehicle "traffic signal lamp appears, and the traffic signal lamp displays as double yellow lamps or yellow flashing yellow", and the corresponding predetermined standard action information includes "right hand is stretched flat, the fist is drawn to the thumb and little finger, and the fist is drawn to the left". Wherein the "traffic light appears, and the traffic light display is a double yellow light or a yellow flashing yellow" may indicate that the traffic light display in front of the rail vehicle requires lateral path operation. Fig. 4 shows a schematic diagram of predetermined standard action information "right hand is stretched flat, a fist is stretched with thumb and little finger, and a fist is left" corresponding to traffic signal information in front of the rail vehicle.

As another example, the traffic signal information in front of the rail vehicle includes "traffic signal lamp appears, and the traffic signal lamp shows as a red light", the traffic signal information in front of the rail vehicle "traffic signal lamp appears, and the traffic signal lamp shows as a red light", and the corresponding predetermined standard action information includes "fist gathered by right arm, and the fist is aligned with eyebrow, and the fist heart is left, and the forearm is rocked up and down 3 times". In this example, the red light may be a fixed red light or a temporary red light. The "traffic light present and traffic light displayed as a red light" may indicate that the traffic light display in front of the rail vehicle requires parking. Fig. 5 shows a schematic diagram of predetermined standard action information "fist is gathered in right arm, lifted and eyebrow, fist is left, and forearm is shaken up and down 3 times" corresponding to traffic signal information in front of the rail vehicle.

As an example of this implementation, the acquiring traffic signal information in front of the rail vehicle includes: and acquiring information of the speed limiting signal in front of the rail vehicle, which is acquired by the train control vehicle-mounted equipment. In this example, the traffic signal information in front of the rail vehicle may include information of a speed limit signal in front of the rail vehicle. In this example, a train control onboard device (LKJ) may also be provided on the rail vehicle. For example, the train control onboard device may send information of a speed limit signal in front of the rail vehicle to a motion detection device (e.g., an intelligent analysis device) of a rail vehicle driver in response to a change in the limit speed in front of the rail vehicle. For example, the train control on-board device may transmit information of the speed limit signal in front of the rail vehicle to the intelligent analysis device through a signal line, so that the intelligent analysis device obtains the information of the speed limit signal in front of the rail vehicle. The information of the speed limit signal in front of the rail vehicle may include numerical information of a limit speed in front of the rail vehicle and/or information of a difference between the limit speed in front of the rail vehicle and a current limit speed. In this example, the action analysis of the driver can be accurately performed by acquiring the information of the speed-limiting signal in front of the rail vehicle, which is acquired by the train-controlled on-board device, and determining the predetermined standard action information based on the information of the speed-limiting signal in front of the rail vehicle.

For example, the traffic signal information in front of the rail vehicle includes information of a speed limit signal in front of the rail vehicle, which is acquired from the column control vehicle-mounted device, and the predetermined standard action information corresponding to the information of the speed limit signal in front of the rail vehicle includes "a right hand stretches out a forefinger and a middle finger to be close together, a fist is left, and points to a display screen of the column control vehicle-mounted device".

In one possible implementation, the matching the action information of the driver with the predetermined standard action information includes: matching the action information of the driver with preset standard action information corresponding to a preset non-signal triggering operation; or, under the condition that the traffic signal information in front of the rail vehicle is not acquired within the preset time range, matching the action information of the driver with preset standard action information corresponding to a preset non-signal triggering operation. In this implementation, the predetermined standard action information corresponding to the preset non-signal triggering operation may represent predetermined standard action information that is preset without triggering according to a traffic signal in front of the rail vehicle or other signals.

For example, the preset predetermined standard action information corresponding to the non-signal triggering operation includes "the right arm is closed to a fist, the big arm and the small arm are 90 °, the lifting fist is aligned with the eyebrow, and the fist is centered to the left". The predetermined standard action information corresponding to the preset non-signal triggering operation may indicate that a vigilance operation is currently required. Fig. 6 shows a schematic diagram of predetermined standard action information "fist drawn on right arm, large arm and small arm forming 90 °, fist drawn to be flush with eyebrow, and fist heart drawn to left" corresponding to a preset non-signal triggering operation.

For another example, the preset predetermined standard action information corresponding to the non-signal triggering operation includes "the five fingers of the right hand are closed and straightened, and the five fingers point to the relevant confirmation device from left to right in sequence". The predetermined standard action information corresponding to the preset non-signal triggering operation may indicate that the driver is confirming the relevant confirmation device. Wherein, the related confirmation equipment can be a display screen, a meter and the like. Fig. 7 is a schematic diagram illustrating predetermined standard motion information "five fingers of right hand are closed and extended, and point to the relevant confirmation device from left to right" corresponding to the preset non-signal triggering operation.

In the implementation manner, the action information of the driver is matched with the preset standard action information corresponding to the preset non-signal triggering operation, or the action information of the driver is matched with the preset standard action information corresponding to the preset non-signal triggering operation under the condition that the traffic signal information in front of the rail vehicle is not acquired within the preset time range, and the action information of the driver is sent to the preset driver management equipment in response to the successful matching of the action information of the driver and the preset standard action information corresponding to the preset non-signal triggering operation, so that relevant personnel can inquire or manage the driving behavior of the rail vehicle driver conveniently, and the safety of rail traffic is improved.

In one possible implementation, the method further includes: in response to the action information of the driver being successfully matched with the preset standard action information, sending relevant data of the action information of the driver to the preset driver management equipment, wherein the relevant data comprises: the video clip corresponding to the action information of the driver in the first video stream, or the video clip corresponding to the action information of the driver in the first video stream, and the video clip corresponding to the action information of the driver in the second video stream in front of the rail vehicle.

In this implementation, the data related to the action information of the driver may represent any type of data related to the action information of the driver. As an example of this implementation, the relevant data includes: and the video clip corresponds to the action information of the driver in the first video stream. As another example of this implementation, the related data includes: the video clip corresponding to the action information of the driver in the first video stream and the video clip corresponding to the action information of the driver in the second video stream in front of the rail vehicle.

In this implementation, the video segment corresponding to the motion information of the driver in the first video stream may represent a video segment related to the detected motion information of the driver in the first video stream. The video clip corresponding to the motion information of the driver in the second video stream in front of the rail vehicle may represent the video clip related to the detected motion information of the driver in the second video stream. The video clip corresponding to the action information of the driver in the first video stream may include a video clip used for detecting and obtaining the action information of the driver in the first video stream. That is, the video clip corresponding to the driver's motion information in the first video stream may represent a video clip determining a detection result of the driver's motion information in the first video stream. In one example, the video clip corresponding to the motion information of the driver in the second video stream in front of the rail vehicle may include a video clip used for detecting and obtaining traffic signal information in front of the rail vehicle corresponding to the motion information of the driver in the second video stream. That is, the video clip corresponding to the motion information of the driver in the second video stream may include a video clip of predetermined standard motion information that determines the motion information of the driver in the second video stream in front of the rail vehicle. In another example, the video clips corresponding to the action information of the driver in the second video stream in front of the rail vehicle may include video clips in the second video stream having the same time range as the video clips corresponding to the action information of the driver in the first video stream. That is, in this example, the time range of the video clip corresponding to the motion information of the driver in the second video stream may be the same as the time range of the video clip corresponding to the motion information of the driver in the first video stream.

In this implementation manner, by responding to the fact that the action information of the driver is successfully matched with the predetermined standard action information, relevant data of the action information of the driver is sent to the preset driver management device, so that relevant personnel can conveniently inquire or manage the driving action of the rail transit driver.

As an example of this implementation, the data related to the driver's action information further includes at least one of: the preset standard action information, the traffic signal information in front of the rail vehicle corresponding to the action information of the driver, and the position information of the rail vehicle corresponding to the action information of the driver. In this example, at least one of the predetermined standard action information, traffic signal information in front of the rail vehicle corresponding to the action information of the driver, and location information of the rail vehicle corresponding to the action information of the driver is transmitted to the preset driver management device, so that the related personnel can be assisted to inquire or manage the driving behavior of the rail vehicle driver, and the safety of the rail vehicle can be improved.

In other possible implementations, the data related to the motion information of the driver may further include one or more video frames corresponding to the motion information of the driver in the first video stream, or one or more video frames corresponding to the motion information of the driver in the first video stream, and one or more video frames corresponding to the motion information of the driver in a second video stream in front of the rail vehicle.

In one possible implementation, the method further includes: in response to a failure to match the driver's action information with the predetermined standard action information, performing at least one of: sending first prompt information to the preset driver management equipment; controlling information output equipment arranged in the rail vehicle to send out second prompt information; wherein the first prompt message and the second prompt message are used for prompting that the action information of the driver is not matched with the preset standard action information. For example, the preset driver management device includes a cloud device and/or a terminal device of an operator of the rail vehicle, and may send the first prompt message to the cloud device and/or the terminal device of the operator of the rail vehicle in response to a failure in matching the action information of the driver with the predetermined standard action information. In one example, after sending the first prompt message to the cloud device, the cloud device may store the first prompt message so as to facilitate query of related people. In an example, the first prompt information may be sent to the terminal device of the operator by an Application (App), a short message, or the like. The terminal device of the operator may be a terminal device (e.g., a mobile phone) of an administrator of the operator, or the like. In this implementation, in response to failure of matching between the action information of the driver and the predetermined standard action information, the information output device provided in the rail vehicle may be further controlled to issue second prompt information for prompting that the action information of the driver does not match with the predetermined standard action information. In this implementation, the information output device may include at least one of a speaker, a display screen, a buzzer, and an LED (Light Emitting Diode). For example, the second prompt message may be played in voice form through a speaker. As another example, the second prompt message may be displayed in text form via a display screen.

In this implementation manner, by responding to failure of matching between the action information of the driver and the predetermined standard action information and sending the first prompt information to the preset driver management device, relevant personnel can conveniently inquire out of the non-normative historical record of the driver of the rail vehicle. And sending first prompt information to the terminal equipment of the operator of the rail vehicle in response to failure of matching between the action information of the driver and the preset standard action information, so that the operator can timely receive the prompt information, and the driving behavior of the rail vehicle driver can be effectively managed. And responding to failure of matching between the action information of the driver and the preset standard action information, and sending second prompt information through an information output device arranged in the rail vehicle, so that the rail vehicle driver can obtain prompt in time, and the driver can be helped to correct driving behaviors in time.

In one possible implementation, the method may further include: detecting voice information of the driver according to the first video stream; matching the voice information of the driver with preset standard voice information; and responding to the fact that the voice information of the driver is successfully matched with the preset standard voice information, and sending the voice information of the driver to preset driver management equipment. In this implementation, the predetermined standard voice information may represent information of a predetermined voice standard. That is, the predetermined standard voice information may embody a predetermined voice standard. For example, the predetermined standard voice information may include at least one of content, volume, speed of voice, and the like of the voice. For example, the content of the speech in the predetermined standard speech information may be determined according to a "mouth call" specification in a "finger mouth call" specification. For example, the predetermined standard voice information corresponding to the traffic signal information "traffic signal appears in front of the rail vehicle, and the traffic signal is displayed as double yellow lights or yellow flashing yellow" includes the display of a call signal and the lateral speed limit value of the call.

In one possible implementation, the method further includes: performing face recognition according to the first video stream to obtain identity information of the driver; in response to the driver's identity information not being a legitimate driver of the rail vehicle, performing at least one of: sending third prompt information to preset driver management equipment and sending fourth prompt information through information output equipment arranged in the rail vehicle, wherein the third prompt information and the fourth prompt information are used for prompting that the driver is not a legal driver of the rail vehicle; and/or, in response to the driver's identity information not being a currently scheduled driver of the rail vehicle, performing at least one of: and sending fifth prompt information to preset driver management equipment, and sending sixth prompt information through information output equipment arranged in the rail vehicle, wherein the fifth prompt information and the sixth prompt information are used for prompting that the driver is not the current scheduling driver of the rail vehicle.

In this implementation, the legal driver of the rail vehicle may refer to a driver who has registered with the management device. The identity information of the driver is not a legal driver of the rail vehicle, and may refer to that the driver is a driver not registered in the management device, that is, the driver does not belong to a driver registered in the management device. The driver of the current shift of the rail vehicle may represent a driver that has been registered with the management device and belongs to the current shift of the rail vehicle.

In this implementation manner, the third prompt message is sent to the management device in response to that the identity information of the driver is not a legal driver of the rail vehicle, so that information that a person who does not belong to the legal driver enters the cockpit of the rail vehicle can be recorded by the management device, and thus, related persons can conveniently inquire the history record that the person who does not belong to the legal driver enters the cockpit of the rail vehicle. And sending third prompt information to the terminal equipment of the operator of the rail vehicle by responding that the identity information of the driver is not a legal driver of the rail vehicle, so that the operator can timely receive the prompt information, and the operator can timely manage the invasion condition of the illegal driver. The identity information of the driver is not a legal driver of the rail vehicle, and the fourth prompt information is sent out through the information output equipment arranged in the rail vehicle, so that the function of warning an illegal driver can be achieved. The fifth prompt message is sent to the cloud device in response to that the identity information of the driver is not the driver of the current scheduling of the rail vehicle, so that the information of the rail vehicle driver on duty can be recorded through the cloud device, and the information of the rail vehicle driver on duty can be conveniently inquired by related personnel. The fifth prompt message is sent to the terminal equipment of the operator of the rail vehicle in response to the fact that the identity information of the driver is not the driver of the current shift of the rail vehicle, so that the operator can timely receive the prompt message, and the operator can timely manage the current shift condition of the rail vehicle driver. The information output device arranged in the rail vehicle sends out sixth prompt information in response to the fact that the identity information of the driver is not the driver of the current scheduling of the rail vehicle, and therefore the information of the current scheduling of the driver can be prompted in time.

The action detection method for the rail transit driver provided by the embodiment of the disclosure does not need human intervention, can improve the operation efficiency of an operator of a rail transit vehicle, and can improve the running safety of the rail transit vehicle.

It is understood that the above-mentioned method embodiments of the present disclosure can be combined with each other to form a combined embodiment without departing from the logic of the principle, which is limited by the space, and the detailed description of the present disclosure is omitted.

It will be understood by those skilled in the art that in the method of the present invention, the order of writing the steps does not imply a strict order of execution and any limitations on the implementation, and the specific order of execution of the steps should be determined by their function and possible inherent logic.

In addition, the present disclosure also provides a device for detecting a track traffic driver's motion, a track traffic tool, an electronic device, a computer-readable storage medium, and a program, which can be used to implement any one of the methods for detecting a track traffic driver's motion provided by the present disclosure, and corresponding technical solutions and technical effects can be referred to corresponding descriptions in the methods section and are not described again.

Fig. 8 shows a block diagram of a motion detection device for rail transit drivers provided by an embodiment of the present disclosure. As shown in fig. 8, the motion detection device for rail transit drivers includes:

a first obtaining module 21, configured to obtain a first video stream of a driving area of a rail vehicle;

a detection module 22, configured to detect motion information of a driver of the rail vehicle according to the first video stream;

the matching module 23 is used for matching the action information of the driver with preset standard action information;

and the first sending module 24 is configured to send the action information of the driver to a preset driver management device in response to that the action information of the driver is successfully matched with the predetermined standard action information.

In one possible implementation, the apparatus further includes:

In one possible implementation, the determining module is configured to:

In a possible implementation manner, the matching module 23 is configured to:

In one possible implementation, the apparatus further includes:

alternatively, the first and second electrodes may be,

In one possible implementation, the apparatus further includes:

sending first prompt information to the preset driver management equipment;

Fig. 9 illustrates an exemplary block diagram of a rail vehicle provided by an embodiment of the present disclosure. As shown in fig. 9, the rail vehicle includes: a first camera 31 for acquiring a first video stream of a driving area of a rail vehicle; and the intelligent analysis device 32 is configured to acquire the first video stream from the first camera 31, detect motion information of a driver of the rail vehicle according to the first video stream, match the motion information of the driver with predetermined standard motion information, and send the motion information of the driver to a preset driver management device in response to successful matching between the motion information of the driver and the predetermined standard motion information.

In an embodiment of the present disclosure, at least one first camera may be mounted on the rail vehicle. The first camera may be mounted within a cockpit of the rail vehicle and may be mounted towards a driver. The intelligent analysis device can be installed at any position of the rail vehicle. For example, the intelligent analysis device may be mounted within a cockpit of the rail vehicle. As another example, the intelligent analysis device may be mounted within a chassis of the rail vehicle.

In one possible implementation, the rail vehicle further includes: the second camera is used for collecting a second video stream in front of the rail vehicle; the intelligent analysis device 32 is further configured to: and acquiring a second video stream in front of the rail vehicle from the second camera, carrying out traffic signal analysis on the second video stream to obtain traffic signal information in front of the rail vehicle, and determining preset standard action information according to the traffic signal information in front of the rail vehicle.

In this implementation, at least one second camera is also mounted on the rail vehicle. The second camera may be mounted towards the front of the rail vehicle, for example, the second camera may be mounted towards a rail to capture a second video stream in front of the rail vehicle. The second camera can be installed in a cockpit of the rail vehicle and can also be installed outside the cockpit of the rail vehicle.

In one possible implementation, the rail vehicle further includes: the train control vehicle-mounted equipment is used for acquiring information of a speed limiting signal in front of the rail vehicle; the intelligent analysis device 32 is further configured to: and acquiring the information of the speed-limiting signal in front of the rail vehicle from the train control vehicle-mounted equipment, and determining preset standard action information according to the information of the speed-limiting signal in front of the rail vehicle.

In this implementation, the train control onboard equipment may be mounted within a cockpit of the rail vehicle. For example, the train control onboard equipment is mounted on a driving console.

In one possible implementation, the intelligent analysis device 32 is further configured to: in response to the action information of the driver being successfully matched with the preset standard action information, sending relevant data of the action information of the driver to the preset driver management equipment, wherein the relevant data comprises: the video clip corresponding to the action information of the driver in the first video stream, or the video clip corresponding to the action information of the driver in the first video stream, and the video clip corresponding to the action information of the driver in the second video stream in front of the rail vehicle.

In one possible implementation, the intelligent analysis device 32 is further configured to: and when the action information of the driver is failed to be matched with the preset standard action information, executing at least one of the following steps: sending first prompt information to the preset driver management equipment; controlling information output equipment arranged in the rail vehicle to send out second prompt information; wherein the first prompt message and the second prompt message are used for prompting that the action information of the driver is not matched with the preset standard action information; the rail vehicle further comprises: and the information output equipment is used for sending the second prompt information.

In this implementation, the information output device may be disposed within a cockpit of the rail vehicle.

Fig. 10 illustrates another exemplary block diagram of a rail vehicle provided by an embodiment of the present disclosure. In the example shown in fig. 10, the rail vehicle includes a first camera 31, a smart analysis device 32, a second camera 33, a train-mounted device 34, and an information output device 35. The first camera 31, the second camera 33, the train control onboard equipment 34 and the information output equipment 35 are respectively connected with the intelligent analysis equipment 32.

In the embodiment of the disclosure, a first video stream of the driving area of the rail vehicle is acquired through a first camera, an intelligent analysis device acquires the first video stream of the driving area of the rail vehicle from the first camera, detecting motion information of a driver of the rail vehicle according to the first video stream, matching the motion information of the driver with predetermined standard motion information, and in response to the driver's action information being successfully matched with the predetermined standard action information, transmitting the driver's action information to a preset driver management device, thereby enabling a high-precision, low-cost, perceptually-free analysis of rail vehicle driver actions based on a video stream of the driving area of the rail vehicle, therefore, the method is beneficial to managing the driving behaviors of rail transit drivers and improving the safety of rail transit.

In some embodiments, functions or modules included in the apparatus provided in the embodiments of the present disclosure may be used to execute the method described in the above method embodiments, and specific implementations and technical effects thereof may refer to the description of the above method embodiments, which are not described herein again for brevity.

Embodiments of the present disclosure also provide a computer-readable storage medium having stored thereon computer program instructions, which when executed by a processor, implement the above-described method. The computer-readable storage medium may be a non-volatile computer-readable storage medium, or may be a volatile computer-readable storage medium.

The embodiment of the present disclosure also provides a computer program, which includes computer readable code, and when the computer readable code runs in an electronic device, a processor in the electronic device executes the computer program to implement the method described above.

The disclosed embodiments also provide another computer program product for storing computer readable instructions, which when executed, cause a computer to execute the operations of the rail transit driver motion detection method provided in any one of the above embodiments.

An embodiment of the present disclosure further provides an electronic device, including: one or more processors; a memory for storing executable instructions; wherein the one or more processors are configured to invoke the memory-stored executable instructions to perform the above-described method.

The electronic device may be provided as a terminal, server, or other form of device. For example, the electronic device may be a smart analysis device, an in-vehicle device, or the like.

Fig. 11 illustrates a block diagram of an electronic device 800 provided by an embodiment of the disclosure. For example, the electronic device 800 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, or the like terminal.

Referring to fig. 11, electronic device 800 may include one or more of the following components: processing component 802, memory 804, power component 806, multimedia component 808, audio component 810, input/output (I/O) interface 812, sensor component 814, and communication component 816.

The processing component 802 generally controls overall operation of the electronic device 800, such as operations associated with display, telephone calls, data communications, camera operations, and recording operations. The processing components 802 may include one or more processors 820 to execute instructions to perform all or a portion of the steps of the methods described above. Further, the processing component 802 can include one or more modules that facilitate interaction between the processing component 802 and other components. For example, the processing component 802 can include a multimedia module to facilitate interaction between the multimedia component 808 and the processing component 802.

The memory 804 is configured to store various types of data to support operations at the electronic device 800. Examples of such data include instructions for any application or method operating on the electronic device 800, contact data, phonebook data, messages, pictures, videos, and so forth. The memory 804 may be implemented by any type or combination of volatile or non-volatile memory devices such as Static Random Access Memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic or optical disks.

The power supply component 806 provides power to the various components of the electronic device 800. The power components 806 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the electronic device 800.

The multimedia component 808 includes a screen that provides an output interface between the electronic device 800 and a user. In some embodiments, the screen may include a Liquid Crystal Display (LCD) and a Touch Panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive an input signal from a user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundary of a touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 808 includes a front facing camera and/or a rear facing camera. The front camera and/or the rear camera may receive external multimedia data when the electronic device 800 is in an operation mode, such as a shooting mode or a video mode. Each front camera and rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.

The audio component 810 is configured to output and/or input audio signals. For example, the audio component 810 includes a Microphone (MIC) configured to receive external audio signals when the electronic device 800 is in an operational mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signals may further be stored in the memory 804 or transmitted via the communication component 816. In some embodiments, audio component 810 also includes a speaker for outputting audio signals.

The I/O interface 812 provides an interface between the processing component 802 and peripheral interface modules, which may be keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to: a home button, a volume button, a start button, and a lock button.

The sensor assembly 814 includes one or more sensors for providing various aspects of state assessment for the electronic device 800. For example, the sensor assembly 814 may detect an open/closed state of the electronic device 800, the relative positioning of components, such as a display and keypad of the electronic device 800, the sensor assembly 814 may also detect a change in the position of the electronic device 800 or a component of the electronic device 800, the presence or absence of user contact with the electronic device 800, orientation or acceleration/deceleration of the electronic device 800, and a change in the temperature of the electronic device 800. Sensor assembly 814 may include a proximity sensor configured to detect the presence of a nearby object without any physical contact. The sensor assembly 814 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 814 may also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

The communication component 816 is configured to facilitate wired or wireless communication between the electronic device 800 and other devices. The electronic device 800 may access a wireless network based on a communication standard, such as Wi-Fi, 2G, 3G, 4G/LTE, 5G, or a combination thereof. In an exemplary embodiment, the communication component 816 receives a broadcast signal or broadcast related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 816 further includes a Near Field Communication (NFC) module to facilitate short-range communications. For example, the NFC module may be implemented based on Radio Frequency Identification (RFID) technology, infrared data association (IrDA) technology, Ultra Wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

In an exemplary embodiment, the electronic device 800 may be implemented by one or more Application Specific Integrated Circuits (ASICs), Digital Signal Processors (DSPs), Digital Signal Processing Devices (DSPDs), Programmable Logic Devices (PLDs), Field Programmable Gate Arrays (FPGAs), controllers, micro-controllers, microprocessors or other electronic components for performing the above-described methods.

In an exemplary embodiment, a non-transitory computer-readable storage medium, such as the memory 804, is also provided that includes computer program instructions executable by the processor 820 of the electronic device 800 to perform the above-described methods.

Fig. 12 shows a block diagram of an electronic device 1900 provided by an embodiment of the disclosure. For example, the electronic device 1900 may be provided as a server. Referring to fig. 12, electronic device 1900 includes a processing component 1922 further including one or more processors and memory resources, represented by memory 1932, for storing instructions, e.g., applications, executable by processing component 1922. The application programs stored in memory 1932 may include one or more modules that each correspond to a set of instructions. Further, the processing component 1922 is configured to execute instructions to perform the above-described method.

The electronic device 1900 may also include a power component 1926 configured to perform power management of the electronic device 1900, a wired or wireless network interface 1950 configured to connect the electronic device 1900 to a network, and an input/output (I/O) interface 1958. The electronic device 1900 may operate based on an operating system, such as Windows, stored in memory 1932

Mac OS

Or the like.

In an exemplary embodiment, a non-transitory computer readable storage medium, such as the memory 1932, is also provided that includes computer program instructions executable by the processing component 1922 of the electronic device 1900 to perform the above-described methods.

The present disclosure may be systems, methods, and/or computer program products. The computer program product may include a computer-readable storage medium having computer-readable program instructions embodied thereon for causing a processor to implement various aspects of the present disclosure.

The computer readable storage medium may be a tangible device that can hold and store the instructions for use by the instruction execution device. The computer readable storage medium may be, for example, but not limited to, an electronic memory device, a magnetic memory device, an optical memory device, an electromagnetic memory device, a semiconductor memory device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the computer readable storage medium would include the following: a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a Static Random Access Memory (SRAM), a portable compact disc read-only memory (CD-ROM), a Digital Versatile Disc (DVD), a memory stick, a floppy disk, a mechanical coding device, such as punch cards or in-groove projection structures having instructions stored thereon, and any suitable combination of the foregoing. Computer-readable storage media as used herein is not to be construed as transitory signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide or other transmission medium (e.g., optical pulses through a fiber optic cable), or electrical signals transmitted through electrical wires.

The computer-readable program instructions described herein may be downloaded from a computer-readable storage medium to a respective computing/processing device, or to an external computer or external storage device via a network, such as the internet, a local area network, a wide area network, and/or a wireless network. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers and/or edge servers. The network adapter card or network interface in each computing/processing device receives computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in a computer-readable storage medium in the respective computing/processing device.

The computer program instructions for carrying out operations of the present disclosure may be assembler instructions, Instruction Set Architecture (ISA) instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source or object code written in any combination of one or more programming languages, including an object oriented programming language such as Smalltalk, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The computer-readable program instructions may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider). In some embodiments, the electronic circuitry that can execute the computer-readable program instructions implements aspects of the present disclosure by utilizing the state information of the computer-readable program instructions to personalize the electronic circuitry, such as a programmable logic circuit, a Field Programmable Gate Array (FPGA), or a Programmable Logic Array (PLA).

Various aspects of the present disclosure are described herein with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the disclosure. It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer-readable program instructions.

These computer-readable program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks. These computer-readable program instructions may also be stored in a computer-readable storage medium that can direct a computer, programmable data processing apparatus, and/or other devices to function in a particular manner, such that the computer-readable medium storing the instructions comprises an article of manufacture including instructions which implement the function/act specified in the flowchart and/or block diagram block or blocks.

The computer readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other devices to cause a series of operational steps to be performed on the computer, other programmable apparatus or other devices to produce a computer implemented process such that the instructions which execute on the computer, other programmable apparatus or other devices implement the functions/acts specified in the flowchart and/or block diagram block or blocks.

The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of instructions, which comprises one or more executable instructions for implementing the specified logical function(s). In some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

The computer program product may be embodied in hardware, software or a combination thereof. In an alternative embodiment, the computer program product is embodied in a computer storage medium, and in another alternative embodiment, the computer program product is embodied in a Software product, such as a Software Development Kit (SDK), or the like.

Having described embodiments of the present disclosure, the foregoing description is intended to be exemplary, not exhaustive, and not limited to the disclosed embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen in order to best explain the principles of the embodiments, the practical application, or improvements made to the technology in the marketplace, or to enable others of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A rail transit driver action detection method is characterized by comprising the following steps:

acquiring a first video stream of a driving area of a rail vehicle;

2. The method of claim 1,

3. The method of claim 2, wherein determining predetermined standard action information from the traffic signal information ahead of the rail vehicle comprises:

4. The method of claim 3, wherein the obtaining traffic signal information ahead of the rail vehicle comprises: and analyzing the traffic signal of the second video stream in front of the rail vehicle to obtain the traffic signal information in front of the rail vehicle.

5. The method according to any one of claims 2 to 4, characterized in that the traffic signal information in front of the rail vehicle comprises at least one of: the information of the traffic signal lamp in front of the rail vehicle, the information of the gesture traffic guidance signal in front of the rail vehicle, and the information of the protection signal machine in front of the rail vehicle.

6. The method according to any one of claims 2 to 5, wherein the obtaining of traffic signal information ahead of the rail vehicle comprises:

7. The method according to any one of claims 1 to 6, wherein the matching of the driver's action information with predetermined standard action information comprises:

8. The method according to any one of claims 1 to 7, further comprising:

alternatively, the first and second electrodes may be,

9. The method of claim 8, wherein the data relating to the driver's action information further comprises at least one of: the preset standard action information, the traffic signal information in front of the rail vehicle corresponding to the action information of the driver, and the position information of the rail vehicle corresponding to the action information of the driver.

10. The method according to any one of claims 1 to 9, further comprising:

sending first prompt information to the preset driver management equipment;

11. A rail transit driver's action detection device, characterized by comprising:

12. An electronic device, comprising:

one or more processors;

a memory for storing executable instructions;

wherein the one or more processors are configured to invoke the memory-stored executable instructions to perform the method of any one of claims 1 to 10.

13. A computer readable storage medium having computer program instructions stored thereon, which when executed by a processor implement the method of any one of claims 1 to 10.

14. A rail vehicle, comprising:

15. The rail vehicle according to claim 14,

16. Rail vehicle according to claim 14 or 15,

17. The rail vehicle of any one of claims 14 to 16, wherein the intelligent analysis device is further configured to:

alternatively, the first and second electrodes may be,

18. Rail vehicle according to any one of claims 14 to 17,