CN111432229B - Method and device for recording, analyzing and directly broadcasting driving command - Google Patents
Method and device for recording, analyzing and directly broadcasting driving command Download PDFInfo
- Publication number
- CN111432229B CN111432229B CN202010247428.1A CN202010247428A CN111432229B CN 111432229 B CN111432229 B CN 111432229B CN 202010247428 A CN202010247428 A CN 202010247428A CN 111432229 B CN111432229 B CN 111432229B
- Authority
- CN
- China
- Prior art keywords
- module
- video
- time
- audio
- information
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 39
- 238000004458 analytical method Methods 0.000 claims abstract description 135
- 230000002159 abnormal effect Effects 0.000 claims abstract description 23
- 238000004806 packaging method and process Methods 0.000 claims abstract description 8
- 238000005538 encapsulation Methods 0.000 claims abstract description 6
- 230000001360 synchronised effect Effects 0.000 claims abstract description 6
- 230000006399 behavior Effects 0.000 claims description 58
- 238000004891 communication Methods 0.000 claims description 54
- 238000007726 management method Methods 0.000 claims description 35
- 230000008859 change Effects 0.000 claims description 20
- 230000009471 action Effects 0.000 claims description 18
- 238000012163 sequencing technique Methods 0.000 claims description 16
- 238000006243 chemical reaction Methods 0.000 claims description 10
- 239000000284 extract Substances 0.000 claims description 9
- 230000000903 blocking effect Effects 0.000 claims description 7
- 238000005206 flow analysis Methods 0.000 claims description 6
- 238000002372 labelling Methods 0.000 claims description 5
- 238000013459 approach Methods 0.000 claims description 3
- 230000015572 biosynthetic process Effects 0.000 claims description 3
- 238000010586 diagram Methods 0.000 claims description 3
- 238000012216 screening Methods 0.000 claims description 3
- 230000035945 sensitivity Effects 0.000 claims description 3
- 238000003786 synthesis reaction Methods 0.000 claims description 3
- RWSOTUBLDIXVET-UHFFFAOYSA-N Dihydrogen sulfide Chemical compound S RWSOTUBLDIXVET-UHFFFAOYSA-N 0.000 claims 1
- 230000008901 benefit Effects 0.000 abstract description 5
- 230000006870 function Effects 0.000 description 8
- 230000008569 process Effects 0.000 description 6
- 230000009286 beneficial effect Effects 0.000 description 4
- 230000007547 defect Effects 0.000 description 3
- 230000008878 coupling Effects 0.000 description 2
- 238000010168 coupling process Methods 0.000 description 2
- 238000005859 coupling reaction Methods 0.000 description 2
- 238000004519 manufacturing process Methods 0.000 description 2
- 230000005540 biological transmission Effects 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000013079 data visualisation Methods 0.000 description 1
- 238000004141 dimensional analysis Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 238000009434 installation Methods 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 230000002085 persistent effect Effects 0.000 description 1
- 238000004088 simulation Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 239000013589 supplement Substances 0.000 description 1
- 230000001502 supplementing effect Effects 0.000 description 1
- 238000012549 training Methods 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/21—Server components or server architectures
- H04N21/218—Source of audio or video content, e.g. local disk arrays
- H04N21/2187—Live feed
-
- G—PHYSICS
- G04—HOROLOGY
- G04R—RADIO-CONTROLLED TIME-PIECES
- G04R20/00—Setting the time according to the time information carried or implied by the radio signal
- G04R20/02—Setting the time according to the time information carried or implied by the radio signal the radio signal being sent by a satellite, e.g. GPS
- G04R20/04—Tuning or receiving; Circuits therefor
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/40—Scenes; Scene-specific elements in video content
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L15/00—Speech recognition
- G10L15/26—Speech to text systems
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
- H04N21/4402—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display
- H04N21/440236—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display by media transcoding, e.g. video is transformed into a slideshow of still pictures, audio is converted into text
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/47—End-user applications
- H04N21/472—End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content
- H04N21/47217—End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content for controlling playback functions for recorded or on-demand content, e.g. using progress bars, mode or play-point indicators or bookmarks
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/222—Studio circuitry; Studio devices; Studio equipment
- H04N5/262—Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/222—Studio circuitry; Studio devices; Studio equipment
- H04N5/262—Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects
- H04N5/265—Mixing
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/222—Studio circuitry; Studio devices; Studio equipment
- H04N5/262—Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects
- H04N5/278—Subtitling
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/76—Television signal recording
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Physics & Mathematics (AREA)
- Human Computer Interaction (AREA)
- Databases & Information Systems (AREA)
- General Physics & Mathematics (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Health & Medical Sciences (AREA)
- Acoustics & Sound (AREA)
- Radar, Positioning & Navigation (AREA)
- Remote Sensing (AREA)
- Computational Linguistics (AREA)
- Theoretical Computer Science (AREA)
- Train Traffic Observation, Control, And Security (AREA)
Abstract
The invention relates to a method and a device for recording, analyzing and directly broadcasting driving command, comprising the following steps: 1) Dynamic image modeling is carried out on various state meanings of railway signal equipment and a driving plan graph, and standardized flows of driving operation and command expression are defined; 2) Receiving HDMI/VDI digital video signals output by an operation terminal and converting the HDMI/VDI digital video signals into RGB images; collecting voice instructions of commanders; 3) Searching and analyzing abnormal operation and key flow operation on images, voices and characters; 4) Encoding, compressing and packaging related images and voices, and adding corresponding voice subtitles to form a video file set; 5) Performing multipath synchronous playback on the formed video file set; 6) And (3) carrying out video coding encapsulation on the image and the voice formed in the step (2), and pushing the image and the voice to a video live broadcast system. Compared with the prior art, the invention has the advantages of improving the comprehensive analysis capability of the operation behavior, the emergency processing capability of the dispatching command and the like.
Description
Technical Field
The invention relates to a monitoring and retrieval analysis technology of operation behaviors in a railway transportation process, in particular to a method and a device for recording, analyzing and directly broadcasting driving command.
Background
Safety and high efficiency are two persistent topics for traffic control. The achievement of these two topics mainly depends on the functional completeness of the traffic control device and the operating level of the dispatch commander, in particular in emergency situations. The TDCS3.0 and CTC3.0 are applied to realize mishandling alarming and safety clamping control of driving operations such as receiving and sending vehicles on a system level, and a new step is formed in the aspect of safety command. In terms of the operation level of dispatch commanders, dispatch and train training simulation systems are being developed and applied, with the primary objective of improving operator understanding of driving codes and the level of operation under abnormal conditions.
But in the actual transportation command process, how does the actual level of the traffic dispatch command? How does a commander operate the device, give commands to command the vehicle to travel under various emergency and abnormal conditions? In this respect, a certain method and equipment are urgently needed in the railway transportation department to find out the violations, improper or unreasonable operations in the actual process in time, so as to inform operators to correct in time, and prevent potential safety hazards.
Chinese patent publication No. CN110110983a discloses an automatic analysis system and method for operation behavior in the process of railway transportation scheduling. The method comprises the following steps: acquiring state change information of CTC signal equipment, interlocking route sequence information, CTC operation alarm information and dispatcher operation information; identifying and analyzing the acquired data, retrieving the operation behaviors with dangerous hidden dangers and related alarms generated by the behaviors, and establishing the association relationship between the operation behaviors and the alarms; automatically analyzing state change information of signal equipment and identifying specific operation behaviors with dangerous hidden danger; filtering dangerous operation behavior records in non-production states of construction and exercise; intuitively displaying the counted operation behaviors with dangerous hidden danger through a data visualization means of the chart; providing a one-key playback function for the retrieval analysis process, and playing back relevant information when the operation behavior occurs, wherein the relevant information comprises signal equipment state change information, route sequence change information and alarm change information, and is used for truly restoring the site situation when the operation behavior occurs; the patent can identify certain improper operation behaviors and functions for analyzing the operation behaviors, but has certain limitations, and is characterized in that: 1) The method takes alarm information generated by CTC as clues, the behavior retrieval range depends on the type of the alarm information and the integrity of the information, and an unreasonable operation behavior, such as the retrieval related to a dispatching command, cannot be identified under the condition that the alarm information is incomplete and a TDCS system without the alarm information. 2) The method cannot record and play back the operation process of the operator, cannot play back menu operation and button operation, namely does not know how the operator operates, and is not beneficial to carrying out specific analysis on operation behaviors and habits. 3) The method can only play back the station diagram, and can not record other terminal scenes related to the operation, so that the method can not play back, and can not analyze the correlation of logic before and after the operation. 4) The method does not record voice command instructions of operators and can not carry out standardized recognition on the command instructions. 5) The method cannot realize live broadcast of field operation and cannot complete a remote command function.
In summary, how to perform whole-course undistorted recording, live broadcasting, playback and comprehensive three-dimensional comprehensive analysis of all-dimensional scenes on various operation behaviors and intentions of operators, and improve the service level of the operators are important problems for ensuring the safety, high efficiency and orderly driving operation of trains.
Disclosure of Invention
The invention aims to overcome the defects of the prior art and provide a method and a device for recording, analyzing and directly broadcasting driving command.
The aim of the invention can be achieved by the following technical scheme:
a method for recording, analyzing and directly broadcasting driving command comprises the following steps:
Step 1), dynamic image modeling is carried out on various state meanings of railway signal equipment and a driving plan diagram, and standardized flows of driving operation and command expression are defined;
Step 2) receiving HDMI/VDI digital video signals output by an operation terminal and converting the HDMI/VDI digital video signals into RGB images; collecting voice instructions of commanders and converting the voice instructions into text information;
step 3) carrying out abnormal operation and retrieval and analysis of key flow operation on the images, voices and characters obtained in the step 2) according to the model and flow defined in the step 1), and recording analysis results;
Step 4) according to the analysis result of the step 3), encoding, compressing and packaging related images and voices, and simultaneously adding corresponding voice subtitles to form a group of video file sets for post analysis and viewing;
Step 5) carrying out multipath synchronous playback on the video file set formed in the step 4), and finding out equipment elements, menu operations and file contents corresponding to unreasonable operations to mark on a video playback picture according to an analysis result;
And 6) carrying out video coding encapsulation on the image and the voice formed in the step 2), then pushing the image and the voice to a video live broadcast system, and carrying out video live broadcast on the live command operation scene.
Preferably, the dynamic image modeling of step 1) includes: the meanings of various states of the annunciator, the turnout, the stock way, the route, the train, the power supply arm and the blocking area and the train running line are expressed in a graphical mode; when the route is established and unlocked, the front and back sequence of the turnout and the annunciator are changed, the change of the track section and the change of the train number are expressed by the relation between time sequence images when the train moves, and the time sequence images are used for video screening;
The standardized flow definition is to explain various operation flows and operation terms related to receiving and dispatching vehicles, equipment faults and dispatching commands, and is used for checking actual operation conditions.
Preferably, when the video signal is converted into RGB image in the step 2), a system clock is acquired from the TDCS/CTC satellite timing system, the image size and frame rate information are acquired from the HDMI/VDI signal, a continuous image set is formed, and time labeling is performed on each frame of image;
step 2) acquiring a system clock from a TDCS/CTC satellite time service system when collecting voice of a commander, and recording the starting time and the ending time of the voice; when converting speech into text information, the start time and end time of the text are recorded.
Preferably, the step 3) performs searching and analysis of abnormal operation and key flow operation on the images, the voice and the text, and records the analysis result specifically as follows:
31 Forming an image sequence according to time sequencing of images output by the station yard graph display terminal, and forming an image sequence according to time sequencing of images output by the planning terminal;
32 Scanning the image sequence, identifying alarm information in an alarm information frame, and extracting equipment state information, operation information and time;
identifying the fault type and number of the signal equipment according to the dynamic fault model of the signal equipment, and recording the time when the fault starts;
Identifying the scheduled command to be written, issued, signed and received operation and the type of the scheduled command, and recording the corresponding time;
33 For abnormal operation of equipment faults, using equipment fault time as a starting point, identifying the time of receiving and sending a vehicle and handling the fault equipment, identifying menu operation commands, button operation commands and time, extracting voice commands from voice word records according to the time, and sequencing the events according to time order to form an abnormal operation action set under the condition of equipment faults;
34 For operation of the scheduling command, identifying the content of the scheduling command and checking with the standard term of the scheduling command; extracting the command sending and receiving person, finding out the image of the related personnel operation terminal according to the time, identifying the operation of the command menu, the button operation of the operator, recording the time, extracting the voice command in the voice text record according to the time, and sequencing the events according to the time order to form an action set of the related operation of the command;
35 Comparing the action sets in step 33) and step 34) with the normal operation flow to form an analysis result, wherein the analysis result information comprises the time of operation, operators and conclusions, the conclusions are consistent or inconsistent, and for inconsistent conclusions, the contents of the basis equipment such as elements, menus, characters and the like for judging the inconsistency are recorded.
Preferably, the video file set formed in the step 4) is that when related images are output from HDMI/VGI signals of different terminals, the images are separately encoded to form video files; when there are multiple voices and subtitles, the voices and subtitles are added into different channels; all videos are used for synchronizing multiple paths of videos during playback by taking clock information of a TDCS/CTC satellite time service system when the videos occur as a basis;
Preferably, the video coding parameters in the step 6) are configurable, including configuring the code rate, frame rate, encoder and video format of the video, so as to ensure the smoothness and definition of playing.
A device for recording, analyzing and live broadcasting driving command, comprising: the system comprises a TDCS/CTC satellite time service system, a TDCS/CTC operation terminal, a voice acquisition module, an image recognition and rule module, an audio and video recording and pushing module, an operation behavior analysis module, an audio and video live broadcast system, a file storage management module and a multipath video playing analysis module;
the voice acquisition module is connected with the TDCS/CTC satellite time service system and is used for receiving a TDCS/CTC system clock;
the audio and video recording and pushing module is connected with the TDCS/CTC operation terminal and is used for receiving HDMI/VDI digital video signals;
The audio and video recording plug flow module is connected with the voice acquisition module and is used for receiving real-time voice data;
The audio and video recording plug flow module is connected with the TDCS/CTC satellite time service system and is used for receiving a TDCS/CTC system clock;
The audio and video recording plug flow module is connected with the file storage management module and is used for sending audio and video files;
The audio and video recording and pushing module is connected with the audio and video live broadcasting system and used for pushing the video stream of the operation terminal;
the operation behavior analysis module is connected with the image recognition and rule module and is used for acquiring railway signal equipment information, route establishment and unlocking, train movement information and standard operation flow information;
The operation behavior analysis module is connected with the audio and video recording and pushing module and is used for receiving the graphic set, the audio and the characters of the operation terminal and sending file storage information;
The operation behavior analysis module is connected with the file storage management module and is used for sending analysis result information;
The multi-channel video playing analysis module is connected with the audio-video live broadcast system and is used for receiving live video streams of all operation terminals;
the multi-channel video playing analysis module is connected with the audio and video recording plug flow module and is used for sending live broadcast selection commands;
The multi-channel video playing analysis module is connected with the file storage management module and is used for sending a playing selection command; receiving a playback video file;
The multipath video playing and analyzing module is connected with the image recognition and rule module and is used for obtaining railway signal equipment, a route and a chart recognition model.
Preferably, the voice acquisition module comprises a first information communication module and a recording setting module;
The first information communication module is connected with the TDCS/CTC satellite time service system interface, acquires system time information, and sends audio data, system time when recording starts and recording equipment ID to the audio-video recording plug flow module;
the voice setting module is used for setting a recording mode, microphone sensitivity and sound control recording, wherein the recording mode comprises mono and double sound channels and sound quality.
Preferably, the audio and video recording and pushing module comprises a second information communication module, a signal conversion module, a subtitle module, a video editing module and an encapsulation pushing module;
The second information communication module includes: an interface with a TDCS/CTC satellite time service system for acquiring system time information; the interface is used for acquiring real-time audio acquisition data; the interface is connected with the multipath video playing analysis module and is used for receiving a dispatching desk or station live broadcast selection control instruction to be played; the interface is used for sending the video live stream; the interface is connected with the operation behavior analysis module and is used for sending the image sequence set and receiving the stored file information; the interface is connected with the file storage management module and is used for sending video files;
The signal conversion module is provided with an interface with the TDCS/CTC terminal and is used for receiving HDMI/VDI signal data, the signal conversion module extracts image size and frame rate information, simultaneously extracts RGB or YUV channel image data, synthesizes a complete RGB image sequence, calculates the actual time of image generation according to the obtained TDCS/CTC system clock and image frame rate and marks the actual time, and caches the generated image sequence set and sends the image sequence set to the operation behavior analysis module through the second information communication module;
the caption module converts recorded voice into characters, sets the time corresponding to the characters according to the system time of the beginning of the voice and voice parameters, and converts the characters into caption information; identifying a speaker according to the recording equipment ID and adding the speaker to the caption, and generating voice and the caption to the operation behavior analysis module through the second information communication module;
The video editing module is used for setting the size, frame rate and encoder of a video image to be generated, selecting an image sequence set, audio and subtitles in a cache according to time information given by the operation behavior analysis module to perform video synthesis, adding different channels into the audio and the subtitles when a plurality of audio and subtitles exist at the same moment, guaranteeing the integrity of information, adding the starting time of video occurrence to a synthesized video file, and then sending the video file to the file storage management module for storage through the second information communication module;
The encapsulation plug flow module carries out video coding on the image sequence set, the audio frequency and the caption in the buffer memory in real time according to the live broadcast requirement, encapsulates the image sequence set, the audio frequency and the caption into a streaming media protocol data packet and pushes the streaming media protocol data packet to the audio and video live broadcast system in real time through the second information communication module.
Preferably, the image recognition and rule module comprises a third information communication module, a dynamic graph modeling module and a standard flow definition module;
The third information communication module includes: the interface is used for sending an identification judgment basis; the interface is used for sending an identification judgment basis;
the dynamic image modeling module expresses various states of the annunciator, the turnout, the stock way, the approach, the train, the power supply arm, the blocking area and the plan chart by using actually displayed graphs, and expresses the meanings of the states by using characters; when the train route and the shunting route are established and unlocked, the front and back sequences of the turnout, the annunciator and the train are changed, the change of the track section and the change of the train number are expressed by a group of image sequences when the train moves, and the characteristics of the train are identified through the change between images;
The standard flow definition module expresses key business operations of various abnormal operations and scheduling commands under equipment faults in the form of operation steps, including which posts are to be notified by telephone, which registrations are to be made, which equipment operations are to be made and the precedence relationship among the operations.
Preferably, the operation behavior analysis module comprises a fourth information communication module, a data scheduling module, an operation identification module and an operation flow analysis module;
The fourth information communication module comprises an interface with the image recognition and rule module and is used for acquiring a retrieval recognition judgment basis; the interface is connected with the audio and video recording plug flow module and is used for receiving audio and video and subtitle data and sending file storage information; the interface is used for sending the analysis result file;
the data scheduling module groups the audio and video subtitle data according to the ID of the operation terminal, synthesizes the audio and video subtitle data belonging to the same scheduling station or station into a group, and inputs the data into the operation identification module with the same ID;
The operation identification module scans the image sequence, identifies alarm information in an alarm information frame, and extracts equipment state information, operation information and time; identifying the fault type and number of the signal equipment according to the dynamic fault model of the signal equipment, and recording the time when the fault starts; identifying the scheduled command to be written, issued, signed and received operation and the type of the scheduled command, and recording the corresponding time; for abnormal operation of equipment failure, taking equipment failure time as a starting point, identifying the receiving and sending vehicle access and time of handling the equipment failure, identifying menu operation commands, button operation commands and time, extracting voice commands from voice character records according to the times, and sequencing the events according to time order to form an abnormal operation action set under the condition of equipment failure; for the operation of the scheduling command, identifying the content of the scheduling command and checking with the standard term of the scheduling command; extracting the command sending and receiving person, finding out the image of the related personnel operation terminal according to the time, identifying the operation of the command menu, the button operation of the operator, recording the time, extracting the voice command in the voice text record according to the time, and sequencing the events according to the time order to form an operation action set related to the command;
The operation flow analysis module compares the operation action set with the normal operation flow to form an analysis result, wherein analysis result information comprises the time of operation occurrence, an operator and a conclusion, corresponding equipment elements, menu operation and file content are recorded for the conclusion inconsistent with the operation standard, and the analysis result file is sent to the file storage management module through the fourth information communication module.
Preferably, the file storage management module comprises a fifth information communication module and a file management module;
the fifth information communication module includes: the interface is connected with the audio and video recording plug flow module and is used for receiving audio and video files; the interface is used for receiving the analysis result file; the interface is connected with the multipath video playing analysis module and is used for receiving application information of playing a dispatching desk or station video playback, and sending a video file and an analysis result file;
The file management module forms a group of audio and video files and analysis result files belonging to the same dispatching desk or station, and stores the audio and video files and the analysis result files according to time sequence; and selecting all audio and video files and analysis results meeting the requirements according to the dispatching desk, station and time period information provided by the multi-channel video playing module, and sending the audio and video files and the analysis results to the multi-channel video playing analysis module through the fifth information communication module.
Preferably, the multi-path video playing and analyzing module comprises a sixth information communication module, a playing module and an analyzing and labeling module;
The sixth information communication module includes: the interface is used for sending application information for live broadcasting of the dispatching desk or station video; the interface is used for acquiring a plurality of video live streams; the interface is used for sending application information for video playback of the dispatching desk or the station, receiving video files and analysis result files; the interface is used for receiving the identification judgment basis;
the playing module is used for simultaneously caching and decoding a plurality of received video streams or video file sets, synchronously playing a plurality of videos according to the same playing clock, playing voices of different audio channels and displaying different subtitles according to the need;
The analysis marking module analyzes the analysis result file, finds out equipment, an operation menu, text information and occurrence time which need marking, finds out corresponding elements in an image buffer area of video decoding according to the identification judgment basis, records corresponding display positions of the elements, and marks various boxes and lines which are defined in advance at the corresponding positions when the frame images are displayed.
Compared with the prior art, the invention has the following advantages:
1) The invention adopts the method of recording all operation behaviors and equipment states of the operation terminal in real time by converting HDMI/VDI signals into images, solves the problems of memory, performance and resource consumption caused by hard disk read-write and further influences on safe production caused by the installation of recording software on the operation terminal, fills the blank that the current driving record playback lacks menu operation, keyboard operation and mouse operation of operators, plays an important role in supplementing on-site information, and lays a foundation for analysis of operation behaviors. Secondly, the method collects the voices of a plurality of traffic commanders, converts the voices into words, synthesizes the images, the voices and the words into videos according to a unified system clock, divides a plurality of related videos into a group for storage, and performs whole-course undistorted recording and playback, thereby solving the problems of scattered and lack of correlation and difficult storage management of field information, and being beneficial to comprehensive analysis of operation behaviors in an omnibearing manner. In addition, the method carries out dynamic image modeling through elements such as railway signal equipment, a driving plan chart and the like, defines standard flow of driving operation and voice instructions, can find unreasonable operation behaviors in real time through voice recognition and image recognition methods, solves the problem of coupling between the conventional retrieval method and data and protocols of various factories, and has the advantages of strong universality, wide retrieval range and easiness in expansion. Furthermore, the method realizes the video live broadcast function of the field operation terminal interface and the driving command voice based on the real operation terminal interface acquired in real time, fills the blank that the driving command has no remote live broadcast, and is beneficial to the safe clamping control of the operation guidance of the field operation personnel by experienced personnel through watching live broadcast pictures under emergency conditions. Finally, the method realizes synchronous playing of multiple paths of video data by a method based on a unified system clock of all video streams and video files, and identifies images according to image models and flow definitions and highlights equipment, menu operations and command operations related to unreasonable operation behaviors while playing, thereby facilitating analysis.
2) The device is formed on the basis of the invention, and the device is characterized in that a voice acquisition module is added on the basis of the traditional TDCS/CTC satellite time service system, a TDCS/CTC operation terminal and an audio-video live broadcast system to complete voice acquisition and transmission of traffic commanders; adding an audio and video recording and pushing module, completing the conversion from an operation terminal HDVI/VDI signal to an image signal, completing the conversion from voice data to text data, completing the real-time audio and video coding of images, voice and subtitles, completing the pushing of an audio and video stream according to a live broadcast command, and completing the sending of a video file according to a stored file command; the method comprises the steps of adding an image recognition and rule module to complete the definition of dynamic graph modeling and driving command behavior standard flow of annunciators, turnouts, tracks, routes, trains, power supply arms, blocking areas, planning charts, routes, train movement and train number tracking, and the basis for voice recognition, image recognition and unreasonable behavior recognition; the added operation behavior analysis module is used for completing scanning of the image sequence, identifying unreasonable operation behaviors, marking related equipment elements, menu operation and button operation, completing information recording and driving record analysis files; adding a file storage management module, completing grouping storage of audio and video files and analysis result files according to the correlation and time, and sending grouped files according to a playing instruction; and adding a multi-path video playing analysis module to finish the caching and decoding of a plurality of videos, finishing the synchronous playing of the videos according to the same clock, finding out corresponding elements in an image cache area of video decoding according to an analysis result file and an identification judgment basis, and marking the elements during playing. Because the device is provided with the equipment and the functional modules, the defect that the menu operation, button operation and command voice recording function of operators cannot be realized in the prior art is completely overcome, the whole-process undistorted whole-process recording is realized, and the comprehensive three-dimensional analysis of the behaviors of the operators for operating the TDCS/CTC system is facilitated; meanwhile, the device realizes the retrieval function of operation behaviors by utilizing a video analysis method, overcomes the defects of tight coupling and serious dependence on equipment in the prior art and has the advantages of strong universality, small investment and easy function expansion; finally, the device can synchronously play back live broadcast and multiple videos of field operation, and mark unreasonable operation behaviors on a play picture, so that experience personnel can remotely conduct operation guidance on field operators by watching live broadcast pictures under emergency conditions, and the accuracy of operation is ensured. The device is applied, is beneficial to supplement of a TDCS/CTC dispatching supervision function, can improve the operation level and emergency command level of operators, ensures transportation safety and improves transportation efficiency, and has great economic and social benefits.
Drawings
Fig. 1 is a schematic structural view of the present invention.
Wherein 1 is TDCS/CTC satellite time service system, 2 is TDCS/CTC operation terminal, 3 is voice collection module, 4 is image recognition and rule module, 5 is audio and video recording and pushing module, 6 is operation behavior analysis module, 7 is audio and video live broadcast system, 8 is file storage management module, 9 is multichannel video play analysis module.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and fully with reference to the accompanying drawings, in which it is evident that the embodiments described are some, but not all embodiments of the invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the present invention without making any inventive effort, shall fall within the scope of the present invention.
As shown in fig. 1, a device for recording, analyzing and directly broadcasting traffic command includes: the system comprises a TDCS/CTC satellite time service system 1, a TDCS/CTC operation terminal 2, a voice acquisition module 3, an image recognition and rule module 4, an audio and video recording and pushing module 5, an operation behavior analysis module 6, an audio and video live broadcast system 7, a file storage management module 8 and a multipath video playing analysis module 9.
The voice acquisition module is connected with the TDCS/CTC satellite time service system and is used for receiving a TDCS/CTC system clock; the audio and video recording and pushing module is connected with the TDCS/CTC operation terminal and is used for receiving HDMI/VDI digital video signals; the audio and video recording plug flow module is connected with the voice acquisition module and is used for receiving real-time voice data; the audio and video recording plug flow module is connected with the TDCS/CTC satellite time service system and is used for receiving a TDCS/CTC system clock; the audio and video recording plug flow module is connected with the file storage management module and is used for sending audio and video files; the audio and video recording and pushing module is connected with the audio and video live broadcasting system and used for pushing the video stream of the operation terminal; the operation behavior analysis module is connected with the image recognition and rule module and is used for acquiring railway signal equipment information, route establishment and unlocking, train movement information and standard operation flow information; the operation behavior analysis module is connected with the audio and video recording and pushing module and is used for receiving the graphic set, the audio and the characters of the operation terminal; transmitting file storage information; the operation behavior analysis module is connected with the file storage management module and is used for sending analysis result information; the multi-channel video playing analysis module is connected with the audio-video live broadcast system and is used for receiving live video streams of all operation terminals; the multi-channel video playing analysis module is connected with the audio and video recording plug flow module and is used for sending live broadcast selection commands; the multi-path video playing analysis module is connected with the file storage management module and is used for sending a playing selection command and receiving a playback video file; the multi-path video playing analysis module is connected with the image recognition and rule module and is used for obtaining railway signal equipment, a route and a chart recognition model;
The voice acquisition module comprises a first information communication module and a recording setting module; the first information communication module is connected with the TDCS/CTC satellite time service system interface, acquires system time information, and sends audio data, system time when recording starts and recording equipment ID to the audio-video recording plug flow module; the voice setting module is used for setting a recording mode (mono-double sound channel and sound quality), microphone sensitivity and sound control recording;
The audio and video recording and pushing module comprises a second information communication module, a signal conversion module, a subtitle module, a video editing module and a packaging pushing module; the second information communication module is connected with the TDCS/CTC satellite time service system interface to acquire system time information; the voice acquisition module is connected with the voice acquisition module to acquire real-time audio acquisition data; the interface is connected with a multi-path video playing analysis module, and receives a dispatching desk or station live broadcast selection control instruction to be played; the method comprises the steps of interfacing with an audio and video live broadcast system, and sending a video live broadcast stream; the interface is connected with the operation behavior analysis module, the image sequence set is sent, and the stored file information is received; the interface is connected with the file storage management module to send video files; the signal conversion module is connected with the TDCS/CTC terminal interface, receives HDMI/VDI signal data, extracts image size and frame rate information, extracts RGB or YUV channel image data, synthesizes a complete RGB image sequence, calculates the actual time of image generation according to the TDCS/CTC system clock and the image frame rate, marks the actual time, caches the generated image sequence set and sends the image sequence set to the operation behavior analysis module through the second information communication module; the caption module converts recorded voice into characters, sets the time corresponding to the characters according to the system time of the beginning of the voice and voice parameters, and converts the characters into caption information; identifying a speaker according to the recording equipment ID and adding the speaker to the caption, and generating voice and the caption to the operation behavior analysis module through the second information communication module; the video editing module is used for setting the size, the frame rate and the encoder of a video image to be generated, selecting an image sequence set, audio and subtitles in a cache according to time information given by the operation behavior analysis module to perform video synthesis, adding different channels into the audio and the subtitles when a plurality of audio and subtitles exist at the same moment, guaranteeing the integrity of information, adding the starting time of video occurrence to a synthesized video file, and then sending the video file to the file storage management module for storage through the second information communication module; the encapsulation plug flow module carries out video coding on the image sequence set, the audio and the caption in the cache in real time according to the live broadcast requirement, encapsulates the image sequence set, the audio and the caption into a streaming media protocol data packet and pushes the streaming media protocol data packet to the audio and video live broadcast system in real time through the second information communication module;
The image recognition and rule module comprises a third information communication module, a dynamic graph modeling module and a standard flow definition module; the third information communication module is connected with the operation behavior analysis module and sends an identification judgment basis; the interface is connected with the multi-path video playing analysis module, and an identification judgment basis is sent; the dynamic image modeling module expresses various states of the annunciator, the turnout, the stock way, the approach, the train, the power supply arm, the blocking area and the plan chart by using actually displayed graphs, and expresses the meanings of the states by using characters; when the train route and the shunting route are established and unlocked, the front and back sequences of the turnout, the annunciator and the train are changed, the change of the track section and the change of the train number are expressed by a group of image sequences when the train moves, and the characteristics of the train are identified through the change between images; the standard flow definition is expressed in the form of operation steps for various abnormal operation and scheduling command and other key business operations under equipment failure, including which posts are to be notified by telephone, which registrations are to be made, which equipment operations are to be made, and the precedence relationship between the operations;
The operation behavior analysis module comprises a fourth information communication module, a data scheduling module, an operation identification module and an operation flow analysis module; the fourth information communication module is connected with the image recognition and rule module to acquire a retrieval recognition judgment basis; receiving audio and video and subtitle data through an interface of the audio and video recording and pushing module, and sending file storage information; the interface is connected with the file storage management module, and an analysis result file is sent; the data scheduling module groups the audio and video caption data according to the ID of the operation terminal and synthesizes a group of audio and video caption data belonging to one scheduling station or station. And inputting the data into an operation recognition module having the same ID; the operation identification module scans the image sequence, identifies alarm information in an alarm information frame, and extracts equipment state information, operation information and time; identifying the fault type and number of the signal equipment according to the dynamic fault model of the signal equipment, and recording the time when the fault starts; identifying the scheduled command to be written, issued, signed and received operation and the type of the scheduled command, and recording the corresponding time; for abnormal operation of equipment failure, taking equipment failure time as a starting point, identifying the receiving and sending vehicle access and time of handling the equipment failure, identifying menu operation commands, button operation commands and time, extracting voice commands from voice character records according to the times, and sequencing the events according to time order to form an abnormal operation action set under the condition of equipment failure; for the operation of the scheduling command, identifying the content of the scheduling command and checking with the standard term of the scheduling command; extracting the command sending and receiving person, finding out the image of the related personnel operation terminal according to the time, identifying the operation of the command menu, the button operation of the operator, recording the time, extracting the voice command in the voice text record according to the time, and sequencing the events according to the time order to form an action set of the related operation of the command; the operation flow analysis module compares the operation action set with the normal operation flow to form an analysis result. The analysis result information comprises the time of operation, operators and conclusions, corresponding equipment elements, menu operations and file contents are recorded for the conclusions inconsistent with the operation standard, and analysis result files are sent to the file storage management module through the fourth information communication module;
The file storage management module comprises a fifth information communication module and a file management module; the fifth information communication module is connected with the audio and video recording plug flow module to receive the audio and video files; the interface is connected with the operation behavior analysis module and used for receiving an analysis result file; the interface is connected with the multipath video playing analysis module, receives application information of playing a dispatching desk or station video playback, and sends a video file and an analysis result file; the file management module forms a group of audio and video files and analysis result files belonging to the same dispatching desk or station, and stores the audio and video files and the analysis result files according to time sequence; according to the information of the dispatching desk, the station and the time period provided by the multi-channel video playing module, all audio and video files and analysis results meeting the requirements are selected and sent to the multi-channel video playing analysis module through the fifth information communication module;
The multi-channel video playing and analyzing module comprises a sixth information communication module, a playing module and an analyzing and labeling module; the sixth information communication module is connected with the audio and video recording and pushing module and is used for sending application information for live broadcasting of the dispatching desk or station video; the method comprises the steps of interfacing with an audio and video live broadcast system to obtain a plurality of video live broadcast streams; the interface is connected with the file management module, transmits application information for video playback of a dispatching desk or a station, and receives a video file and an analysis result file; the interface is connected with the image recognition and rule module and receives a recognition judgment basis; the playing module is used for simultaneously caching and decoding a plurality of received video streams or video file sets, synchronously playing a plurality of videos according to the same playing clock, playing voices of different audio channels and displaying different subtitles according to the need; the analysis marking module analyzes the analysis result file, finds out equipment, an operation menu, text information and occurrence time which need marking, finds out corresponding elements in an image buffer area of video decoding according to the identification judgment basis, records corresponding display positions of the elements, and marks various boxes and lines which are defined in advance at the corresponding positions when the frame images are displayed.
The invention discloses a method for recording, analyzing and directly broadcasting driving command, which comprises the following steps:
1) Dynamic image modeling is carried out on various state meanings of railway signal equipment and a driving plan graph, and standardized flow of driving operation and command expression is defined; the dynamic image modeling is to express various states of a signal machine, a turnout, a stock way, a route, a train, a power supply arm and a blocking area in a graphical mode; when the route is established and unlocked, the front-back sequence of the turnout and the annunciator is changed, the change of the track section and the change of the train number are expressed by the relation between time sequence images when the train moves, and the time sequence images are used for video screening; the standardized flow definition is to explain various operation flows and operation terms related to receiving and dispatching vehicles, equipment faults and dispatching commands and is used for checking with actual operation conditions;
2) Receiving HDMI/VDI digital video signals output by an operation terminal and converting the HDMI/VDI digital video signals into RGB images; collecting voice instructions of commanders and converting the voice instructions into text information; when converting video signals into RGB images, acquiring a system clock from a TDCS/CTC satellite time service system, acquiring image size and frame rate information from HDMI/VDI signals, forming a continuous image set, and performing time marking on each frame of image; when collecting the voice of commander, acquiring a system clock from a TDCS/CTC satellite time service system, and recording the voice start time and the voice end time; when converting voice into character information, recording the starting time and ending time of characters;
3) Searching and analyzing abnormal operation and key flow operation of images, voices and characters according to the model and the flow defined in the step 1), and recording analysis results; in particular to
A. The images output by the station yard graph display terminal are sequenced according to time to form an image sequence, and the images output by the planning terminal are sequenced according to time to form an image sequence;
b. Scanning the image sequence, identifying alarm information in an alarm information frame, and extracting equipment state information, operation information and time; identifying the fault type and number of the signal equipment according to the dynamic fault model of the signal equipment, and recording the time when the fault starts; identifying the scheduled command to be written, issued, signed and received operation and the type of the scheduled command, and recording the corresponding time;
c. For abnormal operation of equipment failure, taking equipment failure time as a starting point, identifying the receiving and sending vehicle access and time of handling the equipment failure, identifying menu operation commands, button operation commands and time, extracting voice commands from voice character records according to the times, and sequencing the events according to time order to form an abnormal operation action set under the condition of equipment failure;
d. For the operation of the scheduling command, identifying the content of the scheduling command and checking with the standard term of the scheduling command; extracting the command sending and receiving person, finding out the image of the related personnel operation terminal according to the time, identifying the operation of the command menu, the button operation of the operator, recording the time, extracting the voice command in the voice text record according to the time, and sequencing the events according to the time order to form an action set of the related operation of the command;
e. Comparing the operation action set with the normal operation flow to form an analysis result. The analysis result information comprises the time of operation, operators and conclusion (consistent or inconsistent), and for inconsistent conclusion, the content according to the equipment elements, menus, characters and the like for inconsistent judgment is recorded.
4) According to the analysis result, encoding, compressing and packaging related images and voices, and adding corresponding voice subtitles to form a video file set for post analysis and viewing; a group of video file sets is formed, namely when related images are output from HDMI/VGI signals of different terminals, the images are independently encoded to form video files; when there are multiple voices and subtitles, the voices and subtitles are added into different channels; all videos are used for synchronizing multiple paths of videos during playback by taking clock information of a TDCS/CTC satellite time service system when the videos occur as a basis;
5) Carrying out multipath synchronous playback on the formed video file set, and finding out equipment elements, menu operations and file contents corresponding to unreasonable operations to mark on a video playback picture according to an analysis result;
6) And (3) video encoding and packaging the image and the audio formed in the step (2), pushing the image and the audio to a video live broadcast system, and carrying out video live broadcast on the live command operation scene. The video coding parameters can be configured to ensure the smoothness and definition of the playing. The video coding parameters include the code rate, frame rate, encoder, video format of the video.
While the invention has been described with reference to certain preferred embodiments, it will be understood by those skilled in the art that various changes and substitutions of equivalents may be made and equivalents will be apparent to those skilled in the art without departing from the scope of the invention. Therefore, the protection scope of the invention is subject to the protection scope of the claims.
Claims (10)
1. The method for recording, analyzing and directly broadcasting the driving command is characterized by comprising the following steps of:
Step 1), dynamic image modeling is carried out on various state meanings of railway signal equipment and a driving plan diagram, and standardized flows of driving operation and command expression are defined;
Step 2) receiving HDMI/VDI digital video signals output by an operation terminal and converting the HDMI/VDI digital video signals into RGB images; collecting voice instructions of commanders and converting the voice instructions into text information;
step 3) carrying out abnormal operation and retrieval and analysis of key flow operation on the images, voices and characters obtained in the step 2) according to the model and flow defined in the step 1), and recording analysis results;
Step 4) according to the analysis result of the step 3), encoding, compressing and packaging related images and voices, and simultaneously adding corresponding voice subtitles to form a group of video file sets for post analysis and viewing;
Step 5) carrying out multipath synchronous playback on the video file set formed in the step 4), and finding out equipment elements, menu operations and file contents corresponding to unreasonable operations to mark on a video playback picture according to an analysis result;
Step 6) video coding and packaging the image and the voice formed in the step 2), and then pushing the image and the voice to a video live broadcast system to conduct video live broadcast on the live command operation scene;
Step 3) search and analysis of abnormal operation and key flow operation are carried out on images, voices and characters, and recording of analysis results is specifically as follows:
31 Forming an image sequence according to time sequencing of images output by the station yard graph display terminal, and forming an image sequence according to time sequencing of images output by the planning terminal;
32 Scanning the image sequence, identifying alarm information in an alarm information frame, and extracting equipment state information, operation information and time;
identifying the fault type and number of the signal equipment according to the dynamic fault model of the signal equipment, and recording the time when the fault starts;
Identifying the scheduled command to be written, issued, signed and received operation and the type of the scheduled command, and recording the corresponding time;
33 For abnormal operation of equipment faults, using equipment fault time as a starting point, identifying time for receiving and sending a vehicle and handling the route through the faulty equipment, identifying menu operation commands, button operation commands and time, extracting voice commands from voice word records according to the time, and sequencing the events according to time order to form an abnormal operation action set under the condition of equipment faults;
34 For operation of the scheduling command, identifying the content of the scheduling command and checking with the standard term of the scheduling command; extracting the command sending and receiving person, finding out the image of the related personnel operation terminal according to the time, identifying the operation of the command menu, the button operation of the operator, recording the time, extracting the voice command in the voice text record according to the time, and sequencing the events according to the time order to form an action set of the related operation of the command;
35 Comparing the action sets in the step 33) and the step 34) with a normal operation flow to form an analysis result, wherein the analysis result information comprises the time of operation, operators and conclusions, the conclusions are consistent or inconsistent, and for inconsistent conclusions, the consistency judgment basis such as equipment elements, menus, characters and the like is recorded.
2. The method for recording, analyzing and broadcasting traffic command according to claim 1, wherein the dynamic image modeling comprises: the meanings of various states of the annunciator, the turnout, the stock way, the route, the train, the power supply arm and the blocking area and the train running line are expressed in a graphical mode; when the route is established and unlocked, the front and back sequence of the turnout and the annunciator are changed, the change of the track section and the change of the train number are expressed by the relation between time sequence images when the train moves, and the time sequence images are used for video screening;
The standardized flow definition is to explain various operation flows and operation terms related to receiving and dispatching vehicles, equipment faults and dispatching commands, and is used for checking actual operation conditions.
3. The method for recording, analyzing and directly broadcasting traffic command according to claim 1, wherein when the video signal is converted into RGB image in the step 2), a system clock is acquired from a TDCS/CTC satellite timing system, image size and frame rate information are acquired from HDMI/VDI signals, a continuous image set is formed, and time labeling is performed on each frame of image;
step 2) acquiring a system clock from a TDCS/CTC satellite time service system when collecting voice of a commander, and recording the starting time and the ending time of the voice; when converting speech into text information, the start time and end time of the text are recorded.
4. The method for recording, analyzing and live broadcasting traffic command according to claim 1, wherein the video file set formed in the step 4) is that when related images are output from HDMI/VGI signals of different terminals, the images are encoded separately to form video files; when there are multiple voices and subtitles, the voices and subtitles are added into different channels; all videos are used for synchronizing multiple paths of videos during playback by taking clock information of a TDCS/CTC satellite time service system when the videos occur as a basis;
the video coding parameters in the step 6) can be configured, including configuring the code rate, the frame rate, the encoder and the video format of the video, so as to ensure the smoothness and the definition of playing.
5. The device for recording, analyzing and directly broadcasting the driving command is characterized by comprising: the system comprises a TDCS/CTC satellite time service system, a TDCS/CTC operation terminal, a voice acquisition module, an image recognition and rule module, an audio and video recording and pushing module, an operation behavior analysis module, an audio and video live broadcast system, a file storage management module and a multipath video playing analysis module;
the voice acquisition module is connected with the TDCS/CTC satellite time service system and is used for receiving a TDCS/CTC system clock;
the audio and video recording and pushing module is connected with the TDCS/CTC operation terminal and is used for receiving HDMI/VDI digital video signals;
The audio and video recording plug flow module is connected with the voice acquisition module and is used for receiving real-time voice data;
The audio and video recording plug flow module is connected with the TDCS/CTC satellite time service system and is used for receiving a TDCS/CTC system clock;
The audio and video recording plug flow module is connected with the file storage management module and is used for sending audio and video files;
The audio and video recording and pushing module is connected with the audio and video live broadcasting system and used for pushing the video stream of the operation terminal;
the operation behavior analysis module is connected with the image recognition and rule module and is used for acquiring railway signal equipment information, route establishment and unlocking, train movement information and standard operation flow information;
The operation behavior analysis module is connected with the audio and video recording and pushing module and is used for receiving the graphic set, the audio and the characters of the operation terminal and sending file storage information;
The operation behavior analysis module is connected with the file storage management module and is used for sending analysis result information;
The multi-channel video playing analysis module is connected with the audio-video live broadcast system and is used for receiving live video streams of all operation terminals;
the multi-channel video playing analysis module is connected with the audio and video recording plug flow module and is used for sending live broadcast selection commands;
The multi-channel video playing analysis module is connected with the file storage management module and is used for sending a playing selection command; receiving a playback video file;
The multi-path video playing analysis module is connected with the image recognition and rule module and is used for obtaining railway signal equipment, a route and a chart recognition model;
the operation behavior analysis module comprises a fourth information communication module, a data scheduling module, an operation identification module and an operation flow analysis module;
The fourth information communication module comprises an interface with the image recognition and rule module and is used for acquiring a retrieval recognition judgment basis; the interface is connected with the audio and video recording plug flow module and is used for receiving audio and video and subtitle data and sending file storage information; the interface is used for sending the analysis result file;
the data scheduling module groups the audio and video subtitle data according to the ID of the operation terminal, synthesizes the audio and video subtitle data belonging to the same scheduling station or station into a group, and inputs the data into the operation identification module with the same ID;
The operation identification module scans the image sequence, identifies alarm information in an alarm information frame, and extracts equipment state information, operation information and time; identifying the fault type and number of the signal equipment according to the dynamic fault model of the signal equipment, and recording the time when the fault starts; identifying the scheduled command to be written, issued, signed and received operation and the type of the scheduled command, and recording the corresponding time; for abnormal operation of equipment failure, using equipment failure time as a starting point, identifying the time of receiving and sending a vehicle to enter and transact through the failed equipment, identifying menu operation commands, button operation commands and time, extracting voice commands from voice character records according to the times, and sequencing the events according to time order to form an abnormal operation action set under the condition of equipment failure; for the operation of the scheduling command, identifying the content of the scheduling command and checking with the standard term of the scheduling command; extracting the command sending and receiving person, finding out the image of the related personnel operation terminal according to the time, identifying the operation of the command menu, the button operation of the operator, recording the time, extracting the voice command in the voice text record according to the time, and sequencing the events according to the time order to form an operation action set related to the command;
The operation flow analysis module compares the operation action set with the normal operation flow to form an analysis result, wherein analysis result information comprises the time of operation occurrence, an operator and a conclusion, corresponding equipment elements, menu operation and file content are recorded for the conclusion inconsistent with the operation standard, and the analysis result file is sent to the file storage management module through the fourth information communication module.
6. The apparatus of claim 5, wherein the voice acquisition module comprises a first information communication module and a recording setting module;
The first information communication module is connected with the TDCS/CTC satellite time service system interface, acquires system time information, and sends audio data, system time when recording starts and recording equipment ID to the audio-video recording plug flow module;
The recording setting module is used for setting a recording mode, microphone sensitivity and sound control recording, wherein the recording mode comprises mono and double sound channels and sound quality.
7. The apparatus of claim 5, wherein the audio-video recording and streaming module comprises a second information communication module, a signal conversion module, a caption module, a video editing module and a packaging streaming module;
The second information communication module includes: an interface with a TDCS/CTC satellite time service system for acquiring system time information; the interface is used for acquiring real-time audio acquisition data; the interface is connected with the multipath video playing analysis module and is used for receiving a dispatching desk or station live broadcast selection control instruction to be played; the interface is used for sending the video live stream; the interface is connected with the operation behavior analysis module and is used for sending the image sequence set and receiving the stored file information; the interface is connected with the file storage management module and is used for sending video files;
The signal conversion module is provided with an interface with the TDCS/CTC terminal and is used for receiving HDMI/VDI signal data, the signal conversion module extracts image size and frame rate information, simultaneously extracts RGB or YUV channel image data, synthesizes a complete RGB image sequence, calculates the actual time of image generation according to the obtained TDCS/CTC system clock and image frame rate and marks the actual time, and caches the generated image sequence set and sends the image sequence set to the operation behavior analysis module through the second information communication module;
The caption module converts recorded voice into characters, sets the time corresponding to the characters according to the system time of the beginning of the voice and voice parameters, and converts the characters into caption information; identifying a speaker according to the recording equipment ID and adding the speaker to the caption, and sending the voice and the caption to the operation behavior analysis module through the second information communication module;
The video editing module is used for setting the size, frame rate and encoder of a video image to be generated, selecting an image sequence set, audio and subtitles in a cache according to time information given by the operation behavior analysis module to perform video synthesis, adding different channels into the audio and the subtitles when a plurality of audio and subtitles exist at the same moment, guaranteeing the integrity of information, adding the starting time of video occurrence to a synthesized video file, and then sending the video file to the file storage management module for storage through the second information communication module;
The encapsulation plug flow module carries out video coding on the image sequence set, the audio frequency and the caption in the buffer memory in real time according to the live broadcast requirement, encapsulates the image sequence set, the audio frequency and the caption into a streaming media protocol data packet and pushes the streaming media protocol data packet to the audio and video live broadcast system in real time through the second information communication module.
8. The apparatus of claim 5, wherein the image recognition and rule module comprises a third information communication module, a dynamic graphic modeling module, and a standard flow definition module;
The third information communication module includes: the interface is used for sending an identification judgment basis; the interface is used for sending an identification judgment basis;
The dynamic graph modeling module expresses various states of the annunciator, the turnout, the stock way, the approach, the train, the power supply arm, the blocking area and the plan chart by actually displayed graphs, and expresses the meanings of the states by words; when the train route and the shunting route are established and unlocked, the front and back sequences of the turnout, the annunciator and the train are changed, the change of the track section and the change of the train number are expressed by a group of image sequences when the train moves, and the characteristics of the train are identified through the change between images;
The standard flow definition module expresses key business operations of various abnormal operations and scheduling commands under equipment faults in the form of operation steps, including which posts are to be notified by telephone, which registrations are to be made, which equipment operations are to be made and the precedence relationship among the operations.
9. The apparatus of claim 5, wherein the file storage management module comprises a fifth information communication module and a file management module;
the fifth information communication module includes: the interface is connected with the audio and video recording plug flow module and is used for receiving audio and video files; the interface is used for receiving the analysis result file; the interface is connected with the multipath video playing analysis module and is used for receiving application information of playing a dispatching desk or station video playback, and sending a video file and an analysis result file;
The file management module forms a group of audio and video files and analysis result files belonging to the same dispatching desk or station, and stores the audio and video files and the analysis result files according to time sequence; and selecting all audio and video files and analysis results meeting the requirements according to the dispatching desk, station and time period information provided by the multi-channel video playing module, and sending the audio and video files and the analysis results to the multi-channel video playing analysis module through the fifth information communication module.
10. The device according to claim 5, wherein the multi-channel video playing and analyzing module comprises a sixth information communication module, a playing module and an analyzing and labeling module;
The sixth information communication module includes: the interface is used for sending application information for live broadcasting of the dispatching desk or station video; the interface is used for acquiring a plurality of video live streams; the interface is used for sending application information for video playback of the dispatching desk or the station, receiving video files and analysis result files; the interface is used for receiving the identification judgment basis;
the playing module is used for simultaneously caching and decoding a plurality of received video streams or video file sets, synchronously playing a plurality of videos according to the same playing clock, playing voices of different audio channels and displaying different subtitles according to the need;
The analysis marking module analyzes the analysis result file, finds out equipment, an operation menu, text information and occurrence time which need marking, finds out corresponding elements in an image buffer area of video decoding according to the identification judgment basis, records corresponding display positions of the elements, and marks various boxes and lines which are defined in advance at the corresponding positions when the frame images are displayed.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010247428.1A CN111432229B (en) | 2020-03-31 | 2020-03-31 | Method and device for recording, analyzing and directly broadcasting driving command |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010247428.1A CN111432229B (en) | 2020-03-31 | 2020-03-31 | Method and device for recording, analyzing and directly broadcasting driving command |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111432229A CN111432229A (en) | 2020-07-17 |
| CN111432229B true CN111432229B (en) | 2024-05-10 |
Family
ID=71551797
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010247428.1A Active CN111432229B (en) | 2020-03-31 | 2020-03-31 | Method and device for recording, analyzing and directly broadcasting driving command |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111432229B (en) |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112463996A (en) * | 2020-11-19 | 2021-03-09 | 长城计算机软件与***有限公司 | Audio and video file playing method and device, terminal and storage medium |
| CN114359271B (en) * | 2022-03-10 | 2022-06-03 | 天津市北海通信技术有限公司 | Method and device for detecting image playing quality of train display equipment |
| CN114845066B (en) * | 2022-05-05 | 2024-07-05 | 重庆赛力斯凤凰智创科技有限公司 | Driving recording method, device, equipment and storage medium |
| CN116506673B (en) * | 2023-06-30 | 2023-09-12 | 安格利(成都)仪器设备有限公司 | Audio-video recording and detecting equipment data synchronous linkage and sound control searching method |
Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2009255654A (en) * | 2008-04-14 | 2009-11-05 | Toshiba Corp | Train operation monitoring system |
| CN103607467A (en) * | 2013-11-28 | 2014-02-26 | 东莞中国科学院云计算产业技术创新与育成中心 | Air-land integrated emergency command platform |
| CN108028967A (en) * | 2015-09-23 | 2018-05-11 | 三星电子株式会社 | Video process apparatus, the method for driving video process apparatus, video relay, the method for driving video relay and computer readable recording medium storing program for performing |
| CN108216252A (en) * | 2017-12-29 | 2018-06-29 | 中车工业研究院有限公司 | A kind of subway driver vehicle carried driving behavior analysis method, car-mounted terminal and system |
| CN108401192A (en) * | 2018-04-25 | 2018-08-14 | 腾讯科技（深圳）有限公司 | Video stream processing method, device, computer equipment and storage medium |
| CN108583592A (en) * | 2017-12-30 | 2018-09-28 | 西安市地下铁道有限责任公司 | A kind of subway service on buses or trains job information acquisition intelligent detecting method |
| CN108614853A (en) * | 2018-03-15 | 2018-10-02 | 中国人民解放军63895部队 | A kind of multi-data source synchronizing information mixing storage and playback system and method |
| CN108765398A (en) * | 2018-05-23 | 2018-11-06 | 深圳市阡丘越科技有限公司 | A kind of rail traffic monitoring management platform |
| CN108872796A (en) * | 2018-07-31 | 2018-11-23 | 广州科易光电技术有限公司 | Ground control platform based on vehicle-mounted contact net condition monitoring |
| CN212519244U (en) * | 2020-03-31 | 2021-02-09 | 卡斯柯信号有限公司 | Device for recording, analyzing and live broadcasting driving command |
-
2020
- 2020-03-31 CN CN202010247428.1A patent/CN111432229B/en active Active
Patent Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2009255654A (en) * | 2008-04-14 | 2009-11-05 | Toshiba Corp | Train operation monitoring system |
| CN103607467A (en) * | 2013-11-28 | 2014-02-26 | 东莞中国科学院云计算产业技术创新与育成中心 | Air-land integrated emergency command platform |
| CN108028967A (en) * | 2015-09-23 | 2018-05-11 | 三星电子株式会社 | Video process apparatus, the method for driving video process apparatus, video relay, the method for driving video relay and computer readable recording medium storing program for performing |
| CN108216252A (en) * | 2017-12-29 | 2018-06-29 | 中车工业研究院有限公司 | A kind of subway driver vehicle carried driving behavior analysis method, car-mounted terminal and system |
| CN108583592A (en) * | 2017-12-30 | 2018-09-28 | 西安市地下铁道有限责任公司 | A kind of subway service on buses or trains job information acquisition intelligent detecting method |
| CN108614853A (en) * | 2018-03-15 | 2018-10-02 | 中国人民解放军63895部队 | A kind of multi-data source synchronizing information mixing storage and playback system and method |
| CN108401192A (en) * | 2018-04-25 | 2018-08-14 | 腾讯科技（深圳）有限公司 | Video stream processing method, device, computer equipment and storage medium |
| CN108765398A (en) * | 2018-05-23 | 2018-11-06 | 深圳市阡丘越科技有限公司 | A kind of rail traffic monitoring management platform |
| CN108872796A (en) * | 2018-07-31 | 2018-11-23 | 广州科易光电技术有限公司 | Ground control platform based on vehicle-mounted contact net condition monitoring |
| CN212519244U (en) * | 2020-03-31 | 2021-02-09 | 卡斯柯信号有限公司 | Device for recording, analyzing and live broadcasting driving command |
Non-Patent Citations (1)
| Title |
|---|
| 基于分布式云架构的铁路客运站智能应急指挥***设计;韩旭辉;;铁路计算机应用;20181125(第11期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111432229A (en) | 2020-07-17 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111432229B (en) | Method and device for recording, analyzing and directly broadcasting driving command | |
| CN103065503B (en) | Reappearing method and system controlling river reach ship traveling dynamic and signal revealing process | |
| CN101631237B (en) | Video monitoring data storing and managing system | |
| EP0679315B1 (en) | Video facility management system | |
| CN103069812B (en) | There is provided based on the method for the multimedia streaming service of segmentation and device, reception based on the method for the multimedia streaming service of segmentation and device | |
| CN104035404B (en) | The development approach of operation order real-time status monitoring | |
| CN104363429B (en) | A kind of locomotive operation monitoring system | |
| CN103761985B (en) | A kind of hyperchannel video and audio is online performs in a radio or TV programme editing system | |
| CN108945009B (en) | Train broadcasting monitoring system | |
| CN101998138A (en) | Television channel monitoring system and real-time monitoring method thereof | |
| US7647141B2 (en) | Advanced simulation capture and reporting tools | |
| CN114363562B (en) | Cloud distribution-based 5G distribution network monitoring system and distribution network monitoring method | |
| CN112722016B (en) | Automatic test system and method for rail transit train automatic control system | |
| CN102862587A (en) | Method and equipment for analyzing rolling stock and locomotive inter-control voice of railways | |
| CN100418373C (en) | Rules for a digital video recorder | |
| CN101335037A (en) | Recording apparatus, recording/reproducing system, and recording method | |
| CN212519244U (en) | Device for recording, analyzing and live broadcasting driving command | |
| CN115303336A (en) | Safety management and control system and method for station shunting operation | |
| US20230365174A1 (en) | Automatic train tracking method and automatic train tracking system | |
| CN112601052B (en) | Video resource integration system applied to internal sharing platform | |
| CN110379130A (en) | A kind of Medical nursing shatter-resistant adjustable voltage system based on multi-path high-definition SDI video | |
| CN100438568C (en) | Control field event detection in a digital video recorder | |
| CN112172877A (en) | Station field representation information identification method based on video image | |
| CN106791536A (en) | The recording player method and terminal of multimedia file | |
| CN108259805B (en) | Multi-channel video decoding method, playing method and device and automobile data recorder |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| REG | Reference to a national code |
Ref country code: HK Ref legal event code: DE Ref document number: 40024935 Country of ref document: HK |
|
| GR01 | Patent grant | ||
| GR01 | Patent grant |