CN110008804B - Elevator monitoring key frame obtaining and detecting method based on deep learning - Google Patents
Elevator monitoring key frame obtaining and detecting method based on deep learning Download PDFInfo
- Publication number
- CN110008804B CN110008804B CN201811519288.8A CN201811519288A CN110008804B CN 110008804 B CN110008804 B CN 110008804B CN 201811519288 A CN201811519288 A CN 201811519288A CN 110008804 B CN110008804 B CN 110008804B
- Authority
- CN
- China
- Prior art keywords
- monitoring video
- monitoring
- elevator
- key frame
- frame
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/40—Scenes; Scene-specific elements in video content
- G06V20/46—Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Indicating And Signalling Devices For Elevators (AREA)
- Maintenance And Inspection Apparatuses For Elevators (AREA)
Abstract
The invention discloses an elevator monitoring key frame obtaining and detecting method based on deep learning, which comprises a monitoring video acquisition module, a monitoring video preprocessing module, a monitoring video key frame intercepting module, a monitoring video post-processing application module and a user application module; the invention provides a method for acquiring and detecting an elevator monitoring key frame based on deep learning, which is simple in design and can complete monitoring tasks of various scenes.
Description
Technical Field
The invention relates to the field of elevator application, in particular to an elevator monitoring key frame obtaining and detecting method based on deep learning.
Background
Elevators are closely related to people's lives and are commonly found in residential buildings, shopping malls, office buildings and other areas. No matter the elevator is in normal operation or when the elevator breaks down, the real-time monitoring video or the historical video can be obtained through the camera arranged in the elevator or in the elevator waiting area. In the past, aiming at elevator monitoring real-time video or historical video, a mode of manual real-time monitoring or review is generally adopted, and by analyzing the real-time video or the historical video, the real-time analysis of the running condition of an elevator and the real-time analysis of elevator passengers in the elevator can be realized, wherein the tasks comprise the number of elevator passengers, the attributes of the elevator passengers, whether the elevator passengers are trapped or not, whether the elevator passengers have abnormal behaviors or not and the like, and the analysis of the elevator historical monitoring video can help solve the tasks of elevator security protection, elevator passenger search and the like, so that short boards for cell monitoring and mall monitoring are completed. However, there is no such content on the market, so there is an urgent need for such processing technology.
Disclosure of Invention
The invention overcomes the defects of the prior art and provides the elevator monitoring key frame acquisition and detection method based on deep learning, which is simple in design and can complete monitoring tasks of various scenes.
In order to solve the technical problems, the technical scheme of the invention is as follows:
the elevator monitoring key frame acquisition and detection method based on deep learning comprises a monitoring video acquisition module, a monitoring video preprocessing module, a monitoring video key frame intercepting module, a monitoring video post-processing application module and a user application module;
101) a pretreatment step: acquiring a plurality of continuous frame images of a monitoring video acquired by a monitoring video acquisition module, and pre-initializing a detection model of a person taking a ladder or a dangerous article through a monitoring video preprocessing module and a monitoring video key frame intercepting module to form a preprocessed frame image;
102) frame image processing: recording, analyzing and comparing the preprocessed frame images in the step 101), and judging whether the number of elevator passengers or dangerous goods in the current preprocessed frame images is changed compared with the number of the previous frame images; if the detected number of the elevator passengers or dangerous goods is not changed, outputting the current preprocessed frame image to a monitoring video post-processing application module and clearing cache, and if the detected number of the elevator passengers or dangerous goods is changed, comparing the current preprocessed frame image with the previous frame image, wherein the comparison content comprises the number of the elevator passengers or goods and the weight of the elevator passengers or dangerous goods; if the weight detection part displays that the weight of the elevator taking person or the dangerous goods in the current frame image is heavier, replacing the previously stored key frame with the current frame, and if the weight of the elevator taking person or the dangerous goods in the current frame image is lighter, maintaining the previous key frame information;
103) the processing steps are as follows: and starting to process the frame image of the next frame of monitoring video, and executing the step 101) until the frame image is completely processed.
Furthermore, the monitoring video acquisition module comprises an elevator and a camera, the camera is arranged at the top in the elevator car, and the acquired monitoring video is sent to the subsequent monitoring video preprocessing module for analysis in real time;
the monitoring video preprocessing module comprises a monitoring video storage server and a video preprocessing server, wherein the video preprocessing server is responsible for processing historical videos in the monitoring video storage server or monitoring videos acquired in real time, and the preprocessed monitoring video data are transmitted to a subsequent monitoring video key frame intercepting module for processing;
the method is based on the weight of a monitoring video picture, scores people or objects interested in monitoring, accurately obtains a required key frame for elevator monitoring in real time, stores the obtained key frame in a specified storage server and then transmits the key frame to a subordinate key frame application module;
the monitoring video post-processing application module comprises a monitoring video key frame application module and is responsible for accurately distributing and displaying a plurality of monitoring video-based application results in real time;
the user application module comprises a property monitoring, supervising and displaying platform, a display screen in the elevator and a human-computer interaction platform for operation and maintenance personnel.
Further, the camera adopts a monitoring camera and/or an industrial camera.
Further, the processing types include video size conversion, video frame conversion, and video format conversion.
Compared with the prior art, the invention has the advantages that: according to the method, the key frame of the monitoring video is automatically acquired by analyzing the real-time monitoring video or the historical monitoring video by utilizing the video key frame intercepting method based on deep learning, and then according to the acquired monitoring video key frame, the method is matched with other operation data of the elevator, such as the state information, the speed information or the position information of the elevator door, and can be used for processing various elevator problems in real time or quasi-real time, for example, the acquired elevator monitoring key frame is matched with a target detection method and a classification method based on deep learning, so that the task of detecting the number of people in the elevator, namely elevator people flow statistics, can be solved; the obtained elevator monitoring key frame is matched with a multi-target tracking method, so that a pedestrian re-identification task based on elevator monitoring video data and other camera data of a cell can be processed, and a short board for security protection of the cell is perfected; the obtained elevator monitoring key frame is matched with a behavior recognition method or a dangerous object classification method, so that whether people are trapped in the elevator or not can be analyzed in real time, or some dangerous elevator taking behaviors of people in operation and maintenance departments such as real-time physical property and the like can be analyzed, for example, an electric vehicle is parked on a floor of a non-underground floor, dangerous objects such as gas tanks are brought into a cell, and the like. Therefore, by the method based on the weight of the monitoring video picture, people or objects interested in monitoring are scored, the interested elevator monitoring key frame is accurately monitored in real time, the monitoring tasks of various scenes can be effectively completed, and the method has high practicability and commercial value.
Drawings
FIG. 1 is a block diagram of the system of the present invention;
FIG. 2 is a flow diagram of a key frame detection module according to the present invention;
FIG. 3 is a detailed flowchart of key frame extraction according to the present invention.
Detailed Description
The invention is further described with reference to the following figures and detailed description.
As shown in fig. 1 to 3, the elevator monitoring key frame obtaining and detecting method based on deep learning includes a monitoring video acquisition module, a monitoring video preprocessing module, a monitoring video key frame intercepting module, a monitoring video post-processing application module and a user application module;
101) a pretreatment step: the method comprises the steps of obtaining a plurality of continuous frame images of a monitoring video collected by a monitoring video collecting module, and pre-initializing a detection model of a person taking the elevator or a dangerous article through a monitoring video preprocessing module and a monitoring video key frame intercepting module to form a preprocessed frame image.
102) Frame image processing: recording, analyzing and comparing the preprocessed frame images in the step 101), and judging whether the number of elevator passengers or dangerous goods in the current preprocessed frame images is changed compared with the number of the previous frame images; if the detected number of the elevator passengers or dangerous goods is not changed, outputting the current preprocessed frame image to a monitoring video post-processing application module and clearing cache, if the detected number of the elevator passengers or dangerous goods is changed, comparing the current preprocessed frame image with the previous frame image, wherein the comparison content comprises the number of the elevator passengers or goods and the weight of the elevator passengers or dangerous goods (weight detection is performed by arranging a weighing sensor and the like); if the weight detection part displays that the weight of the elevator taking person or the dangerous goods in the current frame image is heavier, the key frame stored before is replaced by the current frame, and if the weight of the elevator taking person or the dangerous goods in the current frame image is lighter or unchanged, the previous key frame information is maintained.
103) The processing steps are as follows: and starting to process the frame image of the next frame of monitoring video, and executing the step 101) until the frame image is completely processed.
The monitoring video acquisition module comprises but not limited to an elevator, a monitoring camera, an industrial camera and the like, is arranged at the top in an elevator car, shoots the elevator against the maximum visual field range in the elevator, and transmits the monitoring video acquired in real time to the subsequent monitoring video preprocessing module for analysis.
The monitoring video preprocessing module comprises but is not limited to a monitoring video storage server, a video preprocessing server based on a CPU, a DSP, an FPGA and other processors and is responsible for processing monitoring videos or historical videos acquired in real time, the processing types comprise video size conversion, video frame conversion, video format conversion and the like, and the preprocessed monitoring video data are transmitted to a subsequent monitoring video key frame intercepting module for processing. The weighing sensor is electrically connected with the video preprocessing server, and judgment and selection are carried out through the change of data transmitted by the weighing sensor.
The method is based on a method for monitoring the weight of video pictures, scores people or objects interested in monitoring, accurately acquires a required key frame for elevator monitoring in real time, stores the acquired key frame in a designated storage server and then transmits the key frame to a next key frame application module.
The monitoring video post-processing application module comprises a key frame application module of the monitoring video and is responsible for accurately distributing and displaying a plurality of application results based on the monitoring video in real time.
The user application module comprises a property monitoring, supervising and displaying platform, a display screen in the elevator and a human-computer interaction platform for operation and maintenance personnel.
In summary, the scheme mainly includes that a monitoring video acquisition module is used for acquiring a monitoring video image data unit to acquire a complete monitoring video in real time or quasi-real time, then the monitoring video is transmitted to a monitoring video preprocessing module and a monitoring video key frame intercepting module, the intercepting key frame is combined with a deep learning method, a deep learning database of TB level is matched according to project tasks (such as re-identification of elevator passengers, dangerous behaviors of elevator passengers or detection of dangerous goods carried in the monitoring video), the key frame in monitoring is selected in real time, and then the acquired key frame is matched with other algorithms of the next level (such as an abnormal behavior identification algorithm, an advertisement real-time pushing algorithm, a trapped behavior identification algorithm, a dangerous goods detection algorithm and the like) to realize various applications based on elevator monitoring. Namely, when the elevator normally runs, the monitoring video acquisition module acquires video streams inside the elevator in real time through a camera arranged in the elevator, intercepts all the monitoring video streams into a plurality of continuous video frame images, detects and tracks elevator riders (or dangerous goods) by matching with a weight grading network of a direct frame image to obtain required key frames, wherein the intercepted frame image weight grading module mainly grades the elevator riders (or dangerous goods) in the images according to different measurement standards, such as whether the elevator riders are on the front, whether light rays in the elevator are clear, how the image contrast is, and the like, and finally obtains the required key frames, so that people or goods interested in monitoring are graded by a method based on the weight of the monitoring video images, the key frames of the elevator monitoring which is accurately interested in real time can effectively complete monitoring security tasks of various scenes, and has high practicability and commercial value.
The foregoing is only a preferred embodiment of the present invention, and it should be noted that, for those skilled in the art, several modifications and decorations can be made without departing from the spirit of the present invention, and these modifications and decorations should also be regarded as being within the scope of the present invention.
Claims (3)
1. The elevator monitoring key frame acquisition and detection method is characterized by comprising a monitoring video acquisition module, a monitoring video preprocessing module, a monitoring video key frame intercepting module, a monitoring video post-processing application module and a user application module;
101) a pretreatment step: acquiring a plurality of continuous frame images of a monitoring video acquired by a monitoring video acquisition module, pre-initializing a detection model of a person taking a ladder or a dangerous article through a monitoring video preprocessing module and a monitoring video key frame intercepting module, and forming a preprocessed frame image through the monitoring video preprocessing module;
102) frame image processing: recording, analyzing and comparing the preprocessed frame images in the step 101), and judging whether the number of elevator passengers or dangerous goods in the current preprocessed frame images is changed from that in the previous frame images; if the detected number of the elevator passengers or the dangerous goods is not changed, outputting the current preprocessed frame image to a monitoring video post-processing application module and clearing the cache, if the detected number of the elevator passengers or the dangerous goods is changed, comparing the current preprocessed frame image with the previous frame image, wherein the comparison content comprises the number of the elevator passengers or the dangerous goods and the weight of the elevator passengers or the dangerous goods; if the weight detection part displays that the weight of the elevator taking person or the dangerous goods in the current frame image is heavier, replacing the previously stored key frame with the current frame, and if the weight of the elevator taking person or the dangerous goods in the current frame image is lighter, maintaining the previous key frame information;
103) the processing steps are as follows: and starting to process the frame image of the next frame of monitoring video, and executing the step 101) until all the frame images are processed.
2. The elevator monitoring key frame acquisition and detection method according to claim 1, characterized in that: the monitoring video acquisition module comprises an elevator and a camera, the camera is arranged at the top in the elevator car, and transmits the acquired monitoring video to the subsequent monitoring video preprocessing module for analysis in real time;
the monitoring video preprocessing module comprises a monitoring video storage server and a video preprocessing server, wherein the video preprocessing server is responsible for processing historical videos in the monitoring video storage server or monitoring videos acquired in real time, and the preprocessed monitoring video data are transmitted to a subsequent monitoring video key frame intercepting module for processing;
the monitoring video key frame intercepting module comprises a GPU server and a video key frame extracting module, scores people or objects interested in monitoring, accurately obtains a key frame of required elevator monitoring in real time, stores the obtained key frame in a specified storage server and then transmits the key frame to a subordinate key frame application module;
the monitoring video post-processing application module comprises a monitoring video key frame application module and is responsible for accurately distributing and displaying a plurality of monitoring video-based application results in real time;
the user application module comprises a property monitoring, supervising and displaying platform, a display screen in the elevator and a human-computer interaction platform for operation and maintenance personnel.
3. The elevator monitoring key frame acquisition and detection method according to claim 2, characterized in that: the camera adopts a monitoring camera and/or an industrial camera.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811519288.8A CN110008804B (en) | 2018-12-12 | 2018-12-12 | Elevator monitoring key frame obtaining and detecting method based on deep learning |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811519288.8A CN110008804B (en) | 2018-12-12 | 2018-12-12 | Elevator monitoring key frame obtaining and detecting method based on deep learning |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110008804A CN110008804A (en) | 2019-07-12 |
| CN110008804B true CN110008804B (en) | 2021-07-06 |
Family
ID=67165115
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811519288.8A Active CN110008804B (en) | 2018-12-12 | 2018-12-12 | Elevator monitoring key frame obtaining and detecting method based on deep learning |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110008804B (en) |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111259855A (en) * | 2020-02-09 | 2020-06-09 | 天津博宜特科技有限公司 | Mobile safety helmet wearing detection method based on deep learning |
| CN111738117B (en) * | 2020-06-12 | 2023-12-19 | 鞍钢集团矿业有限公司 | Deep learning-based detection method for electric bucket tooth video key frame |
| CN112418127B (en) * | 2020-11-30 | 2022-05-03 | 浙江大学 | Video sequence coding and decoding method for video pedestrian re-identification |
| CN114463928B (en) * | 2021-12-29 | 2022-11-25 | 上海瑞琨计算机***集成有限公司 | Intelligent alarm method and system |
Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103552890A (en) * | 2013-11-07 | 2014-02-05 | 深圳市汇川技术股份有限公司 | System and method for detecting number of people in lift car |
| CN103771207A (en) * | 2014-01-09 | 2014-05-07 | 深圳市汇川技术股份有限公司 | Elevator car passenger detection system and method |
| CN105929696A (en) * | 2015-12-18 | 2016-09-07 | 许昌学院 | Intelligent home security method |
| CN106559631A (en) * | 2015-09-30 | 2017-04-05 | 小米科技有限责任公司 | Method for processing video frequency and device |
| CN106586738A (en) * | 2017-01-24 | 2017-04-26 | 沈阳建筑大学 | Multi-car elevator dispatching method based on visual inspection |
| US20170148223A1 (en) * | 2014-10-31 | 2017-05-25 | Fyusion, Inc. | Real-time mobile device capture and generation of ar/vr content |
| WO2017166089A1 (en) * | 2016-03-30 | 2017-10-05 | Intel Corporation | Techniques for determining a current location of a mobile device |
| CN108038869A (en) * | 2017-11-20 | 2018-05-15 | 江苏省特种设备安全监督检验研究院 | Passenger falls down to the ground behavioral value method in a kind of lift car |
| CN108275524A (en) * | 2018-01-12 | 2018-07-13 | 东北大学 | A kind of elevator maintenance operation monitoring and guiding device based on the assessment of the first multi-view video series of operations |
| CN108726300A (en) * | 2018-06-25 | 2018-11-02 | 牛乾 | Elevator security protection control system based on load condition |
| CN108805093A (en) * | 2018-06-19 | 2018-11-13 | 华南理工大学 | Escalator passenger based on deep learning falls down detection algorithm |
-
2018
- 2018-12-12 CN CN201811519288.8A patent/CN110008804B/en active Active
Patent Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103552890A (en) * | 2013-11-07 | 2014-02-05 | 深圳市汇川技术股份有限公司 | System and method for detecting number of people in lift car |
| CN103771207A (en) * | 2014-01-09 | 2014-05-07 | 深圳市汇川技术股份有限公司 | Elevator car passenger detection system and method |
| US20170148223A1 (en) * | 2014-10-31 | 2017-05-25 | Fyusion, Inc. | Real-time mobile device capture and generation of ar/vr content |
| CN106559631A (en) * | 2015-09-30 | 2017-04-05 | 小米科技有限责任公司 | Method for processing video frequency and device |
| CN105929696A (en) * | 2015-12-18 | 2016-09-07 | 许昌学院 | Intelligent home security method |
| WO2017166089A1 (en) * | 2016-03-30 | 2017-10-05 | Intel Corporation | Techniques for determining a current location of a mobile device |
| CN106586738A (en) * | 2017-01-24 | 2017-04-26 | 沈阳建筑大学 | Multi-car elevator dispatching method based on visual inspection |
| CN108038869A (en) * | 2017-11-20 | 2018-05-15 | 江苏省特种设备安全监督检验研究院 | Passenger falls down to the ground behavioral value method in a kind of lift car |
| CN108275524A (en) * | 2018-01-12 | 2018-07-13 | 东北大学 | A kind of elevator maintenance operation monitoring and guiding device based on the assessment of the first multi-view video series of operations |
| CN108805093A (en) * | 2018-06-19 | 2018-11-13 | 华南理工大学 | Escalator passenger based on deep learning falls down detection algorithm |
| CN108726300A (en) * | 2018-06-25 | 2018-11-02 | 牛乾 | Elevator security protection control system based on load condition |
Non-Patent Citations (2)
| Title |
|---|
| Emotion Recognition via Body Gesture: Deep Learning Model Coupled with Keyframe Selection;Son Thai Ly等;《Proceedings of the 2018 International Conference on Machine Learning and Machine Intelligence》;20180930;第27-31页 * |
| 基于片段关键帧的视频行为识别方法;李鸣晓等;《***仿真学报》;20180731;第30卷(第7期);第2787-2793页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110008804A (en) | 2019-07-12 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110008804B (en) | Elevator monitoring key frame obtaining and detecting method based on deep learning | |
| CN110188644B (en) | Escalator passenger dangerous behavior monitoring system and method based on visual analysis | |
| CN107679471B (en) | Indoor personnel air post detection method based on video monitoring platform | |
| CN102521578B (en) | Method for detecting and identifying intrusion | |
| CN111488799B (en) | Falling object identification method and system based on image identification | |
| CN102750709B (en) | Video is utilized to detect the method and apparatus of behavior of fighting | |
| GB2502687A (en) | Video-based detection for short-term parking violation | |
| CN108304816B (en) | Identity recognition method and device, storage medium and electronic equipment | |
| CN111563396A (en) | Method and device for online identifying abnormal behavior, electronic equipment and readable storage medium | |
| CN111770317B (en) | Video monitoring method, device, equipment and medium for intelligent community | |
| CN112744174B (en) | Vehicle collision monitoring method, device, equipment and computer readable storage medium | |
| CN111523397A (en) | Intelligent lamp pole visual identification device, method and system and electronic equipment | |
| CN112016528A (en) | Behavior recognition method and device, electronic equipment and readable storage medium | |
| CN112188164A (en) | AI vision-based violation real-time monitoring system and method | |
| CN112287875A (en) | Abnormal license plate recognition method, device, equipment and readable storage medium | |
| CN112949457A (en) | Maintenance method, device and system based on augmented reality technology | |
| CN112580470A (en) | City visual perception method and device, electronic equipment and storage medium | |
| CN112383756A (en) | Video monitoring alarm processing method and device | |
| CN111783700A (en) | Automatic recognition early warning method and system for road foreign matters | |
| CN113052125B (en) | Construction site violation image recognition and alarm method | |
| CN111126112B (en) | Candidate region determination method and device | |
| CN111062238A (en) | Escalator flow monitoring method and system based on human skeleton information and multi-target tracking | |
| CN111191498A (en) | Behavior recognition method and related product | |
| CN106919889B (en) | Method and device for detecting human head in video image | |
| CN115083229B (en) | Intelligent recognition and warning system of flight training equipment based on AI visual recognition |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |