WO2021137687A1 - A system and method for monitoring behaviour - Google Patents
A system and method for monitoring behaviour Download PDFInfo
- Publication number
- WO2021137687A1 WO2021137687A1 PCT/MY2020/050143 MY2020050143W WO2021137687A1 WO 2021137687 A1 WO2021137687 A1 WO 2021137687A1 MY 2020050143 W MY2020050143 W MY 2020050143W WO 2021137687 A1 WO2021137687 A1 WO 2021137687A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- event
- video
- unit
- video stream
- monitoring
- Prior art date
Links
Classifications
-
- G—PHYSICS
- G08—SIGNALLING
- G08B—SIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
- G08B13/00—Burglar, theft or intruder alarms
- G08B13/18—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength
- G08B13/189—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems
- G08B13/194—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems using image scanning and comparing systems
- G08B13/196—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems using image scanning and comparing systems using television cameras
- G08B13/19602—Image analysis to detect motion of the intruder, e.g. by frame subtraction
- G08B13/19613—Recognition of a predetermined image pattern or behaviour pattern indicating theft or intrusion
- G08B13/19615—Recognition of a predetermined image pattern or behaviour pattern indicating theft or intrusion wherein said pattern is defined by the user
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/40—Scenes; Scene-specific elements in video content
- G06V20/41—Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/40—Scenes; Scene-specific elements in video content
- G06V20/44—Event detection
-
- G—PHYSICS
- G08—SIGNALLING
- G08B—SIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
- G08B13/00—Burglar, theft or intruder alarms
- G08B13/18—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength
- G08B13/189—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems
- G08B13/194—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems using image scanning and comparing systems
- G08B13/196—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems using image scanning and comparing systems using television cameras
- G08B13/19665—Details related to the storage of video surveillance data
- G08B13/19671—Addition of non-video data, i.e. metadata, to video stream
-
- G—PHYSICS
- G08—SIGNALLING
- G08B—SIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
- G08B13/00—Burglar, theft or intruder alarms
- G08B13/18—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength
- G08B13/189—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems
- G08B13/194—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems using image scanning and comparing systems
- G08B13/196—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems using image scanning and comparing systems using television cameras
- G08B13/19665—Details related to the storage of video surveillance data
- G08B13/19671—Addition of non-video data, i.e. metadata, to video stream
- G08B13/19673—Addition of time stamp, i.e. time metadata, to video stream
Definitions
- the present invention relates to a system and a method for monitoring behaviour. More specifically, the invention relates to a system and a method for monitoring the behaviour of a person in a confined space.
- Camera surveillance and monitoring system is a crucial part of a security system. Although conventional camera surveillance and monitoring system allows a user to monitor multiple locations at once, usually the user can only evaluate any situation using one camera at a time. There is a need for a camera surveillance and monitoring system that can evaluate behaviour and alert the user about any detected behaviour that can be deemed as aggressive and dangerous, especially in an environment such as a prison. Therefore, many systems for monitoring behaviour have been invented that can perform behaviour analysis and alert the user about any detected behaviour.
- a Chinese Patent Publication No. 206921109 U discloses a behaviour analysis system.
- the system comprises a video capture module, a video display module, an analysis module, a manual module, and an output alarm module.
- the analysis module performs behaviour analysis on a video stream to determine whether any abnormal behaviour can be detected in the video stream. Any detected abnormal behaviour is then further analysed by a human in the manual module. If the human confirmed that an abnormal behaviour has been detected, an alarm signal is transmitted by the system.
- a United States Patent No. US 6678413 B1 discloses a system and a method for finding the position and shape of an object using video.
- the system uses a decision tree classification for behaviour identification.
- the system is able to analyse live video stream or recorded video stream.
- a United States Patent Publication No. US 2004/0240542 A1 discloses a surveillance system and method for automatic detection of a potential alarm situation. Apart from analysing behaviour, the system is able to assign a parameter to a detected event or object which then can be searched by using the parameter to search for a specific event or object. Therefore, the user is able to retrieve either event associated with an object or objects associated with an event.
- the invention relates to a system (1000) and a method for monitoring behaviour.
- the system (1000) comprising a video acquisition unit (300) configured to acquire at least one video stream from a plurality of capturing devices (100), a configuration unit (400) configured to produce configuration information relating to the plurality of capturing devices (100) and video analytics information relating to the plurality of capturing devices (100), a configuration database (401) configured to store configuration information for the plurality of capturing devices (100) and video analytics information, a video decoder (500) converts the at least one video stream into a sequence of still images, and a recording unit (200) configured to record the at least one video stream from the plurality of capturing devices (100).
- the configuration unit (400) is connected to the video acquisition unit (300).
- the configuration database (401) and the video decoder (500) are connected to the configuration unit (400).
- the system (1000) further comprising a video processing unit (600) configured to analyse the at least one video stream for behaviour detection and a smart client unit (700) configured to display and visualise active and inactive monitoring.
- the video processing unit (600) and the smart client unit (700) are each connected to the configuration database (401) and the video decoder (500).
- the smart client unit (700) further comprising an event module (701) configured to output the at least one video stream with an overlay of event metadata.
- the smart client unit (700) further comprising an integrated viewing module (702) configured to display multiple video streams from the plurality of capturing devices (100) with an overlay of event metadata.
- the integrated viewing module (702) enables the user to communicate with a person seen in a live stream.
- a method for monitoring behaviour comprising the steps of acquiring at least one video stream from a plurality of capturing devices (100) or a recording unit (200) by a video acquisition unit (300) and decoding the at least one video stream into a sequence of still images by a video decoder (500).
- the method further comprising the steps of detecting a plurality of behaviour displayed within the sequence of still images by the video processing unit (600), visualising active monitoring by a smart client unit (700), and visualising inactive monitoring by the smart client unit (700).
- the plurality of behaviours being detected relates to cell violation, perimeter violation, and presence detection.
- the active monitoring refers to monitoring at least one real-time live feed video stream from the plurality of capturing devices (100).
- the inactive monitoring refers to monitoring at least one video stream from a recording unit (200) or a media storage.
- the step of detecting the plurality of behaviour displayed within the sequence of still images by the video processing unit (600) further includes the substeps of identifying behaviour displayed within the sequence of still images by a video analytics agent, VAA, creating an alert trigger tag, ATT, for each behaviour that has been identified, recording each ATT in a configuration database (401), generating event clip and snapshot for each ATT, storing the event clip and snapshot for each ATT in the media storage, overlaying the event metadata on the at least one video stream, and transmitting the at least one video stream overlaid with the event metadata.
- a video analytics agent VAA
- creating an alert trigger tag ATT
- the step of visualising active monitoring by the smart client unit (700) further comprising sub-steps of receiving configuration from the video processing unit (600), receiving an alert trigger tag, ATT, for each detected event, extracting information from the ATT, and displaying the at least one video stream overlaid with the event metadata based on the ATT.
- the information includes capturing device information and the event metadata
- the step of visualising inactive monitoring by the smart client unit (700) further comprising sub-steps of receiving at least one keyword for database search from a user, matching the at least one keyword with an alert trigger tag, ATT, in a configuration database (401), displaying search result of the ATT that matches the at least one keyword, selecting at least one event from the search result, retrieving the event clip and event snapshot of at least one selected event from the media storage based on the information from the ATT, displaying the event clip and snapshot of the at least one selected event, and compiling the search result, the event clip, and the event snapshot in a report.
- FIG. 1 illustrates a block diagram of a system (1000) for monitoring behaviour according to an embodiment of the present invention.
- FIG. 3 illustrates a flowchart for a step of detecting a plurality of behaviours displayed within the sequence of still images by the video processing unit (600) of the method of
- FIG. 5 illustrates a flow chart for a step of visualising inactive monitoring by the smart client unit (700) of the method of FIG. 2.
- FIG. 1 illustrates a block diagram of a system (1000) for monitoring behaviour according to an embodiment of the present invention.
- the system (1000) is configured to produce an alert signal if at least one person being monitored in a confined space exhibits a certain number of prohibited behaviours.
- the prohibited behaviours are predetermined by an operator of the system (1000).
- the system (1000) comprises a recording unit (200), a video acquisition unit (300), a configuration unit (400), a configuration database (401), a video processing unit (600), and a smart client unit (700).
- the video processing unit (600) further comprises a plurality of video analytics agents, VAA, configured to analyse the video stream for any behaviour that is considered as cell violation, perimeter violation, and presence detection.
- VAA video analytics agents
- Each capturing devices (100) is assigned a VAA for detecting cell violation, a VAA for detecting perimeter violation, and a VAA for presence detection.
- the video processing unit (600) outputs event metadata regarding detected behaviour in the video stream. Any behaviour that is considered as cell violation, perimeter violation, and presence detection is considered as an event. Examples of the event metadata include but not limited to a bounding box of a detected object or behaviour, timestamp of a detected event, and event alert message.
- the event metadata is stored in the configuration database (401).
- the video processing unit (600) also overlays the event metadata onto the video stream.
- the video processing unit (600) creates an event clip and event snapshot of detected behaviour and stores the event clip and event snapshot in a media storage.
- the event clip is a video clip that only shows the portion of the video stream that contains the detected event.
- the event snapshot is a sequence of still images that shows the detected event.
- the video processing unit (600) is connected to the smart client unit (700).
- the smart client unit (700) visualises inactive monitoring as in step 3500.
- Inactive monitoring refers to monitoring the video stream from the recording unit (200) or the media storage.
- the media storage contains a video clip or snapshot of detected behaviours. Specific detected behaviour may then be searched by the keyword by the user.
- the step of visualising inactive monitoring by the smart client unit (700) is further described in relation to FIG. 5.
- the smart client unit (700) then receives the ATT for each event from the video processing unit (600) as in step 3420. If a behaviour is simultaneously detected in multiple video streams from multiple capturing devices (100), the smart client unit (700) receives multiple ATT from the video processing unit (600). The user is able to view any behaviour that has been detected from multiple points of view if the behaviour is simultaneously detected in multiple video streams from multiple capturing devices
- At least one event from the search result is selected by the user as in step 3531.
- the user may select from the search result any event that the user wants to investigate.
- the event clip and event snapshot of the selected event are retrieved from the media storage based on the information from the ATT as in step 3532.
- the event clip and the event snapshot of the selected event are displayed as in step 3540.
- the event clip and event snapshot may be displayed in the integrated viewing mode, wherein all event clip and event snapshot from multiple capturing devices (100) that shows a similar detected event may be displayed simultaneously. Alternatively, only one event clip or event snapshot is displayed at a time.
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Theoretical Computer Science (AREA)
- Computational Linguistics (AREA)
- Software Systems (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The present invention relates to a system (1000) and a method for monitoring behaviour. The system (1000) comprising a video acquisition unit (300) configured to acquire at least one video stream from a plurality of capturing devices (100), a configuration unit (400) configured to produce configuration information relating to the plurality of capturing devices (100) and video analytics information relating to the plurality of capturing devices (100), a video decoder (500) converts the at least one video stream into a sequence of still images, and a recording unit (200) configured to record the at least one video stream from the plurality of capturing devices (100). The system (1000) further comprising a video processing unit (600) configured to analyse the at least one video stream for behaviour detection and a smart client unit (700) configured to display and visualise active and inactive monitoring.
Description
A SYSTEM AND METHOD FOR MONITORING BEHAVIOUR
FIELD OF INVENTION
The present invention relates to a system and a method for monitoring behaviour. More specifically, the invention relates to a system and a method for monitoring the behaviour of a person in a confined space.
BACKGROUND OF THE INVENTION
Camera surveillance and monitoring system is a crucial part of a security system. Although conventional camera surveillance and monitoring system allows a user to monitor multiple locations at once, usually the user can only evaluate any situation using one camera at a time. There is a need for a camera surveillance and monitoring system that can evaluate behaviour and alert the user about any detected behaviour that can be deemed as aggressive and dangerous, especially in an environment such as a prison. Therefore, many systems for monitoring behaviour have been invented that can perform behaviour analysis and alert the user about any detected behaviour.
A Chinese Patent Publication No. 206921109 U discloses a behaviour analysis system. The system comprises a video capture module, a video display module, an analysis module, a manual module, and an output alarm module. The analysis module performs behaviour analysis on a video stream to determine whether any abnormal behaviour can be detected in the video stream. Any detected abnormal behaviour is then further analysed by a human in the manual module. If the human confirmed that an abnormal behaviour has been detected, an alarm signal is transmitted by the system.
A United States Patent No. US 6678413 B1 discloses a system and a method for finding the position and shape of an object using video. The system uses a decision tree classification for behaviour identification. The system is able to analyse live video stream or recorded video stream.
A United States Patent Publication No. US 2004/0240542 A1 discloses a surveillance system and method for automatic detection of a potential alarm situation. Apart from analysing behaviour, the system is able to assign a parameter to a detected
event or object which then can be searched by using the parameter to search for a specific event or object. Therefore, the user is able to retrieve either event associated with an object or objects associated with an event.
Although there are many systems for monitoring behaviour, most systems are designed to monitor open spaces. Therefore, the behaviour analysis in most systems for monitoring behaviour may not be suitable for conditions and parameters of a confined space. Thus, there is a need for a system for monitoring behaviour that is able to solve the aforementioned problems.
SUMMARY OF INVENTION
The invention relates to a system (1000) and a method for monitoring behaviour. The system (1000) comprising a video acquisition unit (300) configured to acquire at least one video stream from a plurality of capturing devices (100), a configuration unit (400) configured to produce configuration information relating to the plurality of capturing devices (100) and video analytics information relating to the plurality of capturing devices (100), a configuration database (401) configured to store configuration information for the plurality of capturing devices (100) and video analytics information, a video decoder (500) converts the at least one video stream into a sequence of still images, and a recording unit (200) configured to record the at least one video stream from the plurality of capturing devices (100). The configuration unit (400) is connected to the video acquisition unit (300). The configuration database (401) and the video decoder (500) are connected to the configuration unit (400). The system (1000) further comprising a video processing unit (600) configured to analyse the at least one video stream for behaviour detection and a smart client unit (700) configured to display and visualise active and inactive monitoring. The video processing unit (600) and the smart client unit (700) are each connected to the configuration database (401) and the video decoder (500).
Preferably, the smart client unit (700) further comprising an event module (701) configured to output the at least one video stream with an overlay of event metadata.
Preferably, the smart client unit (700) further comprising an integrated viewing module (702) configured to display multiple video streams from the plurality of capturing devices (100) with an overlay of event metadata.
Preferably, the integrated viewing module (702) enables the user to communicate with a person seen in a live stream.
Preferably, the smart client unit (700) further comprising a forensic module (703) configured to enable a user to search for an event based on at least one keyword and retrieve at least one video stream depicting the event.
A method for monitoring behaviour comprising the steps of acquiring at least one video stream from a plurality of capturing devices (100) or a recording unit (200) by a video acquisition unit (300) and decoding the at least one video stream into a sequence of still images by a video decoder (500). The method further comprising the steps of detecting a plurality of behaviour displayed within the sequence of still images by the video processing unit (600), visualising active monitoring by a smart client unit (700), and visualising inactive monitoring by the smart client unit (700). The plurality of behaviours being detected relates to cell violation, perimeter violation, and presence detection. The active monitoring refers to monitoring at least one real-time live feed video stream from the plurality of capturing devices (100). The inactive monitoring refers to monitoring at least one video stream from a recording unit (200) or a media storage.
Preferably, the step of detecting the plurality of behaviour displayed within the sequence of still images by the video processing unit (600) further includes the substeps of identifying behaviour displayed within the sequence of still images by a video analytics agent, VAA, creating an alert trigger tag, ATT, for each behaviour that has been identified, recording each ATT in a configuration database (401), generating event clip and snapshot for each ATT, storing the event clip and snapshot for each ATT in the media storage, overlaying the event metadata on the at least one video stream, and transmitting the at least one video stream overlaid with the event metadata.
Preferably, the step of visualising active monitoring by the smart client unit (700) further comprising sub-steps of receiving configuration from the video processing unit (600), receiving an alert trigger tag, ATT, for each detected event, extracting information from the ATT, and displaying the at least one video stream overlaid with
the event metadata based on the ATT. The information includes capturing device information and the event metadata
Preferably, the step of visualising inactive monitoring by the smart client unit (700) further comprising sub-steps of receiving at least one keyword for database search from a user, matching the at least one keyword with an alert trigger tag, ATT, in a configuration database (401), displaying search result of the ATT that matches the at least one keyword, selecting at least one event from the search result, retrieving the event clip and event snapshot of at least one selected event from the media storage based on the information from the ATT, displaying the event clip and snapshot of the at least one selected event, and compiling the search result, the event clip, and the event snapshot in a report.
BRIEF DESCRIPTION OF THE DRAWINGS
The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate embodiments of the invention and, together with the description, serve to explain the principles of the invention.
FIG. 1 illustrates a block diagram of a system (1000) for monitoring behaviour according to an embodiment of the present invention.
FIG. 2 illustrates a flow chart of a method for monitoring behaviour according to an embodiment of the present invention.
FIG. 3 illustrates a flowchart for a step of detecting a plurality of behaviours displayed within the sequence of still images by the video processing unit (600) of the method of
FIG. 2.
FIG. 4 illustrates a flow chart fora step of visualising active monitoring by a smart client unit (700) of the method of FIG. 2.
FIG. 5 illustrates a flow chart for a step of visualising inactive monitoring by the smart client unit (700) of the method of FIG. 2.
DESCRIPTION OF THE PREFERRED EMBODIMENT
A preferred embodiment of the present invention will be described hereinbelow with reference to the accompanying drawings. In the following description, well-known functions or constructions are not described in detail since they would obscure the description with unnecessary detail.
An initial reference is made to FIG. 1 which illustrates a block diagram of a system (1000) for monitoring behaviour according to an embodiment of the present invention. The system (1000) is configured to produce an alert signal if at least one person being monitored in a confined space exhibits a certain number of prohibited behaviours. The prohibited behaviours are predetermined by an operator of the system (1000). The system (1000) comprises a recording unit (200), a video acquisition unit (300), a configuration unit (400), a configuration database (401), a video processing unit (600), and a smart client unit (700).
The recording unit (200) is connected to a plurality of capturing devices (100). The capturing devices (100) may be in the form of an analogue, digital, or smart video camera. The recording unit (200) records at least one video stream from the capturing devices (100).
The video acquisition unit (300) is connected to the capturing devices (100). The video acquisition unit (300) acquires at least one video stream from the capturing devices (100). Alternatively, the video acquisition unit retrieves at least one recorded video stream from the recording unit (200). The video acquisition unit (300) is connected to the configuration unit (400).
The configuration unit (400) produces configuration information relating to the capturing devices (100) and video analytics for the capturing devices (100). The configuration information for the capturing devices (100) includes but not limited to the location of the capturing device, Internet Protocol address, and MAC address. The configuration information relating to the video analytics for the capturing devices (100) is produced by determining suitable video analytics types with respect to the capturing devices (100), monitoring area that includes a region of interest and line of interest, and video analytics parameter values. Examples of video analytic types are motion detection, object recognition, and video tracking. The configuration unit (400) is
connected to the video decoder (500) and the configuration database (401). The configuration information for the capturing devices (100) and video analytics information is sent to the configuration database (401). The configuration database (401) stores configuration information for the plurality of capturing devices (100) and video analytics information.
The video decoder (500) converts the video stream into a sequence of still images. The video decoder (500) is connected to the video processing unit (600).
The video processing unit (600) is connected to the configuration database (401). The video processing unit (600) receives the still images from the video decoder (500) and video analytics information from the configuration database (401). The video processing unit (600) analyses the still images for behaviour detection based on the video analytic type assigned to the video stream by the configuration unit (400). The video processing unit (600) analyses the type of behaviour showed in the video stream and determined whether any detected behaviour is deemed to be against the security and safety policy of the building.
The video processing unit (600) further comprises a plurality of video analytics agents, VAA, configured to analyse the video stream for any behaviour that is considered as cell violation, perimeter violation, and presence detection. Each capturing devices (100) is assigned a VAA for detecting cell violation, a VAA for detecting perimeter violation, and a VAA for presence detection.
Cell violation is any safety violation that happens within a cell or confined area. Action that constitutes cell violation includes but not limited to climbing, loitering, aggressive behaviour or fighting and sudden stand-up. Loitering is considered as cell violation only if the loitering happens for longer than a predetermined period.
Perimeter violation is any safety violation that happens within a building except the cell or confined area. Perimeter violation includes but not limited to entering a predefined restricted area, moving against a valid direction, and crowding behaviours.
Presence detection constitutes direction detection and appearance detection. Direction detection involves observing any people entering and leaving a monitored
area while appearance detection involves monitoring movement of people inside the monitored area. By detecting the presence of people at various location, a statistic of traffic flow of people is generated.
The video processing unit (600) outputs event metadata regarding detected behaviour in the video stream. Any behaviour that is considered as cell violation, perimeter violation, and presence detection is considered as an event. Examples of the event metadata include but not limited to a bounding box of a detected object or behaviour, timestamp of a detected event, and event alert message. The event metadata is stored in the configuration database (401). The video processing unit (600) also overlays the event metadata onto the video stream. The video processing unit (600) creates an event clip and event snapshot of detected behaviour and stores the event clip and event snapshot in a media storage. The event clip is a video clip that only shows the portion of the video stream that contains the detected event. The event snapshot is a sequence of still images that shows the detected event. The video processing unit (600) is connected to the smart client unit (700).
The smart client unit (700) displays and visualises active and inactive monitoring. Active monitoring refers to monitoring a real-time live feed video stream while inactive monitoring refers to monitoring a video stream from an archive such as the recording unit (200) or the media storage. The smart client unit (700) visualises the detected behaviour analysed by the video processing unit (600) by interpreting the event metadata assigned to the video stream by the video processing unit (600). The smart client unit (700) displays the video stream with an overlay of information. The smart client unit (700) also enables the user to receive an alert, communicate, and search a database for a specific event.
The smart client unit (700) further comprises three modules. The modules are an event module (701), integrated viewing module (702), and a forensic module (703). The event module (701) outputs the video stream with an overlay of the event metadata. The event metadata includes but not limited to alert notification message, visual alert alarm, audio alert alarm, and monitoring area. The event module (701) also displays the location of the capturing devices (100) that detect an event by showing the location of the camera within a 3-dimensional model of the building. The event
module (701) also highlights the location of the cell which has the detected event. The user is able to obtain information about a detainee occupying each cell.
The event module (701) also summarises all detected events in a timeline sequence. Each detected event is summarised into an event detail. The event detail includes a snapshot of the detected event, date, time, location of the event, and the type of event. The event module (701) also enables live reporting and charting of statistical information of the event while depicting the distribution of detainees. The event module (701) is connected to the integrated viewing module (702).
The integrated viewing module (702) retrieves video stream overlaid with the event metadata and other information from the event module (701). The integrated viewing module (702) displays multiple video stream from multiple capturing devices (100) with an overlay of the event metadata. When an event is detected, the integrated viewing module (702) visualises and displays the location of the capturing device (100) on a 2-dimensional map and event details with event snapshot. The integrated viewing module (702) also enables the user to verbally communicate with a person seen in a live stream.
The forensic module (703) enables the user to use at least one keyword to search for an event and retrieve the video stream depicting that event. The forensic module (703) matches the keyword with the event metadata stored in the configuration database (401) and retrieves the event clip and event snapshot from the media storage. The forensic module (703) is connected to the configuration database (401).
Reference is now made to FIG. 2 which illustrates a flow chart of a method for monitoring behaviour according to an embodiment of the present invention. The method detects a plurality of behaviours that violates security and safety policy of a building. The behaviours are detected by tracking the action of at least one object. Initially, at least one video stream is acquired by the video acquisition unit (300) as in step 3100. The video stream may be acquired either directly from the capturing devices (100) or from the recording unit (200).
The video stream is then converted into a sequence of still images by the video decoder (500) as in step 3200.
A plurality of behaviour displayed within the sequence of still images is then detected by the video processing unit (600) as in step 3300. The video processing unit (600) retrieves the configuration information and video analytics information from the configuration database (401). Based on the video analytics type assigned to the video stream, behaviour detection is performed by the video processing unit (600). Any behaviours detected in the video stream is analysed to determine whether the behaviours are against the security and safety policy of the building. The event metadata regarding any behaviours detected in the video stream is produced by the video processing unit (600). The step of detecting the plurality of behaviour displayed within the sequence of still images by the video processing unit (600) is further described in relation to FIG. 3.
The smart client unit (700) visualises active monitoring as in step 3400. Active monitoring refers to monitoring the video stream in real time. The event metadata regarding any behaviours detected in the video stream is interpreted by the smart client unit (700). The video stream is displayed by the smart client unit (700) with the overlay of the event metadata regarding detected behaviours. Preferably, a plurality of video stream is simultaneously displayed by the smart client unit (700). The step of visualising active monitoring by the smart client unit (700) is further described in relation to FIG. 4.
The smart client unit (700) visualises inactive monitoring as in step 3500. Inactive monitoring refers to monitoring the video stream from the recording unit (200) or the media storage. The media storage contains a video clip or snapshot of detected behaviours. Specific detected behaviour may then be searched by the keyword by the user. The step of visualising inactive monitoring by the smart client unit (700) is further described in relation to FIG. 5.
Reference is now made to FIG. 3 which illustrates a flowchart for the step of detecting the plurality of behaviour displayed within the sequence of still images by the video processing unit (600) of the method of FIG. 2. Initially, any behaviour that is considered as cell violation, perimeter violation, and presence detection displayed within the sequence of still images is identified by the VAA as in step 3310.
If any behaviour that is considered as cell violation, perimeter violation, and presence detection is detected, the video processing unit (600) creates an alert trigger tag, ATT, as in step 3320. The ATT contains the information on the type of detected behaviour and the information of the capturing device (100) that produced the video stream that shows the detected behaviour. Each ATT is recorded in the configuration database (401) as in step 3330.
The event clip and snapshot of detected behaviour for each ATT is generated as in step 3340. The event clip is a video clip that only shows the portion of the video stream that contains detected behaviour. The event snapshot is a still image that shows the detected behaviour. The event clip and snapshot is stored in the media storage for record purposes as in step 3350. An address of the event clip and snapshot is tagged to the ATT. The address is a location of the event clip and snapshot within the media storage. The address enables the smart client unit (700) to retrieve the event clip and snapshot from the media storage.
All event metadata from detected behaviour are then overlaid on the video stream as in step 3360. The event metadata includes but not limited to the bounding box of detected object and behaviour, timestamp, and event alert message.
The video stream overlaid with the event metadata is then transmitted to the smart client unit (700) as in step 3370.
Reference is now made to FIG. 4 which illustrates a flow chart for a step of visualising active monitoring by the smart client unit (700) of the method of FIG. 2. Initially, the smart client unit (700) receives configuration information of the capturing devices (100), video analytics information, and the video stream overlaid with the event metadata from the video processing unit (600) as in step 3410.
The smart client unit (700) then receives the ATT for each event from the video processing unit (600) as in step 3420. If a behaviour is simultaneously detected in multiple video streams from multiple capturing devices (100), the smart client unit (700) receives multiple ATT from the video processing unit (600). The user is able to view any behaviour that has been detected from multiple points of view if the behaviour is
simultaneously detected in multiple video streams from multiple capturing devices
(100).
The information within the ATT is then extracted as in step 3430. The information includes the event metadata and capturing device information.
The video stream overlaid with the event metadata is then displayed as in step 3440. The video stream overlaid with the event metadata is displayed in either an integrated viewing mode or a high-resolution mode. In the integrated viewing mode, a plurality of video stream overlaid with the event metadata is displayed simultaneously. In the high-resolution mode, only one video stream overlaid with the event metadata is displayed at a time.
Reference is now made to FIG. 5 which illustrates a flow chart for a step of visualising inactive monitoring by the smart client unit (700) of the method of FIG. 2. Initially, the keyword for database search is received from a user as in step 3510. The keyword may include the date, time, location of the capturing device (100), IP address of capturing device (100), and type of detected behaviour. The keyword is then matched with the ATTs in the configuration database (410) as in step 3520.
A search result is displayed as in step 3530. The search result is a list of ATTs that matches the keyword.
At least one event from the search result is selected by the user as in step 3531. The user may select from the search result any event that the user wants to investigate. The event clip and event snapshot of the selected event are retrieved from the media storage based on the information from the ATT as in step 3532.
The event clip and the event snapshot of the selected event are displayed as in step 3540. The event clip and event snapshot may be displayed in the integrated viewing mode, wherein all event clip and event snapshot from multiple capturing
devices (100) that shows a similar detected event may be displayed simultaneously. Alternatively, only one event clip or event snapshot is displayed at a time.
The search result, the event clip, and the event snapshot are then compiled in a report as in step 3550.
While embodiments of the invention have been illustrated and described, it is not intended that these embodiments illustrate and describe all possible forms of the invention. Rather, the words used in the specifications are words of description rather than limitation and various changes may be made without departing from the scope of the invention.
Claims
1 . A system (1000) for monitoring behaviour comprising: a) a video acquisition unit (300) configured to acquire at least one video stream from a plurality of capturing devices (100), b) a configuration unit (400) configured to produce configuration information relating to the plurality of capturing devices (100) and video analytics information relating to the plurality of capturing devices (100), wherein the configuration unit (400) is connected to the video acquisition unit (300), c) a configuration database (401) configured to store configuration information for the plurality of capturing devices (100) and video analytics information, wherein the configuration database (401) is connected to the configuration unit (400), d) a video decoder (500) converts the at least one video stream into a sequence of still images, wherein the video decoder (500) is connected to the configuration unit (400), and e) a recording unit (200) configured to record the at least one video stream from the plurality of capturing devices (100), characterised in that the system (1000) further comprising: f) a video processing unit (600) configured to analyse the at least one video stream for behaviour detection, wherein the video processing unit (600) is connected to the configuration database (401) and the video decoder (500); and g) a smart client unit (700) configured to display and visualise active and inactive monitoring, wherein the smart client unit (700) is connected to the video processing unit (600) and the configuration database (401).
2. The system (1000) as claimed in claim 1 , wherein the smart client unit (700) further comprising an event module (701) configured to output the at least one video stream with an overlay of event metadata.
3. The system (1000) as claimed in claim 1 , wherein the smart client unit (700) further comprising an integrated viewing module (702) configured to display multiple video streams from the plurality of capturing devices (100) with an
overlay of event metadata and enable the user to communicate with a person seen in a live stream.
4. The system (1000) as claimed in claim 1 , wherein the smart client unit (700) further comprising a forensic module (703) configured to enable the user to search for an event based on at least one keyword and retrieve at least one video stream depicting the event.
5. A method for monitoring behaviour comprising the steps of: a) acquiring at least one video stream from a plurality of capturing devices (100) or a recording unit (200) by a video acquisition unit (300), and b) decoding the at least one video stream into a sequence of still images by a video decoder (500), characterised in that the method further comprising the steps of: c) detecting a plurality of behaviours displayed within the sequence of still images by the video processing unit (600), wherein the plurality of behaviours being detected relates to cell violation, perimeter violation, and presence detection; d) visualising active monitoring by a smart client unit (700), wherein the active monitoring refers to monitoring at least one real-time live feed video stream from the plurality of capturing devices (100); and e) visualising inactive monitoring by the smart client unit (700), wherein the inactive monitoring refers to monitoring at least one video stream from a recording unit (200) or a media storage.
6. The method as claimed in claim 5, wherein the step of detecting the plurality of behaviour displayed within the sequence of still images by the video processing unit (600) further includes the sub-steps of: a) identifying behaviour displayed within the sequence of still images by a video analytics agent, VAA; b) creating an alert trigger tag, ATT, for each behaviour that has been identified; c) recording each ATT in a configuration database (401); d) generating event clip and snapshot for each ATT; e) storing the event clip and snapshot for each ATT in the media storage; f) overlaying event metadata on the at least one video stream; and
g) transmitting the at least one video stream overlaid with the event metadata.
7. The method as claimed in claim 5, wherein the step of visualising active monitoring by the smart client unit (700) further comprising sub-steps of: a) receiving configuration from the video processing unit (600); b) receiving the ATT, for each detected event; c) extracting information from the ATT, wherein the information includes capturing device information and the event metadata; and d) displaying the at least one video stream overlaid with the event metadata based on the ATT.
8. The method as claimed in claim 5, wherein the step of visualising inactive monitoring by the smart client unit (700) further comprising the sub-steps of: a) receiving at least one keyword for database search from a user; b) matching the at least one keyword with the ATT, in a configuration database (401); c) displaying search result of the ATT that matches the at least one keyword; d) selecting at least one event from the search result; e) retrieving the event clip and event snapshot of at least one selected event from the media storage based on the information from the ATT; f) displaying the event clip and snapshot of the at least one selected event; and g) compiling the search result, the event clip, and the event snapshot in a report.
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
MYPI2019007939 | 2019-12-31 | ||
MYPI2019007939 | 2019-12-31 |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2021137687A1 true WO2021137687A1 (en) | 2021-07-08 |
Family
ID=76686938
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/MY2020/050143 WO2021137687A1 (en) | 2019-12-31 | 2020-11-11 | A system and method for monitoring behaviour |
Country Status (1)
Country | Link |
---|---|
WO (1) | WO2021137687A1 (en) |
Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20040240542A1 (en) * | 2002-02-06 | 2004-12-02 | Arie Yeredor | Method and apparatus for video frame sequence-based object tracking |
US8331611B2 (en) * | 2009-07-13 | 2012-12-11 | Raytheon Company | Overlay information over video |
US9087386B2 (en) * | 2012-11-30 | 2015-07-21 | Vidsys, Inc. | Tracking people and objects using multiple live and recorded surveillance camera video feeds |
KR101793636B1 (en) * | 2017-05-31 | 2017-11-03 | 주식회사 투윈스컴 | Image processing system |
CN206921109U (en) * | 2017-04-24 | 2018-01-23 | 浙江中博信息工程有限公司 | A kind of behavior analysis system |
-
2020
- 2020-11-11 WO PCT/MY2020/050143 patent/WO2021137687A1/en active Application Filing
Patent Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20040240542A1 (en) * | 2002-02-06 | 2004-12-02 | Arie Yeredor | Method and apparatus for video frame sequence-based object tracking |
US8331611B2 (en) * | 2009-07-13 | 2012-12-11 | Raytheon Company | Overlay information over video |
US9087386B2 (en) * | 2012-11-30 | 2015-07-21 | Vidsys, Inc. | Tracking people and objects using multiple live and recorded surveillance camera video feeds |
CN206921109U (en) * | 2017-04-24 | 2018-01-23 | 浙江中博信息工程有限公司 | A kind of behavior analysis system |
KR101793636B1 (en) * | 2017-05-31 | 2017-11-03 | 주식회사 투윈스컴 | Image processing system |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
KR101133924B1 (en) | Active image monitoring system using motion pattern database, and method thereof | |
KR102058452B1 (en) | IoT Convergence Intelligent Video Analysis Platform System | |
US11308777B2 (en) | Image capturing apparatus with variable event detecting condition | |
EP1346577B1 (en) | Method and apparatus to select the best video frame to transmit to a remote station for cctv based residential security monitoring | |
EP3016382B1 (en) | Monitoring methods and devices | |
US8675074B2 (en) | Custom video composites for surveillance applications | |
CA2545535C (en) | Video tripwire | |
US20130011014A1 (en) | Surveillance system and method | |
TW201826141A (en) | A method for generating alerts in a video surveillance system | |
KR20070116715A (en) | Home security applications for television with digital video cameras | |
CA2460426A1 (en) | Video tripwire | |
US20200099892A1 (en) | Surveillance systems and methods for automatic real-time monitoring | |
US20170300751A1 (en) | Smart history for computer-vision based security system | |
KR101075550B1 (en) | Image sensing agent and security system of USN complex type | |
JP6893204B2 (en) | Methods, devices and systems for detecting wandering events | |
JP2006093955A (en) | Video processing apparatus | |
US20110157355A1 (en) | Method and System for Detecting Events in Environments | |
CN110097722A (en) | Vehicle monitoring alarm method and device based on image procossing | |
CN110557676B (en) | System and method for determining and recommending video content active areas of a scene | |
WO2021137687A1 (en) | A system and method for monitoring behaviour | |
US20110157431A1 (en) | Method and System for Directing Cameras | |
KR20100077662A (en) | Inteligent video surveillance system and method thereof | |
US9111237B2 (en) | Evaluating an effectiveness of a monitoring system | |
US20030004913A1 (en) | Vision-based method and apparatus for detecting an event requiring assistance or documentation | |
CN110855932A (en) | Alarm method and device based on video data, electronic equipment and storage medium |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 20910163 Country of ref document: EP Kind code of ref document: A1 |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 20910163 Country of ref document: EP Kind code of ref document: A1 |