WO2018198373A1 - 映像監視システム - Google Patents
映像監視システム Download PDFInfo
- Publication number
- WO2018198373A1 WO2018198373A1 PCT/JP2017/017077 JP2017017077W WO2018198373A1 WO 2018198373 A1 WO2018198373 A1 WO 2018198373A1 JP 2017017077 W JP2017017077 W JP 2017017077W WO 2018198373 A1 WO2018198373 A1 WO 2018198373A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- person
- image
- target person
- abandoned object
- video surveillance
- Prior art date
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/172—Classification, e.g. identification
- G06V40/173—Classification, e.g. identification face re-identification, e.g. recognising unknown faces across different face tracks
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
- H04N7/181—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast for receiving images from a plurality of remote sources
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
Definitions
- the present invention relates to a video monitoring system that monitors a video taken by an imaging apparatus, and more particularly to a system that identifies a person who has left an abandoned object and tracks the movement of the person.
- an abandoned object detection system for example, Patent Document 1 that detects an abandoned object by monitoring an image
- a person search system for example, Patent Document 2 that searches for a designated person from within the image.
- the abandoned object detection system and the person search system have been operated as separate systems and operated independently of each other. For this reason, for example, in order to identify a person who left an abandoned object and track the movement of the person, the user has to perform the following procedure. (1) Confirm the report (message to notify the detection of abandoned objects) from the abandoned object detection system (2) Reverse playback of the image showing the abandoned object (3) Visually observe the video when the abandoned object is left Identifies a person who has been neglected (4) Performs a person search specifying a person who has been neglected in the person search system (5) Checks the result display of the person search by the person search system
- the present invention has been made in view of the above-described conventional circumstances, and is an image that can reduce a user's burden of identifying a person who has left an abandoned object and tracking the movement of the person.
- the purpose is to provide a monitoring system.
- a video surveillance system or video surveillance method is configured as follows.
- a video surveillance system including a plurality of imaging devices, an abandoned object detection unit that detects an abandoned object reflected in an image captured by any of the plurality of imaging devices, and the abandoned object is detected
- a person specifying unit that specifies an image immediately before or after the appearance of the abandoned object and specifies at least one person reflected in the image as a target person
- the information indicating the movement of the target person is displayed on the screen based on the person search unit that searches for the image showing the target person, the imaging device that captured each image showing the target person, and the imaging time.
- a result display unit is configured as follows.
- a person feature amount acquisition unit that acquires a person feature amount including a face feature amount and a clothing feature amount of the target person based on the specified image is further provided.
- the person retrieval unit retrieves an image showing the target person based on the person feature amount.
- the result display unit displays a route on which the target person has moved on a map as a display indicating the movement of the target person, or , And displaying on the screen a display in which each image showing the target person is arranged in time-series order for each imaging device.
- the person specifying unit has a distance from the abandoned object most when a plurality of persons are detected from the specified image.
- a close person is specified as the target person.
- the distance from the abandoned object is predetermined. It is characterized in that the number of persons below the threshold value or the top several persons with the shorter distance are specified as the target person.
- a step of detecting an abandoned object reflected in an image captured by any of the plurality of imaging devices, and a case where the abandoned object is detected Identifying an image immediately before or immediately after the abandoned object appears, identifying at least one person reflected in the image as a target person, and images taken by each of the plurality of imaging devices, A step of searching for an image showing the target person, and a step of outputting a display showing the movement of the target person on the screen based on an imaging device and an imaging time for capturing each image showing the target person. It is characterized by.
- a video surveillance system capable of reducing the user burden of the work of identifying a person who has left an abandoned object and tracking the movement of the person.
- FIG. 1 It is a figure which shows the structural example of the video surveillance system which concerns on one Embodiment of this invention. It is a figure which shows the example of the hardware constitutions of the monitoring server shown in FIG. It is a figure which shows the example of the processing flow of the object person tracking by the monitoring server shown in FIG. It is a figure which shows the example of a display of the result of object person tracking. It is a figure which shows another example of a display of the result of object person tracking.
- FIG. 1 shows a configuration example of a video surveillance system according to an embodiment of the present invention.
- the video monitoring system of this example includes a plurality of N imaging devices 100 (1) to 100 (N), a monitoring server 200, and a user terminal 300, and communication between these devices is performed via a network 400. Done.
- Video surveillance systems are deployed in facilities visited by an unspecified number of people, such as airports, train stations, shopping malls, and the like.
- the network 400 is used for exchanging signals and data between devices.
- any network form such as a LAN (Local Area Network), a WAN (Wide Area Network), the Internet, or the like can be used.
- Each device of the video surveillance system can communicate with other devices via the network 400 using a wired line, a wireless line, or a combination thereof.
- the imaging devices 100 (1) to 100 (N) are devices such as IP cameras (network cameras) installed at various locations in the facility where the video surveillance system is installed.
- Each of the imaging devices 100 includes an imaging device such as a charge coupled device (CCD) or a complementary metal oxide semiconductor (CMOS), and digitally converts the captured image and transmits the image to the monitoring server 200 via the network 400.
- CCD charge coupled device
- CMOS complementary metal oxide semiconductor
- the user terminal 300 is a device used by a user (monitor) of this system.
- the user terminal 300 has a function of receiving an operation input from a user and transmitting input data to the monitoring server 200, and a function of receiving provided data transmitted from the monitoring server 200 and displaying the screen.
- the user terminal 300 may be a stationary device or a portable device such as a smart phone or a tablet terminal, and various devices can be used.
- the monitoring server 200 identifies a person (for example, a suspicious person) who left an abandoned object as a target person based on videos (images) captured by each of the plurality of imaging devices 100 and tracks the movement of the target person.
- This is a server having a target person tracking function.
- the monitoring server 200 includes an image storage unit 201, an abandoned object detection unit 202, a person specifying unit 203, a person feature amount acquisition unit 204, a person search unit 205, and a result display unit as processing units related to the target person tracking function. 206.
- the image storage unit 201 performs storage processing for images transmitted from each of the plurality of imaging devices 100.
- the image storage unit 201 stores the image received from the imaging apparatus 100 in a randomly accessible storage medium such as an HDD (Hard Disk Disk Drive). Further, attribute data such as an imaging device number for identifying the imaging device 100 that captured the image and an imaging time of the image are assigned to the image received from the imaging device 100, and these attribute data are also stored in the storage medium. .
- HDD Hard Disk Disk Drive
- the abandoned object detection unit 202 performs an abandoned object detection process for detecting an abandoned object reflected in an image captured by any of the plurality of imaging devices 100.
- the abandoned object is detected by comparing a captured image with a reference image prepared in advance. More specifically, the captured image and the reference image are binarized, and a difference for each pixel is obtained between the binarized images. Next, a pixel area having a difference value equal to or greater than a predetermined threshold is extracted, and when the extracted pixel area is a cluster larger than a predetermined size, it is estimated that there is an abandoned object in the pixel area.
- the same processing is repeated for subsequent captured images from the same imaging apparatus 100, and it is determined that there is an abandoned object when there is no change in the pixel area that is estimated to be an abandoned object even after a predetermined time has elapsed. . That is, what is reflected in the corresponding pixel area is detected as an abandoned object.
- the extracted pixel area is subjected to expansion / contraction processing, numbering, noise component removal processing, etc., and then a pixel area having a block larger than a predetermined size is detected as an abandoned object. It is preferable to do.
- the person specifying unit 203 reflects the image at the time when the abandoned object is placed as the image at the time of leaving and the image when left unattended.
- a person specifying process for specifying at least one person as a target person is performed.
- the unattended image is specified by processing a series of images received from the imaging apparatus 100 that has captured an image in which an unattended object has been detected, in time-series order or vice versa.
- trace the abandoned object detection processing log to identify the image at the time when the abandoned object was found (when it was estimated that there was an abandoned object), and process the image in the reverse playback direction (from the past) from that point I will do it.
- the image at the moment when the abandoned object disappears and the original background image appears, that is, the image immediately before the abandoned object is placed is specified as the image at the time of leaving.
- the image is processed in the forward playback direction from that point (toward the future). To go. Then, the image at the moment when the original background image disappears and the abandoned object appears, that is, the image immediately after the abandoned object is placed is specified as the image at the time of the abandonment.
- a person detection process is performed on the image when left unattended, and a person reflected in the image when left unattended is detected.
- the person detection process can be performed using any method, and for example, a known face recognition technique can be used. If the detected person is one person, the person is specified as the target person. On the other hand, when there are a plurality of detected persons, the distance between the position of the left object and the position of each person in the left image is calculated, and the person with the shortest calculated distance is specified as the target person. Alternatively, a person whose calculated distance is equal to or smaller than a predetermined threshold or a higher number of persons from a shorter calculated distance is specified as a target person.
- the position of the abandoned object is a position representative of the image area of the abandoned object in the image when left unattended.
- the center coordinates of the image area of the abandoned object can be used.
- the position of the person is a position representing the image area of the person in the image when left unattended.
- the center coordinates of the image area of the entire person, the center coordinates of the image area of the head or face, the hand closer to the left object can be used.
- the person feature amount acquisition unit 204 performs a person feature amount acquisition process for acquiring a person feature amount based on an image showing a person.
- a person feature amount related to the person shown in the received image is acquired.
- the person feature amount acquisition unit 204 causes the image storage unit 201 to store the acquired person feature amount in association with the acquisition source image.
- a plurality of person feature amounts are associated with an image showing a plurality of persons.
- the person feature amount acquisition unit 204 acquires the person feature amount of the target person based on the left image specified by the person specifying unit 203.
- the person feature amount of the target person is used as a person search search key in the person search unit 205.
- the person feature amount is a value that represents the feature of the person included in the image.
- the feature amount of the face indicates the feature of the face, or the clothing that indicates the characteristics of the clothing (including hats, belongings, accessories, etc.). It contains at least feature values.
- the facial feature amount can be acquired by, for example, quantifying the shape and direction of the face outline, the color of the skin, the size, shape, and arrangement relationship of the main components (eyes, nose, mouth, etc.) of the face.
- the clothing feature amount is, for example, the position of the person's head, upper body, lower body, etc., and the color or shape of the clothing worn by the person (eg, hat, outerwear, pants, belongings, accessories) It can be obtained by digitizing the above.
- the face feature amount and the clothing feature amount can be acquired using an arbitrary method, and for example, can be acquired using the method disclosed in Patent Document 1.
- the person search unit 205 selects a target from among images captured by each of the plurality of imaging devices 100 based on the person feature amount of the target person acquired based on the left image specified by the person specifying unit 203.
- a person search process is performed to search for an image showing a person.
- the person search process for example, when the person feature amount acquired for the target person is compared with the person feature amount acquired from another image to calculate the similarity, and the similarity is equal to or greater than a predetermined threshold, the target person Is determined to be an image in which
- a past search in which an image in the image storage unit 201 is searched backward from the current time, and a real-time in which the search processing is performed on an image transmitted from the imaging apparatus 100 thereafter. Search and do.
- the result display unit 206 performs tracking result display processing for causing the user terminal 300 to output a display indicating the movement of the target person based on the search result by the person search unit 205.
- the movement of the target person can be tracked based on the imaging device number and the imaging time associated with each image of the search result. For example, it is possible to track the movement of the target person in the order of the shooting area of the imaging device 100 corresponding to the imaging device number by specifying the arrangement order of the imaging device numbers when the images of the search result are sorted in order of the imaging time. .
- a display example of information indicating the movement of the target person (result of target person tracking) will be described later.
- FIG. 2 shows an example of the hardware configuration of the monitoring server 200.
- the monitoring server 200 includes, for example, a CPU (Central Processing Unit) 221, a main storage unit 222, an auxiliary storage unit 223, and a network interface (I / F) 224 connected by a bus 225.
- a CPU Central Processing Unit
- main storage unit 222 main storage unit
- auxiliary storage unit 223 main storage unit
- network interface (I / F) 224 connected by a bus 225.
- the CPU 221 controls each part of the monitoring server 200 and executes programs for realizing various functions according to the present invention.
- the main storage unit 222 is realized by a semiconductor device such as a DRAM (Dynamic Random Access Memory), and is an intermediate memory for loading and storing programs executed by the CPU 221.
- the auxiliary storage unit 223 is realized by an HDD, a flash memory, or the like, has a larger capacity than the main storage unit 222, and stores various data and programs.
- the network I / F 224 performs reception of an image transmitted from the imaging apparatus 100, reception of various input data transmitted from the user terminal 300, transmission of various provided data to the user terminal 300, and the like via the network 400. Interface.
- the monitoring server 200 in this example stores a program that defines the operation of each of the above-described processing units 201 to 206 in the auxiliary storage unit 223.
- the program is loaded into the main storage unit 222 and executed by the CPU 221 so that each process is executed.
- the units 201 to 206 are configured to be realized. Note that some or all of the processing units 201 to 206 may be realized by a dedicated hardware circuit instead of being realized by the software as described above.
- FIG. 3 is a diagram illustrating an example of a processing flow of target person tracking by the monitoring server 200.
- the monitoring server 200 performs the abandoned object detection process by the abandoned object detection unit 202 (step S101).
- the abandoned object detection process is performed every time an image is received from the imaging apparatus 100 or every time the received image is stored in the image storage unit 201.
- the monitoring server 200 detects an abandoned object, the monitoring server 200 issues a message to that effect to the user terminal 300 to notify the user of the user terminal 300 of the detection of the abandoned object.
- the monitoring server 200 specifies the image at the moment when the abandoned object is placed as the image at the time of leaving by performing the image specifying process at the time of leaving by the person specifying unit 203 (step S102).
- the monitoring server 200 performs person identification processing by the person identification unit 203 to identify at least one person reflected in the image when left as a target person (step S103).
- the monitoring server 200 acquires a person feature amount related to the target person by performing a person feature amount acquisition process by the person feature amount acquisition unit 204 (step S104).
- the monitoring server 200 searches for an image showing the target person by performing a person search process by the person search unit 205 based on the person feature amount related to the target person (step S105).
- the monitoring server 200 displays information indicating the movement of the target person on the screen of the user terminal 300 by performing a tracking result display process by the result display unit 206 (step S106).
- each processing of step S102 to step S106 may be performed after waiting for an instruction from a user who has confirmed a message indicating that an abandoned object has been detected, or may be performed immediately after detection of an abandoned object. If a plurality of target persons are specified in step S103, steps S104 to S106 may be repeated for each target person, or steps S104 to S106 may be performed for any target person. When processing is performed for any one of the target persons, the plurality of specified target persons may be presented to the user of the user terminal 300, and the target person may be selected from among them.
- FIG. 4 shows a display example of the target person tracking result.
- the display example of FIG. 4 includes a map display area 241 that shows a route on which the target person has moved on a map, and a timeline display area 246 that shows images showing the target person in time series.
- the map display area 241 shows the locations C1 to C4 of the imaging device and the points T1 to T6 where the target person is shown on the map of the area including the locations C1 to C4 of the imaging device that captured the target person. It is.
- a map of the facility where the present system is installed and the arrangement locations (C1 to C4) of the imaging devices installed at various locations in the facility may be stored in advance.
- the arrangement location of the imaging device that captured the target person can be specified from the imaging device number assigned to the image showing the target person, and displayed on the map.
- images taken at each point (T1 to T6) are displayed side by side in time series.
- the movement route 242 of the target person is displayed on the map in the map display area 241 in an overlapping manner.
- the movement path 242 can be estimated based on the coordinates of each point (T1 to T6), the time when the target person passed each point (image shooting time), a preset general walking speed, and the like.
- the result display unit 206 estimates the movement route 242.
- the movement route 242 is a route estimated based on the search result by the past search in the person search unit 205, and the subsequent movement route 243 of the target person may be added based on the search result by the subsequent real-time search. .
- a function that predicts a travel route that the target person is likely to pass in the future and displays it as a future travel route may be provided.
- FIG. 5 shows another display example of the target person tracking result.
- the display example of FIG. 5 includes a timeline detail display area 251 that displays each image showing a target person in time series in order of imaging devices.
- the timeline detail display area 251 is obtained by dividing the timeline display area 246 in the first display example (FIG. 4) for each imaging device. In this example, whether to use the display of FIG. 4 or the display of FIG. 5 is set in advance, but the user of the user terminal 300 may be able to select, or the display can be switched by the user's designation. You may do it. Further, the timeline detail display area 251 of FIG. 5 may be used instead of the timeline display area 246 of FIG.
- the abandoned object detection unit 202 detects an abandoned object reflected in an image taken by any of the plurality of imaging devices 100, and the person specifying unit 203
- an image immediately before or immediately after the appearance of an abandoned object is specified as an image when left, and at least one person reflected in the image when left is specified as a target person.
- An image in which the target person is shown is retrieved from images taken by each of the image pickup apparatuses 100, and the result display unit 206 selects the target based on the image pickup apparatus in which each image in which the target person is shown and the image pickup time are taken.
- the user terminal 300 is configured to output a display indicating the movement of the person on the screen. Therefore, according to the video monitoring system of this example, since the detection of the abandoned object to the movement tracking of the target person can be automated, the burden on the user in the facility monitoring is reduced, and the target person can be efficiently tracked in real time. It can be carried out.
- the video monitoring system of the present example further includes a person feature amount acquisition unit 204 that acquires a person feature amount including the face feature amount and the clothing feature amount of the target person based on the image when left unattended. Based on the person feature quantity of the target person, an image showing the target person is searched. Therefore, according to the video monitoring system of this example, since the search is performed using not only the facial feature quantity of the target person but also the clothing feature quantity, even when the target person's face is not sufficiently reflected in the image when left unattended, Other images showing the target person can be searched from the clothes of the person.
- the result display unit 206 displays, as a display indicating the movement of the target person, a display showing the route on which the target person has moved on the map, or each image in which the target person is reflected.
- a display that is arranged in time series order is configured to be output on the screen. Therefore, according to the video monitoring system of the present example, the user of the user terminal 300 allows the target person who left the abandoned object (including a suspicious person left intentionally or a person who was lost unintentionally) and the movement status of the person. Can be easily confirmed.
- the person specifying unit 203 is configured to specify the target person based on the length of time that is reflected together with the abandoned object when the image at the time of leaving cannot be specified. Yes. Therefore, according to the video monitoring system of the present example, even when an image when left unattended cannot be specified, a person who has a high possibility of leaving an abandoned object can be specified as a target person.
- the processing contents by the processing units 201 to 206 described above are merely examples, and various functions according to the present invention may be realized using other methods.
- the result of the person search may include not only the image showing the target person but also an image showing another person. If there are many images of another person, the person feature amount used for the person search is not appropriate. Conceivable. That is, when there are many different persons in the search result, it is assumed that the feature that distinguishes the target person from another person does not appear in the feature amount space. Therefore, we prepared several weight patterns with different weights given to each element of clothing features (hats, outerwear, pants, belongings, accessories, etc.), and applied each weight pattern individually.
- a person search may be tried, and a weight pattern that is easy to distinguish between the person in question and another person may be automatically selected.
- this pattern selection method for example, the following method can be applied.
- a person search is tried by applying each weight pattern, and the number of times searched for each person is recorded.
- a person who has been searched frequently has a high probability of being the person himself / herself.
- a predetermined number of persons having been searched many times are selected in the order of the number of searches. This person is called a candidate person.
- the weight pattern in which the number of principal candidate persons is searched for most among the search results of the respective weight patterns is selected.
- This weight pattern has a high probability of containing a large number of principals, and gives a large weight to a feature quantity that contributes to the distinction between the principal and another person, and a weight pattern that gives a small weight to a feature quantity that does not contribute.
- this weight pattern By using this weight pattern, the accuracy of person search can be improved.
- a person who apparently seems to be another person using the weight pattern that contains a large number of candidate individuals Can also be excluded. That is, for example, the target person can be narrowed down by garbage collection of the search result by increasing the probability that the target person is included by logical product processing of the search results.
- an image of the same person with good face appearance is selected from several frames before and after the image when left unattended taken by the same imaging device.
- the face feature amount (and the clothing feature amount) may be acquired from the selected image. Whether or not the person in the image is the same person can be determined from the positional relationship of the person's image area between the images (frames), the identity of the person's clothing feature quantity in each image, and the like.
- the present invention can also be provided as, for example, a method and method for executing the processing according to the present invention, a program for realizing such a method and method, and a storage medium for storing the program.
- the present invention can be used in a video monitoring system that monitors video captured by an imaging device.
- imaging device 200 monitoring server 300: user terminal 400: network 201: Image storage unit 202: Abandoned object detection unit 203: Person identification unit 204: Person feature amount acquisition unit 205: Person search unit 206: Result display unit 221: CPU, 222: main storage unit, 223: auxiliary storage unit, 224: network I / F
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Signal Processing (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Oral & Maxillofacial Surgery (AREA)
- Human Computer Interaction (AREA)
- Closed-Circuit Television Systems (AREA)
- Image Analysis (AREA)
- Alarm Systems (AREA)
Abstract
Description
(1)放置物検知システムからの発報(放置物の検知を知らせるメッセージ)を確認
(2)放置物を映した映像を逆再生
(3)放置物が放置された時点の映像を目視して、放置した人物を特定
(4)放置した人物を指定した人物検索を人物検索システムで実行
(5)人物検索システムによる人物検索の結果表示を確認
本発明は、上記のような従来の事情に鑑みて為されたものであり、放置物を放置した人物を特定してその人物の移動を追跡する作業のユーザ負担を軽減することが可能な映像監視システムを提供することを目的とする。
(1) 複数の撮像装置を備えた映像監視システムにおいて、前記複数の撮像装置のいずれかで撮影された画像に映った放置物を検知する放置物検知部と、前記放置物が検知された場合に、前記放置物が現れる直前又は直後の画像を特定し、当該画像に映った少なくとも一人の人物を対象人物に特定する人物特定部と、前記複数の撮像装置の各々で撮影された画像の中から、前記対象人物が映った画像を検索する人物検索部と、前記対象人物が映った各画像を撮影した撮像装置及び撮像時刻に基づいて、前記対象人物の移動を示す情報を画面に表示させる結果表示部と、を備えたことを特徴とする。
図1には、本発明の一実施形態に係る映像監視システムの構成例を示してある。本例の映像監視システムは、複数であるN台の撮像装置100(1)~100(N)と、監視サーバ200と、ユーザ端末300とを備え、これら装置間の通信はネットワーク400を介して行われる。映像監視システムは、空港、駅構内、ショッピングモール等のような不特定多数の人物が訪れる施設に配備される。
一例として、放置物検知処理のログを辿って放置物の発見時点(放置物があると推定された時点)の画像を特定し、その時点から逆再生方向に(過去に向かって)画像を処理していく。そして、放置物が消えて元々の背景画像が現れた瞬間の画像、すなわち、放置物が置かれる直前の画像を放置時画像として特定する。
また、別の例として、ある特定の時間を遡った過去の時点には放置物が存在しないことが判明している場合に、その時点から順再生方向に(未来に向かって)画像を処理していく。そして、元々の背景画像が消えて放置物が現れた瞬間の画像、すなわち、放置物が置かれた直後の画像を放置時画像として特定する。
監視サーバ200は、放置物検知部202により放置物検知処理を行う(ステップS101)。放置物検知処理は、撮像装置100から画像を受信する毎、あるいは受信した画像を画像蓄積部201に格納する毎に行われる。
監視サーバ200は、放置物を検知すると、その旨のメッセージをユーザ端末300に発報して、ユーザ端末300のユーザに放置物の検知を知らせる。また、監視サーバ200は、人物特定部203により放置時画像特定処理を行うことで、放置物が置かれた瞬間の画像を放置時画像として特定する(ステップS102)。次いで、監視サーバ200は、人物特定部203により人物特定処理を行うことで、放置時画像に映った少なくとも一人の人物を対象人物として特定する(ステップS103)。
また、ステップS103で複数の対象人物が特定された場合には、各々の対象人物についてステップS104~S106を繰り返してもよいし、いずれかの対象人物についてステップS104~S106を行ってもよい。なお、いずれかの対象人物について処理を行う場合には、特定された複数の対象人物をユーザ端末300のユーザに提示し、その中から対象人物をユーザに選択させればよい。
図4には、対象人物追跡の結果の表示例を示してある。図4の表示例は、対象人物が移動した経路を地図上で示す地図表示領域241と、対象人物が映った各画像を時系列順に並べて示すタイムライン表示領域246とを有している。
地図表示領域241には、対象人物を撮影した撮像装置の配置場所C1~C4を含むエリアの地図上に、撮像装置の配置場所C1~C4と、対象人物が映った地点T1~T6とを示してある。このような地図表示を実現するには、本システムを配備した施設内の地図と、施設内の各所に設置された撮像装置の配置場所(C1~C4)とを予め記憶しておけばよい。これにより、対象人物が映った画像に付与された撮像装置番号から、対象人物を撮影した撮像装置の配置場所を特定して、地図上に表示することができる。
また、タイムライン表示領域246には、各地点(T1~T6)で撮影された画像が、時系列順に並べて表示される。
本例では、図4の表示又は図5の表示のどちらを用いるかを予め設定してあるが、ユーザ端末300のユーザが選択できるようにしてもよいし、ユーザの指定により表示の切り替えができるようにしてもよい。また、図4のタイムライン表示領域246に代えて、図5のタイムライン詳細表示領域251を用いるようにしてもよい。
例えば、人物検索の結果には、対象人物が映った画像だけではなく別人が映った画像も含まれることがあるが、別人の画像が多い場合は、人物検索に用いる人物特徴量が適切でないと考えられる。つまり、検索結果に別人が多い場合には、特徴量空間上で対象人物本人と別人とを区別する特徴が表れていないことが想定される。そこで、着衣特徴量の各要素(帽子、上着、ズボン、持ち物、身飾品など)のそれぞれに与える重みを異ならせた幾つかの重みパターンを用意しておき、各重みパターンをそれぞれ適用して人物検索を試行し、結果の本人と別人が区別しやすい重みパターンを自動的に選択するようにしてもよい。このパターン選択方法として、例えば次のような方法を適用することができる。各重みパターンをそれぞれ適用して人物検索を試行し、人物毎に各試行で検索された回数を記録する。ここで、検索された回数が多い人物は本人である確度が高いと言える。次に、検索された回数の多い人物を、検索回数順に所定人数選択する。この人物を本人候補人物と呼ぶ。次に各重みパターンの検索結果の中で本人候補人物が最も多く検索された重みパターンを選択する。この重みパターンは、本人が多く含まれる確率が高く、本人と別人の区別に寄与する特徴量に対して大きな重みを与え、寄与しない特徴量には小さな重みを与えるような重みパターンとなる。この重みパターンを使用することにより、人物検索の精度を高めることができる。また、一つの重みパターンを用いて対象人物を最終判定するのではなく、対象人物を特定する前段階において、前記本人候補人物が多く含まれる重みパターンを使用して、明らかに別人と思われる人物を除外することもできる。すなわち、例えば検索結果の論理積処理により、対象人物が含まれる確率を高めることで、検索結果のごみ取りによる対象人物の絞り込みを行うこともできる。
また、本発明は、例えば、本発明に係る処理を実行する方法や方式、そのような方法や方式を実現するためのプログラム、そのプログラムを記憶する記憶媒体などとして提供することも可能である。
201:画像蓄積部、 202:放置物検知部、 203:人物特定部、 204:人物特徴量取得部、 205:人物検索部、 206:結果表示部、
221:CPU、 222:主記憶部、 223:補助記憶部、 224:ネットワークI/F
Claims (6)
- 複数の撮像装置を備えた映像監視システムにおいて、
前記複数の撮像装置のいずれかで撮影された画像に映った放置物を検知する放置物検知部と、
前記放置物が検知された場合に、前記放置物が現れる直前又は直後の画像を特定し、当該画像に映った少なくとも一人の人物を対象人物に特定する人物特定部と、
前記複数の撮像装置の各々で撮影された画像の中から、前記対象人物が映った画像を検索する人物検索部と、
前記対象人物が映った各画像を撮影した撮像装置及び撮像時刻に基づいて、前記対象人物の移動を示す表示を画面出力させる結果表示部と、
を備えたことを特徴とする映像監視システム。 - 請求項1に記載の映像監視システムにおいて、
前記特定された画像に基づいて、前記対象人物の顔特徴量及び着衣特徴量を含む人物特徴量を取得する人物特徴量取得部を更に備え、
前記人物検索部は、前記人物特徴量に基づいて、前記対象人物が映った画像を検索することを特徴とする映像監視システム。 - 請求項1又は請求項2に記載の映像監視システムにおいて、
前記結果表示部は、前記対象人物の移動を示す表示として、前記対象人物が移動した経路を地図上で示す表示、又は、前記対象人物が映った各画像を撮像装置別に時系列順に並べて示す表示を画面出力させることを特徴とする映像監視システム。 - 請求項1乃至請求項3のいずれかに記載の映像監視システムにおいて、
前記人物特定部は、前記特定した画像から複数の人物が検出された場合、前記放置物からの距離が最も近い人物を前記対象人物に特定することを特徴とする映像監視システム。 - 請求項1乃至請求項3のいずれかに記載の映像監視システムにおいて、
前記人物特定部は、前記特定した画像から複数の人物が検出された場合、前記放置物からの距離が所定の閾値以下の人物、又は、前記距離が短い方から上位数名を前記対象人物に特定することを特徴とする映像監視システム。 - 複数の撮像装置のいずれかで撮影された画像に映った放置物を検知する工程と、
前記放置物が検知された場合に、前記放置物が現れる直前又は直後の画像を特定し、当該画像に映った少なくとも一人の人物を対象人物に特定する工程と、
前記複数の撮像装置の各々で撮影された画像の中から、前記対象人物が映った画像を検索する工程と、
前記対象人物が映った各画像を撮影した撮像装置及び撮像時刻に基づいて、前記対象人物の移動を示す表示を画面出力させる工程と、
を含むことを特徴とする映像監視方法。
Priority Applications (5)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
JP2019515072A JP6674584B2 (ja) | 2017-04-28 | 2017-04-28 | 映像監視システム |
US16/490,043 US11017537B2 (en) | 2017-04-28 | 2017-04-28 | Image monitoring system |
EP17907868.8A EP3618427B1 (en) | 2017-04-28 | 2017-04-28 | Video monitoring system |
SG11201908010YA SG11201908010YA (en) | 2017-04-28 | 2017-04-28 | Image monitoring system |
PCT/JP2017/017077 WO2018198373A1 (ja) | 2017-04-28 | 2017-04-28 | 映像監視システム |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
PCT/JP2017/017077 WO2018198373A1 (ja) | 2017-04-28 | 2017-04-28 | 映像監視システム |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2018198373A1 true WO2018198373A1 (ja) | 2018-11-01 |
Family
ID=63919576
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/JP2017/017077 WO2018198373A1 (ja) | 2017-04-28 | 2017-04-28 | 映像監視システム |
Country Status (5)
Country | Link |
---|---|
US (1) | US11017537B2 (ja) |
EP (1) | EP3618427B1 (ja) |
JP (1) | JP6674584B2 (ja) |
SG (1) | SG11201908010YA (ja) |
WO (1) | WO2018198373A1 (ja) |
Cited By (9)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20200193166A1 (en) * | 2018-12-12 | 2020-06-18 | Avigilon Corporation | Object left behind detection |
WO2020235584A1 (ja) * | 2019-05-21 | 2020-11-26 | 日本電気株式会社 | エリア管理装置、エリア管理方法及びプログラム |
DE102019134813A1 (de) * | 2019-12-17 | 2021-06-17 | G2K Group GmbH | Ereignis-Überwachung in einem räumlichen Bereich |
US11164006B2 (en) * | 2017-03-30 | 2021-11-02 | Nec Corporation | Information processing apparatus, control method, and program |
JP2022058833A (ja) * | 2018-12-07 | 2022-04-12 | 日本電気株式会社 | 情報処理システム、情報処理装置、情報処理方法、およびプログラム |
EP3934240A4 (en) * | 2019-03-01 | 2022-10-12 | Hitachi, Ltd. | UNATTENDED OBJECT DETECTION DEVICE AND UNATTENDED OBJECT DETECTION METHOD |
WO2022259690A1 (ja) | 2021-06-11 | 2022-12-15 | パナソニックIpマネジメント株式会社 | 作業分析装置及び方法 |
JP2023115087A (ja) * | 2018-12-07 | 2023-08-18 | 日本電気株式会社 | 情報処理装置、情報処理方法、およびプログラム |
US12046075B2 (en) | 2018-12-07 | 2024-07-23 | Nec Corporation | Information processing apparatus, information processing method, and program |
Families Citing this family (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN111291585B (zh) * | 2018-12-06 | 2023-12-08 | 杭州海康威视数字技术股份有限公司 | 一种基于gps的目标跟踪***、方法、装置及球机 |
KR20190137032A (ko) * | 2019-11-19 | 2019-12-10 | 엘지전자 주식회사 | 전자 장치 및 그의 동작 방법 |
KR102413960B1 (ko) * | 2020-10-12 | 2022-06-27 | 서울대학교산학협력단 | 질의 응답 장치 및 방법 |
CN112926431A (zh) * | 2021-02-20 | 2021-06-08 | 北京市商汤科技开发有限公司 | 垃圾检测方法、装置、设备及计算机存储介质 |
US20230141862A1 (en) * | 2021-11-09 | 2023-05-11 | AvaSure, LLC | Predictive system for elopement detection |
US11914749B2 (en) * | 2021-12-07 | 2024-02-27 | Motorola Solutions, Inc. | Selective and protected release of private information to an incident response team |
Citations (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2009199322A (ja) | 2008-02-21 | 2009-09-03 | Hitachi Kokusai Electric Inc | 監視システム、人物検索方法 |
JP2011049646A (ja) * | 2009-08-25 | 2011-03-10 | Secom Co Ltd | 置き去り監視装置 |
JP2011061651A (ja) | 2009-09-14 | 2011-03-24 | Hitachi Kokusai Electric Inc | 不審物検知システム |
JP2012222685A (ja) * | 2011-04-12 | 2012-11-12 | Saxa Inc | 置き去り又は持ち去り検知システム |
JP2012235300A (ja) * | 2011-04-28 | 2012-11-29 | Saxa Inc | 置き去り又は持ち去り検知システム及び置き去り又は持ち去り検知記録の生成方法 |
WO2014045670A1 (ja) * | 2012-09-19 | 2014-03-27 | 日本電気株式会社 | 画像処理システム、画像処理方法及びプログラム |
WO2014050518A1 (ja) * | 2012-09-28 | 2014-04-03 | 日本電気株式会社 | 情報処理装置、情報処理方法および情報処理プログラム |
Family Cites Families (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20060018516A1 (en) * | 2004-07-22 | 2006-01-26 | Masoud Osama T | Monitoring activity using video information |
US10019877B2 (en) * | 2005-04-03 | 2018-07-10 | Qognify Ltd. | Apparatus and methods for the semi-automatic tracking and examining of an object or an event in a monitored site |
US20130282446A1 (en) * | 2010-04-15 | 2013-10-24 | Colin Dobell | Methods and systems for capturing, measuring, sharing and influencing the behavioural qualities of a service performance |
JP5954106B2 (ja) | 2012-10-22 | 2016-07-20 | ソニー株式会社 | 情報処理装置、情報処理方法、プログラム、及び情報処理システム |
US11017901B2 (en) * | 2016-08-02 | 2021-05-25 | Atlas5D, Inc. | Systems and methods to identify persons and/or identify and quantify pain, fatigue, mood, and intent with protection of privacy |
-
2017
- 2017-04-28 EP EP17907868.8A patent/EP3618427B1/en active Active
- 2017-04-28 JP JP2019515072A patent/JP6674584B2/ja active Active
- 2017-04-28 US US16/490,043 patent/US11017537B2/en active Active
- 2017-04-28 WO PCT/JP2017/017077 patent/WO2018198373A1/ja active Application Filing
- 2017-04-28 SG SG11201908010YA patent/SG11201908010YA/en unknown
Patent Citations (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2009199322A (ja) | 2008-02-21 | 2009-09-03 | Hitachi Kokusai Electric Inc | 監視システム、人物検索方法 |
JP2011049646A (ja) * | 2009-08-25 | 2011-03-10 | Secom Co Ltd | 置き去り監視装置 |
JP2011061651A (ja) | 2009-09-14 | 2011-03-24 | Hitachi Kokusai Electric Inc | 不審物検知システム |
JP2012222685A (ja) * | 2011-04-12 | 2012-11-12 | Saxa Inc | 置き去り又は持ち去り検知システム |
JP2012235300A (ja) * | 2011-04-28 | 2012-11-29 | Saxa Inc | 置き去り又は持ち去り検知システム及び置き去り又は持ち去り検知記録の生成方法 |
WO2014045670A1 (ja) * | 2012-09-19 | 2014-03-27 | 日本電気株式会社 | 画像処理システム、画像処理方法及びプログラム |
WO2014050518A1 (ja) * | 2012-09-28 | 2014-04-03 | 日本電気株式会社 | 情報処理装置、情報処理方法および情報処理プログラム |
Non-Patent Citations (1)
Title |
---|
See also references of EP3618427A4 |
Cited By (18)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US11164006B2 (en) * | 2017-03-30 | 2021-11-02 | Nec Corporation | Information processing apparatus, control method, and program |
US12046044B2 (en) | 2017-03-30 | 2024-07-23 | Nec Corporation | Information processing apparatus, control method, and program |
US12046043B2 (en) | 2017-03-30 | 2024-07-23 | Nec Corporation | Information processing apparatus, control method, and program |
US11776274B2 (en) | 2017-03-30 | 2023-10-03 | Nec Corporation | Information processing apparatus, control method, and program |
JP2023115087A (ja) * | 2018-12-07 | 2023-08-18 | 日本電気株式会社 | 情報処理装置、情報処理方法、およびプログラム |
JP7371806B2 (ja) | 2018-12-07 | 2023-10-31 | 日本電気株式会社 | 情報処理装置、情報処理方法、およびプログラム |
US12046075B2 (en) | 2018-12-07 | 2024-07-23 | Nec Corporation | Information processing apparatus, information processing method, and program |
JP2022058833A (ja) * | 2018-12-07 | 2022-04-12 | 日本電気株式会社 | 情報処理システム、情報処理装置、情報処理方法、およびプログラム |
JP7298729B2 (ja) | 2018-12-07 | 2023-06-27 | 日本電気株式会社 | 情報処理システム、情報処理装置、情報処理方法、およびプログラム |
US11600072B2 (en) * | 2018-12-12 | 2023-03-07 | Motorola Solutions, Inc. | Object left behind detection |
US20200193166A1 (en) * | 2018-12-12 | 2020-06-18 | Avigilon Corporation | Object left behind detection |
EP3934240A4 (en) * | 2019-03-01 | 2022-10-12 | Hitachi, Ltd. | UNATTENDED OBJECT DETECTION DEVICE AND UNATTENDED OBJECT DETECTION METHOD |
JP2022173553A (ja) * | 2019-05-21 | 2022-11-18 | 日本電気株式会社 | エリア管理装置、エリア管理方法及びプログラム |
JP7160192B2 (ja) | 2019-05-21 | 2022-10-25 | 日本電気株式会社 | エリア管理装置、エリア管理方法及びプログラム |
JPWO2020235584A1 (ja) * | 2019-05-21 | 2020-11-26 | ||
WO2020235584A1 (ja) * | 2019-05-21 | 2020-11-26 | 日本電気株式会社 | エリア管理装置、エリア管理方法及びプログラム |
DE102019134813A1 (de) * | 2019-12-17 | 2021-06-17 | G2K Group GmbH | Ereignis-Überwachung in einem räumlichen Bereich |
WO2022259690A1 (ja) | 2021-06-11 | 2022-12-15 | パナソニックIpマネジメント株式会社 | 作業分析装置及び方法 |
Also Published As
Publication number | Publication date |
---|---|
US20200043174A1 (en) | 2020-02-06 |
EP3618427A4 (en) | 2020-08-26 |
JPWO2018198373A1 (ja) | 2019-11-14 |
US11017537B2 (en) | 2021-05-25 |
EP3618427A1 (en) | 2020-03-04 |
EP3618427B1 (en) | 2022-04-13 |
JP6674584B2 (ja) | 2020-04-01 |
SG11201908010YA (en) | 2019-09-27 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
WO2018198373A1 (ja) | 映像監視システム | |
JP6741130B2 (ja) | 情報処理システム、情報処理方法及びプログラム | |
JP5227911B2 (ja) | 監視映像検索装置及び監視システム | |
JP4984728B2 (ja) | 被写体照合装置および被写体照合方法 | |
JP5001260B2 (ja) | オブジェクト追跡方法及びオブジェクト追跡装置 | |
TWI430186B (zh) | 影像處理裝置及影像處理方法 | |
JP6172551B1 (ja) | 画像検索装置、画像検索システム及び画像検索方法 | |
KR20160010338A (ko) | 비디오 분석 방법 | |
US11120838B2 (en) | Information processing apparatus, control method, and program | |
JP2014016968A (ja) | 人物検索装置及びデータ収集装置 | |
CN111670456B (zh) | 信息处理装置、追踪方法和记录介质 | |
JP5088463B2 (ja) | 監視システム | |
JP6396682B2 (ja) | 監視カメラシステム | |
JP6724919B2 (ja) | 情報処理装置、情報処理方法及びプログラム | |
US10783365B2 (en) | Image processing device and image processing system | |
JP2019083532A (ja) | 画像処理システム、画像処理方法および画像処理プログラム | |
JP5051885B2 (ja) | 人物検索装置及び人物検索方法 | |
JP2022058833A (ja) | 情報処理システム、情報処理装置、情報処理方法、およびプログラム | |
JP6984942B2 (ja) | 画像処理装置、画像処理システム、画像処理方法及びプログラム | |
JP2018195992A (ja) | 人物グループ追跡装置および人物グループ追跡方法 | |
JP7487888B2 (ja) | 画像処理システム、画像処理方法及びプログラム | |
JP6267350B2 (ja) | データ処理装置、データ処理システム、データ処理方法及びプログラム | |
JP6112346B2 (ja) | 情報収集システム、プログラムおよび情報収集方法 | |
JP7357649B2 (ja) | 識別を容易にするための方法および装置 | |
JP2006165729A (ja) | 撮影プログラム、撮影方法及び撮影システム |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 17907868 Country of ref document: EP Kind code of ref document: A1 |
|
ENP | Entry into the national phase |
Ref document number: 2019515072 Country of ref document: JP Kind code of ref document: A |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
WWE | Wipo information: entry into national phase |
Ref document number: 2017907868 Country of ref document: EP |
|
ENP | Entry into the national phase |
Ref document number: 2017907868 Country of ref document: EP Effective date: 20191128 |