WO2011010490A1 - 監視カメラ端末 - Google Patents
監視カメラ端末 Download PDFInfo
- Publication number
- WO2011010490A1 WO2011010490A1 PCT/JP2010/056161 JP2010056161W WO2011010490A1 WO 2011010490 A1 WO2011010490 A1 WO 2011010490A1 JP 2010056161 W JP2010056161 W JP 2010056161W WO 2011010490 A1 WO2011010490 A1 WO 2011010490A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- terminal
- person
- surveillance camera
- object map
- counterpart
- Prior art date
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
- G06T7/292—Multi-camera tracking
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects
-
- G—PHYSICS
- G08—SIGNALLING
- G08B—SIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
- G08B13/00—Burglar, theft or intruder alarms
- G08B13/18—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength
- G08B13/189—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems
- G08B13/194—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems using image scanning and comparing systems
- G08B13/196—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems using image scanning and comparing systems using television cameras
- G08B13/19639—Details of the system layout
- G08B13/19641—Multiple cameras having overlapping views on a single scene
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20021—Dividing image into blocks, subimages or windows
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20076—Probabilistic image processing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30196—Human being; Person
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30232—Surveillance
Definitions
- the present invention relates to a monitoring camera terminal that processes a frame image obtained by imaging a monitoring target area and tracks a suspicious person or the like who has taken a specific action.
- suspicious persons who take a peculiar action using a plurality of surveillance camera terminals. Detects and tracks detected suspicious individuals.
- the surveillance camera terminal that takes over the tracking needs to identify the suspicious person. If identification of the suspicious person fails, another person who has not been suspiciously tracked so far will be tracked. That is, the detected suspicious person is lost and tracking fails.
- one monitoring camera terminal that is tracking the suspicious person takes over the template information used for detecting the suspicious person, and takes over the tracking of the suspicious person to the other monitoring camera terminal. It has been proposed to identify a suspicious person using the template information notified to the other surveillance camera terminal (see Patent Document 1).
- This feature information is, for example, information for identifying the suspicious person based on the image data, shape, color, size, etc. of the suspicious person cut out from the image, or the shape and position of the facial parts such as eyes, nose, mouth .
- one surveillance camera terminal that tracks the suspicious person and the other surveillance camera terminal that takes over the tracking of the suspicious person have different angles with respect to the suspicious person that is the subject. Therefore, there is a difference between the suspicious person in the frame image captured by one surveillance camera terminal and the suspicious person in the frame image captured by the other surveillance camera terminal in the external shape of the suspicious person, the shape and position of the facial parts, etc. . For this reason, even if the template information or feature information used for detecting a suspicious person is notified from one surveillance camera terminal to the other surveillance camera terminal, matching cannot be performed with the other surveillance camera terminal taking into account the difference in angle. Thus, the identification accuracy of the suspicious person cannot be ensured.
- An object of the present invention is to provide a monitoring camera terminal that sufficiently secures the identification accuracy of an object being tracked with another monitoring camera terminal and that can sufficiently suppress an increase in cost.
- the surveillance camera terminal of the present invention is configured as follows to achieve the above object.
- the object extraction unit processes the frame image of the monitoring target area assigned to the terminal captured by the imaging unit, and extracts the imaged object.
- the object referred to here is an object such as a person or a vehicle.
- the ID assigning means assigns an ID to the object extracted by the object extracting means. This ID may be a unique value that can identify the object.
- the object map creating unit creates an object map in which the ID assigned to each object and the coordinate position on the frame image are associated with each object extracted by the object extracting unit. Then, the tracking unit tracks an object located in the monitoring target area assigned to the own terminal, using the object map created by the object map creating unit.
- the object map acquisition means creates a partner object created by the partner terminal by data communication with the partner terminal whose assigned monitor target area overlaps with a part of the monitor target area of the terminal itself. Get the map.
- the coordinate conversion information storage unit stores coordinate conversion information indicating a relative positional relationship between the coordinate position of the frame image of the own terminal and the coordinate position of the frame image of the counterpart terminal.
- the identification unit converts the position of the object extracted by the own terminal and the position of the object extracted by the counterpart terminal into a common coordinate system position using the coordinate conversion information stored in the coordinate conversion information storage unit. .
- the identifying means identifies the object located in the overlapping area by the distance between the object extracted by the own terminal and the object extracted by the counterpart terminal in the common coordinate system.
- the common coordinate system may be any coordinate system, but is preferably the coordinate system of either one of the surveillance camera terminals (own terminal or counterpart terminal). In this way, only the object registered in one of the object maps has to be subjected to coordinate conversion, so that the processing load for this coordinate conversion can be suppressed.
- the position in the common coordinate system is slightly different depending on factors such as the angle difference and detection accuracy. Although there is an error, they are almost identical. Therefore, the object extracted by the own terminal and the object extracted by the counterpart terminal can be accurately identified by the distance between the positions.
- the identification means for example, in a combination of one-to-one correspondence between the object extracted by the own terminal located in the overlapping area and the object extracted by the counterpart terminal, the distance between the associated objects (common The object located in the overlapping area is identified by the combination having the smallest sum of the distances between the positions in the coordinate system.
- two types of common coordinate systems may be used, and an object located in the overlapping area may be identified by a combination in which the total sum of each type is minimum.
- the two types of common coordinate systems are the coordinate system of the frame image of the own terminal and the coordinate system of the frame image of the counterpart terminal.
- the position of the object may be set to the coordinates on the ground plane of the object.
- a matching unit that temporally matches the object map of the own terminal created by the object map creation unit and the partner object map acquired by the object map acquisition unit according to the distance between objects located in the overlapping area. Also good. Thereby, the identification accuracy can be further improved.
- the identification means is configured such that the matching means identifies an object located in the overlapping area using a plurality of sets of object maps in which the object map of the own terminal and the object map of the counterpart terminal are temporally matched. Also good. In this way, the object identification accuracy can be further improved.
- the identification means calculates the distance between the object extracted by the own terminal and the object extracted by the counterpart terminal, in the counterpart object map that the matching means temporally matches with the object map of the own terminal. For a set in which other object maps are also combined, a predetermined penalty distance may be added when calculating the distance between objects located in the overlapping area.
- the object map creation means adds a time stamp related to the creation time to the created object map.
- the time stamp is, for example, the imaging time of the frame image used to create the object map this time, the time when the object map creation is completed, or the like.
- FIG. 1 is a schematic diagram showing the configuration of this monitoring system.
- This monitoring system is a network system having a plurality of monitoring camera terminals 1 (1A to 1H).
- This monitoring system is, for example, an ad hoc network system.
- Data communication can be performed between the monitoring camera terminals 1 directly or via another monitoring camera terminal 1.
- the data communication between the monitoring camera terminals 1 may be wireless or wired.
- FIG. 1 is an example of a surveillance system having eight surveillance camera terminals 1A to 1H, but the number of surveillance camera terminals 1 constituting this system is not limited to this number.
- a line connecting the monitoring camera terminals 1 shown in FIG. 1 is a link.
- the surveillance camera terminals 1A to 1H are indicated as surveillance camera terminals 1 unless they are particularly distinguished.
- FIG. 2 is a diagram showing the configuration of the main part of the surveillance camera terminal.
- the surveillance camera terminal 1 includes a control unit 11, an imaging unit 12, an image processing unit 13, a storage unit 14, a timer 15, and a communication unit 16.
- the control unit 11 controls the operation of each part of the monitoring camera terminal 1 main body.
- the imaging unit 12 images the monitoring target area assigned to the terminal itself. In other words, the imaging area of the imaging unit 12 is a monitoring target area assigned to the own terminal.
- the imaging unit 12 outputs captured images (frame images) of about 5 to 10 frames per second.
- the image processing unit 13 processes the frame image of the monitoring target area imaged by the imaging unit 12 and extracts the person (object) being imaged. Further, the person moving in the monitoring target area is tracked by a plurality of temporally continuous frame images.
- the image processing unit 13 extracts and tracks the extracted person using a spatio-temporal MRF (Markov Random Field) model.
- the spatiotemporal MRF model is an extension of the MRF model as a spatiotemporal model, focusing on the correlation in the time axis direction of the spatiotemporal image.
- a frame image to be processed is divided into blocks of several pixels ⁇ several pixels (for example, 8 pixels ⁇ 8 pixels), and each block between temporally continuous frame images is divided. This is a model that defines the correlation in the time axis direction with reference to the motion vector.
- the storage unit 14 stores an operation program for operating the main body, setting data used during operation, processing data generated during operation, and the like.
- the timer 15 measures the current time.
- the communication unit 16 controls data communication with other surveillance camera terminals 1.
- FIG. 3 is a diagram illustrating imaging areas (monitoring target areas) of two adjacent monitoring camera terminals 1A and 1B. Two adjacent monitoring camera terminals 1A and 1B have overlapping imaging areas (monitoring target areas). This overlapping imaging area is referred to as an overlapping area here.
- the monitoring target areas of the two adjacent monitoring camera terminals 1 ⁇ / b> A and 1 ⁇ / b> B are illustrated. However, in the combination of the other two adjacent monitoring camera terminals 1, a part of the monitoring target area overlaps.
- the monitoring camera terminal 1 performs an identification process for associating the persons X and Y located in the overlapping area with each other adjacent monitoring camera terminal 1 on a one-to-one basis.
- the persons X and Y are located in the overlapping area of the monitoring camera terminals 1A and 1B, and the person Z is in the monitoring target area of the monitoring camera terminal 1A and outside the overlapping area with the monitoring camera terminal 1B. positioned.
- adjacent monitoring camera terminals 1 can perform data communication directly (without passing through other monitoring camera terminals 1).
- the monitoring camera terminal 1 is not necessarily one adjacent other monitoring camera terminal 1. In other words, there are also monitoring camera terminals 1 in which there are two or three adjacent monitoring camera terminals 1. However, the overlapping area is different for each other adjacent surveillance camera terminal 1.
- Each monitoring camera terminal 1 has, for each adjacent monitoring camera terminal 1, a two-dimensional coordinate system of a frame image captured by the imaging unit 12 of its own terminal, and a frame captured by the imaging unit 12 of the adjacent counterpart monitoring camera terminal 1.
- Coordinate conversion information indicating a relative positional relationship with the two-dimensional coordinate system of the image is stored in the storage unit 14. This coordinate conversion information is common to the two-dimensional coordinate system of the frame image captured by the imaging unit 12 of the own terminal and the two-dimensional coordinate system of the frame image captured by the imaging unit 12 of the adjacent counterpart monitoring camera terminal 1. This is information for projective transformation to the coordinate system.
- the two-dimensional coordinate system of the frame image captured by the imaging unit 12 of the own terminal is changed to the two-dimensional coordinate system of the frame image captured by the imaging unit 12 of the adjacent counterpart monitoring camera terminal 1.
- the first coordinate conversion parameter for projective conversion and conversely, the two-dimensional coordinate system of the frame image captured by the image capturing unit 12 of the adjacent counterpart monitoring camera terminal 1 is the frame image captured by the image capturing unit 12 of the own terminal.
- a second coordinate conversion parameter for projective conversion into a two-dimensional coordinate system is stored in the storage unit 14.
- the coordinate conversion information may be only one of the first coordinate conversion parameter and the second coordinate conversion parameter.
- the first coordinate conversion parameter and the second coordinate conversion parameter are values calculated using a frame image actually captured when the surveillance camera terminal 1 is installed.
- FIG. 4 is a flowchart showing the tracking process in each surveillance camera terminal.
- the surveillance camera terminal 1 captures the frame image of the monitoring target area imaged by the imaging unit 12 into the image processing unit 13 (S1).
- the image processing unit 13 processes the frame image captured in S1 and extracts the person being imaged (S2).
- the image processing unit 13 associates the person extracted from the previously processed frame image with the person extracted in S2 (S3).
- the image processing unit 13 uses a spatio-temporal MRF model to extract a person as a person area with a block of 8 pixels ⁇ 8 pixels as a unit. In S3, by associating the person extracted in the previously processed frame image with the person extracted in S2, the movement direction and movement amount of the person extracted this time can be obtained, and these persons can be tracked.
- a temporary ID is assigned to a person extracted in the current process and not extracted in the previous process (that is, a person extracted for the first time this time) (S4, S5).
- the image processing unit 13 creates an object map for the currently processed frame image (S6). As shown in FIG. 5, this object map associates, for each person extracted this time, the ID assigned to the person (or the temporary ID assigned this time) and the coordinate position on the frame image processed this time. Information. Moreover, the monitoring camera terminal 1 gives a time stamp to the object map. This time stamp may be the time when the frame image is taken into the image processing unit 13 in S1, or may be the time (current time) that the timer 15 measures when giving.
- a position where the person extracted this time is in contact with the floor (hereinafter referred to as a foot position) is detected as the coordinate position of the person.
- the center of the lower side of the person area of the person that is in contact with the plane passing through the four marked points (that is, the floor surface) is detected as the coordinate position of the person.
- the person area is referred to as a circumscribed rectangle of the person.
- the surveillance camera terminal 1 stores the object map created in S6 in the storage unit 14 and also creates the object map created this time (hereinafter referred to as its own terminal object map) for other neighboring surveillance camera terminals 1. Transmit (S7, S8).
- Each surveillance camera terminal 1 stores the counterpart object map in the storage unit 14 when an object map (hereinafter referred to as a counterpart object map) is transmitted from another adjacent surveillance camera terminal 1. .
- the monitoring camera terminal 1 distinguishes and stores a counterpart object map for each adjacent counterpart monitoring camera terminal 1.
- the surveillance camera terminal 1 keeps track of a person located within the surveillance area of the terminal itself by repeatedly performing the above-described processes of S1 to S7.
- an ID confirmation process for confirming the ID of the person assigned the temporary ID in S4 will be described.
- an ID for identifying the person or a temporary ID is assigned to the person registered in the object map.
- a person who has been assigned a temporary ID includes a person who has already been assigned an ID by another surveillance camera terminal 1 (that is, a person who has already started tracking) and a person who has not been assigned an ID at the present time. (That is, a person who has entered the area monitored by this monitoring system).
- the temporary ID given this time is invalidated and has already been given.
- the temporary ID assigned this time is determined as the ID of this person.
- the ID or temporary ID assigned to a person is a unique value that can identify the person.
- This ID or temporary ID may be a number, a code, or a combination thereof.
- the ID includes a number or code that can be identified by the assigned surveillance camera terminal 1 so that different IDs can be assigned to each person without notifying the issued ID between the surveillance camera terminals 1.
- FIG. 6 is a flowchart showing this ID confirmation processing.
- This ID confirmation process is performed when an object map in which a person with a temporary ID is registered is created.
- S11 it is determined whether or not there is a person who is located in the overlapping area with another adjacent monitoring camera terminal 1 among the persons who have been assigned a temporary ID this time (S11). If the person to whom the temporary ID is assigned this time is located within the overlapping area, the person is a person being tracked by the adjacent counterpart monitoring camera terminal 1 that includes this overlapping area in the monitoring area. That is, it is a person who has already been given an ID.
- the person assigned with the temporary ID this time is not located in the overlapping area, the person is a person who has not been tracked by another monitoring camera terminal 1. That is, it is a person who is not assigned an ID at the present time. If there is no person located in the overlapping area in S11, the surveillance camera terminal 1 determines the assigned temporary ID as the ID for each person who is assigned the temporary ID this time (S12).
- the provisional ID assigned to the person who is located outside the overlapping area among the persons assigned with the temporary ID this time is determined as the ID (S13).
- the monitoring camera terminal 1 reads the other party object map corresponding to the self terminal side object map which registered temporary ID this time from the memory
- the partner object map to which the time stamp of the time at which the absolute value of the time difference from the time stamp given to the own terminal side object map is the smallest is given from the storage unit 14.
- the surveillance camera terminal 1 makes a pair of a person located in the overlapping area registered in the local terminal object map and a person located in the overlapping area registered in the counterpart object map. 1 is created (S15).
- the number of combination patterns created in S15 is, for example, two if there are two persons located in the overlap area, and 6 if there are three persons located in the overlap area. Street.
- the surveillance camera terminal 1 uses the first coordinate conversion parameter for each person located in the overlapping area among the persons registered in the self-terminal-side object map created this time, and coordinates of the person The position is converted into the coordinate system of the counterpart terminal (S16).
- the surveillance camera terminal 1 calculates the first distance energy for each combination pattern created in S15 (S17). This first distance energy is the sum of the distances between the persons associated in the corresponding combination pattern in the coordinate system of the counterpart terminal.
- the surveillance camera terminal 1 uses the second coordinate conversion parameter for each person located in the overlap area among the persons registered in the counterpart object map read in S14, and coordinates of the person. The position is converted into the coordinate system of its own terminal (S18). The surveillance camera terminal 1 calculates the second distance energy for each combination pattern created in S15 (S19). This second distance energy is the sum of the distances between the persons associated in the corresponding combination pattern in the coordinate system of the terminal itself.
- the surveillance camera terminal 1 calculates the total distance energy for each combination pattern created in S14 (S20). This total distance energy is the sum of the first distance energy and the second distance energy of the combination pattern for each combination pattern.
- the surveillance camera terminal 1 determines that the combination pattern having the minimum total distance energy obtained in S20 is an appropriate association of persons located in the overlapping area. And the person located in an overlap area is identified by matching of the person by the combination pattern with the minimum total distance energy (S21). Then, the surveillance camera terminal 1 invalidates the assigned temporary ID for the person assigned with the temporary ID, and determines the ID assigned to the identified person (S22). In S22, the temporary ID in the object map is replaced with the ID assigned to the identified person.
- the sum of the first distance energy and the second distance energy is used as the total distance energy in order to ensure the identification accuracy of the person located in the overlapping area.
- one of the second distance energies may be the total distance energy. In this way, one process (S16, S17) or (S18, S19) becomes unnecessary, and the processing load on the monitoring camera terminal 1 can be reduced.
- the foot of the person may not be captured. That is, in some cases, the position of the person's foot cannot be detected in the frame image. In such a case, when the object map is created in S5, the foot position of the person being imaged may be determined by the following process. This process may be performed only for a person located in the overlapping area, or may be performed for all persons located in the monitoring area.
- the image processing unit 13 has a function of learning the reference height of the person with respect to the depth of the frame image captured by the imaging unit 12 and storing this in the storage unit 14.
- FIG. 7 is a flowchart showing processing for determining the foot position.
- the image processing unit 13 determines whether or not a person area corresponding to another person is in contact with the lower side of the circumscribed rectangle that is the person area from which the target person whose foot position is to be determined is extracted (S31). If the person area corresponding to another person is not touching in S31, the image processing unit 13 determines that the foot of the person is captured, and determines the center of the lower side of the circumscribed rectangle of the person as the foot position ( S32).
- the image processing unit 13 detects the height of the circumscribed rectangle that is the person area (S33). The image processing unit 13 compares the reference height stored for this person's imaging position with the height detected in S33 (S34).
- the image processing unit 13 determines that the height is higher than the reference height in S34, the image processing unit 13 proceeds to S32 and determines the center of the lower side of the circumscribed rectangle of the person as the foot position. On the other hand, if it is determined in S34 that it is lower than the reference height, the position below the reference height is determined as the foot position of the person from the center coordinates of the upper side of the circumscribed rectangle of the person area corresponding to this person (S35).
- each surveillance camera terminal 1 performs the tracking process shown in FIG.
- FIG. 8 is a flowchart showing the ID confirmation process in this case.
- This ID confirmation process is performed when an object map in which a person with a temporary ID is registered is created. First, it is determined whether a person who is assigned a temporary ID this time is located in an overlapping area with another adjacent monitoring camera terminal 1 (S41). If the surveillance camera terminal 1 determines that there is no person located in the overlapping area in S41, the surveillance camera terminal 1 determines the assigned temporary ID as the ID for each person assigned the temporary ID this time (S42). S41 and S42 are the same processes as S11 and S12 described above, respectively.
- the assigned temporary ID is determined as the ID only for the person who has been assigned the temporary ID this time and is not located in the overlapping area (S43). .
- S43 is the same processing as S13 described above.
- the surveillance camera terminal 1 searches for a corresponding counterpart object map for each predetermined number (for example, eight) of object maps created after the self-terminal-side object map that registered the temporary ID assigned person this time. (S44). In S44, the following processing is performed for each self-terminal-side object map.
- a combination pattern for creating a one-to-one correspondence with the person located in the overlap area with the local terminal object map is created, and the created combination
- the total distance energy is calculated for each pattern. Since the calculation method of the total distance energy has been described above, the description thereof is omitted here. Then, the minimum total distance energy is set as the total distance energy with the partner object map targeted this time.
- the partner object map having the smallest total distance energy is determined as the partner object map corresponding to the own terminal side object map to be processed this time.
- FIG. 9 shows an example of a search result of the counterpart object map of the adjacent surveillance camera terminal 1B corresponding to the self-terminal side object map to which the surveillance camera terminal 1A has given time stamps of times T1 to T8.
- two counterpart object maps with time stamps t1 and t2 are extracted from the self-terminal side object map with a time stamp of time T1
- the total distance energy and the total distance energy with the counterpart object map whose time stamp is t2 are calculated.
- the total distance energy of the partner object map whose time stamp is t1 is compared with the total distance energy of the partner object map whose time stamp is t2, and the partner object whose time stamp is smaller (minimum) is t1.
- the map is associated with the own-terminal-side object map whose time stamp is time T1.
- a plurality of own terminal side object maps may be associated with the same counterpart object map (t3 and t7).
- the surveillance camera terminal 1 has a total distance of its own terminal object map (T3, T4, T6, T7 shown in FIG. 8) in which the other party terminal object map is also associated with the associated partner object map.
- the energy is updated to a value obtained by adding a predetermined penalty value (S45).
- This penalty value can be arbitrarily set depending on the installation environment or the like.
- a combination in which the total distance energy in the combination exceeds a predetermined reference distance energy is extracted and removed (S46).
- This reference distance energy is detected when the correspondence between the own terminal object map and the partner object map is not properly performed, or when the combination of persons located in the overlapping area is not properly associated.
- the size is set as possible. If the appropriate counterpart object map corresponding to the local terminal object map is missing due to, for example, a communication failure with the counterpart surveillance camera terminal 1 or the like, the surveillance camera terminal 1 is properly associated with the counterpart object map. I can not do it.
- the surveillance camera terminal 1 identifies a person located in the overlapping area by the remaining combination of the self-terminal object map and the counterpart object map (S47).
- S47 for example, the combination having the minimum total distance energy is extracted from the combination of the own terminal object map and the partner object map having the minimum total distance energy.
- the person located in this duplication area is identified by the combination of the person located in the duplication area in the combination of the object map extracted here.
- the person located in this duplication area is identified by the combination of the person located in the most duplication area among the remaining combinations of a self-terminal object map and the other party object map.
- the surveillance camera terminal 1 invalidates the assigned temporary ID for the person assigned with the temporary ID and determines the ID assigned to the identified person (S22). In S22, the temporary ID in the object map is replaced with the ID assigned to the identified person.
- FIG. 10 is a flowchart showing this time synchronization processing.
- the time difference of the time stamp is calculated for each combination of the own terminal object map used for identifying the person located in the overlapping area in S47 and the counterpart object map (S51), and the average time difference is obtained (S52). Then, the surveillance camera terminal 1 corrects the time counted by the timer 15 by the average time difference calculated in S52 (S53). Thereby, it is possible to synchronize the time counted by the timer 15 with the adjacent monitoring camera terminal 1.
- time difference of the time stamp is removed from the average time difference calculated in S52 for a certain time or more, and the average time difference is calculated again with the remaining time to correct the time counted by the timer 15. Also good. This process may be repeated until there is no time difference between the time stamps for a certain time or more with respect to the average time difference, or may be repeated only a predetermined number of times.
- the object to be tracked is described as a person.
- this object is not limited to a person, and may be another type of moving body such as a vehicle.
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Closed-Circuit Television Systems (AREA)
- Alarm Systems (AREA)
- Image Analysis (AREA)
- Burglar Alarm Systems (AREA)
- Image Processing (AREA)
- Studio Devices (AREA)
Abstract
Description
X=(a1x+b1y+c1)/(a0x+b0y+1)
Y=(a2x+b2y+c2)/(a0x+b0y+1)
に代入し、8元連立方程式を得る。この8元連立方程式の解である、a0,b0,a1,b1,c1,a2,b2,c2の8個の計数が、この隣接する相手側監視カメラ端末1との第1の座標変換パラメータである。監視カメラ端末1は、この第1の座標変換パラメータを記憶部14に記憶する。
x=(A1X+B1Y+C1)/(A0X+B0Y+1)
y=(A2X+B2Y+C2)/(A0X+B0Y+1)
に代入し、8元連立方程式を得る。この8元連立方程式の解である、A0,B0,A1,B1,C1,A2,B2,C2の8個の計数が、この隣接する相手側監視カメラ端末1との第2の座標変換パラメータである。監視カメラ端末1は、この第2の座標変換パラメータを記憶部14に記憶する。
11-制御部
12-撮像部
13-画像処理部
14-記憶部
15-タイマ
16-通信部
Claims (7)
- 自端末に割り当てられている監視対象エリアを撮像する撮像手段と、
前記撮像手段が撮像したフレーム画像を処理し、撮像されているオブジェクトを抽出するオブジェクト抽出手段と、
前記オブジェクト抽出手段が抽出したオブジェクトにIDを付与するID付与手段と、
前記オブジェクト抽出手段が抽出したオブジェクト毎に、そのオブジェクトに付与されているIDと、フレーム画像上における座標位置と、を対応付けたオブジェクトマップを作成するオブジェクトマップ作成手段と、
前記オブジェクトマップ作成手段が作成したオブジェクトマップを用いて、自端末に割り当てられている監視対象エリア内に位置するオブジェクトを追跡する追跡手段と、を備えた監視カメラ端末において、
割り当てられている監視対象エリアが、自端末の監視対象エリアの一部と重複している相手側端末とのデータ通信により、この相手側端末が作成した相手側オブジェクトマップを取得する相手側オブジェクトマップ取得手段と、
自端末のフレーム画像の座標位置と、前記相手側端末のフレーム画像の座標位置とにおける、相対的な位置関係を示す座標変換情報を記憶する座標変換情報記憶手段と、
自端末の監視対象エリアと、相手側端末の監視対象エリアとが重複している重複エリア内において、自端末が抽出したオブジェクトと、相手側端末が抽出したオブジェクトとを同定する同定手段と、を備え、
前記同定手段は、前記座標変換情報記憶手段が記憶している前記座標変換情報を用いて、自端末が抽出したオブジェクトの位置、および相手側端末が抽出したオブジェクトの位置を共通の座標系の位置に変換し、この共通の座標系における、自端末が抽出したオブジェクトと、相手側端末が抽出したオブジェクトとの距離によって、前記重複エリアに位置するオブジェクトを同定する、監視カメラ端末。 - 前記同定手段は、前記重複エリアに位置する自端末が抽出したオブジェクトと、相手側端末が抽出したオブジェクトと、を1対1で対応付けた組み合せの中で、対応付けたオブジェクト間の距離の総和が最小である組み合せにより、前記重複エリアに位置するオブジェクトを同定する、請求項1に記載の監視カメラ端末。
- 前記オブジェクトマップ作成手段は、前記オブジェクト抽出手段が抽出したオブジェクト毎に、そのオブジェクトの位置を、当該オブジェクトの接地面上の座標とする、請求項1、または2に記載の監視カメラ端末。
- 前記オブジェクトマップ作成手段が作成した自端末のオブジェクトマップと、前記相手側オブジェクトマップ取得手段が取得した相手側オブジェクトマップと、を前記重複エリアに位置するオブジェクト間の距離によって、時間的に合わせるマッチング手段を備えた請求項1~3のいずれかに記載の監視カメラ端末。
- 前記同定手段は、前記マッチング手段が自端末のオブジェクトマップと、前記相手側オブジェクトマップとを時間的に合わせた、複数組のオブジェクトマップを用いて、前記重複エリアに位置するオブジェクトを同定する、請求項4に記載の監視カメラ端末。
- 前記同定手段は、自端末が抽出したオブジェクトと、相手側端末が抽出したオブジェクトとの距離の算出で、前記マッチング手段が自端末のオブジェクトマップに時間的に合わせた前記相手側オブジェクトマップに、自端末の他のオブジェクトマップも合わせられている組については、前記重複エリアに位置するオブジェクト間の距離を算出する際に、予め定めたペナルティ距離を加算する、請求項5に記載の監視カメラ端末。
- 前記オブジェクトマップ作成手段は、作成したオブジェクトマップに、作成時刻にかかるタイムスタンプを付加し、
前記マッチング手段が時間的に合わせた、自端末のオブジェクトマップと、相手側端末のオブジェクトマップとに付加されているタイムスタンプを用いて、自端末が計時している時刻を修正する時刻合せ手段を備えた請求項4~6のいずれかに記載の監視カメラ端末。
Priority Applications (4)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
EP10802111.4A EP2458553B1 (en) | 2009-07-22 | 2010-04-05 | Surveillance camera terminal |
JP2011523577A JP5453429B2 (ja) | 2009-07-22 | 2010-04-05 | 監視カメラ端末 |
CN201080018875.4A CN102414719B (zh) | 2009-07-22 | 2010-04-05 | 监视摄像机终端 |
US13/266,093 US8503727B2 (en) | 2009-07-22 | 2010-04-05 | Monitoring camera terminal |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
JP2009-171329 | 2009-07-22 | ||
JP2009171329 | 2009-07-22 |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2011010490A1 true WO2011010490A1 (ja) | 2011-01-27 |
Family
ID=43498968
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/JP2010/056161 WO2011010490A1 (ja) | 2009-07-22 | 2010-04-05 | 監視カメラ端末 |
Country Status (5)
Country | Link |
---|---|
US (1) | US8503727B2 (ja) |
EP (1) | EP2458553B1 (ja) |
JP (1) | JP5453429B2 (ja) |
CN (1) | CN102414719B (ja) |
WO (1) | WO2011010490A1 (ja) |
Cited By (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2015033066A (ja) * | 2013-08-06 | 2015-02-16 | 株式会社ニコン | カメラシステム |
JPWO2014155958A1 (ja) * | 2013-03-29 | 2017-02-16 | 日本電気株式会社 | 対象物監視システム、対象物監視方法および監視対象抽出プログラム |
JP2017195617A (ja) * | 2017-05-31 | 2017-10-26 | 株式会社ニコン | カメラ |
JP2018013854A (ja) * | 2016-07-19 | 2018-01-25 | 日本電信電話株式会社 | 追跡装置および追跡方法 |
CN109345748A (zh) * | 2018-10-31 | 2019-02-15 | 北京锐安科技有限公司 | 用户设备关联方法、装置、服务端、检测设备及介质 |
JP2020087180A (ja) * | 2018-11-29 | 2020-06-04 | 技研トラステム株式会社 | 移動体追跡方法及びこれに用いる画像処理装置 |
US11004211B2 (en) | 2017-08-30 | 2021-05-11 | Mitsubishi Electric Corporation | Imaging object tracking system and imaging object tracking method |
Families Citing this family (23)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP5227911B2 (ja) * | 2009-07-22 | 2013-07-03 | 株式会社日立国際電気 | 監視映像検索装置及び監視システム |
TW201206192A (en) * | 2010-07-23 | 2012-02-01 | Hon Hai Prec Ind Co Ltd | Detection device and method |
WO2013088517A1 (ja) * | 2011-12-13 | 2013-06-20 | 株式会社日立製作所 | 停立人物の向き推定方法 |
US9430846B2 (en) * | 2013-04-19 | 2016-08-30 | Ge Aviation Systems Llc | Method of tracking objects using hyperspectral imagery |
KR101645959B1 (ko) * | 2014-07-29 | 2016-08-05 | 주식회사 일리시스 | 복수의 오버헤드 카메라와 사이트 맵에 기반한 객체 추적 장치 및 그 방법 |
JP6495705B2 (ja) * | 2015-03-23 | 2019-04-03 | 株式会社東芝 | 画像処理装置、画像処理方法、画像処理プログラムおよび画像処理システム |
US10810539B1 (en) * | 2015-03-25 | 2020-10-20 | Amazon Technologies, Inc. | Re-establishing tracking of a user within a materials handling facility |
WO2017100173A1 (en) | 2015-12-07 | 2017-06-15 | Cooper Technologies Company | Occupancy detection |
CN107666590B (zh) * | 2016-07-29 | 2020-01-17 | 华为终端有限公司 | 一种目标监控方法、摄像头、控制器和目标监控*** |
WO2018067058A1 (en) * | 2016-10-06 | 2018-04-12 | Modcam Ab | Method for sharing information in system of imaging sensors |
US11068721B2 (en) * | 2017-03-30 | 2021-07-20 | The Boeing Company | Automated object tracking in a video feed using machine learning |
CN107403535A (zh) * | 2017-07-29 | 2017-11-28 | 安徽博威康信息技术有限公司 | 一种基于身份识别的监测报警*** |
CN107274618A (zh) * | 2017-07-29 | 2017-10-20 | 安徽博威康信息技术有限公司 | 一种基于定位识别的监测报警*** |
CN109974667B (zh) * | 2017-12-27 | 2021-07-23 | 宁波方太厨具有限公司 | 一种室内人体定位方法 |
CN109165606B (zh) | 2018-08-29 | 2019-12-17 | 腾讯科技(深圳)有限公司 | 一种车辆信息的获取方法、装置以及存储介质 |
CN109753930B (zh) * | 2019-01-03 | 2021-12-24 | 京东方科技集团股份有限公司 | 人脸检测方法及人脸检测*** |
CN109905679B (zh) * | 2019-04-09 | 2021-02-26 | 梅州讯联科技发展有限公司 | 监控方法、装置以及*** |
TWI688924B (zh) * | 2019-04-15 | 2020-03-21 | 勝品電通股份有限公司 | 追蹤辨識監控系統 |
CN110070566B (zh) * | 2019-04-29 | 2021-07-30 | 武汉睿智视讯科技有限公司 | 信息检测方法、装置、计算机设备及可读存储介质 |
CN111372040B (zh) * | 2019-06-13 | 2021-09-24 | 杭州海康威视***技术有限公司 | 一种多视频监控确定坐标转换参数的方法及装置 |
KR102661644B1 (ko) * | 2019-08-23 | 2024-04-29 | 삼성전자주식회사 | 전자 장치 및 이의 제어 방법 |
KR20210112800A (ko) * | 2020-03-06 | 2021-09-15 | 한국전자통신연구원 | 객체 추적 장치 및 방법 |
CN111768433B (zh) * | 2020-06-30 | 2024-05-24 | 杭州海康威视数字技术股份有限公司 | 一种移动目标追踪的实现方法、装置及电子设备 |
Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JPH10255057A (ja) * | 1997-03-06 | 1998-09-25 | Matsushita Electric Ind Co Ltd | 移動物体抽出装置 |
JP2003276963A (ja) * | 2003-02-03 | 2003-10-02 | Toshiba Corp | 画像監視装置を利用したエレベータ制御装置 |
JP2004072628A (ja) * | 2002-08-08 | 2004-03-04 | Univ Waseda | 複数カメラを用いた移動体追跡システム及びその方法 |
JP3814779B2 (ja) | 2001-03-27 | 2006-08-30 | 三菱電機株式会社 | 侵入者自動追尾方法および装置並びに画像処理装置 |
JP3999561B2 (ja) | 2002-05-07 | 2007-10-31 | 松下電器産業株式会社 | 監視システムと監視カメラ |
Family Cites Families (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6335927B1 (en) * | 1996-11-18 | 2002-01-01 | Mci Communications Corporation | System and method for providing requested quality of service in a hybrid network |
US5999525A (en) * | 1996-11-18 | 1999-12-07 | Mci Communications Corporation | Method for video telephony over a hybrid network |
EP1298524A1 (en) * | 2001-09-28 | 2003-04-02 | Ricoh Company, Ltd. | Conference support apparatus, information processor, teleconference system and computer product |
JP3987013B2 (ja) * | 2003-09-01 | 2007-10-03 | 本田技研工業株式会社 | 車両周辺監視装置 |
JP4759988B2 (ja) * | 2004-11-17 | 2011-08-31 | 株式会社日立製作所 | 複数カメラを用いた監視システム |
JP5041757B2 (ja) * | 2006-08-02 | 2012-10-03 | パナソニック株式会社 | カメラ制御装置およびカメラ制御システム |
JP5084374B2 (ja) * | 2007-07-06 | 2012-11-28 | 株式会社トプコン | 位置計測装置及び位置計測方法 |
-
2010
- 2010-04-05 US US13/266,093 patent/US8503727B2/en active Active
- 2010-04-05 JP JP2011523577A patent/JP5453429B2/ja active Active
- 2010-04-05 CN CN201080018875.4A patent/CN102414719B/zh active Active
- 2010-04-05 EP EP10802111.4A patent/EP2458553B1/en active Active
- 2010-04-05 WO PCT/JP2010/056161 patent/WO2011010490A1/ja active Application Filing
Patent Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JPH10255057A (ja) * | 1997-03-06 | 1998-09-25 | Matsushita Electric Ind Co Ltd | 移動物体抽出装置 |
JP3814779B2 (ja) | 2001-03-27 | 2006-08-30 | 三菱電機株式会社 | 侵入者自動追尾方法および装置並びに画像処理装置 |
JP3999561B2 (ja) | 2002-05-07 | 2007-10-31 | 松下電器産業株式会社 | 監視システムと監視カメラ |
JP2004072628A (ja) * | 2002-08-08 | 2004-03-04 | Univ Waseda | 複数カメラを用いた移動体追跡システム及びその方法 |
JP2003276963A (ja) * | 2003-02-03 | 2003-10-02 | Toshiba Corp | 画像監視装置を利用したエレベータ制御装置 |
Non-Patent Citations (1)
Title |
---|
YASUHIDE HYODO ET AL.: "Robust Multiple-human Tracking against Occlusion through Camera Network", IPSJ SIG TECHNICAL REPORTS 2008-CVIM- 164 COMPUTER VISION AND IMAGE MEDIA, vol. 2008, no. 82, 29 August 2008 (2008-08-29), pages 171 - 176, XP008167429 * |
Cited By (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JPWO2014155958A1 (ja) * | 2013-03-29 | 2017-02-16 | 日本電気株式会社 | 対象物監視システム、対象物監視方法および監視対象抽出プログラム |
JP2015033066A (ja) * | 2013-08-06 | 2015-02-16 | 株式会社ニコン | カメラシステム |
JP2018013854A (ja) * | 2016-07-19 | 2018-01-25 | 日本電信電話株式会社 | 追跡装置および追跡方法 |
JP2017195617A (ja) * | 2017-05-31 | 2017-10-26 | 株式会社ニコン | カメラ |
US11004211B2 (en) | 2017-08-30 | 2021-05-11 | Mitsubishi Electric Corporation | Imaging object tracking system and imaging object tracking method |
CN109345748A (zh) * | 2018-10-31 | 2019-02-15 | 北京锐安科技有限公司 | 用户设备关联方法、装置、服务端、检测设备及介质 |
JP2020087180A (ja) * | 2018-11-29 | 2020-06-04 | 技研トラステム株式会社 | 移動体追跡方法及びこれに用いる画像処理装置 |
Also Published As
Publication number | Publication date |
---|---|
JP5453429B2 (ja) | 2014-03-26 |
US20120045096A1 (en) | 2012-02-23 |
US8503727B2 (en) | 2013-08-06 |
JPWO2011010490A1 (ja) | 2012-12-27 |
CN102414719B (zh) | 2014-10-15 |
EP2458553A4 (en) | 2013-12-18 |
EP2458553B1 (en) | 2018-11-21 |
EP2458553A1 (en) | 2012-05-30 |
CN102414719A (zh) | 2012-04-11 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
JP5453429B2 (ja) | 監視カメラ端末 | |
JP5470111B2 (ja) | 監視カメラ端末 | |
WO2011114799A1 (ja) | 監視カメラ端末 | |
CN107431786B (zh) | 图像处理设备、图像处理***和图像处理方法 | |
US8965050B2 (en) | Behavior analysis device | |
JP4970195B2 (ja) | 人物追跡システム、人物追跡装置および人物追跡プログラム | |
US20130002868A1 (en) | Surveillance camera terminal | |
WO2006009192A1 (ja) | カメラ校正システム及び三次元計測システム | |
JP2010206405A (ja) | 画像監視装置 | |
JP5147760B2 (ja) | 画像監視装置 | |
WO2017187694A1 (ja) | 注目領域画像生成装置 | |
CN114325573A (zh) | 变电站运维人员身份及位置信息快速检测方法 | |
JP2006041939A (ja) | 監視装置及び監視プログラム | |
JP6546057B2 (ja) | 複数のカメラ間での人物の追跡装置、追跡方法及びプログラム | |
KR101464192B1 (ko) | 다시점 보안 카메라 시스템 및 영상 처리 방법 | |
Benedek et al. | Motion-based flexible camera registration | |
Morioka et al. | Color appearance based object identification in intelligent space | |
AU2020436767B2 (en) | Markerless motion capture of hands with multiple pose estimation engines | |
JP2018073044A (ja) | オブジェクト追跡装置、方法およびプログラム | |
CN115410229A (zh) | 一种室内人员的定位方法、装置、电子终端及存储介质 | |
CN116704047A (zh) | 一种基于行人ReID的监控摄像设备位置的标定方法 | |
KATO et al. | Cooperative Distributed Tracking for E ective Face Registration | |
Raman | Study on models for smart surveillance through multi-camera networks | |
Miaou et al. | A Smart Surveillance System for Human Fall-Down Detection using Dual Heterogeneous Cameras. |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
WWE | Wipo information: entry into national phase |
Ref document number: 201080018875.4 Country of ref document: CN |
|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 10802111 Country of ref document: EP Kind code of ref document: A1 |
|
ENP | Entry into the national phase |
Ref document number: 2011523577 Country of ref document: JP Kind code of ref document: A |
|
WWE | Wipo information: entry into national phase |
Ref document number: 2010802111 Country of ref document: EP |
|
WWE | Wipo information: entry into national phase |
Ref document number: 13266093 Country of ref document: US |
|
NENP | Non-entry into the national phase |
Ref country code: DE |