WO2022127181A1 - Procédé et appareil de surveillance de flux de passagers, ainsi que dispositif électronique et support de stockage - Google Patents

Procédé et appareil de surveillance de flux de passagers, ainsi que dispositif électronique et support de stockage Download PDF

Info

Publication number
WO2022127181A1
WO2022127181A1 PCT/CN2021/114965 CN2021114965W WO2022127181A1 WO 2022127181 A1 WO2022127181 A1 WO 2022127181A1 CN 2021114965 W CN2021114965 W CN 2021114965W WO 2022127181 A1 WO2022127181 A1 WO 2022127181A1
Authority
WO
WIPO (PCT)
Prior art keywords
sequence
human head
target
dimensional
head frame
Prior art date
Application number
PCT/CN2021/114965
Other languages
English (en)
Chinese (zh)
Inventor
郝凯旋
黄哲
王孝宇
胡文泽
Original Assignee
深圳云天励飞技术股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 深圳云天励飞技术股份有限公司 filed Critical 深圳云天励飞技术股份有限公司
Publication of WO2022127181A1 publication Critical patent/WO2022127181A1/fr

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects
    • G06V20/53—Recognition of crowd images, e.g. recognition of crowd congestion
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/40—Scenes; Scene-specific elements in video content
    • G06V20/48—Matching video sequences
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/161—Detection; Localisation; Normalisation
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/20—Image signal generators
    • H04N13/204—Image signal generators using stereoscopic image cameras
    • H04N13/239—Image signal generators using stereoscopic image cameras using two 2D image sensors having a relative position equal to or related to the interocular distance
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00—Television systems
    • H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00—Television systems
    • H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
    • H04N7/181—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast for receiving images from a plurality of remote sources

Definitions

  • the invention relates to the field of artificial intelligence, and in particular, to a method, device, electronic device and storage medium for monitoring passenger flow.
  • the embodiment of the present invention provides a method for monitoring passenger flow, which can improve the counting accuracy of passenger flow, thereby improving the monitoring effect of passenger flow.
  • an embodiment of the present invention provides a method for monitoring passenger flow, the method comprising:
  • the first head frame and the second head frame are matched to obtain a paired head frame sequence for each target person, and the
  • the paired head frame sequence includes at least one paired head frame of the target person;
  • three-dimensional reconstruction of the human head is performed in a preset three-dimensional space to obtain a three-dimensional human head sequence of the target person, and the three-dimensional human head sequence includes the three-dimensional human head of the target person;
  • passenger flow monitoring is performed on the target area.
  • the first target image sequence is acquired by a first camera
  • the second target image sequence is acquired by a second camera
  • the method further includes:
  • the three-dimensional space is constructed and obtained.
  • performing ground calibration in the coordinate system of the first camera or the second camera to obtain the calibrated ground including:
  • the calibration object information is calibration object information in the coordinate system of the first camera or the second camera;
  • the ground is calibrated according to the calibration object information, and the calibrated ground is obtained.
  • performing ground calibration in the coordinate system of the first camera or the second camera to obtain the calibrated ground including:
  • performing three-dimensional reconstruction of the human head on the paired human head frame sequence in a preset three-dimensional space to obtain a three-dimensional human head sequence of the target person including:
  • three-dimensional reconstruction of the human head is performed in a preset three-dimensional space to obtain a three-dimensional human head of the current frame;
  • a three-dimensional human head sequence of the target person is obtained.
  • the calculating the effective disparity map of the first head frame and the second head frame in the paired head frame in the current frame includes:
  • the disparity map falls within the effective disparity interval, it is determined that the disparity map is an effective disparity map.
  • performing three-dimensional reconstruction of a human head in a preset three-dimensional space through the effective disparity map to obtain a three-dimensional human head of the current frame including:
  • three-dimensional reconstruction of the human head is performed in the preset three-dimensional space to obtain the three-dimensional human head of the current frame.
  • the three-dimensional space includes a demarcated ground
  • the monitoring of passenger flow in the target area according to the three-dimensional head sequence includes:
  • passenger flow monitoring is performed on the target area.
  • the demarcated ground includes a target demarcation area corresponding to the target area
  • the monitoring of passenger flow in the target area according to the projection trajectory includes:
  • the passenger flow monitoring is performed on the target area.
  • an embodiment of the present invention further provides a device for monitoring passenger flow, the device comprising:
  • a first acquisition module configured to acquire a first target image sequence and a second target image sequence of the target area, where the first target image sequence and the second target sequence are acquired at the same moment and from different angles;
  • a processing module configured to perform human head detection on the first target image sequence and the second target image sequence respectively, to obtain a first human head frame sequence and a second human head frame sequence, the first human head frame sequence including at least one target a first head frame of a person, the sequence of second head frames includes a second head frame of at least one target person;
  • the matching module is used to match the first human head frame and the second human head frame according to the time sequence relationship between the first human head frame sequence and the second human head frame sequence to obtain the paired head of each target person a sequence of frames, the sequence of matched head frames comprising at least one matched head frame of a target person;
  • a three-dimensional reconstruction module configured to perform three-dimensional reconstruction of the human head in a preset three-dimensional space according to the paired human head frame sequence to obtain a three-dimensional human head sequence of the target person, where the three-dimensional human head sequence includes the three-dimensional human head of the target person;
  • the monitoring module is used for monitoring the passenger flow of the target area according to the three-dimensional human head sequence.
  • an embodiment of the present invention provides an electronic device, including: a memory, a processor, and a computer program stored on the memory and executable on the processor, when the processor executes the computer program The steps in the method for monitoring passenger flow provided by the embodiment of the present invention are implemented.
  • an embodiment of the present invention provides a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the method for monitoring passenger flow provided by the embodiment of the present invention is implemented. A step of.
  • the first target image sequence and the second target image sequence of the target area are acquired, and the first target image sequence and the second target image sequence are acquired at the same moment and from different angles;
  • the target image sequence and the second target image sequence are subjected to human head detection, and a first human head frame sequence and a second human head frame sequence are obtained, wherein the first human head frame sequence includes the first head frame of at least one target person, and the first human head frame sequence includes at least one target person.
  • the two-person head frame sequence includes at least one second head frame of the target person; according to the time sequence relationship between the first head frame sequence and the second head frame sequence, the first head frame and the second head frame are performed.
  • the paired head frame sequence includes at least one paired head frame of the target person; according to the paired head frame sequence, perform three-dimensional reconstruction of the head in a preset three-dimensional space, and obtain A three-dimensional head sequence of the target person, the three-dimensional head sequence includes the three-dimensional head of the target person; according to the three-dimensional head sequence, passenger flow monitoring is performed on the target area.
  • the target area Through the head images of different angles of the target person, more accurate target head information is extracted for 3D reconstruction, so that the position of the 3D target head in the 3D space is more accurate, thereby improving the counting accuracy of passenger flow, thereby improving passenger flow. Monitor the effect.
  • FIG. 1 is a flowchart of a method for monitoring passenger flow provided by an embodiment of the present invention
  • FIG. 2 is a flowchart of a method for constructing a three-dimensional space provided by an embodiment of the present invention
  • FIG. 3 is a schematic diagram of a three-dimensional human head reconstruction method provided by an embodiment of the present invention.
  • FIG. 4a is a schematic diagram of a relationship between a region and a state according to an embodiment of the present invention.
  • FIG. 5 is a schematic structural diagram of a device for monitoring passenger flow provided by an embodiment of the present invention.
  • FIG. 6 is a schematic structural diagram of another passenger flow monitoring device provided by an embodiment of the present invention.
  • FIG. 7 is a schematic structural diagram of a calibration module provided by an embodiment of the present invention.
  • FIG. 8 is a schematic structural diagram of another calibration module provided by an embodiment of the present invention.
  • FIG. 9 is a schematic structural diagram of a three-dimensional reconstruction module provided by an embodiment of the present invention.
  • FIG. 10 is a schematic structural diagram of a second computing submodule provided by an embodiment of the present invention.
  • FIG. 11 is a schematic structural diagram of a reconstruction submodule provided by an embodiment of the present invention.
  • FIG. 12 is a schematic structural diagram of a monitoring module provided by an embodiment of the present invention.
  • FIG. 13 is a schematic structural diagram of a monitoring submodule provided by an embodiment of the present invention.
  • FIG. 14 is a schematic structural diagram of an electronic device provided by an embodiment of the present invention.
  • FIG. 1 is a flowchart of a method for monitoring passenger flow provided by an embodiment of the present invention. As shown in FIG. 1, the method is used to monitor passenger flow regularly or in real time, and includes the following steps:
  • the above-mentioned first target image sequence and the above-mentioned second target image sequence are acquired at the same moment and from different angles.
  • the above-mentioned first target image sequence and second target image sequence include at least one target person.
  • the first target image sequence and the second target image sequence can be collected by two cameras with different shooting angles, and the two cameras can be calibrated and associated during installation, so that the two cameras process the same coordinate system for shooting, and Allows two cameras to shoot at the same time.
  • the first target image sequence and the second target image sequence may also be collected through a calibrated binocular camera.
  • the above-mentioned first target image sequence and second target image sequence may be continuous frame image sequences (video stream images) collected in real time by the binocular camera, or may be continuous frame images collected historically by the binocular camera.
  • the first head frame sequence includes a first head frame of at least one target person
  • the second head frame sequence includes a second head frame of at least one target person.
  • the first head frame corresponds to the first target image sequence
  • the second head frame corresponds to the second target image sequence.
  • the first target image sequence and the second target image sequence can be detected by the human head detection model, respectively, to obtain the first human head frame sequence and the second human head frame sequence.
  • the human head detection model may be used to perform human head detection on the first target image sequence and the second target image sequence frame by frame, so as to obtain a human head frame corresponding to each frame of image.
  • head frame tracking may be performed on the first head frame sequence through a head tracking model to obtain a first head frame sequence tracked according to the first head frame corresponding to each frame of the first target image.
  • first human head frame sequence the first human head frame detected in each frame of the first target image is included.
  • second head frame sequence it is also possible to perform head frame tracking on the second head frame sequence through a head tracking model, and obtain a second head frame sequence to be tracked according to the second head frame corresponding to each frame of the second target image.
  • head frame tracking on the first human head frame sequence through the human head tracking model, and at the same time, perform human head frame tracking on the second human head frame sequence through the human head tracking model, to obtain the first target image corresponding to each frame.
  • the first head frame sequence may be a first head frame sequence corresponding to multiple target persons
  • the second head frame sequence may be a second head frame sequence corresponding to multiple target persons
  • the first head frame may be a plurality of first head frames, corresponding to a plurality of target persons
  • the second head frame may also be a plurality of second head frames, corresponding to a plurality of target persons.
  • the head frame of each target person can be ID assigned according to the head frame tracking algorithm, that is, the head frame of each target person corresponds to a head frame ID, and the head frame ID is used to identify whether the corresponding head frame belongs to the same head frame.
  • the same head frame ID corresponds to the head frame of the same target person.
  • different head frame IDs can be assigned to different target persons, and then the first head frame sequences of different target persons can be obtained according to the different head frame IDs.
  • a sequence of second head frames of different target persons can be obtained according to the similarity between the first head frame and the second head frame.
  • the first head frame sequence and the second head frame sequence match the first head frame with the second head frame to obtain a paired head frame sequence for each target person.
  • the above-mentioned paired head frame sequence includes at least one paired head frame of the target person.
  • the timing relationship between the first human head frame sequence and the second human head frame sequence may be determined by the timing of each image frame in the first target image sequence and the second target image sequence.
  • the obtained head frame is specifically the same time sequence as the target image sequence.
  • the frame images corresponding to the first target image sequence and the second target image sequence are synchronized, so the first target image sequence and the second target image sequence are synchronized.
  • the first human head frame sequence and the second human head frame sequence extracted from the image sequence also have the property of synchronization. Therefore, the synchronized first head frame and the second head frame can be matched for similarity to obtain a paired head frame sequence for each target person.
  • the above-mentioned paired head frame sequence includes a paired head frame corresponding to each frame of the first target image and the second target image, and each paired head frame includes the first head frame of the same target person and the Second head frame.
  • the first head frame and the second head frame of the same target person can be used to calculate the head depth information of the target person in the first head frame and the second head frame.
  • the corresponding first human head image can be extracted from the first target image according to the first human head frame
  • the corresponding second human head image can be extracted from the second target image according to the second human head frame
  • the first human head image is the same as the
  • the second head image is the head image of the same target person at the same moment.
  • the depth of field information of the target person's head can be calculated through the first head image and the second head image of the same target person.
  • the 3D reconstruction of the head in the preset 3D space is performed to obtain the 3D head of the target person.
  • the three-dimensional reconstruction of the head of each paired head frame in the sequence of matched head frames of the target person then the three-dimensional head sequence of the target person can be obtained.
  • the position of the reconstructed 3D head of the target person in the 3D space will also change correspondingly, thereby forming the movement trajectory of the 3D head in the 3D space.
  • the movement trajectory of the target person can be obtained according to the movement trajectory of the 3D human head in the 3D space.
  • the movement trajectory of the three-dimensional head in the three-dimensional space can be mapped to the real space, and the movement trajectory of the target person in the real space can be obtained to monitor the passenger flow in the target area.
  • the position of the target area in the three-dimensional space can also be mapped in the three-dimensional space, and the passenger flow of the target area can be monitored according to the movement trajectory of the three-dimensional head in the three-dimensional space and the position of the target area in the three-dimensional space.
  • the above-mentioned monitoring of the passenger flow in the target area may be the statistical monitoring of the passenger flow, for example, the statistical monitoring of the passenger flow in and out of the target area, or the monitoring of the passenger flow trajectory.
  • the first target image sequence and the second target image sequence of the target area are acquired, and the first target image sequence and the second target image sequence are acquired at the same moment and from different angles;
  • the above-mentioned first target image sequence is acquired by a first camera
  • the above-mentioned second target image sequence is acquired by a second camera, which may be installed according to the settings of the first camera and the second camera. Construct a 3D space suitable for 3D human head reconstruction.
  • FIG. 2 is a flowchart of a method for constructing a three-dimensional space provided by an embodiment of the present invention. As shown in FIG. 2, the following steps are included:
  • the first camera and the second camera may be a left-eye camera and a right-eye camera in a binocular camera, respectively.
  • the internal and external parameters of the first camera and the second camera are calibrated, and the ground is calibrated to obtain the calibrated ground.
  • the above-mentioned calibration of the internal and external parameters of the first camera and the second camera may adopt a checkerboard ideal calibration method.
  • the checkerboard is used for single-camera calibration and dual-camera calibration, respectively, to obtain the internal parameters of the first camera, the internal parameters of the second camera, and the external parameters between the first camera and the second camera.
  • the above-mentioned calibrated ground can be obtained based on the coordinate system of the first camera, or can be obtained based on the coordinate system of the second camera, and the image collected by the first camera can be compared with the image collected by the second camera.
  • Coordinate transformation specifically, the coordinates of the image collected by the first camera can be transformed to the coordinates of the second camera through the internal parameters of the first camera, the internal parameters of the second camera, and the external parameters between the first camera and the second camera. in the department. Therefore, the calibrated ground can be calibrated by selecting the coordinate system of any one of the first camera or the second camera.
  • the calibration object information associated with the target area may be acquired, and the calibration object information is information of the calibration objects on the same plane as the ground.
  • the above-mentioned calibration object information is the calibration object information in the coordinate system of the above-mentioned first camera or the second camera; the ground is calibrated according to the above-mentioned calibration object information, and the calibrated ground is obtained.
  • the above-mentioned calibration object information includes corner point information, and the PnP (Perspective-n-Point) algorithm can be used to calculate the camera pose according to the corner point information, so as to obtain ground parameters, and the calibrated ground can be obtained according to the ground parameters.
  • PnP Perspective-n-Point
  • a two-dimensional code can be set on the ground in the target area or near the target area, and the two-dimensional code can be associated with the target area as specific calibration information.
  • the four corner points of the QR code are sub-pixel refined, and the camera pose is obtained through the four corner points PnP algorithm, thereby obtaining the calibrated ground.
  • the camera pose can be calculated through the known camera internal parameters, the coordinates of the four corner points in the world coordinate system, and the coordinates in the image coordinate system, so as to obtain the ground parameters.
  • the ground parameters Get the calibrated ground Get the calibrated ground.
  • the ground feature points corresponding to the first camera or the second camera may be calculated, and the ground feature points may be triangulated to obtain three-dimensional space points corresponding to the ground feature points. ; Fit the plane parameters to the three-dimensional space points to obtain the calibrated ground. Specifically, after calculating and extracting the ground feature points, the ground feature points can be triangulated to obtain the point cloud coordinates of the feature points in the world coordinate system.
  • the Hough transform point cloud plane detection algorithm and the ransac random consistency sampling algorithm fit the plane parameters to obtain the ground parameters, and the calibrated ground is obtained according to the ground parameters.
  • the above two ground calibration methods may be combined, for example, through feature extraction, ground calibration object information is extracted, and the ground calibration object information is a calibration object on the same plane as the ground.
  • the feature points of the ground calibration object are extracted by calculation, the feature points of the ground calibration object are triangulated to obtain the point cloud coordinates of the feature points of the ground calibration object in the world coordinate system.
  • the Hough transform point cloud plane detection algorithm and the ransac random consistency sampling algorithm fit the plane parameters to obtain the ground parameters, and the calibrated ground is obtained according to the ground parameters.
  • the calibrated ground is obtained based on the coordinate system of the first camera or the coordinate system of the second camera. Therefore, the constructed three-dimensional space is also based on the first camera or the three-dimensional space based on the second camera. Since the three-dimensional space is established based on the camera coordinate system, more accurate three-dimensional head information can be obtained by reconstructing the three-dimensional human head through the three-dimensional space.
  • FIG. 3 is a flowchart of a 3D human head reconstruction method provided by an embodiment of the present invention. As shown in FIG. 3, the method includes the following steps:
  • the above-mentioned disparity map refers to the distance between two images, and the same disparity means that the corresponding objects have the same distance from the camera.
  • the effective parallax interval can be obtained by calculation according to the preset prior parallax; and the parallax map of the first head frame and the second head frame in the paired head frame in the current frame is calculated, and it is judged whether the parallax map falls within the effective parallax interval; if the disparity map falls within the effective disparity interval, it is determined that the disparity map is an effective disparity map.
  • the effective disparity interval can be calculated by the preset disparity threshold. For example, if the prior disparity is 64 and the threshold is 10, the effective disparity interval is 54-74, that is, when the value corresponding to the disparity map of the image is within the effective disparity interval. 54-74, the disparity map of the image is the effective disparity map.
  • SGBM Semi-Global Block Matching
  • first head frame includes the head image of the target person in the first target image
  • second head frame includes the head image of the target person in the second target image
  • first head frame is connected to the first head frame.
  • the disparity map of the two-person head frame can be understood as the disparity between the head image of the target person in the first target image and the head image of the target person in the second target image.
  • the human head depth information (also referred to as human-to-field depth information) of the target person can be obtained by calculating the parallax map, and then three-dimensional reconstruction of the human head is performed according to the human head depth information.
  • the final disparity value can be calculated according to the effective disparity map; the head depth information of the target person can be calculated according to the preset first camera internal parameter or the second camera internal parameter; according to the human head depth information, in the preset three-dimensional space Perform three-dimensional reconstruction of the human head to obtain the three-dimensional human head of the current frame.
  • the head depth information of the target person can be calculated according to the internal parameters of the first camera.
  • the camera intrinsic parameter calculates the head depth information of the target person.
  • the above-mentioned final disparity value may be an average value of the effective disparity maps.
  • Camera intrinsic parameters can include camera focal length and baseline length.
  • coordinate transformation of the human head depth information may be performed to obtain the human head depth information in the coordinate system of the first camera or the second camera, so as to perform three-dimensional reconstruction of the human head in a three-dimensional space.
  • the three-dimensional head reconstruction of the target person in each frame of the first target image and the second target image is performed, so that the three-dimensional head sequence of the target person can be obtained.
  • FIG. 4 is a flowchart of another method for monitoring passenger flow provided by the embodiment of the present invention.
  • the three-dimensional space includes the demarcated ground, and the monitoring of passenger flow in the target area may be performed. It is the statistical monitoring of the number of passenger flows, such as the number of passenger flows entering and leaving the target area. As shown in Figure 4, it includes the following steps:
  • the three-dimensional head of each target person can be projected onto the ground demarcated in the three-dimensional space, so as to obtain the projected trajectory of each target person.
  • the above-mentioned monitoring of the passenger flow in the target area may be the quantitative monitoring of the passenger flow.
  • the camera coordinate system and the world coordinate system can also be converted to further convert the projected trajectory of the target person. It is converted into the trajectory of the target person in the world coordinate system, so that according to the trajectory of the target person in the world coordinate system, the quantitative monitoring of the passenger flow in the target area is carried out.
  • the ground to be calibrated in the three-dimensional space includes a target calibration area corresponding to the target area, and the state information of the projection trajectory and the target calibration area at each time sequence point can be calculated to obtain the difference between the projection trajectory and the target calibration area.
  • State sequence according to the state sequence, monitor the passenger flow of the target area.
  • the state sequence of the projected trajectory and the target calibration area includes the state information of each time sequence point.
  • the above state information can be divided into valid area status and invalid area status.
  • the above-mentioned target area can be the area on the left and right sides of the store door, that is, the above-mentioned target calibration area can be the fixed area on the left and right sides of the store door in the demarcated ground, and the above-mentioned effective area status can also be divided into:
  • a Outside the store door, it can indicate that the human head is projected on the outer side of the target calibration area at this time, and further indicates that the target person is in the outer area on both sides of the store door area.
  • c Outside the store, it can indicate that the human head is projected on the outside of the target calibration area at this time, and further indicates that the target person is outside the outside area of the store door, and the inside of the outside area of the store door is the inside area on both sides of the store door area corresponding to state b.
  • d Inside the store, it can indicate that the human head is projected on the inside of the target calibration area at this time, and further indicates that the target person is inside the inside area of the store door, and the outside of the inside area of the store door is the outside area on both sides of the store door area corresponding to state a.
  • n Uncertain area, which can include buffers near a and b, and areas very close and far away from the camera (usually caused by the matching error of the human head frame).
  • the above-mentioned state sequence may be composed of a, b, c, d, t, and n, and a regular search can be performed according to the state sequence, and then the counting and statistics of the passenger flow can be performed.
  • the state sequence of a target person is c...ca...ab...bd...dt, it can indicate a process that the target person goes from outside the store, to the outside of the store, to the inside of the store, to the inside of the store, to disappearing, which can be counted as Add 1 to the customer flow into the store;
  • the state sequence of a target person is d...db...ba...ac...ct, which can indicate a process of the target person from inside the store, to the inside of the store, to the outside of the store, to outside the store, to disappearance , which can be counted as out-of-store customer flow plus 1;
  • the status sequence of a target person is c...ca...ac...ca...ab...ba...ab...bd...dt, it can indicate that the target person goes from outside the store, to the outside of the store, to the store Outside the store, to the outside of the store, to the inside of the store, to the outside of the store, to the inside of the store,
  • the above projected trajectory can also be converted into the world coordinate system, and the state information of the trajectory corresponding to the target person at each time sequence point and the target area can be calculated to obtain the state sequence of the target person and the target area, and then according to the state sequence to The target area is monitored for passenger flow.
  • identifiers of the above states can be set according to the needs of the user, and should not be regarded as a limitation on the embodiments of the present invention.
  • the above a, b, c, d, t, and n may also be identified by 1, 2, 3, 4, 5, and 6.
  • passenger flow statistics are performed on the target person through the state sequence, which can increase the accuracy of the statistics.
  • FIG. 5 is a schematic structural diagram of a device for monitoring passenger flow according to an embodiment of the present invention. As shown in FIG. 5, the device includes:
  • an acquisition module 501 configured to acquire a first target image sequence and a second target image sequence of a target area, wherein the first target image sequence and the second target image sequence are acquired at the same moment and from different angles;
  • the processing module 502 is configured to perform human head detection on the first target image sequence and the second target image sequence, respectively, to obtain a first human head frame sequence and a second human head frame sequence, and the first human head frame sequence includes at least one a first head frame of the target person, the second head frame sequence including at least one second head frame of the target person;
  • the matching module 503 is configured to match the first human head frame and the second human head frame according to the time sequence relationship between the first human head frame sequence and the second human head frame sequence to obtain the pairing of each target person
  • a human head frame sequence, the paired human head frame sequence includes at least one paired human head frame of a target person;
  • a three-dimensional reconstruction module 504 configured to perform three-dimensional reconstruction of the human head in a preset three-dimensional space according to the paired human head frame sequence to obtain a three-dimensional human head sequence of the target person, where the three-dimensional human head sequence includes the three-dimensional human head of the target person;
  • the monitoring module 505 is configured to monitor the passenger flow of the target area according to the three-dimensional human head sequence.
  • the first target image sequence is acquired by a first camera
  • the second target image sequence is acquired by a second camera
  • the device further includes:
  • the calibration module 506 is configured to perform ground calibration under the coordinate system of the first camera or the second camera to obtain the calibrated ground;
  • the construction module 507 is configured to construct and obtain the three-dimensional space based on the marked ground.
  • the calibration module 506 includes:
  • the acquisition sub-module 5061 is used to acquire the calibration object information associated with the target area, and the calibration object information is the calibration object information in the coordinate system of the first camera or the second camera;
  • the first calibration sub-module 5062 is configured to perform ground calibration according to the calibration object information to obtain the calibrated ground.
  • the calibration module 506 includes:
  • the first calculation sub-module 5063 is used to calculate the ground feature points corresponding to the first camera and the second camera, triangulate the ground feature points, and obtain the three-dimensional space points corresponding to the ground feature points;
  • the second calibration sub-module 5064 is configured to perform plane parameter fitting on the three-dimensional space point to obtain the calibrated ground.
  • the three-dimensional reconstruction module 504 includes:
  • the second calculation submodule 5041 is used to calculate the effective disparity map of the first head frame and the second head frame in the paired head frame in the current frame;
  • the reconstruction sub-module 5042 is used for performing three-dimensional reconstruction of the human head in a preset three-dimensional space through the effective disparity map to obtain a three-dimensional human head of the current frame;
  • the sequence sub-module 5043 is configured to obtain the 3D human head sequence of the target person based on the 3D human head of the current frame.
  • the second calculation sub-module 5041 includes:
  • the first calculation unit 50411 is used to calculate the effective parallax interval according to the preset prior parallax
  • the second calculation unit 50412 is configured to calculate the disparity map of the first human head frame and the second human head frame in the paired human head frame in the current frame, and determine whether the disparity map falls within the effective disparity interval;
  • the determining unit 50413 is configured to determine that the disparity map is an effective disparity map if the disparity map falls within the valid disparity interval.
  • the reconstruction sub-module 5042 includes:
  • a third calculation unit 50421, configured to calculate the final disparity value according to the effective disparity map
  • the fourth calculation unit 50422 is used to calculate and obtain the head depth information of the target person according to the preset first camera internal parameter or second camera internal parameter;
  • the reconstruction unit 50423 is configured to perform three-dimensional reconstruction of the human head in the preset three-dimensional space according to the human head depth information to obtain the three-dimensional human head of the current frame.
  • the monitoring module 505 includes:
  • the projection sub-module 5051 is used to project the three-dimensional human head in the three-dimensional human head sequence onto the calibrated ground to obtain the projected trajectory of the target person;
  • the monitoring sub-module 5052 is configured to monitor the passenger flow of the target area according to the projected trajectory.
  • the calibrated ground includes a target calibration area corresponding to the target area
  • the monitoring sub-module 5052 includes:
  • the fifth calculation unit 50521 is used to calculate the state information of the projection trajectory and the target calibration area at each timing point, and obtain the state sequence of the projection trajectory and the target calibration area;
  • the monitoring unit 50522 is configured to monitor the passenger flow of the target area according to the state sequence.
  • the device for monitoring passenger flow provided by the embodiment of the present invention can implement each process implemented by the method for monitoring passenger flow in the above method embodiments, and can achieve the same beneficial effects. To avoid repetition, details are not repeated here.
  • FIG. 14 is a schematic structural diagram of an electronic device provided by an embodiment of the present invention, as shown in FIG. 14, including: a memory 1402, a processor 1401, and a memory 1402 and a processor A computer program running on 1401, which:
  • the processor 1401 is configured to call the computer program stored in the memory 1402, and perform the following steps:
  • first human head frame sequence includes the first person of at least one target person a head frame
  • sequence of second human head frames includes a second head frame of at least one target person
  • the first head frame and the second head frame are matched to obtain a paired head frame sequence for each target person, and the
  • the paired head frame sequence includes at least one paired head frame of the target person;
  • three-dimensional reconstruction of the human head is performed in a preset three-dimensional space to obtain a three-dimensional human head sequence of the target person, and the three-dimensional human head sequence includes the three-dimensional human head of the target person;
  • passenger flow monitoring is performed on the target area.
  • the first target image sequence is acquired by a first camera
  • the second target image sequence is acquired by a second camera
  • the processor 1401 further includes:
  • the three-dimensional space is constructed and obtained.
  • the processor 1401 performs ground calibration in the coordinate system of the first camera or the second camera to obtain the calibrated ground, including:
  • the calibration object information is calibration object information in the coordinate system of the first camera or the second camera;
  • the ground is calibrated according to the calibration object information, and the calibrated ground is obtained.
  • the processor 1401 performs ground calibration in the coordinate system of the first camera or the second camera to obtain the calibrated ground, including:
  • performing three-dimensional reconstruction of the human head on the paired human head frame sequence in a preset three-dimensional space by the processor 1401 to obtain a three-dimensional human head sequence of the target person including:
  • three-dimensional reconstruction of the human head is performed in a preset three-dimensional space to obtain a three-dimensional human head of the current frame;
  • a three-dimensional human head sequence of the target person is obtained.
  • the calculation performed by the processor 1401 of the effective disparity map of the first head frame and the second head frame in the paired head frame in the current frame includes:
  • the disparity map falls within the effective disparity interval, it is determined that the disparity map is an effective disparity map.
  • performing three-dimensional reconstruction of a human head in a preset three-dimensional space through the effective disparity map performed by the processor 1401 to obtain a three-dimensional human head of the current frame including:
  • three-dimensional reconstruction of the human head is performed in the preset three-dimensional space to obtain the three-dimensional human head of the current frame.
  • the three-dimensional space includes a calibrated ground
  • the monitoring of the passenger flow in the target area according to the three-dimensional human head sequence executed by the processor 1401 includes:
  • passenger flow monitoring is performed on the target area.
  • the demarcated ground includes a target demarcation area corresponding to the target area
  • the monitoring of passenger flow in the target area according to the projection trajectory includes:
  • the passenger flow monitoring is performed on the target area.
  • the electronic device provided in the embodiments of the present invention can implement the various processes implemented by the method for monitoring passenger flow in the above method embodiments, and can achieve the same beneficial effects. To avoid repetition, details are not repeated here.
  • Embodiments of the present invention further provide a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium.
  • a computer program is stored on the computer-readable storage medium.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • Human Computer Interaction (AREA)
  • Image Analysis (AREA)
  • Image Processing (AREA)
  • Closed-Circuit Television Systems (AREA)

Abstract

L'invention concerne un procédé et un appareil de surveillance de flux de passagers, ainsi qu'un dispositif électronique et un support de stockage. Le procédé consiste : à acquérir une première séquence d'images cibles et une seconde séquence d'images cibles d'une région cible (101); à réaliser respectivement une détection de tête sur la première séquence d'images cibles et la seconde séquence d'images cibles pour obtenir une première séquence de cadres de tête et une seconde séquence de cadres de tête (102); à mettre en correspondance un premier cadre de tête avec un second cadre de tête selon une relation de synchronisation entre la première séquence de cadres de tête et la seconde séquence de cadres de tête de sorte à obtenir une séquence de cadres de tête appariés de chaque personne cible (103), la séquence de cadres de tête appariés comprenant des cadres de tête appariés d'au moins une personne cible; à réaliser une reconstruction de tête tridimensionnelle dans un espace tridimensionnel prédéfini selon la séquence de cadres de tête appariés de sorte à obtenir une séquence de tête tridimensionnelle de la personne cible (104), la séquence de tête tridimensionnelle comprenant une tête tridimensionnelle de la personne cible; et à surveiller le flux de passagers dans la région cible en fonction de la séquence de têtes tridimensionnelles (105). Au moyen du procédé, l'effet de surveillance de flux de passagers est amélioré.
PCT/CN2021/114965 2020-12-14 2021-08-27 Procédé et appareil de surveillance de flux de passagers, ainsi que dispositif électronique et support de stockage WO2022127181A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202011466662.XA CN112633096A (zh) 2020-12-14 2020-12-14 客流的监测方法、装置、电子设备及存储介质
CN202011466662.X 2020-12-14

Publications (1)

Publication Number Publication Date
WO2022127181A1 true WO2022127181A1 (fr) 2022-06-23

Family

ID=75312656

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/114965 WO2022127181A1 (fr) 2020-12-14 2021-08-27 Procédé et appareil de surveillance de flux de passagers, ainsi que dispositif électronique et support de stockage

Country Status (2)

Country Link
CN (1) CN112633096A (fr)
WO (1) WO2022127181A1 (fr)

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112633096A (zh) * 2020-12-14 2021-04-09 深圳云天励飞技术股份有限公司 客流的监测方法、装置、电子设备及存储介质
CN112651571A (zh) * 2020-12-31 2021-04-13 深圳云天励飞技术股份有限公司 一种商城客流量的预测方法、装置、电子设备及存储介质
CN113326830B (zh) * 2021-08-04 2021-11-30 北京文安智能技术股份有限公司 基于俯视图像的客流统计模型训练方法和客流统计方法
CN114677651B (zh) * 2022-05-30 2022-09-27 山东极视角科技有限公司 一种基于低画质低帧率视频的客流统计方法及相关装置

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103455792A (zh) * 2013-08-20 2013-12-18 深圳市飞瑞斯科技有限公司 一种客流量统计方法及***
CN104103077A (zh) * 2014-07-29 2014-10-15 浙江宇视科技有限公司 一种人头检测方法和装置
CN106709432A (zh) * 2016-12-06 2017-05-24 成都通甲优博科技有限责任公司 基于双目立体视觉的人头检测计数方法
WO2018135510A1 (fr) * 2017-01-19 2018-07-26 パナソニック インテレクチュアル プロパティ コーポレーション オブ アメリカ Procédé de reconstruction tridimensionnelle et dispositif de reconstruction tridimensionnelle
CN108446611A (zh) * 2018-03-06 2018-08-24 深圳市图敏智能视频股份有限公司 一种车门状态关联的双目图像公交客流计算方法
CN112633096A (zh) * 2020-12-14 2021-04-09 深圳云天励飞技术股份有限公司 客流的监测方法、装置、电子设备及存储介质

Family Cites Families (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9001199B2 (en) * 2011-04-29 2015-04-07 Tata Consultancy Services Limited System and method for human detection and counting using background modeling, HOG and Haar features
JP2013093013A (ja) * 2011-10-06 2013-05-16 Ricoh Co Ltd 画像処理装置、車両
WO2014037603A1 (fr) * 2012-09-06 2014-03-13 Nokia Corporation Appareil, procédé et programme informatique pour le traitement d'images
KR20140108828A (ko) * 2013-02-28 2014-09-15 한국전자통신연구원 카메라 트래킹 장치 및 방법
CN104915965A (zh) * 2014-03-14 2015-09-16 华为技术有限公司 一种摄像机跟踪方法及装置
CN105160649A (zh) * 2015-06-30 2015-12-16 上海交通大学 基于核函数非监督聚类的多目标跟踪方法及***
CN107133988B (zh) * 2017-06-06 2020-06-02 科大讯飞股份有限公司 车载全景环视***中摄像头的标定方法及标定***
CN109191504A (zh) * 2018-08-01 2019-01-11 南京航空航天大学 一种无人机目标跟踪方法
CN109785396B (zh) * 2019-01-23 2021-09-28 中国科学院自动化研究所 基于双目相机的写字姿态监测方法、***、装置
CN110222673B (zh) * 2019-06-21 2021-04-06 杭州宇泛智能科技有限公司 一种基于头部检测的客流统计方法
CN111028271B (zh) * 2019-12-06 2023-04-14 浩云科技股份有限公司 基于人体骨架检测的多摄像机人员三维定位跟踪***
CN111160243A (zh) * 2019-12-27 2020-05-15 深圳云天励飞技术有限公司 客流量统计方法及相关产品
CN111354077B (zh) * 2020-03-02 2022-11-18 东南大学 一种基于双目视觉的三维人脸重建方法
CN111899282B (zh) * 2020-07-30 2024-05-14 平安科技（深圳）有限公司 基于双目摄像机标定的行人轨迹跟踪方法及装置

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103455792A (zh) * 2013-08-20 2013-12-18 深圳市飞瑞斯科技有限公司 一种客流量统计方法及***
CN104103077A (zh) * 2014-07-29 2014-10-15 浙江宇视科技有限公司 一种人头检测方法和装置
CN106709432A (zh) * 2016-12-06 2017-05-24 成都通甲优博科技有限责任公司 基于双目立体视觉的人头检测计数方法
WO2018135510A1 (fr) * 2017-01-19 2018-07-26 パナソニック インテレクチュアル プロパティ コーポレーション オブ アメリカ Procédé de reconstruction tridimensionnelle et dispositif de reconstruction tridimensionnelle
CN108446611A (zh) * 2018-03-06 2018-08-24 深圳市图敏智能视频股份有限公司 一种车门状态关联的双目图像公交客流计算方法
CN112633096A (zh) * 2020-12-14 2021-04-09 深圳云天励飞技术股份有限公司 客流的监测方法、装置、电子设备及存储介质

Also Published As

Publication number Publication date
CN112633096A (zh) 2021-04-09

Similar Documents

Publication Publication Date Title
WO2022127181A1 (fr) Procédé et appareil de surveillance de flux de passagers, ainsi que dispositif électronique et support de stockage
US9646212B2 (en) Methods, devices and systems for detecting objects in a video
CN107240124B (zh) 基于时空约束的跨镜头多目标跟踪方法及装置
JP2019075156A (ja) 多因子画像特徴登録及び追尾のための方法、回路、装置、システム、及び、関連するコンピュータで実行可能なコード
JP6295645B2 (ja) 物体検出方法及び物体検出装置
CN104966062B (zh) 视频监视方法和装置
US9189859B2 (en) 3D image generation
CN109949347B (zh) 人体跟踪方法、装置、***、电子设备和存储介质
US20190051036A1 (en) Three-dimensional reconstruction method
Salih et al. Depth and geometry from a single 2d image using triangulation
WO2019225547A1 (fr) Dispositif, procédé et programme de suivi d'objet
CN110458897A (zh) 多摄像头自动标定方法及***、监控方法及***
CN107560592A (zh) 一种用于光电跟踪仪联动目标的精确测距方法
JP5027758B2 (ja) 画像監視装置
JP2018156408A (ja) 画像認識撮像装置
JP5667846B2 (ja) 対象物画像判定装置
JP2010002976A (ja) 画像監視装置
CN108230351A (zh) 基于双目立体视觉行人检测的柜台评价方法与***
CN110910449B (zh) 识别物体三维位置的方法和***
JP2004046464A (ja) 移動物体３次元位置推定装置及びその方法、プログラム並びにその記録媒体
JP2007109126A (ja) 移動体分布推定装置、移動体分布推定方法及び移動体分布推定プログラム
CN115880643A (zh) 一种基于目标检测算法的社交距离监测方法和装置
JP6504711B2 (ja) 画像処理装置
JP6548306B2 (ja) カメラの撮影画像に映る人物を追跡する画像解析装置、プログラム及び方法
Hung et al. Detecting fall incidents of the elderly based on human-ground contact areas

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21905114

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21905114

Country of ref document: EP

Kind code of ref document: A1