CN113077556A - Ticket checking system and method based on pedestrian re-identification - Google Patents
Ticket checking system and method based on pedestrian re-identification Download PDFInfo
- Publication number
- CN113077556A CN113077556A CN202110334262.1A CN202110334262A CN113077556A CN 113077556 A CN113077556 A CN 113077556A CN 202110334262 A CN202110334262 A CN 202110334262A CN 113077556 A CN113077556 A CN 113077556A
- Authority
- CN
- China
- Prior art keywords
- pedestrian
- ticket checking
- data
- frame
- ticket
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 24
- 230000009471 action Effects 0.000 claims abstract description 7
- 238000009432 framing Methods 0.000 claims abstract description 4
- 238000005070 sampling Methods 0.000 claims description 16
- 238000004364 calculation method Methods 0.000 claims description 15
- 230000037237 body shape Effects 0.000 claims description 6
- 238000004458 analytical method Methods 0.000 claims description 2
- 238000002156 mixing Methods 0.000 claims 1
- 230000008569 process Effects 0.000 abstract description 4
- 238000001514 detection method Methods 0.000 description 6
- 238000000605 extraction Methods 0.000 description 4
- 230000036544 posture Effects 0.000 description 4
- 238000013527 convolutional neural network Methods 0.000 description 3
- 238000010586 diagram Methods 0.000 description 2
- 239000000284 extract Substances 0.000 description 2
- 238000011176 pooling Methods 0.000 description 2
- 230000004075 alteration Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000005034 decoration Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000036651 mood Effects 0.000 description 1
- 230000011218 segmentation Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 239000013589 supplement Substances 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G07—CHECKING-DEVICES
- G07B—TICKET-ISSUING APPARATUS; FARE-REGISTERING APPARATUS; FRANKING APPARATUS
- G07B11/00—Apparatus for validating or cancelling issued tickets
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/23—Clustering techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/40—Scenes; Scene-specific elements in video content
- G06V20/46—Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects
- G06V20/53—Recognition of crowd images, e.g. recognition of crowd congestion
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30196—Human being; Person
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Engineering & Computer Science (AREA)
- Evolutionary Computation (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Computational Linguistics (AREA)
- Multimedia (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Mathematical Physics (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Image Analysis (AREA)
Abstract
The embodiment of the invention discloses a ticket checking system and a method based on pedestrian re-identification, wherein the system comprises a tourist video acquisition device, a computer, a comparison database, a ticket checking video acquisition device and a gateway, wherein the tourist video acquisition device acquires video data of a tourist who has purchased a ticket; the ticket checking video acquisition device acquires videos of pedestrians to be detected at all ticket checking points; reading the video data by the computer, framing the video data to extract a pedestrian image, and storing the obtained pedestrian image into a comparison database in real time; acquiring videos of all ticket checking points, detecting in real time to obtain pedestrian data, comparing the pedestrian data with data in a comparison database, and transmitting a comparison result to a gateway gate; and the channel gate makes a release or warning action according to the comparison result. The ticket checking system hides the traditional ticket checking process and replaces a manual process with computer equipment, so that the ticket buying and checking efficiency can be improved, and the playing experience of tourists can be ensured.
Description
Technical Field
The invention relates to the technical field of scenic spot services, in particular to a ticket checking system and method based on pedestrian re-identification.
Background
With the improvement of living standard, the requirement of going out for playing is increasing day by day, and especially in the legal holidays such as Wuyi and national celebration, the phenomena that the scenic spots in each region are congested and the working personnel are not enough due to the arrival of a large number of tourists occur. Due to lack of hands and low working efficiency, long teams can be arranged at ticket selling points and ticket checking points of various scenic spots, and therefore most tourists can be irritated and lose pleasure. Not only the waiting time for buying and checking tickets can affect the playing mood, but also the playing interruption caused by the loss of paper tickets can be caused frequently. Although tourists can pay and purchase tickets on the internet through mobile phones, the paper bills need to be replaced by payment codes, the difficulty of buying and using the tickets is not fundamentally solved, and the fundamental reason is that the paper bills are always used as the unique identification of each tourist.
At present, paper tickets are used as unique identification of tourists in all playing fields needing ticket purchasing. Besides paper bills, fingerprints and human faces are used as unique identifications of people and are also applied to various scenes, but privacy problems are frequently discussed. It is impractical to extract fingerprints and faces of tourists in a playing field.
Disclosure of Invention
The technical problem to be solved by the embodiments of the present invention is to provide a ticket checking system and method based on pedestrian re-identification, so as to improve the work efficiency of the tourist attraction related service field and to replace the use of paper tickets to bring good play experience to tourists.
In order to solve the above technical problems, an embodiment of the present invention provides a ticket checking system based on pedestrian re-identification, which includes a visitor video collecting device, a computer, a comparison database, a ticket checking video collecting device, and a gateway, wherein the computer is connected to the visitor video collecting device, the comparison database, the ticket checking video collecting device, and the gateway,
the tourist video acquisition device acquires video data of the tourist who has purchased the ticket;
the ticket checking video acquisition device acquires videos of pedestrians to be detected at all ticket checking points;
the computer reads the collected video data of the ticket-purchased tourists, frames the video data to extract a pedestrian image, and stores the obtained pedestrian image into a comparison database in real time; acquiring videos of all ticket checking points, detecting in real time to obtain pedestrian data, comparing the pedestrian data with data in a comparison database, and transmitting a comparison result to a gateway gate;
and the channel gate makes a release or warning action according to the comparison result.
Further, the computer reads the frame format of the collected video data of the ticket-purchased tourists passing through the preset position, determines the frame number, extracts the features of the continuous frames of images, analyzes and judges whether the images contain the pedestrian features; the specific position of the pedestrian in the image is calibrated for the image judged to contain the pedestrian, the pedestrian photo is stored in a comparison database, and the image judged not to contain the pedestrian is not processed; the continuous processing of the frame images of the preset position video marks the detected pedestrian data and compares the detected pedestrian data with the number of purchased tickets, thereby avoiding the situation that the number of sold tickets is not matched with the tourists.
Further, using Darknet-19 to combine with residual skip layer link to perform feature extraction on the image of each frame, using the step length of the volume local layer to perform down-sampling, using Yolov3 to perform up-sampling on the finally obtained features and fusing the features with the previous feature layers, and using three different feature layers as the final feature output, wherein the error is calculated by using the following formula in Yolov 3:
wherein the parametersWhether the jth prior frame of the ith grid is responsible for the target object or not is represented, if so, the jth prior frame is 1, and if not, the jth prior frame is 0; lambda [ alpha ]coordIs a coordination coefficient set for coordinating the inconsistency of the contribution of the rectangular frames with different sizes to the error function,is the coordinates of the center of the rectangular box of the network prediction,is the center coordinate of the marked rectangular frame,the size of the width and height of the rectangle predicted for the network,is to mark the width and height of the rectangular frame, CiTo predict the probability score of the target object contained within the frame,representing the true value;indicating that the size of the rectangular box is 1 if it is not responsible for predicting an object, otherwise it is equal to 0; lambda [ alpha ]noobjIs a weight value representing that the confidence error is in the loss when the target is not predicted by the prediction boxThe weight occupied by the lost function;indicates the probability that the prediction box belongs to the category c,and if the true value of the category to which the mark box belongs to the class c, the size of the true value is equal to 1, and otherwise, the true value is 0.
Further, the computer obtains the shape and the size of an anchor frame according with the human body shape by adopting a clustering algorithm to the pedestrian data in each ticket checking point video; extracting global and local features of the detected pedestrian; and calculating the Euclidean distance between the pedestrian characteristics of the ticket checking point and the pedestrian characteristics of the comparison database, considering that the pedestrian is not matched with the current pedestrian if the calculation result is greater than a set threshold value, and considering that the pedestrian of the current ticket checking point and the pedestrian in the comparison database are the same pedestrian if the calculation result is less than the set threshold value.
Further, the loss function in the clustering algorithm adopts a loss of a triplet of sampling difficult samples, for each training set, people with P identities are randomly selected, and each person randomly selects K different pictures, so that each training set contains P × K pictures, and then a triplet is formed by selecting a most difficult positive sample and a most difficult negative sample for each picture in the training set, and the loss function formula is as follows:
wherein d isa,pRepresenting the Euclidean distance, d, between the sample a and the positive sample pa,nRepresenting the euclidean distance between the sample a and the negative sample n, alpha being the set threshold parameter.
Correspondingly, the embodiment of the invention also provides a ticket checking method based on pedestrian re-identification, which comprises the following steps:
step 1: collecting video data of the ticket-purchased tourist;
step 2: reading the collected video data of the ticket-purchased tourists, and framing the video data to extract a pedestrian image;
and step 3: acquiring a pedestrian image, and storing the pedestrian image into a comparison database in real time;
and 4, step 4: acquiring videos of all ticket checking points, detecting in real time to obtain pedestrian data, and comparing the pedestrian data with data in a comparison database;
and 5: and sending the comparison result to a channel gate, and allowing the channel gate to perform a release or warning action according to the comparison result.
Further, in step 2, reading a frame format of collected video data of the ticket-purchased tourist passing through a preset position, determining the number of frames, extracting the characteristics of continuous frames of images, analyzing and judging whether the images contain pedestrian characteristics; the specific position of the pedestrian in the image is calibrated for the image judged to contain the pedestrian, the pedestrian photo is stored in a comparison database, and the image judged not to contain the pedestrian is not processed; the continuous processing of the frame images of the preset position video marks the detected pedestrian data and compares the detected pedestrian data with the number of purchased tickets, thereby avoiding the situation that the number of sold tickets is not matched with the tourists.
Further, in step 2, using the Darknet-19 to combine with the residual skip layer link to perform feature extraction on the image of each frame, using the step size of the volume local layer to perform down-sampling, using Yolov3 to perform up-sampling on the finally obtained features and fusing the features with the previous feature layers, and using three different sized feature layers as the final feature output, wherein the error is calculated by using the following formula in Yolov 3:
wherein the parametersWhether the jth prior frame of the ith grid is responsible for the target object or not is represented, if so, the jth prior frame is 1, and if not, the jth prior frame is 0; lambda [ alpha ]coordIs a coordination set for coordinating the inconsistency of the contribution of different size rectangular frames to the error functionThe coefficient of modulation is adjusted,is the coordinates of the center of the rectangular box of the network prediction,is the center coordinate of the marked rectangular frame,the size of the width and height of the rectangle predicted for the network,is to mark the width and height of the rectangular frame, CiTo predict the probability score of the target object contained within the frame,representing the true value;indicating that the size of the rectangular box is 1 if it is not responsible for predicting an object, otherwise it is equal to 0; lambda [ alpha ]noobjThe weight value represents the weight of the confidence error in the loss function when the target is not predicted by the prediction box;indicates the probability that the prediction box belongs to the category c,and if the true value of the category to which the mark box belongs to the class c, the size of the true value is equal to 1, and otherwise, the true value is 0.
Further, in the step 4, the pedestrian data in the ticket checking point videos are subjected to a clustering algorithm to obtain the shape and the size of an anchor frame according with the human body shape; extracting global and local features of the detected pedestrian; and calculating the Euclidean distance between the pedestrian characteristics of the ticket checking point and the pedestrian characteristics of the comparison database, considering that the pedestrian is not matched with the current pedestrian if the calculation result is greater than a set threshold value, and considering that the pedestrian of the current ticket checking point and the pedestrian in the comparison database are the same pedestrian if the calculation result is less than the set threshold value.
Further, the loss function in the clustering algorithm adopts a loss of a triplet of sampling difficult samples, for each training set, people with P identities are randomly selected, and each person randomly selects K different pictures, so that each training set contains P × K pictures, and then a triplet is formed by selecting a most difficult positive sample and a most difficult negative sample for each picture in the training set, and the loss function formula is as follows:
wherein d isa,pRepresenting the Euclidean distance, d, between the sample a and the positive sample pa,nRepresenting the euclidean distance between the sample a and the negative sample n, alpha being the set threshold parameter.
The invention has the beneficial effects that: the ticket checking system hides the traditional ticket checking process and replaces a manual process with computer equipment, so that the ticket buying and checking efficiency can be improved, and the playing experience of tourists can be ensured. Therefore, the ticket checking method and system based on pedestrian re-identification provided by the invention have high popularization and use values.
Drawings
Fig. 1 is a flowchart of a ticket checking method based on pedestrian re-identification according to an embodiment of the present invention.
Fig. 2 is a detailed structural diagram of an algorithm for extracting a pedestrian image according to an embodiment of the present invention.
Fig. 3 is a schematic diagram of a ticket checking method based on pedestrian re-identification according to an embodiment of the invention.
Detailed Description
It should be noted that the embodiments and features of the embodiments in the present application can be combined with each other without conflict, and the present invention is further described in detail with reference to the drawings and specific embodiments.
The ticket checking system based on pedestrian re-identification comprises a tourist video acquisition device, a computer, a comparison database, a ticket checking video acquisition device and a gateway. The computer is connected with the tourist video acquisition device, the comparison database, the ticket checking video acquisition device and the channel gate.
Preferably, visitor's video acquisition device includes two cameras, during the concrete implementation, places two cameras respectively in fixed point scanning platform's the preceding top and the upper right side, and its shooting range covers: the front and the side of the pedestrian are the whole body. The ticket checking system based on pedestrian re-identification in the embodiment of the invention collects the video data of the ticket-purchased tourists by controlling the cameras arranged at the front and the right sides of the fixed-point scanning table, and stores the video data recorded by the two cameras into the video storage module for calling and video processing.
The tourist video acquisition device acquires video data of the tourist who has bought the ticket.
The ticket checking video acquisition device acquires videos of pedestrians to be checked at all ticket checking points.
The computer reads the collected video data of the ticket-purchased tourists, frames the video data to extract a pedestrian image, and stores the obtained pedestrian image into a comparison database in real time; and acquiring videos of all ticket checking points, detecting in real time to obtain pedestrian data, comparing the pedestrian data with data in a comparison database, and transmitting a comparison result to the gateway.
And the channel gate makes a release or warning action according to the comparison result. For example, if the same pedestrian exists in the comparison result, extracting the data in the comparison library and displaying the data in the detection instrument, and if the comparison result does not exist, prompting; the channel gate is used as a unique channel, if the matching is successful, the gate is opened, and if the matching is failed, the tourist is prompted to purchase the ticket to enter.
As an implementation mode, the computer reads the frame format of the collected video data of the ticket-purchased tourists passing through the preset position and determines the frame number, and extracts the characteristics of the continuous frames of images for analysis and judges whether the images contain the pedestrian characteristics; the specific position of the pedestrian in the image is calibrated for the image judged to contain the pedestrian, the pedestrian photo is stored in a comparison database, and the image judged not to contain the pedestrian is not processed; the continuous processing of the frame images of the preset position video marks the detected pedestrian data and compares the detected pedestrian data with the number of purchased tickets, thereby avoiding the situation that the number of sold tickets is not matched with the tourists.
As an embodiment, using Darknet-19 to combine with residual skip layer link to perform feature extraction on the image of each frame, using the step size of volume local layer to perform down-sampling, using YOLOv3 to perform up-sampling on the finally obtained feature and fusing the feature layers before, and using three different sized feature layers together as the final feature output, wherein the YOLOv3 adopts the following formula to calculate the error:
wherein the parametersWhether the jth prior frame of the ith grid is responsible for the target object or not is represented, if so, the jth prior frame is 1, and if not, the jth prior frame is 0; lambda [ alpha ]coordIs a coordination coefficient set for coordinating the inconsistency of the contribution of the rectangular frames with different sizes to the error function,is the coordinates of the center of the rectangular box of the network prediction,is the center coordinate of the marked rectangular frame,the size of the width and height of the rectangle predicted for the network,is to mark the width and height of the rectangular frame, CiTo predict the probability score of the target object contained within the frame,representing the true value;indicating that the size of the rectangular box is 1 if it is not responsible for predicting an object, otherwise it is equal to 0; lambda [ alpha ]noobjThe weight value represents the weight of the confidence error in the loss function when the target is not predicted by the prediction box;indicates the probability that the prediction box belongs to the category c,and if the true value of the category to which the mark box belongs to the class c, the size of the true value is equal to 1, and otherwise, the true value is 0.
As an implementation mode, the computer adopts a clustering algorithm to the pedestrian data in each ticket checking point video to obtain the shape and the size of an anchor frame according with the human body shape; extracting global and local features of the detected pedestrian; and calculating the Euclidean distance between the pedestrian characteristics of the ticket checking point and the pedestrian characteristics of the comparison database, considering that the pedestrian is not matched with the current pedestrian if the calculation result is greater than a set threshold value, and considering that the pedestrian of the current ticket checking point and the pedestrian in the comparison database are the same pedestrian if the calculation result is less than the set threshold value. The embodiment of the invention adopts a large amount of pedestrian data, obtains the size and the data of the anchor frame more suitable for detecting the pedestrian by using the clustering algorithm, uses the size and the data of the anchor frame for more accurately detecting the specific position of the pedestrian in the pedestrian detection algorithm, and adopts a strategy of combining global and local characteristics for the detected pedestrian so as to effectively match the image of the same pedestrian under different cameras, thereby effectively reducing the false detection, the missed detection and the false detection of the ticket checking system.
As an embodiment, the loss function in the clustering algorithm uses a hard sample sampling triplet loss, for each training set, P persons with identities are randomly selected, and each person randomly selects K different pictures, so that each training set includes P × K pictures, and then a hard positive sample and a hard negative sample are selected for each picture in the training set to form a triplet, where the loss function formula is as follows:
wherein d isa,pRepresenting the Euclidean distance, d, between the sample a and the positive sample pa,nRepresenting the euclidean distance between the sample a and the negative sample n, alpha being the set threshold parameter.
Referring to fig. 1, a ticket checking method based on pedestrian re-identification according to an embodiment of the present invention includes:
step 1: collecting video data of the ticket-purchased tourist;
step 2: reading the collected video data of the ticket-purchased tourists, and framing the video data to extract a pedestrian image;
and step 3: acquiring a pedestrian image, and storing the pedestrian image into a comparison database in real time;
and 4, step 4: acquiring videos of all ticket checking points, detecting in real time to obtain pedestrian data, and comparing the pedestrian data with data in a comparison database;
and 5: and sending the comparison result to a channel gate, and allowing the channel gate to perform a release or warning action according to the comparison result.
As an implementation manner, in step 2, reading a frame format of collected video data of the ticket-purchased tourist passing through a preset position, determining a frame number, extracting features of continuous frames of images, analyzing, and judging whether the images contain pedestrian features; the specific position of the pedestrian in the image is calibrated for the image judged to contain the pedestrian, the pedestrian photo is stored in a comparison database, and the image judged not to contain the pedestrian is not processed; the continuous processing of the frame images of the preset position video marks the detected pedestrian data and compares the detected pedestrian data with the number of purchased tickets, thereby avoiding the situation that the number of sold tickets is not matched with the tourists.
In the present embodiment, the computer reads the frame format of the video data, determines the number of frames, and determines the size of the frame image to be 418 × 418 × 3, where 418 denotes the number of lines and columns of the frame image and 3 denotes the number of layers of the image.
In fig. 2, the image of each frame in the video is feature extracted by a convolutional neural network.
In step 2, in one embodiment, the Darknet-19 is used in combination with the residual jump layer link to perform feature extraction on the image of each frame, and in order to retain more image information, the step size of the volume local layer used by the pooling layer for downsampling is abandoned. Meanwhile, in order to fuse more detailed information, YOLOv3 performs upsampling on the finally obtained features and fuses with the previous feature layer, and three feature layers with different sizes are used as final feature output in common, so that the method makes full use of the detailed information of the features to enable the extracted features to more accurately represent the features and positions of the detected object, wherein the YOLOv3 adopts the following formula to calculate errors:
wherein the parametersWhether the jth prior frame of the ith grid is responsible for the target object or not is represented, if so, the jth prior frame is 1, and if not, the jth prior frame is 0; lambda [ alpha ]coordIs a coordination coefficient set for coordinating the inconsistency of the contribution of the rectangular frames with different sizes to the error function,is the coordinates of the center of the rectangular box of the network prediction,is the center coordinate of the marked rectangular frame,the size of the width and height of the rectangle predicted for the network,is marked with a rectangular frameSize of width and height, CiTo predict the probability score of the target object contained within the frame,representing the true value;indicating that the size of the rectangular box is 1 if it is not responsible for predicting an object, otherwise it is equal to 0; lambda [ alpha ]noobjThe weight value represents the weight of the confidence error in the loss function when the target is not predicted by the prediction box;indicates the probability that the prediction box belongs to the category c,and if the true value of the category to which the mark box belongs to the class c, the size of the true value is equal to 1, and otherwise, the true value is 0. When calculating the error, the error loss function of converting the real object into a value similar to the predicted value includes five parts, as shown in formula (1), the first row represents the center position loss, the calculation is performed only on the frame where the target object exists and the IOU is the largest, and the second row represents the condition that the width and height errors are the same as the condition of the center position calculation. The third and fourth rows represent confidence errors for the presence and absence of the target, respectively, with the absence of a calculation condition being that the maximum IOU is not present and less than a set threshold. The last row represents the category loss.
In the embodiment, in consideration of detection of pedestrians, the pedestrian data is combined with the clustering algorithm to obtain the proper size and shape of the anchor frame aiming at the characteristics of the pedestrians, and personnel data is adopted to carry out fine adjustment under the existing network model, so that the anchor frame can be more accurately returned to the pedestrian to be detected.
As an implementation mode, in step 4, a clustering algorithm is adopted for pedestrian data in each ticket checking point video to obtain the shape and size of an anchor frame according with the human body shape; extracting global and local features of the detected pedestrian; and calculating the Euclidean distance between the pedestrian characteristics of the ticket checking point and the pedestrian characteristics of the comparison database, considering that the pedestrian is not matched with the current pedestrian if the calculation result is greater than a set threshold value, and considering that the pedestrian of the current ticket checking point and the pedestrian in the comparison database are the same pedestrian if the calculation result is less than the set threshold value.
In this example, referring to step 2, the method for acquiring video and detecting pedestrians is shown in fig. 3, and a flow of re-identifying pedestrians with different postures and angles is shown.
In fig. 3, the collected images of the pedestrians are input to a convolutional neural network, the euclidean distance is calculated between the finally extracted features and the pedestrian features in the comparison database, if the distance value is smaller than a set threshold value, the same pedestrian is determined, otherwise, the different pedestrians are determined, and the pedestrians at the ticket checking point and the pedestrians in the comparison database are subjected to traversal comparison.
In the present embodiment, it is considered that human identification in life is often performed from the whole, i.e., the whole, but when the whole form is similar, it is necessary to perform comparison on local features. Therefore, the image features of the pedestrian are fully utilized by combining the AlignedReID and the PCB, the alignment of the posture of the pedestrian is guaranteed, the global features of the pedestrian are effectively combined, and the stable and reliable convolutional neural network is obtained by training through the CUHK03, the Market501, the DukeMTMC-reiD and other public data sets.
As an embodiment, the loss function in the clustering algorithm uses a hard sample sampling triplet loss, for each training set, P persons with identities are randomly selected, and each person randomly selects K different pictures, so that each training set includes P × K pictures, and then a hard positive sample and a hard negative sample are selected for each picture in the training set to form a triplet, where the loss function formula is as follows:
wherein d isa,pRepresenting the Euclidean distance, d, between the sample a and the positive sample pa,nRepresents between the sample a and the negative sample nA is the set threshold parameter.
The local characteristics can adopt characteristics such as head, shoulder, waist, leg, decoration and the like, and experimental comparison is carried out by taking the length of hair and the color of clothes as auxiliary characteristics for personnel heavy identification. In most cases the local features are more detailed and the comparison results are more accurate. The invention also realizes the utilization of the local characteristics of the personnel by methods such as segmentation, posture division and the like of the characteristics of the personnel. Meanwhile, a feature is extracted from each horizontal cutting block through horizontal pooling, but the method for directly performing cutting can cause different positions of personnel to be compared due to the fact that postures are not aligned, and therefore a certain error can be generated in a re-recognition result. For the problem of people misalignment, the embodiment firstly cuts the features extracted from the image, calculates the distance between the feature blocks of different people, and obtains the feature block with the closest feature distance to reorder, so that the people align the features, and further better complete the task of people re-identification. The image is cut into a grid shape, which is a physical area feature with finer granularity and can be used as an auxiliary feature of the global feature to optimize the network structure. The method of combining global and local features optimizes the network structure, uses local features as a detailed supplement to the human global features, and tries a multi-branch network structure.
Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that changes, modifications, substitutions and alterations can be made in these embodiments without departing from the principles and spirit of the invention, the scope of which is defined in the appended claims and their equivalents.
Claims (10)
1. A ticket checking system based on pedestrian re-identification is characterized by comprising a tourist video acquisition device, a computer, a comparison database, a ticket checking video acquisition device and a channel gate, wherein the computer is connected with the tourist video acquisition device, the comparison database, the ticket checking video acquisition device and the channel gate,
the tourist video acquisition device acquires video data of the tourist who has purchased the ticket;
the ticket checking video acquisition device acquires videos of pedestrians to be detected at all ticket checking points;
the computer reads the collected video data of the ticket-purchased tourists, frames the video data to extract a pedestrian image, and stores the obtained pedestrian image into a comparison database in real time; acquiring videos of all ticket checking points, detecting in real time to obtain pedestrian data, comparing the pedestrian data with data in a comparison database, and transmitting a comparison result to a gateway gate;
and the channel gate makes a release or warning action according to the comparison result.
2. The ticket checking system based on pedestrian re-identification as claimed in claim 1, wherein the computer reads the frame format of the collected video data of the purchased tourists passing through the preset position and determines the number of frames, and analyzes and judges whether the images contain the pedestrian features by extracting the features of the images of the continuous frames; the specific position of the pedestrian in the image is calibrated for the image judged to contain the pedestrian, the pedestrian photo is stored in a comparison database, and the image judged not to contain the pedestrian is not processed; the continuous processing of the frame images of the preset position video marks the detected pedestrian data and compares the detected pedestrian data with the number of purchased tickets, thereby avoiding the situation that the number of sold tickets is not matched with the tourists.
3. The system for checking tickets based on pedestrian re-identification as claimed in claim 2, wherein the Darknet-19 is used to combine the residual skip layer link to extract the features of the image of each frame, the step size of the volume local layer is used to perform down-sampling, and the Yolov3 performs up-sampling on the resulting features and blending with the previous feature layer, and three different size feature layers are used together as the final feature output, wherein the error is calculated by the following formula in Yolov 3:
wherein the parametersWhether the jth prior frame of the ith grid is responsible for the target object or not is represented, if so, the jth prior frame is 1, and if not, the jth prior frame is 0; lambda [ alpha ]coordIs a coordination coefficient set for coordinating the inconsistency of the contribution of the rectangular frames with different sizes to the error function,is the coordinates of the center of the rectangular box of the network prediction,is the center coordinate of the marked rectangular frame,the size of the width and height of the rectangle predicted for the network,is to mark the width and height of the rectangular frame, CiTo predict the probability score of the target object contained within the frame,representing the true value;indicating that the size of the rectangular box is 1 if it is not responsible for predicting an object, otherwise it is equal to 0; lambda [ alpha ]noobjThe weight value represents the weight of the confidence error in the loss function when the target is not predicted by the prediction box; pi (c)Indicates the probability that the prediction box belongs to the category c,and if the true value of the category to which the mark box belongs to the class c, the size of the true value is equal to 1, and otherwise, the true value is 0.
4. The system for checking tickets based on pedestrian re-identification as claimed in claim 1, wherein the computer adopts a clustering algorithm to the pedestrian data in each ticket checking point video to obtain the shape and size of the anchor frame according with the human body shape; extracting global and local features of the detected pedestrian; and calculating the Euclidean distance between the pedestrian characteristics of the ticket checking point and the pedestrian characteristics of the comparison database, considering that the pedestrian is not matched with the current pedestrian if the calculation result is greater than a set threshold value, and considering that the pedestrian of the current ticket checking point and the pedestrian in the comparison database are the same pedestrian if the calculation result is less than the set threshold value.
5. The system according to claim 4, wherein the loss function in the clustering algorithm employs a hard sample sampling triple loss, for each training set, a person with P identities is randomly selected, and each person randomly selects K different pictures, so that each training set contains P x K pictures, and then a hard positive sample and a hard negative sample are selected for each picture in the training set to form a triple, and the loss function formula is as follows:
wherein d isa,pRepresenting the Euclidean distance, d, between the sample a and the positive sample pa,nRepresenting the euclidean distance between the sample a and the negative sample n, alpha being the set threshold parameter.
6. A ticket checking method based on pedestrian re-identification is characterized by comprising the following steps:
step 1: collecting video data of the ticket-purchased tourist;
step 2: reading the collected video data of the ticket-purchased tourists, and framing the video data to extract a pedestrian image;
and step 3: acquiring a pedestrian image, and storing the pedestrian image into a comparison database in real time;
and 4, step 4: acquiring videos of all ticket checking points, detecting in real time to obtain pedestrian data, and comparing the pedestrian data with data in a comparison database;
and 5: and sending the comparison result to a channel gate, and allowing the channel gate to perform a release or warning action according to the comparison result.
7. The ticket checking method based on pedestrian re-identification as claimed in claim 6, wherein in step 2, the frame format of the collected video data of the purchased tourists passing through the preset position is read and the number of frames is determined, and the characteristics of the images of the continuous frames are extracted for analysis and whether the images contain the pedestrian characteristics is judged; the specific position of the pedestrian in the image is calibrated for the image judged to contain the pedestrian, the pedestrian photo is stored in a comparison database, and the image judged not to contain the pedestrian is not processed; the continuous processing of the frame images of the preset position video marks the detected pedestrian data and compares the detected pedestrian data with the number of purchased tickets, thereby avoiding the situation that the number of sold tickets is not matched with the tourists.
8. The method for checking tickets based on pedestrian re-identification as claimed in claim 7, wherein in step 2, the Darknet-19 is used to combine with the residual skip-layer link to extract the features of the image of each frame, the step size of the volume local layer is used to perform down-sampling, YOLOv3 performs up-sampling on the last obtained features and fuses with the previous feature layers, and three different size feature layers are used together as the final feature output, wherein the error is calculated by using the following formula in YOLOv 3:
wherein the parametersWhether the jth prior frame of the ith grid is responsible for the target object or not is represented, if so, the jth prior frame is 1, and if not, the jth prior frame is 0; lambda [ alpha ]coordIs a coordination coefficient set for coordinating the inconsistency of the contribution of the rectangular frames with different sizes to the error function,is the coordinates of the center of the rectangular box of the network prediction,is the center coordinate of the marked rectangular frame,the size of the width and height of the rectangle predicted for the network,is to mark the width and height of the rectangular frame, CiTo predict the probability score of the target object contained within the frame,representing the true value;indicating that the size of the rectangular box is 1 if it is not responsible for predicting an object, otherwise it is equal to 0; lambda [ alpha ]noobjThe weight value represents the weight of the confidence error in the loss function when the target is not predicted by the prediction box; pi (c)Indicates the probability that the prediction box belongs to the category c,and if the true value of the category to which the mark box belongs to the class c, the size of the true value is equal to 1, and otherwise, the true value is 0.
9. The ticket checking method based on pedestrian re-identification as claimed in claim 6, wherein in step 4, a clustering algorithm is applied to pedestrian data in each ticket checking point video to obtain the shape and size of an anchor frame according with human body shape; extracting global and local features of the detected pedestrian; and calculating the Euclidean distance between the pedestrian characteristics of the ticket checking point and the pedestrian characteristics of the comparison database, considering that the pedestrian is not matched with the current pedestrian if the calculation result is greater than a set threshold value, and considering that the pedestrian of the current ticket checking point and the pedestrian in the comparison database are the same pedestrian if the calculation result is less than the set threshold value.
10. The ticket checking method based on pedestrian re-identification as claimed in claim 9, wherein the loss function in the clustering algorithm adopts a difficult sample sampling triple loss, for each training set, people with P identities are randomly selected, and each person randomly selects K different pictures, so that each training set contains P × K pictures, and then a most difficult positive sample and a most difficult negative sample are selected for each picture in the training set to form a triple, and the loss function formula is as follows:
wherein d isa,pRepresenting the Euclidean distance, d, between the sample a and the positive sample pa,nRepresenting the euclidean distance between the sample a and the negative sample n, alpha being the set threshold parameter.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202110334262.1A CN113077556A (en) | 2021-03-29 | 2021-03-29 | Ticket checking system and method based on pedestrian re-identification |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202110334262.1A CN113077556A (en) | 2021-03-29 | 2021-03-29 | Ticket checking system and method based on pedestrian re-identification |
Publications (1)
Publication Number | Publication Date |
---|---|
CN113077556A true CN113077556A (en) | 2021-07-06 |
Family
ID=76611106
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN202110334262.1A Pending CN113077556A (en) | 2021-03-29 | 2021-03-29 | Ticket checking system and method based on pedestrian re-identification |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN113077556A (en) |
Citations (17)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN106485217A (en) * | 2016-09-30 | 2017-03-08 | 四川世纪云道科技有限公司 | A kind of method and system of identification visit scenic spot stream of people's saturation degree |
CN108921152A (en) * | 2018-06-29 | 2018-11-30 | 清华大学 | English character cutting method and device based on object detection network |
CN110246244A (en) * | 2019-05-16 | 2019-09-17 | 珠海华园信息技术有限公司 | Intelligent foreground management system based on recognition of face |
CN110781350A (en) * | 2019-09-26 | 2020-02-11 | 武汉大学 | Pedestrian retrieval method and system oriented to full-picture monitoring scene |
CN111105412A (en) * | 2019-12-30 | 2020-05-05 | 郑州大学 | Intelligent auxiliary system for intestinal polyp detection and identification |
CN111192391A (en) * | 2018-10-25 | 2020-05-22 | 杭州海康威视数字技术股份有限公司 | Pedestrian passageway gate control method and device based on images and/or videos |
CN111292305A (en) * | 2020-01-22 | 2020-06-16 | 重庆大学 | Improved YOLO-V3 metal processing surface defect detection method |
CN111476168A (en) * | 2020-04-08 | 2020-07-31 | 山东师范大学 | Cross-domain pedestrian re-identification method and system based on three stages |
CN111507134A (en) * | 2019-01-31 | 2020-08-07 | 北京奇虎科技有限公司 | Human-shaped posture detection method and device, computer equipment and storage medium |
CN111753583A (en) * | 2019-03-28 | 2020-10-09 | 阿里巴巴集团控股有限公司 | Identification method and device |
CN111783753A (en) * | 2020-09-04 | 2020-10-16 | 中国科学院自动化研究所 | Pedestrian re-identification method based on semantic consistency horizontal bar and foreground correction |
CN111862408A (en) * | 2020-06-16 | 2020-10-30 | 北京华电天仁电力控制技术有限公司 | Intelligent access control method |
CN112070135A (en) * | 2020-08-28 | 2020-12-11 | 广东电网有限责任公司 | Power equipment image detection method and device, power equipment and storage medium |
CN112183667A (en) * | 2020-10-31 | 2021-01-05 | 哈尔滨理工大学 | Insulator fault detection method in cooperation with deep learning |
CN112215203A (en) * | 2020-11-02 | 2021-01-12 | 坝道工程医院(平舆) | Pavement disease detection method and device based on deep learning |
CN112307984A (en) * | 2020-11-02 | 2021-02-02 | 安徽工业大学 | Safety helmet detection method and device based on neural network |
CN212624158U (en) * | 2020-06-28 | 2021-02-26 | 中科华豫科技有限公司 | Multifunctional intelligent ticketing system applied through big data technology |
-
2021
- 2021-03-29 CN CN202110334262.1A patent/CN113077556A/en active Pending
Patent Citations (17)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN106485217A (en) * | 2016-09-30 | 2017-03-08 | 四川世纪云道科技有限公司 | A kind of method and system of identification visit scenic spot stream of people's saturation degree |
CN108921152A (en) * | 2018-06-29 | 2018-11-30 | 清华大学 | English character cutting method and device based on object detection network |
CN111192391A (en) * | 2018-10-25 | 2020-05-22 | 杭州海康威视数字技术股份有限公司 | Pedestrian passageway gate control method and device based on images and/or videos |
CN111507134A (en) * | 2019-01-31 | 2020-08-07 | 北京奇虎科技有限公司 | Human-shaped posture detection method and device, computer equipment and storage medium |
CN111753583A (en) * | 2019-03-28 | 2020-10-09 | 阿里巴巴集团控股有限公司 | Identification method and device |
CN110246244A (en) * | 2019-05-16 | 2019-09-17 | 珠海华园信息技术有限公司 | Intelligent foreground management system based on recognition of face |
CN110781350A (en) * | 2019-09-26 | 2020-02-11 | 武汉大学 | Pedestrian retrieval method and system oriented to full-picture monitoring scene |
CN111105412A (en) * | 2019-12-30 | 2020-05-05 | 郑州大学 | Intelligent auxiliary system for intestinal polyp detection and identification |
CN111292305A (en) * | 2020-01-22 | 2020-06-16 | 重庆大学 | Improved YOLO-V3 metal processing surface defect detection method |
CN111476168A (en) * | 2020-04-08 | 2020-07-31 | 山东师范大学 | Cross-domain pedestrian re-identification method and system based on three stages |
CN111862408A (en) * | 2020-06-16 | 2020-10-30 | 北京华电天仁电力控制技术有限公司 | Intelligent access control method |
CN212624158U (en) * | 2020-06-28 | 2021-02-26 | 中科华豫科技有限公司 | Multifunctional intelligent ticketing system applied through big data technology |
CN112070135A (en) * | 2020-08-28 | 2020-12-11 | 广东电网有限责任公司 | Power equipment image detection method and device, power equipment and storage medium |
CN111783753A (en) * | 2020-09-04 | 2020-10-16 | 中国科学院自动化研究所 | Pedestrian re-identification method based on semantic consistency horizontal bar and foreground correction |
CN112183667A (en) * | 2020-10-31 | 2021-01-05 | 哈尔滨理工大学 | Insulator fault detection method in cooperation with deep learning |
CN112215203A (en) * | 2020-11-02 | 2021-01-12 | 坝道工程医院(平舆) | Pavement disease detection method and device based on deep learning |
CN112307984A (en) * | 2020-11-02 | 2021-02-02 | 安徽工业大学 | Safety helmet detection method and device based on neural network |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN110781838B (en) | Multi-mode track prediction method for pedestrians in complex scene | |
CN109447169B (en) | Image processing method, training method and device of model thereof and electronic system | |
CN108334847B (en) | A kind of face identification method based on deep learning under real scene | |
CN108388888B (en) | Vehicle identification method and device and storage medium | |
CN109934176B (en) | Pedestrian recognition system, recognition method, and computer-readable storage medium | |
WO2020181685A1 (en) | Vehicle-mounted video target detection method based on deep learning | |
CN108229397A (en) | Method for text detection in image based on Faster R-CNN | |
CN110287960A (en) | The detection recognition method of curve text in natural scene image | |
CN112926506B (en) | Non-controlled face detection method and system based on convolutional neural network | |
CN108564049A (en) | A kind of fast face detection recognition method based on deep learning | |
CN109214280A (en) | Shop recognition methods, device, electronic equipment and storage medium based on streetscape | |
TW201201107A (en) | Barcode image recognition system and associated method for hand-held device | |
CN104794171B (en) | Mark the method and device of picture geographical location information | |
CN109559362B (en) | Image subject face replacing method and device | |
CN110263768A (en) | A kind of face identification method based on depth residual error network | |
CN112464843A (en) | Accurate passenger flow statistical system, method and device based on human face human shape | |
CN106980855A (en) | Traffic sign quickly recognizes alignment system and method | |
CN110148223A (en) | Monitor video target concentration expression and system in three-dimensional geography model of place | |
CN110533026A (en) | The competing image digitization of electricity based on computer vision and icon information acquisition methods | |
CN107689157A (en) | Traffic intersection based on deep learning can passing road planing method | |
CN111126275A (en) | Pedestrian re-identification method and device based on multi-granularity feature fusion | |
CN110766645B (en) | Target person recurrence map generation method based on person identification and segmentation | |
CN110517285A (en) | The minimum target following of large scene based on estimation ME-CNN network | |
CN113077556A (en) | Ticket checking system and method based on pedestrian re-identification | |
CN111582278B (en) | Portrait segmentation method and device and electronic equipment |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
RJ01 | Rejection of invention patent application after publication | ||
RJ01 | Rejection of invention patent application after publication |
Application publication date: 20210706 |