CN113077556A - Ticket checking system and method based on pedestrian re-identification - Google Patents

Ticket checking system and method based on pedestrian re-identification Download PDF

Info

Publication number
CN113077556A
CN113077556A CN202110334262.1A CN202110334262A CN113077556A CN 113077556 A CN113077556 A CN 113077556A CN 202110334262 A CN202110334262 A CN 202110334262A CN 113077556 A CN113077556 A CN 113077556A
Authority
CN
China
Prior art keywords
pedestrian
ticket checking
data
frame
ticket
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202110334262.1A
Other languages
Chinese (zh)
Inventor
张勇
宗拓
赵东宁
廉德亮
梁长垠
曾庆好
何钦煜
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shenzhen University
Original Assignee
Shenzhen University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shenzhen University filed Critical Shenzhen University
Priority to CN202110334262.1A priority Critical patent/CN113077556A/en
Publication of CN113077556A publication Critical patent/CN113077556A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G07—CHECKING-DEVICES
    • G07B—TICKET-ISSUING APPARATUS; FARE-REGISTERING APPARATUS; FRANKING APPARATUS
    • G07B11/00—Apparatus for validating or cancelling issued tickets
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/23—Clustering techniques
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/40—Scenes; Scene-specific elements in video content
    • G06V20/46—Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects
    • G06V20/53—Recognition of crowd images, e.g. recognition of crowd congestion
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10016—Video; Image sequence
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30196—Human being; Person

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • General Engineering & Computer Science (AREA)
  • Evolutionary Computation (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Computational Linguistics (AREA)
  • Multimedia (AREA)
  • Software Systems (AREA)
  • Health & Medical Sciences (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Mathematical Physics (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • Evolutionary Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Image Analysis (AREA)

Abstract

The embodiment of the invention discloses a ticket checking system and a method based on pedestrian re-identification, wherein the system comprises a tourist video acquisition device, a computer, a comparison database, a ticket checking video acquisition device and a gateway, wherein the tourist video acquisition device acquires video data of a tourist who has purchased a ticket; the ticket checking video acquisition device acquires videos of pedestrians to be detected at all ticket checking points; reading the video data by the computer, framing the video data to extract a pedestrian image, and storing the obtained pedestrian image into a comparison database in real time; acquiring videos of all ticket checking points, detecting in real time to obtain pedestrian data, comparing the pedestrian data with data in a comparison database, and transmitting a comparison result to a gateway gate; and the channel gate makes a release or warning action according to the comparison result. The ticket checking system hides the traditional ticket checking process and replaces a manual process with computer equipment, so that the ticket buying and checking efficiency can be improved, and the playing experience of tourists can be ensured.

Description

Ticket checking system and method based on pedestrian re-identification
Technical Field
The invention relates to the technical field of scenic spot services, in particular to a ticket checking system and method based on pedestrian re-identification.
Background
With the improvement of living standard, the requirement of going out for playing is increasing day by day, and especially in the legal holidays such as Wuyi and national celebration, the phenomena that the scenic spots in each region are congested and the working personnel are not enough due to the arrival of a large number of tourists occur. Due to lack of hands and low working efficiency, long teams can be arranged at ticket selling points and ticket checking points of various scenic spots, and therefore most tourists can be irritated and lose pleasure. Not only the waiting time for buying and checking tickets can affect the playing mood, but also the playing interruption caused by the loss of paper tickets can be caused frequently. Although tourists can pay and purchase tickets on the internet through mobile phones, the paper bills need to be replaced by payment codes, the difficulty of buying and using the tickets is not fundamentally solved, and the fundamental reason is that the paper bills are always used as the unique identification of each tourist.
At present, paper tickets are used as unique identification of tourists in all playing fields needing ticket purchasing. Besides paper bills, fingerprints and human faces are used as unique identifications of people and are also applied to various scenes, but privacy problems are frequently discussed. It is impractical to extract fingerprints and faces of tourists in a playing field.
Disclosure of Invention
The technical problem to be solved by the embodiments of the present invention is to provide a ticket checking system and method based on pedestrian re-identification, so as to improve the work efficiency of the tourist attraction related service field and to replace the use of paper tickets to bring good play experience to tourists.
In order to solve the above technical problems, an embodiment of the present invention provides a ticket checking system based on pedestrian re-identification, which includes a visitor video collecting device, a computer, a comparison database, a ticket checking video collecting device, and a gateway, wherein the computer is connected to the visitor video collecting device, the comparison database, the ticket checking video collecting device, and the gateway,
the tourist video acquisition device acquires video data of the tourist who has purchased the ticket;
the ticket checking video acquisition device acquires videos of pedestrians to be detected at all ticket checking points;
the computer reads the collected video data of the ticket-purchased tourists, frames the video data to extract a pedestrian image, and stores the obtained pedestrian image into a comparison database in real time; acquiring videos of all ticket checking points, detecting in real time to obtain pedestrian data, comparing the pedestrian data with data in a comparison database, and transmitting a comparison result to a gateway gate;
and the channel gate makes a release or warning action according to the comparison result.
Further, the computer reads the frame format of the collected video data of the ticket-purchased tourists passing through the preset position, determines the frame number, extracts the features of the continuous frames of images, analyzes and judges whether the images contain the pedestrian features; the specific position of the pedestrian in the image is calibrated for the image judged to contain the pedestrian, the pedestrian photo is stored in a comparison database, and the image judged not to contain the pedestrian is not processed; the continuous processing of the frame images of the preset position video marks the detected pedestrian data and compares the detected pedestrian data with the number of purchased tickets, thereby avoiding the situation that the number of sold tickets is not matched with the tourists.
Further, using Darknet-19 to combine with residual skip layer link to perform feature extraction on the image of each frame, using the step length of the volume local layer to perform down-sampling, using Yolov3 to perform up-sampling on the finally obtained features and fusing the features with the previous feature layers, and using three different feature layers as the final feature output, wherein the error is calculated by using the following formula in Yolov 3:
Figure BDA0002996697730000021
wherein the parameters
Figure BDA0002996697730000022
Whether the jth prior frame of the ith grid is responsible for the target object or not is represented, if so, the jth prior frame is 1, and if not, the jth prior frame is 0; lambda [ alpha ]coordIs a coordination coefficient set for coordinating the inconsistency of the contribution of the rectangular frames with different sizes to the error function,
Figure BDA0002996697730000023
is the coordinates of the center of the rectangular box of the network prediction,
Figure BDA0002996697730000024
is the center coordinate of the marked rectangular frame,
Figure BDA0002996697730000025
the size of the width and height of the rectangle predicted for the network,
Figure BDA0002996697730000026
is to mark the width and height of the rectangular frame, CiTo predict the probability score of the target object contained within the frame,
Figure BDA0002996697730000027
representing the true value;
Figure BDA0002996697730000028
indicating that the size of the rectangular box is 1 if it is not responsible for predicting an object, otherwise it is equal to 0; lambda [ alpha ]noobjIs a weight value representing that the confidence error is in the loss when the target is not predicted by the prediction boxThe weight occupied by the lost function;
Figure BDA0002996697730000029
indicates the probability that the prediction box belongs to the category c,
Figure BDA00029966977300000210
and if the true value of the category to which the mark box belongs to the class c, the size of the true value is equal to 1, and otherwise, the true value is 0.
Further, the computer obtains the shape and the size of an anchor frame according with the human body shape by adopting a clustering algorithm to the pedestrian data in each ticket checking point video; extracting global and local features of the detected pedestrian; and calculating the Euclidean distance between the pedestrian characteristics of the ticket checking point and the pedestrian characteristics of the comparison database, considering that the pedestrian is not matched with the current pedestrian if the calculation result is greater than a set threshold value, and considering that the pedestrian of the current ticket checking point and the pedestrian in the comparison database are the same pedestrian if the calculation result is less than the set threshold value.
Further, the loss function in the clustering algorithm adopts a loss of a triplet of sampling difficult samples, for each training set, people with P identities are randomly selected, and each person randomly selects K different pictures, so that each training set contains P × K pictures, and then a triplet is formed by selecting a most difficult positive sample and a most difficult negative sample for each picture in the training set, and the loss function formula is as follows:
Figure BDA0002996697730000031
wherein d isa,pRepresenting the Euclidean distance, d, between the sample a and the positive sample pa,nRepresenting the euclidean distance between the sample a and the negative sample n, alpha being the set threshold parameter.
Correspondingly, the embodiment of the invention also provides a ticket checking method based on pedestrian re-identification, which comprises the following steps:
step 1: collecting video data of the ticket-purchased tourist;
step 2: reading the collected video data of the ticket-purchased tourists, and framing the video data to extract a pedestrian image;
and step 3: acquiring a pedestrian image, and storing the pedestrian image into a comparison database in real time;
and 4, step 4: acquiring videos of all ticket checking points, detecting in real time to obtain pedestrian data, and comparing the pedestrian data with data in a comparison database;
and 5: and sending the comparison result to a channel gate, and allowing the channel gate to perform a release or warning action according to the comparison result.
Further, in step 2, reading a frame format of collected video data of the ticket-purchased tourist passing through a preset position, determining the number of frames, extracting the characteristics of continuous frames of images, analyzing and judging whether the images contain pedestrian characteristics; the specific position of the pedestrian in the image is calibrated for the image judged to contain the pedestrian, the pedestrian photo is stored in a comparison database, and the image judged not to contain the pedestrian is not processed; the continuous processing of the frame images of the preset position video marks the detected pedestrian data and compares the detected pedestrian data with the number of purchased tickets, thereby avoiding the situation that the number of sold tickets is not matched with the tourists.
Further, in step 2, using the Darknet-19 to combine with the residual skip layer link to perform feature extraction on the image of each frame, using the step size of the volume local layer to perform down-sampling, using Yolov3 to perform up-sampling on the finally obtained features and fusing the features with the previous feature layers, and using three different sized feature layers as the final feature output, wherein the error is calculated by using the following formula in Yolov 3:
Figure BDA0002996697730000041
wherein the parameters
Figure BDA0002996697730000042
Whether the jth prior frame of the ith grid is responsible for the target object or not is represented, if so, the jth prior frame is 1, and if not, the jth prior frame is 0; lambda [ alpha ]coordIs a coordination set for coordinating the inconsistency of the contribution of different size rectangular frames to the error functionThe coefficient of modulation is adjusted,
Figure BDA0002996697730000043
is the coordinates of the center of the rectangular box of the network prediction,
Figure BDA0002996697730000044
is the center coordinate of the marked rectangular frame,
Figure BDA0002996697730000045
the size of the width and height of the rectangle predicted for the network,
Figure BDA0002996697730000046
is to mark the width and height of the rectangular frame, CiTo predict the probability score of the target object contained within the frame,
Figure BDA0002996697730000047
representing the true value;
Figure BDA0002996697730000048
indicating that the size of the rectangular box is 1 if it is not responsible for predicting an object, otherwise it is equal to 0; lambda [ alpha ]noobjThe weight value represents the weight of the confidence error in the loss function when the target is not predicted by the prediction box;
Figure BDA0002996697730000049
indicates the probability that the prediction box belongs to the category c,
Figure BDA00029966977300000410
and if the true value of the category to which the mark box belongs to the class c, the size of the true value is equal to 1, and otherwise, the true value is 0.
Further, in the step 4, the pedestrian data in the ticket checking point videos are subjected to a clustering algorithm to obtain the shape and the size of an anchor frame according with the human body shape; extracting global and local features of the detected pedestrian; and calculating the Euclidean distance between the pedestrian characteristics of the ticket checking point and the pedestrian characteristics of the comparison database, considering that the pedestrian is not matched with the current pedestrian if the calculation result is greater than a set threshold value, and considering that the pedestrian of the current ticket checking point and the pedestrian in the comparison database are the same pedestrian if the calculation result is less than the set threshold value.
Further, the loss function in the clustering algorithm adopts a loss of a triplet of sampling difficult samples, for each training set, people with P identities are randomly selected, and each person randomly selects K different pictures, so that each training set contains P × K pictures, and then a triplet is formed by selecting a most difficult positive sample and a most difficult negative sample for each picture in the training set, and the loss function formula is as follows:
Figure BDA00029966977300000411
wherein d isa,pRepresenting the Euclidean distance, d, between the sample a and the positive sample pa,nRepresenting the euclidean distance between the sample a and the negative sample n, alpha being the set threshold parameter.
The invention has the beneficial effects that: the ticket checking system hides the traditional ticket checking process and replaces a manual process with computer equipment, so that the ticket buying and checking efficiency can be improved, and the playing experience of tourists can be ensured. Therefore, the ticket checking method and system based on pedestrian re-identification provided by the invention have high popularization and use values.
Drawings
Fig. 1 is a flowchart of a ticket checking method based on pedestrian re-identification according to an embodiment of the present invention.
Fig. 2 is a detailed structural diagram of an algorithm for extracting a pedestrian image according to an embodiment of the present invention.
Fig. 3 is a schematic diagram of a ticket checking method based on pedestrian re-identification according to an embodiment of the invention.
Detailed Description
It should be noted that the embodiments and features of the embodiments in the present application can be combined with each other without conflict, and the present invention is further described in detail with reference to the drawings and specific embodiments.
The ticket checking system based on pedestrian re-identification comprises a tourist video acquisition device, a computer, a comparison database, a ticket checking video acquisition device and a gateway. The computer is connected with the tourist video acquisition device, the comparison database, the ticket checking video acquisition device and the channel gate.
Preferably, visitor's video acquisition device includes two cameras, during the concrete implementation, places two cameras respectively in fixed point scanning platform's the preceding top and the upper right side, and its shooting range covers: the front and the side of the pedestrian are the whole body. The ticket checking system based on pedestrian re-identification in the embodiment of the invention collects the video data of the ticket-purchased tourists by controlling the cameras arranged at the front and the right sides of the fixed-point scanning table, and stores the video data recorded by the two cameras into the video storage module for calling and video processing.
The tourist video acquisition device acquires video data of the tourist who has bought the ticket.
The ticket checking video acquisition device acquires videos of pedestrians to be checked at all ticket checking points.
The computer reads the collected video data of the ticket-purchased tourists, frames the video data to extract a pedestrian image, and stores the obtained pedestrian image into a comparison database in real time; and acquiring videos of all ticket checking points, detecting in real time to obtain pedestrian data, comparing the pedestrian data with data in a comparison database, and transmitting a comparison result to the gateway.
And the channel gate makes a release or warning action according to the comparison result. For example, if the same pedestrian exists in the comparison result, extracting the data in the comparison library and displaying the data in the detection instrument, and if the comparison result does not exist, prompting; the channel gate is used as a unique channel, if the matching is successful, the gate is opened, and if the matching is failed, the tourist is prompted to purchase the ticket to enter.
As an implementation mode, the computer reads the frame format of the collected video data of the ticket-purchased tourists passing through the preset position and determines the frame number, and extracts the characteristics of the continuous frames of images for analysis and judges whether the images contain the pedestrian characteristics; the specific position of the pedestrian in the image is calibrated for the image judged to contain the pedestrian, the pedestrian photo is stored in a comparison database, and the image judged not to contain the pedestrian is not processed; the continuous processing of the frame images of the preset position video marks the detected pedestrian data and compares the detected pedestrian data with the number of purchased tickets, thereby avoiding the situation that the number of sold tickets is not matched with the tourists.
As an embodiment, using Darknet-19 to combine with residual skip layer link to perform feature extraction on the image of each frame, using the step size of volume local layer to perform down-sampling, using YOLOv3 to perform up-sampling on the finally obtained feature and fusing the feature layers before, and using three different sized feature layers together as the final feature output, wherein the YOLOv3 adopts the following formula to calculate the error:
Figure BDA0002996697730000061
wherein the parameters
Figure BDA0002996697730000062
Whether the jth prior frame of the ith grid is responsible for the target object or not is represented, if so, the jth prior frame is 1, and if not, the jth prior frame is 0; lambda [ alpha ]coordIs a coordination coefficient set for coordinating the inconsistency of the contribution of the rectangular frames with different sizes to the error function,
Figure BDA0002996697730000063
is the coordinates of the center of the rectangular box of the network prediction,
Figure BDA0002996697730000064
is the center coordinate of the marked rectangular frame,
Figure BDA0002996697730000065
the size of the width and height of the rectangle predicted for the network,
Figure BDA0002996697730000066
is to mark the width and height of the rectangular frame, CiTo predict the probability score of the target object contained within the frame,
Figure BDA0002996697730000067
representing the true value;
Figure BDA0002996697730000068
indicating that the size of the rectangular box is 1 if it is not responsible for predicting an object, otherwise it is equal to 0; lambda [ alpha ]noobjThe weight value represents the weight of the confidence error in the loss function when the target is not predicted by the prediction box;
Figure BDA0002996697730000069
indicates the probability that the prediction box belongs to the category c,
Figure BDA00029966977300000610
and if the true value of the category to which the mark box belongs to the class c, the size of the true value is equal to 1, and otherwise, the true value is 0.
As an implementation mode, the computer adopts a clustering algorithm to the pedestrian data in each ticket checking point video to obtain the shape and the size of an anchor frame according with the human body shape; extracting global and local features of the detected pedestrian; and calculating the Euclidean distance between the pedestrian characteristics of the ticket checking point and the pedestrian characteristics of the comparison database, considering that the pedestrian is not matched with the current pedestrian if the calculation result is greater than a set threshold value, and considering that the pedestrian of the current ticket checking point and the pedestrian in the comparison database are the same pedestrian if the calculation result is less than the set threshold value. The embodiment of the invention adopts a large amount of pedestrian data, obtains the size and the data of the anchor frame more suitable for detecting the pedestrian by using the clustering algorithm, uses the size and the data of the anchor frame for more accurately detecting the specific position of the pedestrian in the pedestrian detection algorithm, and adopts a strategy of combining global and local characteristics for the detected pedestrian so as to effectively match the image of the same pedestrian under different cameras, thereby effectively reducing the false detection, the missed detection and the false detection of the ticket checking system.
As an embodiment, the loss function in the clustering algorithm uses a hard sample sampling triplet loss, for each training set, P persons with identities are randomly selected, and each person randomly selects K different pictures, so that each training set includes P × K pictures, and then a hard positive sample and a hard negative sample are selected for each picture in the training set to form a triplet, where the loss function formula is as follows:
Figure BDA0002996697730000071
wherein d isa,pRepresenting the Euclidean distance, d, between the sample a and the positive sample pa,nRepresenting the euclidean distance between the sample a and the negative sample n, alpha being the set threshold parameter.
Referring to fig. 1, a ticket checking method based on pedestrian re-identification according to an embodiment of the present invention includes:
step 1: collecting video data of the ticket-purchased tourist;
step 2: reading the collected video data of the ticket-purchased tourists, and framing the video data to extract a pedestrian image;
and step 3: acquiring a pedestrian image, and storing the pedestrian image into a comparison database in real time;
and 4, step 4: acquiring videos of all ticket checking points, detecting in real time to obtain pedestrian data, and comparing the pedestrian data with data in a comparison database;
and 5: and sending the comparison result to a channel gate, and allowing the channel gate to perform a release or warning action according to the comparison result.
As an implementation manner, in step 2, reading a frame format of collected video data of the ticket-purchased tourist passing through a preset position, determining a frame number, extracting features of continuous frames of images, analyzing, and judging whether the images contain pedestrian features; the specific position of the pedestrian in the image is calibrated for the image judged to contain the pedestrian, the pedestrian photo is stored in a comparison database, and the image judged not to contain the pedestrian is not processed; the continuous processing of the frame images of the preset position video marks the detected pedestrian data and compares the detected pedestrian data with the number of purchased tickets, thereby avoiding the situation that the number of sold tickets is not matched with the tourists.
In the present embodiment, the computer reads the frame format of the video data, determines the number of frames, and determines the size of the frame image to be 418 × 418 × 3, where 418 denotes the number of lines and columns of the frame image and 3 denotes the number of layers of the image.
In fig. 2, the image of each frame in the video is feature extracted by a convolutional neural network.
In step 2, in one embodiment, the Darknet-19 is used in combination with the residual jump layer link to perform feature extraction on the image of each frame, and in order to retain more image information, the step size of the volume local layer used by the pooling layer for downsampling is abandoned. Meanwhile, in order to fuse more detailed information, YOLOv3 performs upsampling on the finally obtained features and fuses with the previous feature layer, and three feature layers with different sizes are used as final feature output in common, so that the method makes full use of the detailed information of the features to enable the extracted features to more accurately represent the features and positions of the detected object, wherein the YOLOv3 adopts the following formula to calculate errors:
Figure BDA0002996697730000081
wherein the parameters
Figure BDA0002996697730000082
Whether the jth prior frame of the ith grid is responsible for the target object or not is represented, if so, the jth prior frame is 1, and if not, the jth prior frame is 0; lambda [ alpha ]coordIs a coordination coefficient set for coordinating the inconsistency of the contribution of the rectangular frames with different sizes to the error function,
Figure BDA0002996697730000083
is the coordinates of the center of the rectangular box of the network prediction,
Figure BDA0002996697730000084
is the center coordinate of the marked rectangular frame,
Figure BDA0002996697730000085
the size of the width and height of the rectangle predicted for the network,
Figure BDA0002996697730000086
is marked with a rectangular frameSize of width and height, CiTo predict the probability score of the target object contained within the frame,
Figure BDA0002996697730000087
representing the true value;
Figure BDA0002996697730000088
indicating that the size of the rectangular box is 1 if it is not responsible for predicting an object, otherwise it is equal to 0; lambda [ alpha ]noobjThe weight value represents the weight of the confidence error in the loss function when the target is not predicted by the prediction box;
Figure BDA0002996697730000089
indicates the probability that the prediction box belongs to the category c,
Figure BDA00029966977300000810
and if the true value of the category to which the mark box belongs to the class c, the size of the true value is equal to 1, and otherwise, the true value is 0. When calculating the error, the error loss function of converting the real object into a value similar to the predicted value includes five parts, as shown in formula (1), the first row represents the center position loss, the calculation is performed only on the frame where the target object exists and the IOU is the largest, and the second row represents the condition that the width and height errors are the same as the condition of the center position calculation. The third and fourth rows represent confidence errors for the presence and absence of the target, respectively, with the absence of a calculation condition being that the maximum IOU is not present and less than a set threshold. The last row represents the category loss.
In the embodiment, in consideration of detection of pedestrians, the pedestrian data is combined with the clustering algorithm to obtain the proper size and shape of the anchor frame aiming at the characteristics of the pedestrians, and personnel data is adopted to carry out fine adjustment under the existing network model, so that the anchor frame can be more accurately returned to the pedestrian to be detected.
As an implementation mode, in step 4, a clustering algorithm is adopted for pedestrian data in each ticket checking point video to obtain the shape and size of an anchor frame according with the human body shape; extracting global and local features of the detected pedestrian; and calculating the Euclidean distance between the pedestrian characteristics of the ticket checking point and the pedestrian characteristics of the comparison database, considering that the pedestrian is not matched with the current pedestrian if the calculation result is greater than a set threshold value, and considering that the pedestrian of the current ticket checking point and the pedestrian in the comparison database are the same pedestrian if the calculation result is less than the set threshold value.
In this example, referring to step 2, the method for acquiring video and detecting pedestrians is shown in fig. 3, and a flow of re-identifying pedestrians with different postures and angles is shown.
In fig. 3, the collected images of the pedestrians are input to a convolutional neural network, the euclidean distance is calculated between the finally extracted features and the pedestrian features in the comparison database, if the distance value is smaller than a set threshold value, the same pedestrian is determined, otherwise, the different pedestrians are determined, and the pedestrians at the ticket checking point and the pedestrians in the comparison database are subjected to traversal comparison.
In the present embodiment, it is considered that human identification in life is often performed from the whole, i.e., the whole, but when the whole form is similar, it is necessary to perform comparison on local features. Therefore, the image features of the pedestrian are fully utilized by combining the AlignedReID and the PCB, the alignment of the posture of the pedestrian is guaranteed, the global features of the pedestrian are effectively combined, and the stable and reliable convolutional neural network is obtained by training through the CUHK03, the Market501, the DukeMTMC-reiD and other public data sets.
As an embodiment, the loss function in the clustering algorithm uses a hard sample sampling triplet loss, for each training set, P persons with identities are randomly selected, and each person randomly selects K different pictures, so that each training set includes P × K pictures, and then a hard positive sample and a hard negative sample are selected for each picture in the training set to form a triplet, where the loss function formula is as follows:
Figure BDA0002996697730000091
wherein d isa,pRepresenting the Euclidean distance, d, between the sample a and the positive sample pa,nRepresents between the sample a and the negative sample nA is the set threshold parameter.
The local characteristics can adopt characteristics such as head, shoulder, waist, leg, decoration and the like, and experimental comparison is carried out by taking the length of hair and the color of clothes as auxiliary characteristics for personnel heavy identification. In most cases the local features are more detailed and the comparison results are more accurate. The invention also realizes the utilization of the local characteristics of the personnel by methods such as segmentation, posture division and the like of the characteristics of the personnel. Meanwhile, a feature is extracted from each horizontal cutting block through horizontal pooling, but the method for directly performing cutting can cause different positions of personnel to be compared due to the fact that postures are not aligned, and therefore a certain error can be generated in a re-recognition result. For the problem of people misalignment, the embodiment firstly cuts the features extracted from the image, calculates the distance between the feature blocks of different people, and obtains the feature block with the closest feature distance to reorder, so that the people align the features, and further better complete the task of people re-identification. The image is cut into a grid shape, which is a physical area feature with finer granularity and can be used as an auxiliary feature of the global feature to optimize the network structure. The method of combining global and local features optimizes the network structure, uses local features as a detailed supplement to the human global features, and tries a multi-branch network structure.
Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that changes, modifications, substitutions and alterations can be made in these embodiments without departing from the principles and spirit of the invention, the scope of which is defined in the appended claims and their equivalents.

Claims (10)

1. A ticket checking system based on pedestrian re-identification is characterized by comprising a tourist video acquisition device, a computer, a comparison database, a ticket checking video acquisition device and a channel gate, wherein the computer is connected with the tourist video acquisition device, the comparison database, the ticket checking video acquisition device and the channel gate,
the tourist video acquisition device acquires video data of the tourist who has purchased the ticket;
the ticket checking video acquisition device acquires videos of pedestrians to be detected at all ticket checking points;
the computer reads the collected video data of the ticket-purchased tourists, frames the video data to extract a pedestrian image, and stores the obtained pedestrian image into a comparison database in real time; acquiring videos of all ticket checking points, detecting in real time to obtain pedestrian data, comparing the pedestrian data with data in a comparison database, and transmitting a comparison result to a gateway gate;
and the channel gate makes a release or warning action according to the comparison result.
2. The ticket checking system based on pedestrian re-identification as claimed in claim 1, wherein the computer reads the frame format of the collected video data of the purchased tourists passing through the preset position and determines the number of frames, and analyzes and judges whether the images contain the pedestrian features by extracting the features of the images of the continuous frames; the specific position of the pedestrian in the image is calibrated for the image judged to contain the pedestrian, the pedestrian photo is stored in a comparison database, and the image judged not to contain the pedestrian is not processed; the continuous processing of the frame images of the preset position video marks the detected pedestrian data and compares the detected pedestrian data with the number of purchased tickets, thereby avoiding the situation that the number of sold tickets is not matched with the tourists.
3. The system for checking tickets based on pedestrian re-identification as claimed in claim 2, wherein the Darknet-19 is used to combine the residual skip layer link to extract the features of the image of each frame, the step size of the volume local layer is used to perform down-sampling, and the Yolov3 performs up-sampling on the resulting features and blending with the previous feature layer, and three different size feature layers are used together as the final feature output, wherein the error is calculated by the following formula in Yolov 3:
Figure FDA0002996697720000011
wherein the parameters
Figure FDA0002996697720000021
Whether the jth prior frame of the ith grid is responsible for the target object or not is represented, if so, the jth prior frame is 1, and if not, the jth prior frame is 0; lambda [ alpha ]coordIs a coordination coefficient set for coordinating the inconsistency of the contribution of the rectangular frames with different sizes to the error function,
Figure FDA0002996697720000022
is the coordinates of the center of the rectangular box of the network prediction,
Figure FDA0002996697720000023
is the center coordinate of the marked rectangular frame,
Figure FDA0002996697720000024
the size of the width and height of the rectangle predicted for the network,
Figure FDA0002996697720000025
is to mark the width and height of the rectangular frame, CiTo predict the probability score of the target object contained within the frame,
Figure FDA0002996697720000026
representing the true value;
Figure FDA0002996697720000027
indicating that the size of the rectangular box is 1 if it is not responsible for predicting an object, otherwise it is equal to 0; lambda [ alpha ]noobjThe weight value represents the weight of the confidence error in the loss function when the target is not predicted by the prediction box; pi (c)Indicates the probability that the prediction box belongs to the category c,
Figure FDA0002996697720000028
and if the true value of the category to which the mark box belongs to the class c, the size of the true value is equal to 1, and otherwise, the true value is 0.
4. The system for checking tickets based on pedestrian re-identification as claimed in claim 1, wherein the computer adopts a clustering algorithm to the pedestrian data in each ticket checking point video to obtain the shape and size of the anchor frame according with the human body shape; extracting global and local features of the detected pedestrian; and calculating the Euclidean distance between the pedestrian characteristics of the ticket checking point and the pedestrian characteristics of the comparison database, considering that the pedestrian is not matched with the current pedestrian if the calculation result is greater than a set threshold value, and considering that the pedestrian of the current ticket checking point and the pedestrian in the comparison database are the same pedestrian if the calculation result is less than the set threshold value.
5. The system according to claim 4, wherein the loss function in the clustering algorithm employs a hard sample sampling triple loss, for each training set, a person with P identities is randomly selected, and each person randomly selects K different pictures, so that each training set contains P x K pictures, and then a hard positive sample and a hard negative sample are selected for each picture in the training set to form a triple, and the loss function formula is as follows:
Figure FDA0002996697720000029
wherein d isa,pRepresenting the Euclidean distance, d, between the sample a and the positive sample pa,nRepresenting the euclidean distance between the sample a and the negative sample n, alpha being the set threshold parameter.
6. A ticket checking method based on pedestrian re-identification is characterized by comprising the following steps:
step 1: collecting video data of the ticket-purchased tourist;
step 2: reading the collected video data of the ticket-purchased tourists, and framing the video data to extract a pedestrian image;
and step 3: acquiring a pedestrian image, and storing the pedestrian image into a comparison database in real time;
and 4, step 4: acquiring videos of all ticket checking points, detecting in real time to obtain pedestrian data, and comparing the pedestrian data with data in a comparison database;
and 5: and sending the comparison result to a channel gate, and allowing the channel gate to perform a release or warning action according to the comparison result.
7. The ticket checking method based on pedestrian re-identification as claimed in claim 6, wherein in step 2, the frame format of the collected video data of the purchased tourists passing through the preset position is read and the number of frames is determined, and the characteristics of the images of the continuous frames are extracted for analysis and whether the images contain the pedestrian characteristics is judged; the specific position of the pedestrian in the image is calibrated for the image judged to contain the pedestrian, the pedestrian photo is stored in a comparison database, and the image judged not to contain the pedestrian is not processed; the continuous processing of the frame images of the preset position video marks the detected pedestrian data and compares the detected pedestrian data with the number of purchased tickets, thereby avoiding the situation that the number of sold tickets is not matched with the tourists.
8. The method for checking tickets based on pedestrian re-identification as claimed in claim 7, wherein in step 2, the Darknet-19 is used to combine with the residual skip-layer link to extract the features of the image of each frame, the step size of the volume local layer is used to perform down-sampling, YOLOv3 performs up-sampling on the last obtained features and fuses with the previous feature layers, and three different size feature layers are used together as the final feature output, wherein the error is calculated by using the following formula in YOLOv 3:
Figure FDA0002996697720000031
wherein the parameters
Figure FDA0002996697720000032
Whether the jth prior frame of the ith grid is responsible for the target object or not is represented, if so, the jth prior frame is 1, and if not, the jth prior frame is 0; lambda [ alpha ]coordIs a coordination coefficient set for coordinating the inconsistency of the contribution of the rectangular frames with different sizes to the error function,
Figure FDA0002996697720000033
is the coordinates of the center of the rectangular box of the network prediction,
Figure FDA0002996697720000034
is the center coordinate of the marked rectangular frame,
Figure FDA0002996697720000035
the size of the width and height of the rectangle predicted for the network,
Figure FDA0002996697720000036
is to mark the width and height of the rectangular frame, CiTo predict the probability score of the target object contained within the frame,
Figure FDA0002996697720000037
representing the true value;
Figure FDA0002996697720000038
indicating that the size of the rectangular box is 1 if it is not responsible for predicting an object, otherwise it is equal to 0; lambda [ alpha ]noobjThe weight value represents the weight of the confidence error in the loss function when the target is not predicted by the prediction box; pi (c)Indicates the probability that the prediction box belongs to the category c,
Figure FDA0002996697720000041
and if the true value of the category to which the mark box belongs to the class c, the size of the true value is equal to 1, and otherwise, the true value is 0.
9. The ticket checking method based on pedestrian re-identification as claimed in claim 6, wherein in step 4, a clustering algorithm is applied to pedestrian data in each ticket checking point video to obtain the shape and size of an anchor frame according with human body shape; extracting global and local features of the detected pedestrian; and calculating the Euclidean distance between the pedestrian characteristics of the ticket checking point and the pedestrian characteristics of the comparison database, considering that the pedestrian is not matched with the current pedestrian if the calculation result is greater than a set threshold value, and considering that the pedestrian of the current ticket checking point and the pedestrian in the comparison database are the same pedestrian if the calculation result is less than the set threshold value.
10. The ticket checking method based on pedestrian re-identification as claimed in claim 9, wherein the loss function in the clustering algorithm adopts a difficult sample sampling triple loss, for each training set, people with P identities are randomly selected, and each person randomly selects K different pictures, so that each training set contains P × K pictures, and then a most difficult positive sample and a most difficult negative sample are selected for each picture in the training set to form a triple, and the loss function formula is as follows:
Figure FDA0002996697720000042
wherein d isa,pRepresenting the Euclidean distance, d, between the sample a and the positive sample pa,nRepresenting the euclidean distance between the sample a and the negative sample n, alpha being the set threshold parameter.
CN202110334262.1A 2021-03-29 2021-03-29 Ticket checking system and method based on pedestrian re-identification Pending CN113077556A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110334262.1A CN113077556A (en) 2021-03-29 2021-03-29 Ticket checking system and method based on pedestrian re-identification

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110334262.1A CN113077556A (en) 2021-03-29 2021-03-29 Ticket checking system and method based on pedestrian re-identification

Publications (1)

Publication Number Publication Date
CN113077556A true CN113077556A (en) 2021-07-06

Family

ID=76611106

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110334262.1A Pending CN113077556A (en) 2021-03-29 2021-03-29 Ticket checking system and method based on pedestrian re-identification

Country Status (1)

Country Link
CN (1) CN113077556A (en)

Citations (17)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106485217A (en) * 2016-09-30 2017-03-08 四川世纪云道科技有限公司 A kind of method and system of identification visit scenic spot stream of people's saturation degree
CN108921152A (en) * 2018-06-29 2018-11-30 清华大学 English character cutting method and device based on object detection network
CN110246244A (en) * 2019-05-16 2019-09-17 珠海华园信息技术有限公司 Intelligent foreground management system based on recognition of face
CN110781350A (en) * 2019-09-26 2020-02-11 武汉大学 Pedestrian retrieval method and system oriented to full-picture monitoring scene
CN111105412A (en) * 2019-12-30 2020-05-05 郑州大学 Intelligent auxiliary system for intestinal polyp detection and identification
CN111192391A (en) * 2018-10-25 2020-05-22 杭州海康威视数字技术股份有限公司 Pedestrian passageway gate control method and device based on images and/or videos
CN111292305A (en) * 2020-01-22 2020-06-16 重庆大学 Improved YOLO-V3 metal processing surface defect detection method
CN111476168A (en) * 2020-04-08 2020-07-31 山东师范大学 Cross-domain pedestrian re-identification method and system based on three stages
CN111507134A (en) * 2019-01-31 2020-08-07 北京奇虎科技有限公司 Human-shaped posture detection method and device, computer equipment and storage medium
CN111753583A (en) * 2019-03-28 2020-10-09 阿里巴巴集团控股有限公司 Identification method and device
CN111783753A (en) * 2020-09-04 2020-10-16 中国科学院自动化研究所 Pedestrian re-identification method based on semantic consistency horizontal bar and foreground correction
CN111862408A (en) * 2020-06-16 2020-10-30 北京华电天仁电力控制技术有限公司 Intelligent access control method
CN112070135A (en) * 2020-08-28 2020-12-11 广东电网有限责任公司 Power equipment image detection method and device, power equipment and storage medium
CN112183667A (en) * 2020-10-31 2021-01-05 哈尔滨理工大学 Insulator fault detection method in cooperation with deep learning
CN112215203A (en) * 2020-11-02 2021-01-12 坝道工程医院(平舆) Pavement disease detection method and device based on deep learning
CN112307984A (en) * 2020-11-02 2021-02-02 安徽工业大学 Safety helmet detection method and device based on neural network
CN212624158U (en) * 2020-06-28 2021-02-26 中科华豫科技有限公司 Multifunctional intelligent ticketing system applied through big data technology

Patent Citations (17)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106485217A (en) * 2016-09-30 2017-03-08 四川世纪云道科技有限公司 A kind of method and system of identification visit scenic spot stream of people's saturation degree
CN108921152A (en) * 2018-06-29 2018-11-30 清华大学 English character cutting method and device based on object detection network
CN111192391A (en) * 2018-10-25 2020-05-22 杭州海康威视数字技术股份有限公司 Pedestrian passageway gate control method and device based on images and/or videos
CN111507134A (en) * 2019-01-31 2020-08-07 北京奇虎科技有限公司 Human-shaped posture detection method and device, computer equipment and storage medium
CN111753583A (en) * 2019-03-28 2020-10-09 阿里巴巴集团控股有限公司 Identification method and device
CN110246244A (en) * 2019-05-16 2019-09-17 珠海华园信息技术有限公司 Intelligent foreground management system based on recognition of face
CN110781350A (en) * 2019-09-26 2020-02-11 武汉大学 Pedestrian retrieval method and system oriented to full-picture monitoring scene
CN111105412A (en) * 2019-12-30 2020-05-05 郑州大学 Intelligent auxiliary system for intestinal polyp detection and identification
CN111292305A (en) * 2020-01-22 2020-06-16 重庆大学 Improved YOLO-V3 metal processing surface defect detection method
CN111476168A (en) * 2020-04-08 2020-07-31 山东师范大学 Cross-domain pedestrian re-identification method and system based on three stages
CN111862408A (en) * 2020-06-16 2020-10-30 北京华电天仁电力控制技术有限公司 Intelligent access control method
CN212624158U (en) * 2020-06-28 2021-02-26 中科华豫科技有限公司 Multifunctional intelligent ticketing system applied through big data technology
CN112070135A (en) * 2020-08-28 2020-12-11 广东电网有限责任公司 Power equipment image detection method and device, power equipment and storage medium
CN111783753A (en) * 2020-09-04 2020-10-16 中国科学院自动化研究所 Pedestrian re-identification method based on semantic consistency horizontal bar and foreground correction
CN112183667A (en) * 2020-10-31 2021-01-05 哈尔滨理工大学 Insulator fault detection method in cooperation with deep learning
CN112215203A (en) * 2020-11-02 2021-01-12 坝道工程医院(平舆) Pavement disease detection method and device based on deep learning
CN112307984A (en) * 2020-11-02 2021-02-02 安徽工业大学 Safety helmet detection method and device based on neural network

Similar Documents

Publication Publication Date Title
CN110781838B (en) Multi-mode track prediction method for pedestrians in complex scene
CN109447169B (en) Image processing method, training method and device of model thereof and electronic system
CN108334847B (en) A kind of face identification method based on deep learning under real scene
CN108388888B (en) Vehicle identification method and device and storage medium
CN109934176B (en) Pedestrian recognition system, recognition method, and computer-readable storage medium
WO2020181685A1 (en) Vehicle-mounted video target detection method based on deep learning
CN108229397A (en) Method for text detection in image based on Faster R-CNN
CN110287960A (en) The detection recognition method of curve text in natural scene image
CN112926506B (en) Non-controlled face detection method and system based on convolutional neural network
CN108564049A (en) A kind of fast face detection recognition method based on deep learning
CN109214280A (en) Shop recognition methods, device, electronic equipment and storage medium based on streetscape
TW201201107A (en) Barcode image recognition system and associated method for hand-held device
CN104794171B (en) Mark the method and device of picture geographical location information
CN109559362B (en) Image subject face replacing method and device
CN110263768A (en) A kind of face identification method based on depth residual error network
CN112464843A (en) Accurate passenger flow statistical system, method and device based on human face human shape
CN106980855A (en) Traffic sign quickly recognizes alignment system and method
CN110148223A (en) Monitor video target concentration expression and system in three-dimensional geography model of place
CN110533026A (en) The competing image digitization of electricity based on computer vision and icon information acquisition methods
CN107689157A (en) Traffic intersection based on deep learning can passing road planing method
CN111126275A (en) Pedestrian re-identification method and device based on multi-granularity feature fusion
CN110766645B (en) Target person recurrence map generation method based on person identification and segmentation
CN110517285A (en) The minimum target following of large scene based on estimation ME-CNN network
CN113077556A (en) Ticket checking system and method based on pedestrian re-identification
CN111582278B (en) Portrait segmentation method and device and electronic equipment

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication
RJ01 Rejection of invention patent application after publication

Application publication date: 20210706