CN110188694B - Method for identifying shoe wearing footprint sequence based on pressure characteristics - Google Patents
Method for identifying shoe wearing footprint sequence based on pressure characteristics Download PDFInfo
- Publication number
- CN110188694B CN110188694B CN201910464874.5A CN201910464874A CN110188694B CN 110188694 B CN110188694 B CN 110188694B CN 201910464874 A CN201910464874 A CN 201910464874A CN 110188694 B CN110188694 B CN 110188694B
- Authority
- CN
- China
- Prior art keywords
- footprint
- foot
- image
- pressure energy
- similarity
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/25—Fusion techniques
- G06F18/254—Fusion techniques of classification results, e.g. of results related to same input data
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Physics & Mathematics (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Bioinformatics & Computational Biology (AREA)
- General Engineering & Computer Science (AREA)
- Artificial Intelligence (AREA)
- Life Sciences & Earth Sciences (AREA)
- Human Computer Interaction (AREA)
- Multimedia (AREA)
- Image Analysis (AREA)
- Measurement Of The Respiration, Hearing Ability, Form, And Blood Characteristics Of Living Organisms (AREA)
Abstract
The invention provides a method for identifying a shoe wearing footprint sequence by combining pressure characteristics, which comprises an off-line training process and on-line identification. During off-line training, extracting a single shoe print relative pressure image of the pressure footprint sequence image to be trained, extracting a centroid deviation angle based on front and rear feet, constructing a footprint pressure energy map quadruple of the pressure footprint sequence image to be trained, and finally obtaining a feature database formed by quintuple feature expression of the footprint. In the online identification, similarity scores are calculated by the five-tuple characteristics of the footmarks to be identified and data in a pre-stored characteristic database, and identification of the footmarks sequences of shoes is completed through ranking. According to the method, the footprint pressure energy graph is constructed by two footprints according to the minimum unit capable of reflecting the walking habits of people, the error accumulation is reduced, the identification precision is improved, the matching result can be more stable through the quadruple similarity weighted fusion of the footprint pressure energy graph, the shoe wearing footprint sequence is researched, and the applicability is wider.
Description
Technical Field
The invention relates to the technical field of footprint identification, in particular to a method for identifying shoe wearing footprint sequences based on pressure characteristics.
Background
Feature extraction and feature matching are currently the main steps in biometric identification based on footprint sequences. The feature extraction includes a stride feature and a pressure feature. Acquisition of footprint stride characteristics [1 ,2]The method mainly comprises the following steps: 1) Obtaining a connecting line of centroids of two adjacent footprints on the same side as a walking line; 2) Calculating the distance between corresponding points of the front and rear adjacent left and right footprints (usually selecting the edge point of the heel) as the step length; 3) The distance between the edge point of the heel of the footprint and the opposite side walking line is step width; 3) The included angle between the footprint central line and the walking line in the advancing direction is a step angle.
The method commonly adopted for extracting the pressure characteristics is to extract the footprint centroid pressure in the walking processForce trace (cop) [5] 。
The adopted matching method mainly comprises the following steps: (1) U test method: general way of document [3] Six indexes of stride left by 20 suspects and criminals are counted through experiments, the absolute value of the mean value and the mean value difference of the samples is calculated, and then the absolute value is compared with a calculated mean value difference upper limit table, 18 persons can be excluded, and finally the extreme difference test method of criminals (2) is determined by combining other case solving means: the method comprises the steps of firstly, measuring normal step length, step width and step angle data in a footprint sequence of a field perpetrator, and calculating the mean value of each index. And secondly, measuring normal step length, step width and step angle data in the footprint sequence of the suspect, and calculating the mean value of each characteristic index, wherein the leaving condition of the footprint sequence of the suspect is close to the site as much as possible. And thirdly, respectively calculating the absolute value of the average difference of the indexes corresponding to the stride characteristics of each suspect and the site. And fourthly, checking each corresponding index of the suspect and the site, and judging as positive if the absolute value of the mean value difference of the characteristic checking index is less than or equal to the range value of the index: if the difference is greater than the range of the index, the result is negative. After six indexes of the suspect and the field stride are detected, if one index is not received, the result is judged to be negative; only when the six indexes are all received, the conclusion can be drawn that the stride characteristic of the suspect is consistent with the stride characteristic of the field perpetrator. (3) Membership degree test method [3] : "degree of membership" is a concept in fuzzy mathematics. It represents the closeness of an element to some fuzzy set. Before a case is not checked, the footprint characteristic of the perpetrator is a fuzzy set, and the corresponding domain of interest is all suspicious objects. Each suspect is to some extent affiliated with the fuzzy set of "footprint characteristics of perpetrators", and the difference of the closeness of each element to a fuzzy set can be represented by a data between 0 and 1, which is called the affiliation.
Reference documents:
[1] shandong political law academy of academic interest, a comprehensive index quantitative test method by using stride characteristics, china, application for published invention patent, 201710902633.5.2017
[2] Design and implementation of an intelligent stride characteristic analysis and inspection system [ J ] scientific technology and engineering 2014,14 (3): 64-69.
[3] Yuan Sheng, wang Yang, china footprint inspection technology, yunan academy of police, 2011 (2): 119-121.
[4] Peng, five star, et al, turn into a trip and research of a three-dimensional footprint analysis and inspection system [ D ]. University of kunming, 2016.
[5] The application of the U test method in the identification of steps [ J ] mathematical statistics and management, 1982 (2): 17-21.
[6]Zhou B,Singh M S,Doda Set al.The carpet knows:Identifying people in a smart environment from a single step.IEEE International Conference on Pervasive Computing and Communications Workshops,2017.
Disclosure of Invention
In accordance with the above-mentioned technical problem, there is provided a method for identifying a sequence of footmarks of shoes worn based on pressure characteristics, comprising: an off-line training process and an on-line identification process; the off-line process comprises at least the following steps:
step S11: extracting a single shoe print relative pressure image in a centralized footprint image sequence to be trained;
step S12: calculating a front foot mass center deviation angle I and a rear foot mass center deviation angle I according to the single shoe print relative pressure image extracted in the S11;
step S13: constructing a footprint pressure energy map quadruple I of the footprint sequence set to be trained;
step S14: and constructing a footprint sequence quintuple feature database by the front and rear foot mass center deviation angle I extracted in the S12 and the footprint energy quadruple I constructed in the S13.
Further, the online identification process comprises at least the following steps:
step S21: extracting a single shoe print relative pressure image of the online recognized footprint image sequence;
step S22: calculating a front and rear foot mass center offset angle II according to the single shoe print relative pressure image extracted in the S21 and identified on line;
step S23: constructing a footprint pressure energy graph quadruple II of the online recognized footprint sequence images;
step S24: according to the extracted front and rear foot centroid deviation angle II and pressure energy quadruple II groups identified on line, calculating similarity of the footmark sequence data to be identified and the footmark sequence data stored in the off-line training process respectively in a normalized cross-correlation measurement mode and a cosine distance measurement mode, fusing the calculated similarity scores, sorting the fused similarity scores from large to small, and taking the similarity score with the maximum similarity as the category of the footmarks to be retrieved.
Furthermore, the single shoe print relative pressure image extraction step comprises the following steps:
step S111: preprocessing an original footprint sequence; removing salt and pepper noise and plaque noise from the online recognized/to-be-trained footprint sequence image through median filtering, and enhancing the denoised image to obtain a footprint sequence image F i (x, y) where i ∈ [1, N ]]N is the total number of people in the database;
step S112: extracting the single shoe print image; and binarizing the preprocessed footprint sequence images, solving segmentation errors caused by shoe printing pattern fracture through closed operation of expansion and corrosion, calculating horizontal projection of the obtained images, and segmenting a single footprint by utilizing the interval between the footprint sequences. Marking the divided single footprints with connected domains and calculating the number and the area of the connected domains, deleting the images with the area and the number larger than the threshold value as abnormal data to obtain the jth single shoe print image L of the ith individual ij (x, y) where i ∈ [1, N ]],j∈[1,M]Wherein N represents the total number of people in the database, and M represents the number of single shoe print images obtained by dividing each person.
Step S113: calculating a relative pressure image of the single shoe print image extracted in step S112; scanning the single shoe print image line by line to find a pixel value l of a non-zero point ij (x, y), wherein x, y respectively represent abscissa and ordinate of pixel point of single shoe print image, and mean value of pixel value point obtained：
Making a difference l between the pixel value and the mean value of each non-zero point new (x,y)＝l ij (x,y)-l mean To obtain a relative pressure image R of the single shoe seal ij (x,y)。
Furthermore, the method for calculating the offset angle based on the centroids of the front foot and the rear foot comprises the following steps: setting the deviation angle of the mass centers of the front foot and the rear foot extracted from a single relative pressure distribution diagram as theta, distributing the deviation angle of the mass centers of the front foot and the rear foot as theta according to the main pressure area of a person on the sole and the heel, and obtaining the following data according to a mass center coordinate formula:
separately determine the center of mass of the sole (plot _ x) sole ,plot_y sole ) And the center of mass of the heel (plot _ x) heel ,plot_y heel ) And the arctangent value of the slope of the connecting line of the two centroids is a relative centroid deviation angle:
calculating the centroid offset angle theta = (theta) of each footprint sequence through the centroid offset angles of the front foot and the rear foot 1 ,θ 2 ) Wherein theta 1 Is the front and rear foot mass center offset angle theta of the left foot of the footprint sequence 2 And the front foot mass center offset angle and the rear foot mass center offset angle of the right foot of the footprint sequence are obtained.
Further, the footprint pressure energy map quadruple is constructed by: finding the minimum external rectangle for the obtained single relative pressure distribution image, and respectively obtaining the coordinate values (x) of the top left corners of the external rectangles of the two adjacent single footprints 1 ,y 1 ) And (x) 2 ,y 2 ) If x 1 >x 2 The first picture is the right foot, otherwise the second picture is the right foot.
Furthermore, the obtained single relative pressure image is spliced up and down, the number of spliced single feet is 2 according to the minimum unit of the walking habit of people and the principle of reducing error accumulation, the formed reference is splicing with the step length and splicing without the step length, the precedence relationship is left and right and left, and therefore the ith person can form the spliced image with the step length with the left foot as the reference respectively
Left foot-based spliced image without step length
Spliced image with step length taking right foot as reference
Spliced image without step length with right foot as reference
Wherein i ∈ [1, N ]],j∈[1,N]Wherein N is the total number of people in the database, and M is the number of each type of feature map of each person.
Further, performing scale normalization on the spliced image, respectively traversing the image under each tuple, and finding the maximum size of the image under each tuple as the standard size S of the tuple Q1 ,S Q2 ,S Q3 ,S Q4 And solving a circumscribed rectangle of the footprint area of each image, and normalizing the circumscribed rectangle to the standard size under the corresponding tuple by a zero padding method.
Further, the normalized images are respectively added and averaged to obtain a footprint pressure energy map, and a footprint pressure energy map quadruple of each person is constructed
Wherein the content of the first and second substances,
a kth element footprint pressure energy map representing the ith person, and m represents the number of pictures per tuple for each person.
Further, when the obtained footprint pressure energy map has low contrast, the image is enhanced through gamma conversion
Wherein the content of the first and second substances,
representing a footprint pressure energy map, gamma representing an enhancement factor; γ is obtained by training and is typically taken to be 1.3.
Still further, the five-tuple of the footprint is: obtaining quintuple expression of the footprint sequence according to the relative centroid deviation angle and the four-tuple splicing of the footprint pressure energy diagram
Further, during the online identification, the footprint identification process based on the footprint quintuple is as follows:
step S241: calculating the similarity of the centroid deviation angles of the front and rear feet; calculating Euclidean distance for the centroid deviation angle of the single relative pressure footprint image identified on line and the centroid deviation angle characteristic of the single relative pressure image set in the database:
wherein, theta 01 Representing an on-line identification of the front and rear foot centroid offset angle, θ, of a person's left foot i1 Representing the anterior-posterior foot centroid offset angle, θ, of the left foot of the ith individual in the database 02 Representing an on-line identification of the offset angle, θ, of the front and rear foot centroids of the right foot of the person i2 Representing the anterior-posterior foot centroid offset angle of the ith individual's right foot in the database;
for fusion with the similarity score, normalizing the resulting distance to obtain a normalized similarity:
where k represents a weighting coefficient obtained by training, and is typically 0.06.
A similarity matrix D ' = { D ' based on front and rear foot offset angles can be obtained through a measurement method of front and rear foot centroid offset angles ' i },i∈[1,N]。
Step S242: calculating the similarity of the four-tuple of the footprint pressure energy diagram; calculating similarity scores of the footmark pressure energy graph with the step length and taking the left foot as the reference to be identified and the footmark pressure energy graph with the step length and taking the left foot as the reference in the data set to obtain a similarity score S 1 ＝{s i },i∈[1,N]Wherein N represents the total number of people in the pool, s i Representing the similarity score of the sample to be identified and the ith sample in the library;
similarly, calculating the similarity score S between the footmark pressure energy graph of the step-removing step length based on the left foot to be identified and the footmark pressure energy graph of the step-removing step length based on the left foot in the database 2 ＝{s i },i∈[1,N]Similarity score S between the footmark pressure distribution diagram of the step-removing based on the right foot and the footmark pressure energy diagram of the step-removing based on the right foot in the database 3 ＝{s i },i∈[1,N]Similarity score S between footprint pressure energy map with step length based on right foot and footprint pressure energy map with step length based on right foot in database 4 ＝{s i },i∈[1,N]；
And performing weighted fusion on the similarity scores of the four characteristics, and determining a weighting coefficient through training, wherein the weighting coefficient is generally 0.3,0.2 and 0.2 to obtain the fused similarity score:
S＝0.3S 1 +0.3S 2 +0.2S 3 +0.2S 4 。
further, the similarity score is calculated according to the normalized cross correlation coefficient of the footprint pressure energy map in the online identification stage and the footprint pressure energy map of each sample in the data set.
Step S243: correcting the similarity score of the footprint pressure energy diagram by using the similarity of the centroid offset angles of the front foot and the rear foot;
weighting by using the normalized distance of the centroid offset angle of the front foot and the rear foot, and then correcting the similarity score to obtain a final similarity score S ' = S + D ', wherein S represents the similarity score of the four-tuple of the footprint energy diagram after weighting fusion, and D ' represents the calculated normalized similarity of the centroid offset angle of the front foot and the rear foot; and according to the final scoring sorting result, finding the label corresponding to the maximum similarity as the identification result.
Compared with the prior art, the invention has the following advantages:
(1) The invention fully considers the mass center offset angle of the sole and the heel of the pressure footprint sequence image; and the Euclidean distance is calculated through the deviation angles of the mass centers of the front foot and the rear foot of the left foot and the right foot, then the similarity of the footprint energy diagram is weighted, the error of the footprint pressure energy diagram is corrected, and a more stable similarity score is obtained.
(2) The invention fully considers that the minimum unit reflecting the walking habits of people and errors brought by splicing a plurality of footprints are superposed according to the increase of the number of the footprints, so that images formed by two feet are used for splicing to obtain a footprint pressure energy diagram with stability, and the allowable fluctuation range of each person during walking can be reflected by weighting and fusing the shoe prints at different time points;
(3) The invention fully considers the difference of information reflected by different splicing modes, forms four footprint pressure energy graphs containing different characteristics, namely a footprint pressure energy graph with step length based on the left foot, a foot-length-removing footprint pressure energy graph with step length based on the right foot and a foot-length-removing footprint pressure energy graph with step length based on the right foot, obtains matching scores through similarity calculation, and performs weighting fusion to ensure higher matching precision.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings needed to be used in the description of the embodiments or the prior art will be briefly introduced below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to these drawings without creative efforts.
FIG. 1 is a general flow diagram of the present invention.
FIG. 2 is a foot-to-foot footprint pressure energy map based on the right foot of the present invention.
FIG. 3 is a graph of foot pressure energy without step size based on the right foot of the present invention.
FIG. 4 is a plot of foot-to-foot pressure energy with step size based on the left foot of the present invention.
FIG. 5 is a graph of the foot pressure energy without step size based on the left foot of the present invention.
FIG. 6 is a schematic view of the offset angle of the centroid of the forefoot and hindfoot of the present invention.
Detailed Description
In order to make those skilled in the art better understand the technical solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be obtained by a person skilled in the art without making any creative effort based on the embodiments in the present invention, shall fall within the protection scope of the present invention.
It should be noted that the terms "first," "second," and the like in the description and claims of the present invention and in the drawings described above are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the invention described herein are capable of operation in other sequences than those illustrated or described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
As shown in fig. 1 to 6, the present invention includes a method for identifying sequences of footmarks of shoes worn based on pressure characteristics, comprising: an off-line training process and an on-line identification process.
As a preferred embodiment, the off-line process comprises at least the following steps:
step S11: and extracting the relative pressure image of the single shoe print in the sequence set of the footprint images to be trained.
In this embodiment, the step of extracting the relative pressure image of the single shoe print comprises:
step S111: preprocessing an original footprint sequence; removing salt and pepper noise and plaque noise from the online recognized/to-be-trained footprint sequence image through median filtering, and enhancing the denoised image to obtain a footprint sequence image F i (x, y) where i ∈ [1,N ]]And N is the total number of people in the database.
Step S112: extracting the single shoe print image; and binarizing the preprocessed footprint sequence images, solving segmentation errors caused by shoe printing pattern fracture through closed operation of expansion and corrosion, calculating horizontal projection of the obtained images, and segmenting a single footprint by utilizing the interval between the footprint sequences. Marking the divided single footprints with connected domains and calculating the number and the area of the connected domains, deleting the images with the area and the number larger than the threshold value as abnormal data to obtain the jth single shoe print image L of the ith individual ij (x, y) where i ∈ [1,N ]],j∈[1,M]Wherein N represents the total number of people in the database, and M represents the number of single shoe print images obtained by dividing each person;
step S113: calculating a relative pressure image of the single shoe print image extracted in step S112; scanning the single shoe print image line by line to find a pixel value l of a non-zero point ij (x, y), wherein x, y respectively represent abscissa and ordinate of single piece of shoe print image pixel, and calculate the mean value to the pixel value point that obtains:
making a difference l between the pixel value and the average value of each non-zero point new (x,y)＝l ij (x,y)-l mean Obtaining the relative pressure image R of the single shoe print ij (x,y)。
Further, as a preferred embodiment, step S12: and calculating a front and rear foot mass center deviation angle I according to the relative pressure image of the single shoe mark extracted in the step S11.
The calculation method based on the centroid deviation angle of the front foot and the rear foot comprises the following steps:
setting the deviation angle of the mass centers of the front foot and the rear foot extracted from a single relative pressure distribution map as theta, distributing the deviation angle on the sole and the heel according to the main pressure area of a person, and obtaining the following data by a mass center coordinate formula:
separately determine the sole centroid (plot _ x) sole ,plot_y sole ) And the center of mass of the heel (plot _ x) heel ,plot_y heel ) And the arctangent value of the slope of the connecting line of the two centroids is a relative centroid deviation angle:
calculating the centroid offset angle theta = (theta) of each footprint sequence through the centroid offset angles of the front foot and the rear foot 1 ,θ 2 ) Wherein θ 1 Is the front and rear foot mass center offset angle theta of the left foot of the footprint sequence 2 And the front and rear foot mass center deviation angles of the right foot of the footprint sequence are obtained.
In the present embodiment, step S13: and constructing a footprint pressure energy diagram quadruplet I of the footprint sequence set to be trained. As a preferred embodiment, the footprint pressure energy map quadruple is constructed by:
finding the minimum external rectangle for the obtained single relative pressure distribution image, and respectively obtaining the coordinate values (x) of the top left corners of the external rectangles of the two adjacent single footprints 1 ,y 1 ) And (x) 2 ,y 2 ) If x 1 >x 2 If the first picture is the right foot, otherwise the second picture is the right foot;
the obtained single relative pressure image is spliced up and down, the number of spliced single feet is 2 according to the minimum unit of the walking habit of people and the principle of reducing error accumulation, the formed reference is splicing with the step length and splicing without the step length, the precedence relationship is left and right and left, and therefore the ith person can respectively form the spliced image with the step length with the left foot as the reference
Spliced image without step length with left foot as reference
Spliced image with step length taking right foot as reference
Spliced image without step length with right foot as reference
Wherein i ∈ [1, N ]],j∈[1,N]Wherein N is the total number of people in the database, and M is the number of each type of characteristic graph of each person;
performing scale normalization on the spliced images, respectively traversing the images under each tuple, and finding out the maximum size of the images under each tuple as the standard size S of the tuple Q1 ,S Q2 ,S Q3 ,S Q4 And solving a circumscribed rectangle of the footprint area of each image, and normalizing the circumscribed rectangle to the standard size under the corresponding tuple by a zero padding method.
As a preferred embodiment, the normalized images are respectively added and averaged to obtain a footprint pressure energy map, and a footprint pressure energy map is constructedFootprint pressure energy map quadruplet for each person
Wherein, the first and the second end of the pipe are connected with each other,
a kth element footprint pressure energy map representing the ith person, and m represents the number of pictures per tuple for each person.
For the condition that the obtained footprint pressure energy map has low contrast, the image is enhanced through gamma conversion
Wherein the content of the first and second substances,
representing a footprint pressure energy map, gamma representing an enhancement factor; γ is obtained by training and is typically taken to be 1.3.
Step S14: and constructing a footprint sequence quintuple feature database by the front and rear foot mass center offset angles I extracted in the step S12 and the footprint energy quadruple I constructed in the step S13. Obtaining quintuple expression of the footprint sequence according to the relative centroid deviation angle and the four-tuple splicing of the footprint pressure energy diagram
In this embodiment, the online identification process at least includes the following steps:
step S21: extracting a single shoe print relative pressure image of the online recognized footprint image sequence;
step S22: calculating a front and rear foot mass center offset angle II according to the single shoe print relative pressure image extracted in the step S21 and identified on line;
step S23: constructing a footprint pressure energy graph quadruple II of the online recognized footprint sequence images;
step S24: according to the extracted front and rear foot centroid deviation angle II and pressure energy quadruple II groups identified on line, calculating similarity of the footmark sequence data to be identified and the footmark sequence data stored in the off-line training process respectively in a normalized cross-correlation measurement mode and a cosine distance measurement mode, fusing the calculated similarity scores, sorting the fused similarity scores from large to small, and taking the similarity score with the maximum similarity as the category of the footmarks to be retrieved.
In a preferred embodiment, in the online identification, the footprint identification process based on the footprint quintuple is as follows:
s241: calculating the similarity of the centroid deviation angles of the front and rear feet; calculating Euclidean distance of the centroid offset angle of the single relative pressure footprint image identified on line and the centroid offset angle characteristic of the single relative pressure image set in the database:
wherein, theta 01 Representing an on-line identification of the front and rear foot centroid offset angle, θ, of a person's left foot i1 Representing the anterior-posterior foot centroid offset angle, θ, of the left foot of the ith individual in the database 02 Representing an on-line identification of the offset angle, θ, of the front and rear foot centroids of the right foot of the person i2 Representing the anterior-posterior foot centroid offset angle of the ith individual's right foot in the database;
for fusion with the similarity score, normalizing the resulting distance to obtain a normalized similarity:
where k represents a weighting coefficient obtained by training, typically 0.06; a similarity matrix D ' = { D ' based on front and rear foot offset angles can be obtained through a measurement method of front and rear foot centroid offset angles ' i },i∈[1,N]。
S242: calculating the similarity of the four groups of the footprint pressure energy diagram; left foot referenced foot-left pressure energy map and data set to be identifiedCalculating a similarity score by using the footprint pressure energy map with the step length to obtain a similarity score S 1 ＝{s i },i∈[1,N]Wherein N represents the total number of people in the pool, s i Representing the similarity score of the sample to be identified and the ith sample in the library;
similarly, calculating the similarity score S between the footmark pressure energy graph of the step-removing step length based on the left foot to be identified and the footmark pressure energy graph of the step-removing step length based on the left foot in the database 2 ＝{s i },i∈[1,N]Similarity score S between the footmark pressure distribution diagram of the step-removing based on the right foot and the step-removing footmark pressure energy diagram of the step-removing based on the right foot in the database 3 ＝{s i },i∈[1,N]Similarity score S between footprint pressure energy map with step length based on right foot and footprint pressure energy map with step length based on right foot in database 4 ＝{s i },i∈[1,N]；
And performing weighted fusion on the similarity scores of the four characteristics, and determining a weighting coefficient through training, wherein the weighting coefficient is generally 0.3,0.2 and 0.2 to obtain the fused similarity score:
S＝0.3S 1 +0.3S 2 +0.2S 3 +0.2S 4 ；
calculating the similarity score according to the normalized cross-correlation coefficient of the footprint pressure energy graph in the online identification stage and each sample footprint pressure energy graph in the data set;
s243: correcting the similarity score of the footprint pressure energy map by using the similarity of the offset angles of the centroids of the front foot and the rear foot;
weighting by using the normalized distance of the offset angle of the front and rear foot centroids, and then correcting the similarity score to obtain a final similarity score S ' = S + D ', wherein S represents the similarity score of the weighted and fused footprint energy diagram quadruple, and D ' represents the calculated normalized similarity of the offset angle of the front and rear foot centroids; and according to the final scoring sorting result, finding the label corresponding to the maximum similarity as the identification result.
The above-mentioned serial numbers of the embodiments of the present invention are merely for description and do not represent the merits of the embodiments.
In the above embodiments of the present invention, the description of each embodiment has its own emphasis, and reference may be made to the related description of other embodiments for parts that are not described in detail in a certain embodiment.
In the embodiments provided in the present application, it should be understood that the disclosed technology can be implemented in other ways. The above-described device embodiments are merely illustrative, and for example, the division of the unit may be a logical function division, and there may be another division in actual implementation.
Finally, it should be noted that: the above embodiments are only used to illustrate the technical solution of the present invention, and not to limit the same; while the invention has been described in detail and with reference to the foregoing embodiments, it will be understood by those skilled in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some or all of the technical features may be equivalently replaced; and the modifications or the substitutions do not make the essence of the corresponding technical solutions depart from the scope of the technical solutions of the embodiments of the present invention.
Claims (4)
1. A method for recognizing sequences of shoe wearing footprints based on pressure characteristics is characterized by comprising the following steps: an off-line training process and an on-line identification process;
the off-line process comprises at least the following steps:
s11: extracting a single shoe print relative pressure image in a footprint image sequence set to be trained;
s12: calculating a front foot mass center deviation angle I and a rear foot mass center deviation angle I according to the single shoe print relative pressure image extracted in the S11;
s13: constructing a footprint pressure energy map quadruple I of the footprint sequence set to be trained;
s14: constructing a footprint sequence quintuple feature database by the front and rear foot mass center offset angle I extracted in the S12 and the footprint energy quadruplet I constructed in the S13;
the online identification process comprises at least the following steps:
s21: extracting a single shoe print relative pressure image of the online recognized footprint image sequence;
s22: calculating a front and rear foot mass center offset angle II according to the single shoe print relative pressure image extracted in the S21 and identified on line;
s23: constructing a footprint pressure energy graph quadruple II of the online recognized footprint sequence images;
s24: according to the extracted front and rear foot mass center offset angles II and pressure energy quadruple II groups identified on line, calculating similarity of the footmark sequence data to be identified and the footmark sequence data stored in the off-line training process by respectively adopting a normalized cross-correlation measurement mode and a cosine distance measurement mode, fusing the calculated similarity scores, sequencing the fused similarity scores from large to small, and taking the maximum similarity score as the category of the footmarks to be retrieved;
the construction of the footprint pressure energy diagram quadruple is as follows:
finding the minimum external rectangle for the obtained single relative pressure distribution image, and respectively obtaining the coordinate values (x) of the top left corners of the external rectangles of the two adjacent single footprints 1 ,y 1 ) And (x) 2 ,y 2 ) If x 1 ＞x 2 The first picture is the right foot, otherwise the second picture is the right foot;
the obtained single relative pressure image is spliced up and down, the number of spliced single feet is 2 according to the minimum unit of the walking habit of people and the principle of reducing error accumulation, the formed reference is splicing with the step length and splicing without the step length, the precedence relationship is left and right and left, and therefore the ith person can respectively form the spliced image with the step length with the left foot as the reference
Spliced image without step length with left foot as reference
Spliced image with step length taking right foot as reference
Spliced image without step length with right foot as reference
Wherein i ∈ [1, N ]],j∈[1,N]Wherein N is the total number of people in the database, and M is the number of each type of characteristic graph of each person;
carrying out scale normalization on the spliced images, respectively traversing the images under each tuple, and finding the maximum size of the images under each tuple as the standard size S of the tuple Q1 ,S Q2 ,S Q3 ,S Q4 Calculating a circumscribed rectangle of a footprint area of each image, and normalizing the circumscribed rectangle to a standard size under a corresponding tuple through a zero padding method;
respectively adding and averaging the normalized images to obtain a footprint pressure energy map, and constructing the four-tuple of the footprint pressure energy map of each person
Wherein the content of the first and second substances,
a kth element footprint pressure energy map representing the ith individual, m representing the number of pictures per tuple for each individual;
for the condition that the obtained footprint pressure energy map has low contrast, the image is enhanced through gamma conversion
Wherein, the first and the second end of the pipe are connected with each other,
representing a footprint pressure energy map, gamma representing an enhancement factor; the gamma is obtained by training, and the gamma is 1.3;
during the online identification, the footprint identification process based on the footprint quintuple comprises the following steps:
s241: calculating the similarity of the centroid deviation angles of the front and rear feet; calculating Euclidean distance for the centroid deviation angle of the single relative pressure footprint image identified on line and the centroid deviation angle characteristic of the single relative pressure image set in the database:
wherein, theta 01 Representing an on-line identification of the front and rear foot centroid offset angle, θ, of a person's left foot i1 Representing the anterior-posterior foot centroid offset angle, θ, of the left foot of the ith individual in the database 02 Representing the on-line identification of the offset angle, theta, of the front and rear foot centroids of the right foot of a person i2 Representing the anterior-posterior foot centroid offset angle of the ith individual's right foot in the database;
for fusion with the similarity score, normalizing the obtained distance to obtain a normalized similarity:
where k represents a weighting coefficient obtained by training, typically 0.06; a similarity matrix D ' = { D ' based on front and rear foot offset angles can be obtained through a measurement method of front and rear foot centroid offset angles ' i },i∈[1,N]；
S242: calculating the similarity of the four-tuple of the footprint pressure energy diagram; calculating similarity scores of the footmark pressure energy graph with the step length and taking the left foot as the reference to be identified and the footmark pressure energy graph with the step length and taking the left foot as the reference in the data set to obtain a similarity score S 1 ＝{s i },i∈[1,N]Wherein N represents the total number of people in the bank, s i Representing the similarity score of the sample to be identified and the ith sample in the library;
similarly, calculating the similarity score S between the footmark pressure energy graph of the step-removing step length based on the left foot to be identified and the footmark pressure energy graph of the step-removing step length based on the left foot in the database 2 ＝{s i },i∈[1,N]Step-down footprint pressure distribution map based on right foot and step-down footprint pressure energy based on right foot in databaseSimilarity score S of graph 3 ＝{s i },i∈[1,N]Similarity score S between footprint pressure energy map with step length based on right foot and footprint pressure energy map with step length based on right foot in database 4 ＝{s i },i∈[1,N]；
And performing weighted fusion on the similarity scores of the four characteristics, and determining a weighting coefficient through training, wherein the weighting coefficient is 0.3,0.2 and 0.2 to obtain the fused similarity score:
S＝0.3S 1 +0.3S 2 +0.2S 3 +0.2S 4 ；
calculating the similarity score according to the normalized cross-correlation coefficient of the footprint pressure energy graph in the online identification stage and each sample footprint pressure energy graph in the data set;
s243: correcting the similarity score of the footprint pressure energy diagram by using the similarity of the centroid offset angles of the front foot and the rear foot;
weighting by using the normalized distance of the offset angle of the front and rear foot centroids, and then correcting the similarity score to obtain a final similarity score S ' = S + D ', wherein S represents the similarity score of the weighted and fused footprint energy diagram quadruple, and D ' represents the calculated normalized similarity of the offset angle of the front and rear foot centroids; and according to the final scoring sorting result, finding the label corresponding to the maximum similarity as the identification result.
2. The method for identifying sequences of footmarks for wearing shoes according to claim 1, further characterized in that:
the single shoe print relative pressure image extraction step comprises the following steps:
s111: preprocessing an original footprint sequence; removing salt and pepper noise and plaque noise from the online recognized/to-be-trained footprint sequence image through median filtering, and enhancing the denoised image to obtain a footprint sequence image F i (x, y) where i ∈ [1,N ]]N is the total number of people in the database;
s112: extracting the single shoe print image; binarizing the preprocessed footprint sequence images, solving segmentation errors caused by shoe printing pattern fracture through closed operation of expansion and corrosion, calculating horizontal projection of the obtained images, and segmenting a single footprint by utilizing the interval between the footprint sequences;
marking the divided single footprints with connected domains and calculating the number and the area of the connected domains, deleting the images with the area and the number larger than the threshold value as abnormal data to obtain the jth single shoe print image L of the ith individual ij (x, y) where i ∈ [1,N ]],j∈[1,M]Wherein N represents the total number of people in the database, and M represents the number of single shoe print images obtained by dividing each person;
s113: calculating a relative pressure image of the single shoe print image extracted in step S112; scanning the single shoe print image line by line to find a pixel value l of a non-zero point ij (x, y), wherein x, y respectively represent abscissa and ordinate of the image pixel of single shoe print, calculate the mean value to the pixel value point got:
making a difference l between the pixel value and the average value of each non-zero point new (x,y)＝l ij (x,y)-l mean Obtaining the relative pressure image R of the single shoe print ij (x,y)。
3. The method of identifying sequences of shoe wearing footprints based on pressure characteristics as claimed in claim 1, further characterized by: the calculation method based on the centroid deviation angle of the front foot and the rear foot comprises the following steps:
setting the deviation angle of the mass centers of the front foot and the rear foot extracted from a single relative pressure distribution diagram as theta, distributing the deviation angle of the mass centers of the front foot and the rear foot as theta according to the main pressure area of a person on the sole and the heel, and obtaining the following data according to a mass center coordinate formula:
separately determine the sole centroid (plot _ x) sole ,plot_y sole ) And the center of mass of the heel (plot _ x) heel ,plot_y heel ) The inverse tangent value of the slope of the connecting line of the two centroids is the relative centroid deviation angle:
calculating the centroid offset angle theta = (theta) of each footprint sequence through the centroid offset angles of the front foot and the rear foot 1 ,θ 2 ) Wherein theta 1 Is the offset angle theta of the front and rear foot mass centers of the left foot of the footprint sequence 2 And the front foot mass center offset angle and the rear foot mass center offset angle of the right foot of the footprint sequence are obtained.
4. The method for identifying sequences of footmarks for wearing shoes according to claim 1, further characterized in that: the five-element group of the footprint is as follows:
obtaining quintuple expression of the footprint sequence according to the relative centroid deviation angle and the four-tuple splicing of the footprint pressure energy diagram
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910464874.5A CN110188694B (en) | 2019-05-30 | 2019-05-30 | Method for identifying shoe wearing footprint sequence based on pressure characteristics |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910464874.5A CN110188694B (en) | 2019-05-30 | 2019-05-30 | Method for identifying shoe wearing footprint sequence based on pressure characteristics |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110188694A CN110188694A (en) | 2019-08-30 |
| CN110188694B true CN110188694B (en) | 2022-10-21 |
Family
ID=67719039
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910464874.5A Active CN110188694B (en) | 2019-05-30 | 2019-05-30 | Method for identifying shoe wearing footprint sequence based on pressure characteristics |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110188694B (en) |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110781852B (en) * | 2019-10-31 | 2023-10-27 | 大连海事大学 | Footwear sequence footprint identification method based on multi-resolution feature fusion |
| CN111402202B (en) * | 2020-02-24 | 2023-07-25 | 安徽大学 | Plantar pressure image similarity evaluation method |
| CN112102362A (en) * | 2020-09-14 | 2020-12-18 | 北京数衍科技有限公司 | Pedestrian step track determination method and device |
| CN112633338A (en) * | 2020-12-14 | 2021-04-09 | 辽宁师范大学 | Suspect gender prediction method based on shoe imprints |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2015084868A (en) * | 2013-10-29 | 2015-05-07 | 花王株式会社 | Walking feature display method |
| CN106887019A (en) * | 2017-02-23 | 2017-06-23 | 大连海事大学 | A kind of footprint Pressure Distribution method for expressing |
| CN109325546A (en) * | 2018-10-19 | 2019-02-12 | 大连海事大学 | A kind of combination footwork feature at time footprint recognition method |
-
2019
- 2019-05-30 CN CN201910464874.5A patent/CN110188694B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2015084868A (en) * | 2013-10-29 | 2015-05-07 | 花王株式会社 | Walking feature display method |
| CN106887019A (en) * | 2017-02-23 | 2017-06-23 | 大连海事大学 | A kind of footprint Pressure Distribution method for expressing |
| CN109325546A (en) * | 2018-10-19 | 2019-02-12 | 大连海事大学 | A kind of combination footwork feature at time footprint recognition method |
Non-Patent Citations (1)
| Title |
|---|
| 基于稳定特征的鞋印图像识别方法研究;管燕等;《计算机工程与应用》;20081001(第28期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110188694A (en) | 2019-08-30 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110188694B (en) | Method for identifying shoe wearing footprint sequence based on pressure characteristics | |
| Li et al. | Automatic pavement crack detection by multi-scale image fusion | |
| CN107145829B (en) | Palm vein identification method integrating textural features and scale invariant features | |
| Zamir et al. | Accurate image localization based on *** maps street view | |
| CN106600600A (en) | Wafer defect detection method based on characteristic matching | |
| CN108197587A (en) | A kind of method that multi-modal recognition of face is carried out by face depth prediction | |
| CN111931623A (en) | Face mask wearing detection method based on deep learning | |
| CN109325546B (en) | Step-by-step footprint identification method combining features of step method | |
| CN108960211A (en) | A kind of multiple target human body attitude detection method and system | |
| CN111914642B (en) | Pedestrian re-identification method, device, equipment and medium | |
| CN114332650B (en) | Remote sensing image road identification method and system | |
| CN105160355B (en) | A kind of method for detecting change of remote sensing image based on region correlation and vision word | |
| CN103415871B (en) | Irradiation field identifies | |
| CN108629286A (en) | A kind of remote sensing airport target detection method based on the notable model of subjective perception | |
| CN115272652A (en) | Dense object image detection method based on multiple regression and adaptive focus loss | |
| CN107644227A (en) | A kind of affine invariant descriptor of fusion various visual angles for commodity image search | |
| CN103854278A (en) | Printed circuit board image registration method based on shape context of mass center of communicated region | |
| CN108182705A (en) | A kind of three-dimensional coordinate localization method based on machine vision | |
| JP2001043368A (en) | Device and method for retrieving image and computer readable storage medium | |
| CN105469099B (en) | Pavement crack detection and identification method based on sparse representation classification | |
| CN109146913B (en) | Face tracking method and device | |
| CN110348386B (en) | Face image recognition method, device and equipment based on fuzzy theory | |
| CN106326927A (en) | Shoeprint new class detection method | |
| CN107066943B (en) | A kind of method for detecting human face and device | |
| CN109190489A (en) | A kind of abnormal face detecting method based on reparation autocoder residual error |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |