CN107273873B - Pedestrian based on irregular video sequence recognition methods and system again - Google Patents
Pedestrian based on irregular video sequence recognition methods and system again Download PDFInfo
- Publication number
- CN107273873B CN107273873B CN201710572155.6A CN201710572155A CN107273873B CN 107273873 B CN107273873 B CN 107273873B CN 201710572155 A CN201710572155 A CN 201710572155A CN 107273873 B CN107273873 B CN 107273873B
- Authority
- CN
- China
- Prior art keywords
- subsequence
- sequence
- video sequence
- pedestrian
- video
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/30—Noise filtering
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Theoretical Computer Science (AREA)
- Human Computer Interaction (AREA)
- Image Analysis (AREA)
- Image Processing (AREA)
Abstract
The present invention provides the pedestrian based on irregular video sequence recognition methods and system again, including extracting multiple continuous subsequences from video sequence, obtaining candidate sequence by the stable point in detecting state curve；The reconstructed error that each subsequence is sought using rarefaction representation obtains the noise measurement result of each subsequence；According to the noise measurement of each subsequence as a result, cancelling noise is greater than the subsequence of respective threshold from candidate sequence, candidate pool is constituted；The pedestrian's character representation for carrying out adaptive weighting, obtains the search result based on video sequence.The present invention improves the performance that pedestrian identifies again under irregular sequence, can be widely used for monitoring field, video analysis and other multimedia application, and precision is high, and effect is good, has important market value.
Description
Technical field
The present invention relates to a kind of pedestrians again recognition methods, and in particular to a kind of pedestrian based on irregular video sequence knows again
Other method and system.
Background technique
Pedestrian identifies again, i.e., pedestrian interested is identified under different cameras.In recent years, due to its monitoring field,
Video analysis and key effect in other multimedia application and be widely noticed.It is usually due to inquiry picture and in library picture
It is shot by different camera, therefore, illumination and pedestrian's appearance can have greatly changed, in addition other external rings such as blocking
The interference in border, so that pedestrian, which identifies, becomes a quite challenging job again.Currently, the work identified again based on pedestrian can
To be roughly divided into two kinds: character representation or the effective distance metric model of study with judgement index.Although being based in recent years
The pedestrian of image identifies again has been achieved for significant progress, however, the Limited information having due to single picture, it is difficult to obtain
The external appearance characteristic and space time information of pedestrian's robust, so that this method is difficult to obtain under complex scene identification effect well
Fruit.In actual application, pedestrian is recorded by video, this means that pedestrian has more under each camera
Open continuous image information.Therefore, the performance that pedestrian identifies again is promoted using this sequence image be one naturally select
It selects, this has also pushed directly on the research of the recognition methods again of the pedestrian based on video sequence.
It is identified again compared to the pedestrian based on single image, video sequence has information more abundant.Firstly, video sequence
Column have space time information abundant；Secondly, pedestrian's visual signature in video sequence is more sufficient, so as to construct more Shandong
Pedestrian's appearance of stick indicates；Finally, in the video sequence, it can effectively be mitigated by certain method and be blocked and complex background
Bring influences, and is difficult to overcome background in single picture and blocks the influence to identification.Recently, part work causes
Power identifies work in pedestrian of the research based on video again, these work mainly construct more robust pedestrian using video sequence
Feature or the distance metric method more using the study of pedestrian's video sequence with judgement index.It is existing based on video sequence
Recognition methods ignores noise present in video sequence mostly again, and every image in video sequence is utilized on an equal basis.However,
In practical applications, the parts of images in video sequence has very strong noise jamming mostly, especially blocks and carries on the back
The interference of scape.The present invention by it is this have compared with the video sequence that very noisy interferes be known as irregular video sequence, this sequence pair
It is had a very big impact in the character representation of pedestrian, how effectively to handle irregular video sequence is a skill urgently to be resolved
Art problem.
Summary of the invention
It is directed to conventional method and is difficult to solve the problems, such as noise jamming in video sequence, the invention proposes a kind of feasible
Technical solution, to retain the lesser video sequence of noise, is constructed for detecting and eliminating the irregular subsequence in video
More robust pedestrian's character representation improves retrieval effect.
In order to achieve the above objectives, the technical solution adopted by the present invention provides a kind of pedestrian based on irregular video sequence
Recognition methods again includes the following steps,
Step 1, the segmentation of video sequence, including being extracted from video sequence M more by the stable point in detecting state curve
A continuous subsequence, obtains candidate sequence S；
Step 2, the irregular Sequence Detection based on rarefaction representation, the weight of each subsequence is sought including the use of rarefaction representation
Structure error obtains the noise measurement result of each subsequence；
Step 3, the removal of irregular subsequence, including according to the noise measurement of each subsequence of gained in step 2 as a result, from
Cancelling noise is greater than the subsequence of respective threshold in candidate sequence S, if the subsequence retained is s1,...,sT, constitute candidate pool Q；
Step 4, the pedestrian's character representation for carrying out adaptive weighting, obtains the search result based on video sequence M, including with
Lower sub-step,
Step 4.1, subsequence foundation characteristic indicates, including using corresponding base respectively for each subsequence in candidate pool Q
Plinth character representation, is denoted as f1,...,fN；
Step 4.2, any subsequence s in candidate pool Q is calculatedtWeights omegatIt is as follows,
Wherein, ω*For normalization factor, σtFor subsequence stNoise, t=1 ..., T；
Step 4.3, pedestrian's character representation, including the use of in candidate pool Q subsequence and corresponding weight, weighted calculation row
The final character representation of people, then the character representation of video sequence M is as follows,
Search result is obtained according to the character representation of video sequence M.
Moreover, step 1 includes following sub-step,
Step 1.1, to video sequence M={ I1,I2,...,IN, if IiIt indicates the i-th frame of video, calculates consecutive frame Ii-1
Block information oi, wherein N is frame number, i=1,2,3 ..., N；
Step 1.2, variation of the state of each frame in video sequence M relative to former frame is calculated using stability metric；
Video frame IiStability φiIt is defined as follows,
Wherein, c is constant, and exp is exponential function；
Step 1.3, the sequence stability metric SSM of video sequence M is defined as ε=(φ2,...,φN), if the office in SSM
Portion's maximum value is stationary point；
Step 1.4, if the local maximum in detection SSM curve obtains m stationary point, m sub- sequences are extracted from video sequence M
Column, obtain candidate sequence S={ sj, j=1 ..., m；If j-th of stationary point respective image is It, according to the sub- sequence of stationary point extraction
Arrange sj=(It-L,...,It,...,It+L), wherein ItIndicate that t frame image in video sequence M, L are preset segmentation spacing.
Moreover, step 2 includes following sub-step,
Step 2.1, the building of dictionary, including from subsequence sjIn every frame image extract K topography's block respectively, it is right
In subsequence sjIn k-th of image block in certain image IK=1 ..., K, corresponding dictionary is by except when preceding subsequence sjIt
The image block composition of all video frame same positions in other outer subsequences；
Step 2.2, sparse reconstruct, including by subsequence sjReconstructed error ejIt is defined as follows,
Wherein, ZkFor subsequence sjIn each image k-th of image block formed matrix, XkFor corresponding sparse coefficient；
Step 2.3, by subsequence sjNoise σjIt is defined as σj=exp ((ej)2)。
Moreover, erased noise is more than or equal to the corresponding subsequence of specified threshold θ from candidate sequence S, wherein θ in step 3
It is defined as the intermediate value of all subsequence noises.
Pedestrian's weight identifying system based on irregular video sequence that the present invention also provides a kind of, comprises the following modules,
First module, for the segmentation of video sequence, including passing through the stable point in detecting state curve, from video sequence
M extracts multiple continuous subsequences, obtains candidate sequence S；
Second module seeks every height including the use of rarefaction representation for the irregular Sequence Detection based on rarefaction representation
The reconstructed error of sequence obtains the noise measurement result of each subsequence；
Third module, for the removal of irregular subsequence, including the noise according to each subsequence of gained in the first module
Measurement results, cancelling noise is greater than the subsequence of respective threshold from candidate sequence S, if the subsequence retained is s1,...,sT,
Constitute candidate pool Q；
4th module obtains the retrieval knot based on video sequence M for carrying out pedestrian's character representation of adaptive weighting
Fruit, including with lower unit,
First unit is indicated for subsequence foundation characteristic, including using phase respectively for each subsequence in candidate pool Q
It answers foundation characteristic to indicate, is denoted as f1,...,fN；
Second unit, for calculating any subsequence s in candidate pool QtWeights omegatIt is as follows,
Wherein, ω*For normalization factor, σtFor subsequence stNoise, t=1 ..., T；
Third unit, be used for pedestrian's character representation, including the use of in candidate pool Q subsequence and corresponding weight, weighting
Calculating the final character representation of pedestrian, then the character representation of video sequence M is as follows,
Search result is obtained according to the character representation of video sequence M.
Moreover, the first module include with lower unit,
First unit, for video sequence M={ I1,I2,...,IN, if IiIt indicates the i-th frame of video, calculates adjacent
Frame Ii-1Block information oi, wherein N is frame number, i=1,2,3 ..., N；
Second unit, for calculating the state of each frame in video sequence M using stability metric relative to former frame
Variation；Video frame IiStability φiIt is defined as follows,
Wherein, c is constant, and exp is exponential function；
Third unit, the sequence stability metric SSM for video sequence M are defined as ε=(φ2,...,φN), if SSM
In local maximum be stationary point；
Unit the 4th obtains m stationary point for setting the local maximum in detection SSM curve, extracts m from video sequence M
A subsequence obtains candidate sequence S={ sj, j=1 ..., m；If j-th of stationary point respective image is It, extracted according to the stationary point
Subsequence sj=(It-L,...,It,...,It+L), wherein ItIndicate that t frame image in video sequence M, L are preset segmentation
Spacing.
Moreover, the second module include with lower unit,
First unit, for the building of dictionary, including from subsequence sjIn every frame image extract K topography respectively
Block, for subsequence sjIn k-th of image block in certain image IK=1 ..., K, corresponding dictionary is by except when preceding sub- sequence
Arrange sjExcept other subsequences in all video frame same positions image block composition；
Second unit is used for sparse reconstruct, including by subsequence sjReconstructed error ejIt is defined as follows,
Wherein, ZkFor subsequence sjIn each image k-th of image block formed matrix, XkFor corresponding sparse coefficient；
Third unit is used for subsequence sjNoise σjIt is defined as σj=exp ((ej)2)。
Moreover, erased noise is more than or equal to the corresponding subsequence of specified threshold θ from candidate sequence S in third module,
Middle θ is defined as the intermediate value of all subsequence noises.
The present invention has the positive effect that and advantage:
1, the present invention is multiple with subsequence by extracting in the stability metric method of sequence from video sequence.
2, the present invention is detected and is removed the biggish subsequence of noise based on the method for sparse reconstruct using a kind of, thus structure
Build the lesser candidate pool of noise.
3, the present invention comprehensively utilizes subsequence all in candidate pool and carrys out structure using a kind of adaptive weight calculation scheme
Clarification of objective expression is built, so that target signature is more robust and has judgement index.
4, the present invention improves the performance that pedestrian identifies again under irregular sequence, can be widely used for monitoring field, video point
Analysis and other multimedia application, precision is high, and effect is good, has important market value.
Detailed description of the invention
Fig. 1 is the flow chart of the embodiment of the present invention.
Specific embodiment
Firstly, the pedestrian of traditional video based on image again recognition methods due to its limited information, it is difficult to overcome row
People blocks, background interference etc. pedestrian under complex scenes identify problem again.And the identification again based on video sequence can use
Its visual information abundant and space-time characteristic, effective solution is blocked and the influence pedestrians such as background interference weigh asking for recognition effect
Topic.
It is done secondly, the existing recognition methods again based on video sequence ignores noise present in video sequence mostly
It disturbs, same utilizes all information in video.And this noise is generally existing and identifies have very again for pedestrian
Big negative influence.
It is different from the consistency that conventional method assumes sequence of pictures, the present invention pays close attention to the noise in video sequence for knowing again
Other performance bring adverse effect.Observation for image state differentiation in video sequence, the present invention are found through experiments that
A universal phenomenon present in sequence: the variation of sequence image state has certain rule, the change of dbjective state in sequence
Change the stability for reflecting target to a certain extent.
A kind of recognition methods again of the pedestrian based on irregular video sequence proposed by the present invention, including having for irregular sequence
Effect detection and removal.Firstly, video sequence has time continuity, therefore the removal and detection of noise image should be based on continuous
The measurement of video image, i.e. video noise should be based on the subsequence of video.It is worth noting that, video that the present invention herein refers to
Sequence refers to that the image in the same subsequence should have similar state (its noise size of state instruction).
Further, to the valid metric of dbjective state in sequence, the present invention utilizes blocking between consecutive frame in sequence
Information measures the stability of its sequence, effectively realizes that the subsequence of video sequence is extracted using its stability change, thus
So that the image state having the same in same subsequence.
It further, is the detection and removal for realizing irregular subsequence, the present invention is using a kind of based on rarefaction representation
The noise of method sub-sequences is measured, and the size of its noise is characterized using the reconstructed error of subsequence.It makes an uproar finally, rejecting
The excessive subsequence of sound constructs the lesser target candidate pond of noise.
It further, is the effective building for realizing pedestrian's character representation, the present invention is calculated often using the size of its noise
The weight of a subsequence, it is final to comprehensively utilize all subsequences in candidate pool to weight pedestrian's character representation of building robust,
It is greatly improved the effect that pedestrian identifies again.The adaptive weighing computation method of one kind proposed by the present invention, can be effective
Comprehensively utilize the lesser all subsequences of noise so that pedestrian's feature that the present invention constructs describe it is more robust and more with sentencing
Other power.
The present invention is capable of the dry of effective less noise by detecting and eliminating the biggish subsequence of noise in video sequence
It disturbs, to be greatly improved the effect that pedestrian identifies again.
Referring to Fig. 1, a kind of recognition methods again of the pedestrian based on irregular video sequence that embodiment provides is specifically included:
Step 1, the segmentation of video sequence.The state change of image has the continuity in time domain, therefore this hair in video
It is bright to propose a kind of sequence stability metric method, to effectively indicate the state change of pedestrian in video sequence.Pass through detection
Stable point in condition curve, the present invention extract multiple continuous subsequences from video sequence, the row in the same subsequence
People's image has similar state (noiseless or having noise).
For most of video sequences, noise is larger in the subsequence of part, and remaining subsequence relative noise compared with
Small, therefore, video sequence is divided into multiple subsequences according to the variation of its state by the present invention, thus the figure that will have unified state
As sequences segmentation is unified subsequence.The step 1 of embodiment is specific as follows:
Step 1.1: to video sequence M={ I1,I2,...,IN, if IiThe i-th frame for indicating video, calculates its consecutive frame
Ii-1Block information oi, N is frame number, i=1,2,3 ..., N.
Preferably, embodiment optimizes optical flow computation and occlusion detection as a united optimization problem, from
And a reliable occlusion detection is obtained, specific implementation can be found in Alper Ayvaci, Michalis Raptis, and
Stefano Soatto.2010.Occlusion detection and motion estimation with convex
optimization.In Advances in Neural Information Processing Systems.100–108
Step 1.2: the block information o being calculated in step 1.1iReflection is that interframe is blocked, i.e. oiShow present frame Ii
Compared to former frame Ii-1Circumstance of occlusion.From the perspective of certain, oiIllustrate the shape in video sequence M between successive frame
State variation.Therefore, the present invention proposes a kind of stability metric method to calculate the state of each frame in video sequence M relative to it
The variation of former frame.Video frame IiStability φiIt can be defined as:
Wherein, c is a constant, and exp is exponential function.When it is implemented, empirical value can be used in c, embodiment takes 2.
Step 1.3: at this point, the sequence stability metric (SSM) of video sequence M is defined as: ε=(φ2,...,φN),
It can be observed that the local maximum in SSM, the present invention are called stationary point, i.e., the video frame around stationary point is all having the same
State；
Step 1.4: setting and several stationary points are obtained by the local maximum in detection SSM curve, the present invention is each by extracting
Video frame around stationary point extracts multiple subsequences from video sequence M, obtains collection of candidate sequences S={ sj, j=1 ...,
M, m are the number (i.e. the number in stationary point) of subsequence.If j-th of stationary point respective image is It, according to the sub- sequence of stationary point extraction
Arrange sj=(It-L,...,It,...,It+L), wherein ItIndicate that t frame image, L are preset segmentation spacing, the son in each stationary point
Sequence includes the stationary point and front and back L adjacent video frames.When it is implemented, L can be arranged according to the distance before stationary point,
Or empirical value, such as L is used to take 10.
Step 2, the irregular Sequence Detection based on rarefaction representation.The present invention is using rarefaction representation to each of video
Subsequence carries out its noise measurement, and the size of its noise is indicated with the reconstructed error of each subsequence.
Subsequence is extracted from video sequence by the front present invention, the picture in same subsequence has similar state
(noiseless or having noise).On this basis, the step 2 of embodiment detects irregular subsequence using rarefaction representation, tool
It is divided into following steps for body:
Step 2.1: the building of dictionary, for subsequence sj, the present invention is every wherein with certain spatial mesh structure
A video frame picture I (including It-L,...,It,...,It+L) in extract K topography's block respectively, it is such as that every frame picture is uniform
The grid for being divided into 10 × 10, obtain K=100 topography's block.These image blocks are used to construct corresponding dictionary.Tool
Body is said, for subsequence sjAny image I in k-th of image blockK=1 ..., K, corresponding dictionary are as follows:That is,Dictionary be by except when preceding subsequence sjExcept other subsequences in
The image block of all video frame same positions forms.It must be noted that the state of image has height in the same subsequence
Similitude, therefore the dictionary of building corresponding image block cannot be used to, the present invention uses each video frame in other subsequences
The image block of same position, dictionary are stablized, and not will receive the interference of noise.
Step 2.2: sparse reconstruct, for subsequence sjAny image I in arbitrary image blockSimplification is denoted as image
Block P, sparse representation model can indicate are as follows:
Wherein, β indicates sparse coefficient, and λ is regularization factors, and D is corresponding dictionary, then the reconstructed error of image block P at this time
It is defined as:
Correspondingly, the present invention is by subsequence sjReconstructed error ejIt is defined as follows:
Wherein, ZkFor subsequence sjIn each image the matrix that is formed of k-th of image block, including 2L+1 image block, each column
It is all pixels of an image block by the result being from left to right unfolded from top to bottom；XkIt is 2L+1 for corresponding sparse coefficient
The vector that the rarefaction representation calculated result of image block is constituted.
Step 2.3: 2.2 reconstructed error for having obtained each subsequence through the above steps, the present invention is by subsequence sj's
Noise σjIs defined as:
σj=exp ((ej)2)；
Step 3, the removal of irregular subsequence.According to the noise size of subsequence each in step 2, the present invention rejects those
Noise is greater than the subsequence of specified threshold, retains the lesser subsequence of noise.
In above-mentioned steps 2.3, it can be observed that sjNoise reconstructed error ejIt is bigger, then its noise σjIt is bigger, then sub- sequence
Arrange sjThe poorer therefore deleted probability of availability should be bigger, step 3 specific steps of embodiment are as follows:
Step 3.1: erased noise is more than or equal to the corresponding subsequence of specified threshold θ from collection of candidate sequences S, wherein θ
Is defined as:
θ=median (σj), j=1 ..., m
That is the threshold θ intermediate value that is defined as all subsequence noises；
Step 3.2: setting the lesser subsequence of noise retained after step 3.1 processing and (meet σjThe subsequence of < θ) be
s1,...,sT, construct candidate pool Q={ s1,...,sT, T indicates the lesser subsequence number of noise retained.This step basis
It is retaining after step 3.1 processing as a result, the lesser subsequence of noise is constructed candidate pool.
Step 4, the pedestrian's character representation for carrying out adaptive weighting, obtains the search result based on video sequence M.The present invention
To the lesser subsequence of the noise of reservation, its feature weight of adaptive calculating is finally added the feature of each subsequence
Power fusion, to construct more robust pedestrian's character representation.
In above-mentioned steps, the present invention deletes the biggish subsequence of noise, is waited using the lesser subsequence building of noise
Scavenger, in order to obtain more sufficient pedestrian's external appearance characteristic, all subsequences of comprehensive candidate pool are constructed pedestrian by the present invention
Character representation.Step 4 specific steps of embodiment are as follows:
Step 4.1: subsequence foundation characteristic indicates that, for each subsequence in candidate pool Q, the present invention uses accordingly respectively
Foundation characteristic f is indicated, is denoted as f respectively1,...,fN.I.e. the feature of Q can indicate are as follows:
FQ={ f1,...,fT}；
When constructing foundation characteristic f, in order to obtain more robust character representation, preferably by the visual signature of sequence
The space-time characteristic (HOG3D) of (color characteristic, textural characteristics etc.) and sequence indicates to integrate building pedestrian's foundation characteristic.Such as
When constructing the visual signature of sequence, color characteristic (such as hsv color histogram) is extracted to every frame picture in sequence first, so
It is indicated afterwards using the visual signature of maxpooling building sequence
Step 4.2: calculating any subsequence s in candidate pooltWeight, weights omegatIt is defined as follows:
Wherein, ω*For normalization factor, for ensuring the weight of each subsequence and being 1；σtFor subsequence stNoise, t
=1 ..., T.
Step 4.3: pedestrian's character representation, the present invention using in candidate pool subsequence and its corresponding weight come weight meter
The final character representation of pedestrian is calculated, then the character representation of video sequence M are as follows:
After obtaining the character representation of sequence, it can use existing characteristic measure method and be trained and test.It is surveying
When examination, for each search sequence, the search sequence is calculated using existing measure (such as XQDA, KISSME) and in library
The characteristic distance of pedestrian's sequence, finally according to the search result that the search sequence can be obtained apart from ascending sort.
When it is implemented, method provided by the present invention can realize automatic running process based on software technology, mould can also be used
Block mode realizes corresponding system.
Pedestrian's weight identifying system based on irregular video sequence that the present invention also provides a kind of, comprises the following modules,
First module, for the segmentation of video sequence, including passing through the stable point in detecting state curve, from video sequence
M extracts multiple continuous subsequences, obtains candidate sequence S；
Second module seeks every height including the use of rarefaction representation for the irregular Sequence Detection based on rarefaction representation
The reconstructed error of sequence obtains the noise measurement result of each subsequence；
Third module, for the removal of irregular subsequence, including the noise according to each subsequence of gained in the first module
Measurement results, cancelling noise is greater than the subsequence of respective threshold from candidate sequence S, if the subsequence retained is s1,...,sT,
Constitute candidate pool Q；
4th module obtains the retrieval knot based on video sequence M for carrying out pedestrian's character representation of adaptive weighting
Fruit, including with lower unit,
First unit is indicated for subsequence foundation characteristic, including using phase respectively for each subsequence in candidate pool Q
It answers foundation characteristic to indicate, is denoted as f1,...,fN；
Second unit, for calculating any subsequence s in candidate pool QtWeights omegatIt is as follows,
Wherein, ω*For normalization factor, σtFor subsequence stNoise, t=1 ..., T；
Third unit, be used for pedestrian's character representation, including the use of in candidate pool Q subsequence and corresponding weight, weighting
Calculating the final character representation of pedestrian, then the character representation of video sequence M is as follows,
Search result is obtained according to the character representation of video sequence M.
Each module specific implementation can be found in corresponding steps, and it will not go into details by the present invention.
Specific embodiment described herein only illustrates that spirit of the invention.Technology belonging to the present invention
The technical staff in field can do various modifications or additions to described specific embodiment or use similar side
Formula substitution, however, it does not deviate from the spirit of the invention or beyond the scope of the appended claims.
Claims (6)
1. a kind of recognition methods again of the pedestrian based on irregular video sequence, it is characterised in that: include the following steps,
Step 1, the segmentation of video sequence, including extracting multiple companies from video sequence M by the stable point in detecting state curve
Continuous subsequence obtains candidate sequence S；
Step 2, the irregular Sequence Detection based on rarefaction representation is missed including the use of the reconstruct that rarefaction representation seeks each subsequence
Difference obtains the noise measurement result of each subsequence；
Step 3, the removal of irregular subsequence, including according to the noise measurement of each subsequence of gained in step 2 as a result, from candidate
Cancelling noise is greater than the subsequence of respective threshold in sequence S, if the subsequence retained is s1,...,sT, constitute candidate pool Q；
Step 4, the pedestrian's character representation for carrying out adaptive weighting obtains the search result based on video sequence M, including following son
Step,
Step 4.1, subsequence foundation characteristic indicates, including using corresponding basis special respectively each subsequence in candidate pool Q
Sign indicates, is denoted as f1,...,fN；
Step 4.2, any subsequence s in candidate pool Q is calculatedtWeights omegatIt is as follows,
Wherein, ω*For normalization factor, σtFor subsequence stNoise, t=1 ..., T；
Step 4.3, pedestrian's character representation, including the use of in candidate pool Q subsequence and corresponding weight, weighted calculation pedestrian is most
Whole character representation, then the character representation of video sequence M is as follows,
Search result is obtained according to the character representation of video sequence M；
Wherein, step 1 includes following sub-step,
Step 1.1, to video sequence M={ I1,I2,...,IN, if IiIt indicates the i-th frame of video, calculates consecutive frame Ii-1Block
Information οi, wherein N is frame number, i=1,2,3 ..., N；
Step 1.2, variation of the state of each frame in video sequence M relative to former frame is calculated using stability metric；Video
Frame IiStability φiIt is defined as follows,
Wherein, c is constant, and exp is exponential function；
Step 1.3, the sequence stability metric SSM of video sequence M is defined as ε=(φ2,...,φN), if the part in SSM is most
Big value is stationary point；
Step 1.4, if the local maximum in detection SSM curve obtains m stationary point, m subsequence is extracted from video sequence M,
Obtain candidate sequence S={ sj, j=1 ..., m；If j-th of stationary point respective image is It, according to the subsequence of stationary point extraction
sj=(It-L,...,It,...,It+L), wherein ItIndicate that t frame image in video sequence M, L are preset segmentation spacing.
2. pedestrian's recognition methods again according to claim 1 based on irregular video sequence, it is characterised in that: step 2 packet
Following sub-step is included,
Step 2.1, the building of dictionary, including from subsequence sjIn every frame image extract K topography's block respectively, for son
Sequence sjIn k-th of image block in certain image ICorresponding dictionary is by except when preceding subsequence sjExcept
The image block composition of all video frame same positions in other subsequences；
Step 2.2, sparse reconstruct, including by subsequence sjReconstructed error ejIt is defined as follows,
Wherein, ZkFor subsequence sjIn each image k-th of image block formed matrix, XkFor corresponding sparse coefficient；
Step 2.3, by subsequence sjNoise σjIt is defined as σj=exp ((ej)2)。
3. the recognition methods again of the pedestrian based on irregular video sequence according to claim 1 or claim 2, it is characterised in that: step 3
In, erased noise is more than or equal to the corresponding subsequence of specified threshold θ from candidate sequence S, and wherein θ is defined as all subsequences and makes an uproar
The intermediate value of sound.
4. a kind of pedestrian's weight identifying system based on irregular video sequence, it is characterised in that: it comprises the following modules,
First module, for the segmentation of video sequence, including being mentioned from video sequence M by the stable point in detecting state curve
Multiple continuous subsequences are taken, candidate sequence S is obtained；
Second module seeks each subsequence including the use of rarefaction representation for the irregular Sequence Detection based on rarefaction representation
Reconstructed error, obtain the noise measurement result of each subsequence；
Third module, for the removal of irregular subsequence, including the noise measurement according to each subsequence of gained in the first module
As a result, cancelling noise is greater than the subsequence of respective threshold from candidate sequence S, if the subsequence retained is s1,...,sT, constitute
Candidate pool Q；
4th module obtains the search result based on video sequence M, wraps for carrying out pedestrian's character representation of adaptive weighting
It includes with lower unit,
First unit is indicated for subsequence foundation characteristic, including using corresponding base respectively for each subsequence in candidate pool Q
Plinth character representation, is denoted as f1,...,fN；
Second unit, for calculating any subsequence s in candidate pool QtWeights omegatIt is as follows,
Wherein, ω*For normalization factor, σtFor subsequence stNoise, t=1 ..., T；
Third unit, be used for pedestrian's character representation, including the use of in candidate pool Q subsequence and corresponding weight, weighted calculation
The final character representation of pedestrian, then the character representation of video sequence M is as follows,
Search result is obtained according to the character representation of video sequence M；
Wherein, the first module include with lower unit,
First unit, for video sequence M={ I1,I2,...,IN, if IiIt indicates the i-th frame of video, calculates consecutive frame Ii-1
Block information οi, wherein N is frame number, i=1,2,3 ..., N；
Second unit, for calculating variation of the state of each frame in video sequence M relative to former frame using stability metric；
Video frame IiStability φiIt is defined as follows,
Wherein, c is constant, and exp is exponential function；
Third unit, the sequence stability metric SSM for video sequence M are defined as ε=(φ2,...,φN), if in SSM
Local maximum is stationary point；
Unit the 4th obtains m stationary point for setting the local maximum in detection SSM curve, extracts m son from video sequence M
Sequence obtains candidate sequence S={ sj, j=1 ..., m；If j-th of stationary point respective image is It, according to the son of stationary point extraction
Sequence sj=(It-L,...,It,...,It+L), wherein ItIndicate t frame image in video sequence M, L is between preset segmentation
Away from.
5. pedestrian's weight identifying system based on irregular video sequence according to claim 4, it is characterised in that: the second module
Including with lower unit,
First unit, for the building of dictionary, including from subsequence sjIn every frame image extract K topography's block respectively, it is right
In subsequence sjIn k-th of image block in certain image ICorresponding dictionary is by except when preceding subsequence sjIt
The image block composition of all video frame same positions in other outer subsequences；
Second unit is used for sparse reconstruct, including by subsequence sjReconstructed error ejIt is defined as follows,
Wherein, ZkFor subsequence sjIn each image k-th of image block formed matrix, XkFor corresponding sparse coefficient；
Third unit is used for subsequence sjNoise σjIt is defined as σj=exp ((ej)2)。
6. pedestrian's weight identifying system according to claim 4 or 5 based on irregular video sequence, it is characterised in that: third
In module, erased noise is more than or equal to the corresponding subsequence of specified threshold θ from candidate sequence S, and wherein θ is defined as all sub- sequences
The intermediate value of column noise.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710572155.6A CN107273873B (en) | 2017-07-13 | 2017-07-13 | Pedestrian based on irregular video sequence recognition methods and system again |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710572155.6A CN107273873B (en) | 2017-07-13 | 2017-07-13 | Pedestrian based on irregular video sequence recognition methods and system again |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN107273873A CN107273873A (en) | 2017-10-20 |
| CN107273873B true CN107273873B (en) | 2019-09-17 |
Family
ID=60073090
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710572155.6A Active CN107273873B (en) | 2017-07-13 | 2017-07-13 | Pedestrian based on irregular video sequence recognition methods and system again |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107273873B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109145744B (en) * | 2018-07-19 | 2020-08-18 | 西安理工大学 | LSTM network pedestrian re-identification method based on self-adaptive prediction mode |
| CN109101913A (en) * | 2018-08-01 | 2018-12-28 | 北京飞搜科技有限公司 | Pedestrian recognition methods and device again |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103984915A (en) * | 2014-02-28 | 2014-08-13 | 中国计量学院 | Pedestrian re-recognition method in monitoring video |
| CN104298992A (en) * | 2014-10-14 | 2015-01-21 | 武汉大学 | Self-adaptive scale pedestrian re-identification method based on data driving |
| CN104537686A (en) * | 2014-12-12 | 2015-04-22 | 中国科学院自动化研究所 | Tracing method and device based on target space and time consistency and local sparse representation |
| CN105989358A (en) * | 2016-01-21 | 2016-10-05 | 中山大学 | Natural scene video identification method |
| CN106254722A (en) * | 2016-07-15 | 2016-12-21 | 北京邮电大学 | A kind of video super-resolution method for reconstructing and device |
| CN106846378A (en) * | 2017-01-23 | 2017-06-13 | 中山大学 | Across video camera object matching and tracking that a kind of combination topology of spacetime is estimated |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7218756B2 (en) * | 2004-03-24 | 2007-05-15 | Cernium, Inc. | Video analysis using segmentation gain by area |
| US9600738B2 (en) * | 2015-04-07 | 2017-03-21 | Xerox Corporation | Discriminative embedding of local color names for object retrieval and classification |
-
2017
- 2017-07-13 CN CN201710572155.6A patent/CN107273873B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103984915A (en) * | 2014-02-28 | 2014-08-13 | 中国计量学院 | Pedestrian re-recognition method in monitoring video |
| CN104298992A (en) * | 2014-10-14 | 2015-01-21 | 武汉大学 | Self-adaptive scale pedestrian re-identification method based on data driving |
| CN104537686A (en) * | 2014-12-12 | 2015-04-22 | 中国科学院自动化研究所 | Tracing method and device based on target space and time consistency and local sparse representation |
| CN105989358A (en) * | 2016-01-21 | 2016-10-05 | 中山大学 | Natural scene video identification method |
| CN106254722A (en) * | 2016-07-15 | 2016-12-21 | 北京邮电大学 | A kind of video super-resolution method for reconstructing and device |
| CN106846378A (en) * | 2017-01-23 | 2017-06-13 | 中山大学 | Across video camera object matching and tracking that a kind of combination topology of spacetime is estimated |
Non-Patent Citations (1)
| Title |
|---|
| "Person Re-Identification by Discriminative Selection in Video Ranking";Taiqing Wang et al.;《IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE》;20160127;摘要、第2-3章 |
Also Published As
| Publication number | Publication date |
|---|---|
| CN107273873A (en) | 2017-10-20 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108921107B (en) | Pedestrian re-identification method based on sequencing loss and Simese network | |
| CN107067413B (en) | A kind of moving target detecting method of time-space domain statistical match local feature | |
| CN109145708B (en) | Pedestrian flow statistical method based on RGB and D information fusion | |
| CN110097115B (en) | Video salient object detection method based on attention transfer mechanism | |
| CN110120064B (en) | Depth-related target tracking algorithm based on mutual reinforcement and multi-attention mechanism learning | |
| CN110163041A (en) | Video pedestrian recognition methods, device and storage medium again | |
| CN108280421B (en) | Human behavior recognition method based on multi-feature depth motion map | |
| CN114241517B (en) | Cross-mode pedestrian re-recognition method based on image generation and shared learning network | |
| CN109472191A (en) | A kind of pedestrian based on space-time context identifies again and method for tracing | |
| Karaman et al. | Human daily activities indexing in videos from wearable cameras for monitoring of patients with dementia diseases | |
| CN104376334B (en) | A kind of pedestrian comparison method of multi-scale feature fusion | |
| CN106157330B (en) | Visual tracking method based on target joint appearance model | |
| CN109544592A (en) | For the mobile moving object detection algorithm of camera | |
| CN104063871B (en) | The image sequence Scene Segmentation of wearable device | |
| CN109271932A (en) | Pedestrian based on color-match recognition methods again | |
| CN109063549A (en) | High-resolution based on deep neural network is taken photo by plane video moving object detection method | |
| CN112580525A (en) | Case activity track monitoring method based on pedestrian re-identification | |
| CN114241511A (en) | Weak supervision pedestrian detection method, system, medium, equipment and processing terminal | |
| Zhu et al. | Action recognition in broadcast tennis video using optical flow and support vector machine | |
| CN107273873B (en) | Pedestrian based on irregular video sequence recognition methods and system again | |
| CN113033468A (en) | Specific person re-identification method based on multi-source image information | |
| CN114550208A (en) | Cross-modal pedestrian re-identification method based on global level and local level combined constraint | |
| CN116977937A (en) | Pedestrian re-identification method and system | |
| Singh et al. | Animal localization in camera-trap images with complex backgrounds | |
| Huo et al. | 3DVSD: An end-to-end 3D convolutional object detection network for video smoke detection |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |