CN106204477B - Video frequency sequence background restoration methods based on online low-rank background modeling - Google Patents
Video frequency sequence background restoration methods based on online low-rank background modeling Download PDFInfo
- Publication number
- CN106204477B CN106204477B CN201610526248.0A CN201610526248A CN106204477B CN 106204477 B CN106204477 B CN 106204477B CN 201610526248 A CN201610526248 A CN 201610526248A CN 106204477 B CN106204477 B CN 106204477B
- Authority
- CN
- China
- Prior art keywords
- background
- frame
- video
- matrix
- indicate
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 64
- 239000011159 matrix material Substances 0.000 claims abstract description 59
- 239000013598 vector Substances 0.000 claims abstract description 47
- 238000011084 recovery Methods 0.000 claims abstract description 19
- 238000013507 mapping Methods 0.000 claims abstract description 15
- 238000005457 optimization Methods 0.000 claims abstract description 13
- 238000006467 substitution reaction Methods 0.000 claims abstract description 7
- 238000000354 decomposition reaction Methods 0.000 claims description 16
- 230000008602 contraction Effects 0.000 claims description 9
- 230000003287 optical effect Effects 0.000 claims description 6
- NAWXUBYGYWOOIX-SFHVURJKSA-N (2s)-2-[[4-[2-(2,4-diaminoquinazolin-6-yl)ethyl]benzoyl]amino]-4-methylidenepentanedioic acid Chemical compound C1=CC2=NC(N)=NC(N)=C2C=C1CCC1=CC=C(C(=O)N[C@@H](CC(=C)C(O)=O)C(O)=O)C=C1 NAWXUBYGYWOOIX-SFHVURJKSA-N 0.000 claims description 4
- 230000006870 function Effects 0.000 description 24
- 238000002474 experimental method Methods 0.000 description 6
- 238000012545 processing Methods 0.000 description 4
- 230000000694 effects Effects 0.000 description 3
- 230000017105 transposition Effects 0.000 description 3
- 238000009795 derivation Methods 0.000 description 2
- 238000001514 detection method Methods 0.000 description 2
- 230000007812 deficiency Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000000513 principal component analysis Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
Classifications
-
- G06T5/73—
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
Abstract
The invention belongs to video analysis fields, to realize that the background to video sequence carries out accurately online recovery.The technical solution adopted by the present invention is that; video frequency sequence background restoration methods based on online low-rank background modeling; the low-rank matrix that nuclear norm is introduced on the basis of traditional background modeling is decomposed and the motion information of binaryzation is estimated, to solve the problems, such as that prior art can not be handled.The present invention includes the following steps: 1) to be specifically expressed as background recovery problem in video sequence to solve following unconstrained optimization equation: 2) constructing kth frame movement mapping weight vectors wk: 3) it is solved using alternating direction method: 4) being solved5) it solves6) it repeats the above steps 4), 5) until algorithmic statement；7) it is solved using substitution of variable: 8) being solved9) it solves10) it repeats the above steps 8), 9) until algorithmic statement；11) video background is solved.Present invention is mainly applied to video analysis occasions.
Description
Technical field
The invention belongs to video analysis fields.In particular to the video frequency sequence background based on online low-rank background modeling restores
Method.
Background technique
With the fast development of internet and mass media, the availability of video data increases sharply, far beyond people
Manual analyzing range.Therefore, interested information is excavated in a large amount of video using automatic video frequency analysis has extremely
Important meaning.Background recovery is widely used in many views as a preconditioning technique for extracting interesting target in video
Feel in application, such as target detection, target following and Activity recognition.
In recent years, there are many research achievements about the method for solving the problems, such as video background recovery, it is a kind of most important
Method is the method based on Robust Principal Component Analysis (RPCA), the general idea is that: it is assumed that being linear phase between video background frame
Close, video is resolved into low-rank background and sparse prospect two parts, wherein for low-rank background approximation using nuclear norm come
Constraint.Such methods can be realized simultaneously the recovery of video background and the detection of sport foreground.But it solves nuclear norm to require pair
All video frames carry out singular value decomposition simultaneously, and therefore, limitation the method can only be solved in a manner of batch processing, Wu Fashi
Now real-time background recovery.In addition, every iteration is primary when solving such method, require at all video frames
Reason, causes algorithm time and memory efficient lower, is not suitable for stream-type video and big video.So in the epoch of big data
Under background, it is very necessary for finding a kind of effective online background recovery method.
At this stage, for above-mentioned traditional batch processing RPCA method the shortcomings that, some online back have been proposed both at home and abroad
Scape restoration methods, it can processing in real time is carried out to single frame video image and obtains its corresponding background.Its thinking is: to limitation
The nuclear norm item of batch processing implementation carries out low-rank matrix decomposition, obtains the base and coefficient corresponding with base phase of background.Herein
On the basis of, the decomposition of matrix low-rank sparse is carried out to each frame video image respectively, the background finally restored is to solve obtained base
With the product of coefficient transposition.This kind of online restoration methods can restore the background of each frame image in real time, greatly improve
The memory efficient of algorithm and the applicability of streaming data.But because it ignores the motion information of video, in prospect
Slow region is moved, smear effect is easy to appear in the background of recovery.To overcome this disadvantage, the present invention uses in a model
Optical flow algorithm estimates motion information.
Summary of the invention
The invention is intended to make up the deficiencies in the prior art, that is, realize the background of video sequence is carried out it is accurately online extensive
It is multiple.The technical solution adopted by the present invention is that the video frequency sequence background restoration methods based on online low-rank background modeling, traditional
The low-rank matrix that nuclear norm is introduced on the basis of background modeling is decomposed and the motion information of binaryzation is estimated, to solve prior art
The problem of can not handling.The present invention includes the following steps:
1) background recovery problem in video sequence is specifically expressed as solving following unconstrained optimization equation:
Wherein | | | |FThis black norm of the not Luo Beini of representing matrix, | | | |*The nuclear norm of representing matrix, | | | |1
One norm of representing matrix, ο indicate that the point multiplication operation of two matrixes, D are that actual video sequence frame successively presses column vector arrangement
Made of matrix, B indicates background to be restored, and E represents foreground part, and W indicates that two-value sports ground maps weight matrix, λ1, λ2Point
Not Biao Shi in video low-rank background and prospect weight coefficient；
To solve nuclear norm needs singular value decomposition SVD is carried out to all frames together, low-rank square is carried out to background B to be restored
Battle array is decomposed:
Wherein inf { } expression takes infimum to { }, and L is the base of the corresponding lower-dimensional subspace of video frequency sequence background, and C is
Video sequence corresponds to the coefficient matrix of base L；Obtain following decomposition model:
When solving this equation, using alternate optimization method, following loss function is minimized with incremental mode:
Wherein n indicates the current totalframes of video sequence, dkIndicate the column vector of video sequence kth frame,It is
K frame image dkThe corresponding loss function at base L, is defined as follows:
Wherein | | | |2Indicate two norms of vector, w indicates present frame dkMovement map weight vectors, c is dkIn base L
Under coefficient vector, e be present frame foreground part；
Solution procedure is, online to each frame image dkSuccessively: estimation motion information calculates kth frame movement mapping weight
Vector wk, update kth frame coefficient vector ck, update kth frame prospect ek, the corresponding base L of k frame before updatingk；
2) building kth frame movement mapping weight vectors wk: w is solved using backward motion estimation scheme onlinek；
It is based on sparse prospect it is assumed that two-value movement weight is all assigned a value of 1 to first frame；For subsequent frame, respectively
It is then that movement is reflected by its binaryzation using the dense sports ground of optical flow algorithm estimation present frame using first frame as reference frame
It penetrates；
3) equation (5) following sequence is converted into using alternating direction method ADM to solve:
In above formulaIndicate the value of variable c when being minimized objective function,Expression makes
The value of variable e when objective function is minimized, l are the number of iterations；Then method according to step 4), 5) is iterated solution
Obtain coefficient vector ckWith prospect ek；
4) it solvesClosed solutions by solving least square problem acquire coefficient
Remove and solves kth frame coefficient vector c in formula (7)kObjective function in ckUnrelated item obtains following equation:
It is acquired using least square methodSolution are as follows:
WhereinIndicate movement mapping wkDiagonal matrix, i.e., by vector wkEach element be successively put into matrix's
On leading diagonal, I indicates unit matrix；
5) it solvesKth frame prospect is acquired using contraction operator
Remove and solves kth frame prospect e in formula (7)kObjective function in ekUnrelated item obtains following equation:
Formula (10) is solved using contraction operator:
Wherein eik, sik, wikRespectively indicate ek, sk, wkIth pixel value,
6) repeat the above steps 4), 5) until algorithmic statement, the at this moment result of iterationWithIt is exactly present frame dkBe
Number vector ckWith prospect ek, byThe coefficient matrix C of k frame image before obtainingk；
7) equation (3) following unconstrained optimization problem is converted to using substitution of variable to solve:
WhereinFor the variable newly introduced, Rk=[r1,...,rk] indicate the present frame that has acquired and
The background of video frame before, rk=dk-ekIndicate the background of the kth frame image acquired, λ3For weight coefficient；
Equation (12) following sequence is converted into using alternating direction method ADM to solve:
Then method according to step 8), 9) is iterated solution and obtains variable YkWith the base of video sequence lower-dimensional subspace
Lk；
8) it solvesMode acquires pixel-by-pixel
Remove and solves Y in formula (13)kObjective function in YkUnrelated item obtains following equation:
Using least square method, calculates solve pixel-by-pixel:
WhereinIndicate base LkThe i-th row, yikIt indicatesThe element value of i-th row kth column, rikIndicate rkIth pixel
Value；
9) it solvesClosed solutions by solving least square problem acquire base
Remove and solves L in formula (13)kObjective function in LkUnrelated item obtains following equation:
It is acquired using least square methodSolution are as follows:
Further, intermediate variable P is introducedk=RkCk,It is efficiently solved with incremental mode
Wherein:
10) repeat the above steps 8), 9) until algorithmic statement, the at this moment result of iterationExactly preceding k frame image background
Base Lk；
6) and 10) 11) video background B is solved: by the C respectively obtainedkAnd LkIt acquiresBkExactly former problem
Last solution B.
The specific formula of step 2) is:
Wherein wikIndicate that the movement of kth frame image ith pixel maps weight, as wkIth pixel value,WithThe horizontal motion components and vertical motion component of kth frame image ith pixel are respectively indicated, τ is for binaryzation sports ground
Threshold value.
Technical characterstic and effect of the invention:
The method of the present invention restores problem for the background of video sequence online, by introducing the low-rank matrix point to nuclear norm
Solution and the estimation frame by frame to video motion information, realize the solution for restoring problem online to the background of video sequence.The present invention
It has the following characteristics that
1, the advantages of having used alternating direction method (ADM) algorithm, contraction operator to solve subproblem, incorporate existing algorithm.
2, the online recovery to video frequency sequence background is realized.Core is introduced in traditional matrix low-rank sparse decomposition model
The low-rank decomposition of norm can carry out online recovery to background, greatly improve memory efficient, be suitable for streaming video
It is handled with big video data.
3, low-rank matrix is decomposed and motion information estimation combines, the movement letter being introduced into optical flow algorithm estimation video
Breath, and be movement weight matrix by its binaryzation, the sport foreground in video background can be more effectively separated, so that video sequence
Column background recovery result is more accurate.
Detailed description of the invention
Fig. 1 is algorithm flow chart；
Fig. 2 is the background recovery result figure of original browser1 video frame and use the method for the present invention；
Fig. 3 is the background recovery result figure of original browser1 video frame and use the method for the present invention.
Specific embodiment
Low-rank decomposition and the motion information estimation of nuclear norm are introduced in traditional matrix low-rank sparse decomposition model, so that
The accurate background of video sequence, i.e. the video frequency sequence background recovery side based on online low-rank background modeling can be recovered online
Method, to solve the problems, such as that prior art can not be handled.It elaborates with reference to the accompanying drawings and examples to the present invention.
1) background recovery problem in video sequence is specifically expressed as solving following unconstrained optimization equation:
Wherein | | | |FThis black (Frobenius) norm of the not Luo Beini of representing matrix.||·||*The core model of representing matrix
Number.||·||1One norm of representing matrix.ο indicates the point multiplication operation of two matrixes.D is that actual video sequence frame is successively pressed
Matrix made of column vector arrangement.B indicates background to be restored.E represents foreground part.W indicates that two-value sports ground maps weight
Matrix.λ1, λ2Respectively indicate the weight coefficient of low-rank background and prospect in video.
11) all frames are carried out with singular value decomposition (SVD) together to solve nuclear norm needs, the present invention is to background to be restored
B carries out low-rank matrix decomposition:
Wherein inf { } indicates to take { } infimum, and L is the base of lower-dimensional subspace, and C is that video frame corresponds to base L
Coefficient, CTThe transposition of representing matrix C.The matrix low-rank sparse that model (1) is converted into following online motion information auxiliary is decomposed
Model:
12) when solving this equation, the present invention uses alternate optimization method, minimizes following loss function with incremental mode:
Wherein n indicates the current totalframes of video sequence, dkIndicate the column vector of video sequence kth frame,For kth
The corresponding loss function of frame image, is defined as follows:
Wherein | | | |2Two norms of representing matrix.W indicates present frame dkMovement map weight vectors.C is dkIn base L
Under coefficient vector.E is the foreground part of present frame.
13) solution procedure is, online to each frame image dkSuccessively: estimation motion information calculates kth frame movement mapping
Weight vectors wk, update kth frame coefficient vector ck, update kth frame prospect ek, the corresponding base L of k frame before updatingk。
2) building kth frame movement mapping weight vectors wk: w is solved using backward motion estimation scheme onlinek；
21) based on sparse prospect it is assumed that two-value movement weight is all assigned a value of 1 to first frame.
22) for subsequent frame, respectively using first frame as reference frame, using the dense movement of optical flow algorithm estimation present frame
, it is then movement mapping by its binaryzation, specific formula is as follows:
Wherein wikIndicate that the movement of kth frame image ith pixel maps weight, as wkIth pixel value,WithRespectively indicate the horizontal motion components and vertical motion component of kth frame image ith pixel.τ is for binaryzation sports ground
Threshold value, according to the average pixel value of the sports ground of estimation set.
3) present invention solves equation (5) using alternating direction method (ADM), i.e., equation (5) is converted into following sequence and carried out
It solves:
In above formulaIndicate the value of variable c when being minimized objective function,Expression makes
The value of variable e when objective function is minimized.L is the number of iterations.Set each initial parameter values, then according to step 4), 5)
Method be iterated solution, obtain the corresponding coefficient vector c of kth frame imagekWith prospect ek。
4) it solvesClosed solutions by solving least square problem acquire kth frame coefficient
Remove and solves coefficient c in formula (7)kObjective function in ckUnrelated item obtains following equation:
The corresponding base L of k frame image before givenkWith kth frame prospectIt is acquired using least square methodSolution are as follows:
WhereinIndicate the movement map vector w of kth frame imagekDiagonal matrix, i.e., by vector wkEach element according to
It is secondary to be put into matrixLeading diagonal on,Other positions element value be 0.I indicates unit matrix.
5) it solvesKth frame prospect is acquired using contraction operator
Remove and solves e in formula (7)kObjective function in ekUnrelated item obtains following equation:
Given base LkWith iteration result 4)Formula (10) is solved using contraction operator:
Wherein eik, sik, wikRespectively indicate ek, sk, wkIth pixel,
6) repeat the above steps 4), 5) until algorithmic statement, the at this moment result of iterationWithIt is exactly present frame dkBe
Number vector ckWith prospect ek.It willIt is assigned to the row k of C, the coefficient C of k frame image before then obtainingk；
7) substitution of variable and alternating direction method (ADM) solving model (3) are used.
71) because of the base L of direct solution backgroundkIt needs to carry out derivation to matrix dot product, is not easy to solve, so the present invention makes
Equation (3) following unconstrained optimization problem is converted to substitution of variable to solve:
WhereinIt is for convenience of the variable for calculating and newly introducing, Rk=[r1,...,rk] indicate to have asked
The background of the preceding k frame obtained, rk=dk-ekIndicate the background of the kth frame image acquired, λ3For weight coefficient.
72) equation (12) following sequence is converted into using alternating direction method (ADM) to solve:
Set variable YkInitial value, then method according to step 8), 9) is iterated solution and obtains variableAnd view
The base L of frequency sequence lower-dimensional subspacek。
8) it solvesRemove and solves Y in formula (13)kObjective function in YkUnrelated item obtains following equation:
Using least square method, calculates solve pixel-by-pixel:
WhereinIndicate base LkThe i-th row, yikIt indicatesThe element value of i-th row kth column, rikIndicate rkIth pixel
Value.
9) it solvesClosed solutions by solving least square problem acquire base
91) remove and solve L in formula (13)kObjective function in LkUnrelated item obtains following equation:
It is acquired using least square methodSolution are as follows:
92) in order to improve memory efficient, the present invention is further introduced into intermediate variable Pk=RkCk,It is efficiently solved with incremental modeEquation is as follows:
Wherein intermediate variable Pk、XkAnd ZkUpdate rule are as follows:
10) repeat the above steps 8), 9) until algorithmic statement, the at this moment result of iterationExactly preceding k frame image background
Base Lk；
6) and 10) 11) the background B of video sequence is solved: by the C respectively obtainedkAnd LkIt acquiresBkIt is exactly former
Problem last solution B.
The method of the present invention decomposes low-rank matrix and motion information estimation combines, and decomposes in traditional matrix low-rank sparse
The low-rank decomposition of nuclear norm and the motion information weight of two-value are introduced in model, so that solve that prior art can not handle asks
Topic is realized and carries out accurately restoring (experiment flow figure is as shown in Figure 1) online to video frequency sequence background.In conjunction with attached drawing and implementation
Detailed description are as follows for example:
1) the browser1 video frame (as shown in Figure 2) of 115 × 86 pixels is used to input in experiment as initial data, often
One frame image slices vegetarian refreshments is m=115 × 86.Original video frame is processed into column vector form and handled by the present invention, by it
It is arranged in order composition matrix D, restores the video frequency sequence background problem and is specifically expressed as solving following unconstrained optimization equation:
Wherein | | | |FThis black (Frobenius) norm of the not Luo Beini of representing matrix.||·||*The core model of representing matrix
Number.||·||1One norm of representing matrix.ο indicates the point multiplication operation of two matrixes.B indicates background to be restored.Before E is represented
Scape part.W indicates that two-value sports ground maps weight matrix.λ1, λ2Respectively indicate the weight system of low-rank background and prospect in video
Number, value is respectively in an experiment1。
11) need to carry out all frames together singular value decomposition to solve nuclear norm, the present invention carries out low-rank matrix point to B
Solution:
Wherein inf { } indicates to take { } infimum, and L is the base of lower-dimensional subspace, and C is that video frame corresponds to base L
Coefficient, CTThe transposition of representing matrix C.The matrix low-rank sparse that model (1) is converted into following online motion information auxiliary is decomposed
Model:
The order value of L is 5 in experiment.
12) when solving this equation, the present invention uses alternate optimization method, minimizes following loss function with incremental mode:
Wherein n indicates the current totalframes of video sequence, dkIndicate the column vector of video sequence kth frame,For kth
The corresponding loss function of frame image, is defined as follows:
Wherein | | | |2Two norms of representing matrix.W indicates present frame dkMovement map weight vectors.C is dkIn base L
Under coefficient vector.E is the foreground part of present frame.
13) solution procedure is, online to each frame image dkSuccessively: estimation motion information calculates kth frame movement mapping
Weight vectors wk, update kth frame coefficient vector ck, update kth frame prospect ek, the corresponding base L of k frame before updatingk。
2) building kth frame movement mapping weight vectors wk: w is solved using backward motion estimation scheme onlinek。
21) based on sparse prospect it is assumed that two-value movement weight is all assigned a value of 1 to first frame.
22) for subsequent frame, respectively using first frame as reference frame, using the dense movement of optical flow algorithm estimation present frame
, it is then movement mapping by its binaryzation, specific formula is as follows:
Wherein wikIndicate that the movement of kth frame image ith pixel maps weight, as wkIth pixel value,WithRespectively indicate the horizontal motion components and vertical motion component of kth frame image ith pixel.τ is for binaryzation sports ground
Threshold value, set according to the average pixel value of the sports ground of estimation, take τ=0.1 in experiment.
3) present invention solves equation (5) using alternating direction method (ADM), i.e., equation (5) is converted into following sequence and carried out
It solves:
In above formulaIndicate the value of variable c when being minimized objective function.Expression makes
The value of variable e when objective function is minimized.L is the number of iterations.Set each initial parameter values, then according to step 4), 5)
Method be iterated solution, obtain coefficient vector ckWith prospect ek.Initialization in experiment are as follows: l=1；K=1；w1=1；L1The preceding 5 principal component assignment obtained after singular value decomposition are done by ten frame background frames of the video sequence.
4) correspond to base L by solving the closed solutions of least square problem and solvingkCoefficient ck。
Remove and solves coefficient c in formula (7)kObjective function in ckUnrelated item obtains following equation:
The corresponding base L of k frame image before givenkAnd prospectIt is acquired using least square methodSolution are as follows:
WhereinIndicate movement mapping wkDiagonal matrix, i.e., by vector wkEach element be successively put into matrix's
On leading diagonal,Other positions element value be 0.I indicates unit matrix.
5) it solvesKth frame prospect is acquired using contraction operator
Remove and solves e in formula (7)kObjective function in ekUnrelated item obtains following equation:
Given base LkWith iteration result 4)Formula (10) is solved using contraction operator:
Wherein eik, sik, wikRespectively indicate ek, sk, wkIth pixel,
6) repeat the above steps 4), 5) until algorithmic statement, the at this moment result of iterationWithIt is exactly present frame dkBe
Number vector ckWith prospect ek.It willIt is assigned to the row k of C, the coefficient C of k frame image before then obtainingk。
7) substitution of variable and alternating direction method (ADM) solving model (3) are used.
71) because of the base L of direct solution backgroundkIt needs to carry out derivation to matrix dot product, is not easy to solve, so the present invention makes
Equation (3) following unconstrained optimization problem is converted to substitution of variable to solve:
WhereinIt is for convenience of the variable for calculating and newly introducing, Rk=[r1,...,rk] indicate to have asked
The background of the preceding k frame obtained, rk=dk-ekIndicate the background of the kth frame image acquired, λ3For weight coefficient.
72) equation (12) following sequence is converted into using alternating direction method (ADM) to solve:
Variable YkInitial value be set as 0, then method according to step 8), 9) is iterated solution and obtains variableAnd view
The base L of frequency sequence lower-dimensional subspacek。
8) it solvesRemove and solves Y in formula (13)kObjective function in YkUnrelated item obtains following equation:
Using least square method, calculates solve pixel-by-pixel:
WhereinIndicate base LkThe i-th row, yikIt indicatesThe element value of i-th row kth column, rikIndicate rkIth pixel
Value.
9) it solvesClosed solutions by solving least square problem acquire base
91) remove and solve L in formula (13)kObjective function in LkUnrelated item obtains following equation:
It is acquired using least square methodSolution are as follows:
92) in order to improve memory efficient, the present invention is further introduced into intermediate variable Pk=RkCk,It is efficiently solved with incremental modeEquation is as follows:
Wherein intermediate variable Pk、XkAnd ZkUpdate rule are as follows:
Each initial guess is respectively as follows: P0=0, X0=0, Z0=0.
10) repeat the above steps 8), 9) until algorithmic statement, the at this moment result of iterationExactly preceding k frame image background
Base Lk；
6) and 10) 11) B is solved: by the C respectively obtainedkAnd LkIt acquiresBkIt is exactly the last solution B of former problem.
12) each column of B are successively adjusted to the image of 115 × 86 pixels, finally obtain the back of the video sequence of recovery
Scape (as shown in Figure 2).106 × 86 hallmonitor video frame is reused to input as initial data, other parameters are constant,
Obtain the background image (as shown in Figure 3) of its recovery.
Claims (2)
1. a kind of video frequency sequence background restoration methods based on online low-rank background modeling, characterized in that steps are as follows:
1) background recovery problem in video sequence is specifically expressed as solving following unconstrained optimization equation:
Wherein | | | |FThis black norm of the not Luo Beini of representing matrix, | | | |*The nuclear norm of representing matrix, | | | |1It indicates
One norm of matrix,Indicate that the point multiplication operation of two matrixes, D are that actual video sequence frame is successively arranged by column vector
Matrix, B indicates background to be restored, and E represents foreground part, and W indicates that two-value sports ground maps weight matrix, λ1, λ2Table respectively
Show the weight coefficient of low-rank background and prospect in video；
To solve nuclear norm needs singular value decomposition SVD is carried out to all frames together, low-rank matrix point is carried out to background B to be restored
Solution:
Wherein inf { } expression takes infimum to { }, and L is the base of the corresponding lower-dimensional subspace of video frequency sequence background, and C is video
Sequence corresponds to the coefficient matrix of base L；Obtain following decomposition model:
When solving this equation, using alternate optimization method, following loss function is minimized with incremental mode:
Wherein n indicates the current totalframes of video sequence, dkIndicate the column vector of video sequence kth frame,For kth frame figure
As dkThe corresponding loss function at base L, is defined as follows:
Wherein | | | |2Indicate two norms of vector, w indicates present frame dkMovement map weight vectors, c is dkAt base L
Coefficient vector, e are the foreground part of present frame；
Solution procedure is, online to each frame image dkSuccessively: estimation motion information calculates kth frame movement mapping weight vectors
wk, update kth frame coefficient vector ck, update kth frame prospect ek, the corresponding base L of k frame before updatingk；
2) building kth frame movement mapping weight vectors wk: w is solved using backward motion estimation scheme onlinek；
It is based on sparse prospect it is assumed that two-value movement weight is all assigned a value of 1 to first frame；For subsequent frame, respectively with
One frame is as reference frame, is then movement mapping by its binaryzation using the dense sports ground of optical flow algorithm estimation present frame；
3) equation (5) following sequence is converted into using alternating direction method ADM to solve:
In above formulaIndicate the value of variable c when being minimized objective function,Expression makes target letter
The value of variable e when number is minimized, l is the number of iterations；Then method according to step 4), 5) is iterated solution and is
Number vector ckWith prospect ek；
4) it solvesClosed solutions by solving least square problem acquire coefficient
Remove and solves kth frame coefficient vector c in formula (7)kObjective function in ckUnrelated item obtains following equation:
It is acquired using least square methodSolution are as follows:
WhereinIndicate movement mapping wkDiagonal matrix, i.e., by vector wkEach element be successively put into matrixMaster couple
On linea angulata, I indicates unit matrix；
5) it solvesKth frame prospect is acquired using contraction operator
Remove and solves kth frame prospect e in formula (7)kObjective function in ekUnrelated item obtains following equation:
Formula (10) is solved using contraction operator:
Wherein eik, sik, wikRespectively indicate ek, sk, wkIth pixel value,
6) repeat the above steps 4), 5) untilWithThe threshold value of setting is converged to, at this moment the result of iterationWithIt is exactly
Present frame dkCoefficient vector ckWith prospect ek, byThe coefficient matrix C of k frame image before obtainingk；
7) equation (3) following unconstrained optimization problem is converted to using substitution of variable to solve:
WhereinFor the variable newly introduced, Rk=[r1,...,rk] indicate the present frame that has acquired and before
Video frame background, rk=dk-ekIndicate the background of the kth frame image acquired, WkTo map weight by the movement of preceding k frame
The two-value sports ground that vector is constituted maps weight matrix, λ3For weight coefficient；
Equation (12) following sequence is converted into using alternating direction method ADM to solve:
Then method according to step 8), 9) is iterated solution and obtains variable YkWith the base L of video sequence lower-dimensional subspacek；
8) it solvesMode acquires pixel-by-pixel
Remove and solves Y in formula (13)kObjective function in YkUnrelated item obtains following equation:
Using least square method, calculates solve pixel-by-pixel:
WhereinIndicate base LkThe i-th row, yikIt indicatesThe element value of i-th row kth column, rikIndicate rkIth pixel value；
9) it solvesClosed solutions by solving least square problem acquire base
Remove and solves L in formula (13)kObjective function in LkUnrelated item obtains following equation:
It is acquired using least square methodSolution are as follows:
Further, intermediate variable P is introducedk=RkCk,It is efficiently solved with incremental mode
Wherein:
10) repeat the above steps 8), 9) untilWithThe threshold value of setting is converged to, at this moment the result of iterationIt is exactly preceding k
The base L of frame image backgroundk；
6) and 10) 11) video background B is solved: by the C respectively obtainedkAnd LkIt acquiresBkIt is exactly the final of former problem
Solve B.
2. the video frequency sequence background restoration methods as described in claim 1 based on online low-rank background modeling, characterized in that step
Rapid 2) specific formula is:
Wherein wikIndicate that the movement of kth frame image ith pixel maps weight, as wkIth pixel value,WithPoint
Not Biao Shi kth frame image ith pixel horizontal motion components and vertical motion component, τ is threshold for binaryzation sports ground
Value.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610526248.0A CN106204477B (en) | 2016-07-06 | 2016-07-06 | Video frequency sequence background restoration methods based on online low-rank background modeling |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610526248.0A CN106204477B (en) | 2016-07-06 | 2016-07-06 | Video frequency sequence background restoration methods based on online low-rank background modeling |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN106204477A CN106204477A (en) | 2016-12-07 |
| CN106204477B true CN106204477B (en) | 2019-05-31 |
Family
ID=57466399
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610526248.0A Active CN106204477B (en) | 2016-07-06 | 2016-07-06 | Video frequency sequence background restoration methods based on online low-rank background modeling |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106204477B (en) |
Families Citing this family (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106815854B (en) * | 2016-12-30 | 2020-05-15 | 西安交通大学 | On-line video foreground and background separation method based on regular error modeling |
| CN106815858B (en) * | 2016-12-30 | 2020-01-03 | 首都师范大学 | Moving target extraction method and device |
| CN109949252B (en) * | 2019-04-15 | 2020-12-25 | 北京理工大学 | Infrared image light spot removing method based on compensation coefficient fitting |
| CN111626942A (en) * | 2020-03-06 | 2020-09-04 | 天津大学 | Method for recovering dynamic video background based on space-time joint matrix |
| CN111798485B (en) * | 2020-06-30 | 2022-07-19 | 武汉大学 | Event camera optical flow estimation method and system enhanced by IMU |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102881002A (en) * | 2012-07-11 | 2013-01-16 | 天津大学 | Video background recovery method based on movement information and matrix completion |
| CN104867119A (en) * | 2015-05-21 | 2015-08-26 | 天津大学 | Structural lack image filling method based on low rank matrix reconstruction |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8935308B2 (en) * | 2012-01-20 | 2015-01-13 | Mitsubishi Electric Research Laboratories, Inc. | Method for recovering low-rank matrices and subspaces from data in high-dimensional matrices |
-
2016
- 2016-07-06 CN CN201610526248.0A patent/CN106204477B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102881002A (en) * | 2012-07-11 | 2013-01-16 | 天津大学 | Video background recovery method based on movement information and matrix completion |
| CN104867119A (en) * | 2015-05-21 | 2015-08-26 | 天津大学 | Structural lack image filling method based on low rank matrix reconstruction |
Also Published As
| Publication number | Publication date |
|---|---|
| CN106204477A (en) | 2016-12-07 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106204477B (en) | Video frequency sequence background restoration methods based on online low-rank background modeling | |
| CN111814719B (en) | Skeleton behavior recognition method based on 3D space-time diagram convolution | |
| CN112434655B (en) | Gait recognition method based on adaptive confidence map convolution network | |
| CN110287819B (en) | Moving target detection method based on low rank and sparse decomposition under dynamic background | |
| CN105931264B (en) | A kind of sea infrared small target detection method | |
| CN110570481A (en) | calligraphy word stock automatic repairing method and system based on style migration | |
| CN110139046B (en) | Tensor-based video frame synthesis method | |
| CN104867119B (en) | The structural missing image fill method rebuild based on low-rank matrix | |
| US9247139B2 (en) | Method for video background subtraction using factorized matrix completion | |
| CN107229920B (en) | Behavior identification method based on integration depth typical time warping and related correction | |
| CN104732566B (en) | Compression of hyperspectral images cognitive method based on non-separation sparse prior | |
| CN110827312B (en) | Learning method based on cooperative visual attention neural network | |
| CN107154064B (en) | Natural image compressed sensing method for reconstructing based on depth sparse coding | |
| CN107680116A (en) | A kind of method for monitoring moving object in video sequences | |
| CN105761251A (en) | Separation method of foreground and background of video based on low rank and structure sparseness | |
| CN110675422B (en) | Video foreground and background separation method based on generalized non-convex robust principal component analysis | |
| CN106815854A (en) | A kind of Online Video prospect background separation method based on normal law error modeling | |
| CN109887050A (en) | A kind of code aperture spectrum imaging method based on self-adapting dictionary study | |
| CN105957022A (en) | Recovery method of low-rank matrix reconstruction with random value impulse noise deletion image | |
| CN104376533A (en) | Image denoising method based on regularization principal component tracking | |
| CN109658362B (en) | Data recovery method based on supplied core norm | |
| CN104820967A (en) | On-orbit calculation imaging method | |
| CN110826500A (en) | Method for estimating 3D human body posture based on antagonistic network of motion link space | |
| CN110113607A (en) | A kind of compressed sensing video method for reconstructing based on part and non-local constraint | |
| Sagonas et al. | Recovering joint and individual components in facial data |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |