CN102547477B - Video fingerprint method based on contourlet transformation model - Google Patents

Abstract

The invention discloses a video fingerprint method based on a contourlet transformation model, which is characterized by comprising the following steps of: standardizing a piece of video and segmenting each image; transforming each image block through a multi-scale and multi-directional contourlet, and using a hidden Markov tree technology to extract model parameters; decomposing singular values of a standard variance matrix of each block, extracting the maximum singular value, and cascading and normalizing the singular value to construct a video fingerprint vector; using a two-step search strategy in the fingerprint matching stage; and randomly selecting the fingerprint of each frame in the video to be checked, rapidly determining candidate video, and then measuring the similarity of the video to be tested and the candidate video according to the Euclidean distance. The stability, identification efficiency and search efficiency of video fingerprints can be improved, and the method can be widely applied in copy detection, retrieval, identification and copyright protection of video files.

Description

A kind of video fingerprint method based on profile wave convert model

Technical field

The invention belongs to multimedia information field, be specifically related to can be used for the video fingerprint method of copy detection and video identification, video frequency searching, copyright protection and tracing pirate.

Background technology

Electric electronic engineering association (IEEE) information evidence obtaining in 2007 and safe transactions the 12nd interim pointing out, variation along with the universal and circulation way of digital device, there is every day thousands of video file uploaded and share in the Internet, TV and mobile communication equipment, video file is by bootlegging and to process be very general phenomenon, the copyright infringement extensively the existing content-based fast and accurately video copy detection method of will seeking development.By the video finger print technology of dealing with this challenge; fingerprint described here is different from somatic fingerprint; it means a kind of signature technology that video information is content-based; concrete video data can be used for identifying uniquely and representing, management, protection and retrieval to video file can be effectively realized.

The interim video finger print of having introduced of video technique transactions the 7th in electric electronic engineering association (IEEE) Circuits and Systems in 2008 has three main designing requirements: one, robustness: video finger print will keep keeping stable in operation in various contents, do not changing under the perception properties prerequisite of video, video finger print remains unchanged or less change.Video user can have a mind to or by mistake video be carried out to some contents keep attacking or processing, as the change of brightness/contrast, add make an uproar, compress, rotation, shearing, sequential adjustment, frame per second change, increase logo etc., video data not only can look like image generation spatial variations, and can change if having time, make to keep robustness to become more difficult; Two, sensitiveness: the different video of perception should have different fingerprints, this is particularly important for video labeling and retrieval; Three, fail safe: in the situation that there is no key, fingerprint is not easy forged by opponent or estimate.This point is only emphasized in the application of copyright protection, does not need to consider this requirement in the application of video frequency searching, video automatic marking.

Domestic and international existing fingerprint method is mainly included in the method based on gradient direction barycenter of video technique transactions the 7th interim proposition in electric electronic engineering association (IEEE) Circuits and Systems in 2008, the feature that this method is extracted can not effective expression image principal character, data volume is larger simultaneously, to various contents, keeps operation robustness not high; The method based on wavelet transform and PCA decomposition of electric electronic engineering association (IEEE) information evidence obtaining in 2010 and the 3rd interim proposition of safe transactions, video identification with retrieve application in recognition correct rate not high, the false alarm probability occurring under certain detection probability is larger; Electric electronic engineering association (IEEE) information evidence obtaining in 2011 and the interim method that proposes the temporal information presentation video technology in DCT territory of safe transactions the 1st, the global search decision search speed that this method adopts in matching process is slow, real-time is poor, and database search efficiency is not high.

Summary of the invention

The object of the invention is to propose a kind of video fingerprint method based on profile wave convert model, to overcome the above-mentioned defect of prior art, realize video dubbing detection, content recognition, video frequency searching and copyright protection.

The present invention is based on the video fingerprint method of profile wave convert model, comprise two parts of fingerprint extraction and fingerprint matching;

It is characterized in that:

Described fingerprint extraction step is as follows:

First by input original video files V (w, h, f), the width that wherein w is frame, the height that h is frame, f is frame per second, converts reference format V to _norm(W, H, F), wherein W, H and F get respectively 320,240 and 10; If the color video of rgb format, converts brightness, colourity and saturation to and represent, wherein luminance component is I=(R+G+B)/3, using luminance component as processing object;

To in every two field picture, carry out non-overlapping piecemeal, produce N * M number of sub images piece;

Each image block is carried out to J yardstick, m _j, j=1,2 ... the profile wave convert of J direction, adopts the concealed Markov tree technology that IEEE image processing the 6th phase of transactions in 2006 proposes to carry out image modeling to conversion coefficient, obtains following model parameter:

P _{1, k}, k=1 ..., m ₁, the state probability of each direction in the thickest yardstick of expression;

A _j,k, j=2 ... J, k=1 ..., m _j, represent the transition probability from yardstick j-1 to j;

B _j,k, j=1 ... J, k=1 ..., m _j, Gauss's standard variance at the direction k place of expression yardstick j;

Using standard variance matrix as intermediate features, carry out singular value decomposition, B _{i, j, k}represent k picture frame, the standard variance matrix that i is capable, the image block at j row place forms, its singular value decomposition is expressed as B _{i, j, k}=USV ^h, 1≤i≤N wherein, 1≤j≤M, U, V is respectively left and right singular value matrix, S is diagonal matrix;

The maximum singular value of all diagonal matrix S in one frame is taken out to constitutive characteristic vector, and k frame feature vector is expressed as

s _k=[s _1,1,k,s _1,2,k,…,s _N,M,k]；

The characteristic vector of cascade P frame, is then normalized μ _sand σ _sbe respectively average and the variance of characteristic vector, each value in characteristic vector deducts average, divided by variance, produces last fingerprint sequence:

$z_{i,j,k}=\frac{S_{i,j,k}-{\mathrm{\μ}}_s}{{\mathrm{\σ}}_s};$

Described fingerprint matching step is:

If video v to be checked _qafter standardization, have P frame, k frame fingerprint is

in video database according to k nearest neighbor criterion select with a most close L fingerprint

c=1,2 ... L, then take k frame fingerprint as L the candidate fingerprint sequence that length is NMP of benchmark formation, the length that NMP is video finger print to be checked, and each candidate fingerprint is expressed as:

$z^c=[z_{k^{\text{'}}-k+1}^c\·\·\·z_{k^{\text{'}}}^c\·\·\·z_{k^{\text{'}}+P-k}^c],c=\mathrm{1,2},\·\·\·L;$

Calculate the Euclidean distance of video finger print to be checked and L candidate's video finger print:

$D(v_q,v_c)=\frac{1}{\mathrm{NMP}}\underset{i=1}{\overset{P}{\mathrm{\Σ}}}{(z_i^q-z_i^c)}^2;$

If be less than default thresholding T apart from minimum value, export the video corresponding with minimum value

for matching result; If be greater than default thresholding T apart from minimum value, video to be checked does not exist in video database;

The desired false alarm probability P of set basis of thresholding T _fAby gauss of distribution function formula, determined, work as N=3, during M=4, μ=1.62, σ=0.22; Work as P _fA=3.1 * 10 ^-9time, T ≈ 0.4; False alarm probability

$P_{\mathrm{FA}}={\∫}_{-\∞}^T\frac{1}{\sqrt{2\mathrm{\π}}\mathrm{\σ}}\exp \left[\frac{-{(x-\mathrm{\μ})}^2}{2{\mathrm{\σ}}^2}\right]\mathrm{dx}=\frac{1}{2}\mathrm{erfc}\left(\frac{\mathrm{\μ}-T}{\sqrt{2}\mathrm{\σ}}\right).$

The present invention is based on the video fingerprint method of profile wave convert model owing to first one section of video being carried out to standardization, every two field picture is carried out to piecemeal; Then to each image block, adopt multiple dimensioned and multi-direction profile wave convert to process, and adopt concealed Markov tree technology extraction model parameter; Again the standard variance matrix of every is carried out to singular value decomposition, extract maximum singular value cascade and be normalized rear formation video finger print vector; In the fingerprint matching stage, adopt two step search strategies: in first choosing at random video to be checked, the fingerprint of arbitrary frame is determined candidate's video fast; adopt again Euclidean distance to estimate the similitude of video to be measured and candidate's video; improve robustness, discrimination and the search efficiency of video finger print, can be widely used in copy detection, retrieval, identification and the copyright protection of video file.Owing to taking to extract by the mode of image modeling the principal character of image, and modeling parameters is done to further dimension-reduction treatment, thereby reduced the data volume of presentation video, improved the robustness that similarity is differentiated.

Compared with prior art, because the present invention adopts multi-direction, the multiple dimensioned characteristic information of profile wave convert capture video images, can effectively represent the main contents of video, improve the accuracy of detection and Identification;

Because the present invention adopts concealed Markov tree modeling technique, by less model parameter, represent the statistical relationship between conversion coefficient, further improved video finger print and content has been kept to robustness and the sensitiveness to video variance information of operation;

The inventive method, by the luminance video component through standardization is processed, can effectively be resisted the attacks such as frame per second variation, convergent-divergent, color space variation.

Because the present invention first adopts from the picture frame of choosing at random of video to be checked, select limited candidate item set, then candidate item set is carried out to two step search strategies of whole frame couplings, effectively improved speed and the efficiency of retrieval with identification.

The video finger print that adopts the inventive method to generate meets Gaussian Profile, thereby can facilitate various application systems according to the adaptive definite threshold value of desired false alarm probability.

Accompanying drawing explanation

Fig. 1 is the schematic flow sheet that the present invention is based on the fingerprint extraction part in the video fingerprint method of profile wave convert model, and Fig. 2 is the schematic flow sheet of fingerprint matching part.

Embodiment

Embodiment 1:

The video fingerprint method that the present invention is based on profile wave convert model is divided into fingerprint extraction and fingerprint matching two parts, and Fig. 1 is the schematic flow sheet of fingerprint extraction part, and Fig. 2 is the schematic flow sheet of fingerprint matching part.Now to the present invention is based on the specific operation process of the video fingerprint method of profile wave convert model, be described below by reference to the accompanying drawings:

Described fingerprint extraction step is as follows:

The first step, normalization step A, original video V comprises P two field picture, first by sdi video and time standard, be about to resolution and be scaled 320 * 240, frame per second is adjusted into 10 frames per second, standardization is conducive to resist convergent-divergent, frame per second changes attack, and between every frame, the expansion of time can be removed change small between pixel simultaneously, expands the otherness between each frame.

Second step, switch process B, if every two field picture is colored, adopt RGB to represent, convert thereof into brightness and chrominance representation, luminance component I=(R+G+B)/3, the present invention adopts luminance component to be for further processing, and makes the suitable various black and white of method and color format video.

The 3rd step, image block step C, do non-overlapping piecemeal to every two field picture, generates N * M number of sub images piece, N=3 for example, and M=4, every block size is 80 * 80, this is conducive to local message and extracts.

The 4th step, profile wave convert step D, every image block is amplified to 128 * 128 through bilinear interpolation, then carry out profile wave convert, profile wave convert is a kind of multiple dimensioned multidirectional transform method, it is mainly divided into two steps, first adopt the laplacian pyramid of IEEE signal processing transactions in 1992 proposition to carry out sub-band division, then, with the conversion of directional filter group travel direction, the result of decomposing is like this sparse.

The 5th step, modeling procedure E, process according to IEEE image in 2006 dependence that concealed Markov tree model that the 6th phase of transactions proposes has been described parent and filial generation coefficient between different scale, can be by expectation maximum solution acquisition model parameter.Parameter is divided into three classes:

(1) P _{1, k}, k=1 ..., m ₁, the state probability of each direction in the thickest yardstick of expression;

(2) A _j,k, j=2 ... J, k=1 ..., m _j, represent the transition probability from yardstick j-1 to j;

(3) B _j,k, j=1 ... J, k=1 ..., m _j, Gauss's standard variance at the direction k place of expression yardstick j.

By model parameter is kept to the research of the stability under operation in various parameters, find that Gauss's standard variance matrix has same content and has good stability.

The 6th step, singular value decomposition step F, carry out singular value decomposition (SVD) using standard variance matrix as intermediate features, the singular value of matrix has fully reflected the internal characteristics of matrix, B _{i, j, k}represent k picture frame in video, the standard variance matrix that i is capable, the image block at j row place forms, its singular value decomposition SVD is expressed as: B _{i, j, k}=USV ^h, 1≤i≤N wherein, 1≤j≤M; U, V is respectively left and right singular value matrix, S is diagonal matrix.

The 7th step, singular value concatenation step G, each image block can obtain a maximum singular value, and the maximum singular value in a frame is taken out to constitutive characteristic vector, and k frame feature vector is expressed as:

s _k=[s _1,1,k,s _1,2,k,…,s _N,M,k]；

The 8th step, normalized step H, the characteristic vector of first cascade P frame, is then normalized μ _sand σ _sbe respectively average and the variance of characteristic vector, each value in characteristic vector deducts average, divided by variance, produces last fingerprint sequence.

$z_{i,j,k}=\frac{S_{i,j,k}-{\mathrm{\μ}}_s}{{\mathrm{\σ}}_s};$

The detection essence of video identification and retrieval, replicating video is exactly to realize the fingerprint matching process of video to be checked and video database.

Described fingerprint matching step is as follows:

The 9th step, choose fingerprint step I at random, establishing video U to be checked has P frame, through above-mentioned fingerprint extraction process, generates fingerprint sequence, chooses at random wherein certain frame fingerprint as next step processing object.

The tenth step, selection candidate frame step J, the k frame fingerprint of establishing random selection is

with K nearest neighbor method, in video finger print database W, select with a most close L fingerprint, then take k frame fingerprint as L the candidate fingerprint sequence that length is NMP of benchmark formation, the length that NMP is video finger print to be checked, and each candidate fingerprint is expressed as:

$z^c=[z_{k^{\text{'}}-k+1}^c\·\·\·z_{k^{\text{'}}}^c\·\·\·z_{k^{\text{'}}+P-k}^c],c=\mathrm{1,2},\·\·\·L;$

This step by single image in the location of database, dwindled candidate item set, improved search efficiency.

The 11 step, compute euclidian distances step K, calculate the Euclidean distance of video finger print to be checked and L candidate's video finger print

$D(v_q,v_c)=\frac{1}{\mathrm{NMP}}\underset{i=1}{\overset{P}{\mathrm{\Σ}}}{(z_i^q-z_i^c)}^2;$

The 12 step, thresholding comparison step L, make comparisons the minimum value of all distances and default thresholding T, if be less than thresholding T apart from minimum value, video to be checked is present in video database; If be greater than default thresholding T apart from minimum value, video to be checked does not exist in video database.

The 13 step, Output rusults step M, export the video corresponding with minimum value for matching result

$v_c^{*}=\underset{v_c\∈V}{\arg \min }D(v_q,v_c);$

Through the statistical research to distance between the fingerprint sequence forming, find that it meets Gaussian Profile rule, the relation of false alarm probability and thresholding T is determined by following gauss of distribution function, for example, works as N=3, during M=4, μ=1.62, σ=0.22; Work as P _fA=3.1 * 10 ^-9time, T ≈ 0.4; False alarm probability

$P_{\mathrm{FA}}={\∫}_{-\∞}^T\frac{1}{\sqrt{2\mathrm{\π}}\mathrm{\σ}}\exp \left[\frac{-{(x-\mathrm{\μ})}^2}{2{\mathrm{\σ}}^2}\right]\mathrm{dx}=\frac{1}{2}\mathrm{erfc}\left(\frac{\mathrm{\μ}-T}{\sqrt{2}\mathrm{\σ}}\right);$

Thresholding T is larger, and the video of search coupling is more, but false alarm probability is larger; Thresholding T is less, and the video of search coupling is fewer, but false alarm probability is less.The selection of thresholding T can be according to the Location of requirement of practical application.

Advantage of the present invention can further illustrate by following emulation experiment:

Emulation experiment selects video file as tested object from Reefvid video database, and each video is attacked and produced 22 bias versions by some common Video processing, specific as follows:

1) 3 * 3 and 5 * 5 Gaussian Blur is processed;

2) image scaling 50%, 200%;

3) Gamma correction-20%, 20%;

4) additive Gaussian noise, signal to noise ratio-10dB, 0dB, 10dB, 20dB, 30dB, 40dB;

5) JPEG lossy compression method, quality factor 10,30,50,70,90;

6) be trimmed to full-sized 90%, 80%;

7) insert icon to full-sized 5%, 10%;

8) frame per second changes from 25fps to 15fps;

The present invention adopts false alarm probability to come the robustness of balancing method and the recognition capability of employing recognition correct rate balancing method, video is in above 8 kinds of situations, false alarm probability is respectively 0.0036,0.0025,0.0064,0.0561,0.0021,0.0341,0.0068,0.0180, correct recognition rata is respectively 0.991,0.992,0.970,0.912,0.992,0.954,0.957,0.952, better to common Video processing robustness from the known the present invention of above data, test has also confirmed gratifying recognition capability all sidedly.

Because the present invention adopts multi-direction, the multiple dimensioned characteristic information of profile wave convert capture video images, can effectively represent the main contents of video, improve the accuracy of detection and Identification;

Because the present invention adopts concealed Markov tree modeling technique, by less model parameter, represent the statistical relationship between conversion coefficient, further improved video finger print and content has been kept to robustness and the sensitiveness to video variance information of operation;

The inventive method, by the luminance video component through standardization is processed, can effectively be resisted the attacks such as frame per second variation, convergent-divergent, color space variation.

Because the present invention first adopts from the picture frame of choosing at random of video to be checked, select limited candidate item set, then candidate item set is carried out to two step search strategies of whole frame couplings, effectively improved speed and the efficiency of retrieval with identification.

The video finger print that adopts the inventive method to generate meets Gaussian Profile, thereby can facilitate various application systems according to the adaptive definite threshold value of desired false alarm probability.

Claims (1)

1. the video fingerprint method based on profile wave convert model, comprises two parts of fingerprint extraction and fingerprint matching; It is characterized in that:

Described fingerprint extraction step is as follows:

First by input original video files V (w, h, f), the width that wherein w is frame, the height that h is frame, f is frame per second, converts reference format V to _norm(W, H, F), wherein W, H and F get respectively 320,240 and 10; If the color video of rgb format, converts brightness, colourity and saturation to and represent, wherein luminance component is I=(R+G+B)/3, using luminance component as processing object;

To in every two field picture, carry out non-overlapping piecemeal, produce N * M number of sub images piece;

Each image block is carried out to J yardstick, m _j, j=1,2 ... the profile wave convert of J direction, adopts concealed Markov tree technology to carry out image modeling to conversion coefficient, obtains following model parameter:

P _{1, k}, k=1 ..., m ₁, the state probability of each direction in the thickest yardstick of expression;

A _j,k, j=2 ... J, k=1 ..., m _j, represent the transition probability from yardstick j-1 to j;

B _j,k, j=1 ... J, k=1 ..., m _j, Gauss's standard variance at the direction k place of expression yardstick j;

Using standard variance matrix as intermediate features, carry out singular value decomposition, B _{i, j, k}represent k picture frame, the standard variance matrix that i is capable, the image block at j row place forms, its singular value decomposition is expressed as B _{i, j, k}=USV ^h, 1≤i≤N wherein, 1≤j≤M, U, V is respectively left and right singular value matrix, S is diagonal matrix;

The maximum singular value of all diagonal matrix S in one frame is taken out to constitutive characteristic vector, and k frame feature vector is expressed as

s _k=[s _1,1,k,s _1,2,k,…,s _N,M,k]；

The characteristic vector of cascade P frame, is then normalized μ _sand σ _sbe respectively average and the variance of characteristic vector, each value in characteristic vector deducts average, divided by variance, produces last fingerprint sequence:

$z_{i,j,k}=\frac{S_{i,j,k}-{\mathrm{\μ}}_s}{{\mathrm{\σ}}_s}.$

Described fingerprint matching step is:

If video v to be checked _qafter standardization, have P frame, k frame fingerprint is , in video database according to k nearest neighbor criterion select with

a most close L fingerprint

then take k frame fingerprint as L the candidate fingerprint sequence that length is NMP of benchmark formation, the length that NMP is video finger print to be checked, each candidate fingerprint is expressed as:

$z^c=[z_{k^{\text{'}}-k+1}^c\·\·\·z_{k^{\text{'}}}^c\·\·\·z_{k^{\text{'}}+P-k}^c],c=\mathrm{1,2},\·\·\·L;$

Calculate the Euclidean distance of video finger print to be checked and L candidate's video finger print:

$D(v_q,v_c)=\frac{1}{\mathrm{NMP}}\underset{i=1}{\overset{P}{\mathrm{\Σ}}}{(z_i^q-z_i^c)}^2;$

If be less than default thresholding T apart from minimum value, export the video corresponding with minimum value

for matching result; If be greater than default thresholding T apart from minimum value, video to be checked does not exist in video database;

The desired false alarm probability P of set basis of thresholding T _fAby gauss of distribution function formula, determined, work as N=3, during M=4, μ=1.62, σ=0.22; Work as P _fA=3.1 * 10 ^-9time, T ≈ 0.4; False alarm probability

$P_{\mathrm{FA}}={\∫}_{-\∞}^T\frac{1}{\sqrt{2\mathrm{\π}}\mathrm{\σ}}\exp \left[\frac{-{(x-\mathrm{\μ})}^2}{2{\mathrm{\σ}}^2}\right]\mathrm{dx}=\frac{1}{2}\mathrm{erfc}\left(\frac{\mathrm{\μ}-T}{\sqrt{2}\mathrm{\σ}}\right).$

Images