CN105678243A - On-line extraction method of monitoring video feature frames - Google Patents

Abstract

The invention provides an on-line extraction method of monitoring video feature frames. Under an increment sliding window technical framework, the on-line extraction method detects sequential variation points of video sub sequences and utilizes the detected variation points to segment the video into video clips containing different contents so as to utilize a clustering method to complete extraction of key frames in the obtained video clips. The on-line extraction method of monitoring video feature frames does not need any man-made preset parameters, and can realize extraction of the key frames of the monitoring video without any monitoring.

Description

A kind of monitor video characteristic frame On-line testing method

Technical field

The present invention relates to a kind of monitor video characteristic frame On-line testing method, belong to the technical field of intelligent monitoring.

Background technology

Video frequency abstract (videosummarization) technology allows user to pass through to browse video features hardwood in finite time can grasp event in observation time. But, the existing video frame extraction method based on scene changes, shot transition detection can not be applicable to monitor video.

Summary of the invention

For the deficiencies in the prior art, the present invention provides a kind of monitor video characteristic frame On-line testing method. The method is under increment sliding window (incrementalsliding-window) technological frame, first the timing variations point of video sequence is detected, and utilize the change point detected that Video segmentation becomes the video segment comprising different content, and then clustering method is utilized to complete the extraction of key frame in the video segment obtained. The method does not need any artificial parameter set in advance, it may be achieved complete unsupervised monitor video key-frame extraction.

Technical scheme is as follows:

A kind of monitor video characteristic frame On-line testing method, comprises the following steps that and first video sequence carries out timing variations point detection; And utilize the change point detected that Video segmentation becomes the video segment comprising different content, and then clustering method is utilized to complete the extraction of key frame in the video segment obtained. Wherein said video is the video of N frame: F={f₁,f₂,...,f_N},f_iRepresent the frame of video at time i place, extract the key frame F={f in described video_r1,f_r2,...,f_rk, and carry out arrangement chronologically and obtain.

According to currently preferred, the described method that video sequence carries out timing variations point detection comprises the steps:

Step (1-1): set up sliding window model

Initialize the start frame n of video change point detection₁=1 and the frame length L of corresponding sliding window₁=L₀;

Step (1-2)

A detection it is changed in the video sliding window set up in described step (1-1);

Step (1-3)

If sequential change point η having been detected in video sequence window, then the start frame that detects with time point η for next round also reinitializes sliding window frame length for L₀, i.e. n_i+1=η and L₁=L₀, subsequent video is carried out next round change point detection; If being not detected by timing variations point in video sequence window, then still with initialized n_iFor detection start frame, it may be assumed that n_i+1=n_i, and sliding window length is updated to L_i+1=L_i+ Δ L, Δ L are sliding window length increment step-length:Proceed change point detection;

Step (1-4)

Whole change point detection process is until all sequence of frames of video have all detected or arrived T preassigned deadline₀Terminate, i.e. L > N or i > T₀, wherein N is given complete monitor video totalframes, T₀For preassigned deadline;Otherwise, i=i+1 return step (1-2).

According to currently preferred, Video segmentation is become the method for video segment comprising different content to be by the change point that described utilization detects: realizing the sequential to video split by detecting the timing variations point of video sequence in each window, wherein the timing variations point of video sequence detects and includes:

Step (2-1): video feature extraction

In hsv color space, frame of video is carried out the feature extraction based on color histogram, and adopts quantization method to form and aspect, saturation, lightness dimensionality reduction respectively to 16 dimensions, 8 dimensions, 8 dimensions, finally give the video time sequence feature of 32 dimensions, be designated as F={f equally₁, f₂..., f_N}∈^32×N;

Step (2-2): diversity detects

Assume Y_iIt is that in given video F, time span is one section of video sequence of LFrom time i, terminate to time i+L-1, for each reference change point η ∈ Y_i, similarity measurement formula is:

$L\left\{Y_i\right\}=\underset{{\mathrm{\η}}^{\′}\∈\left\{{\mathrm{\η}}_i\right\}}{\sup }\left(L\right\{Y_i\left|{\mathrm{\η}}^{\′}\right\})=\underset{{\mathrm{\η}}^{\′}\∈\left\{{\mathrm{\η}}_i\right\}}{\sup }(\frac{1}{m^2}\underset{i,j=1}{\overset{m}{\Σ}}k(x_i,y_i)-\frac{2}{mn}\underset{i,j=1}{\overset{m,n}{\Σ}}k(x_i,y_j)+\frac{1}{n^2}k(x_i,y_i))$

WhereinIt is by sample point (x_i, y_j) be mapped in gaussian kernel; The sample point making former sample point different classes of in new space by such mapping has bigger separation property, and makes the sample point in new space that former data are had better descriptive power;

Step (2-3): hypothesis testing

Based on it is assumed hereinafter that inspection carries out timing variations point detection:

H₀: L{Y_i| η ' } < λ_i

H_A: L{Y_i|η′}≥λ_i

Wherein λ_iIt is a setting threshold value, self adaptation can obtain in algorithm performs, if H₀Set up, be then not changed in a little; Otherwise, H_AFor true time, there is timing variations point at η ' place, and at η ' place to Y_iSplit.

According to currently preferred, the described extracting method utilizing clustering method to complete key frame in the video segment obtained, including following content:

Obtaining change point η ' and to Y_iAfter splitting, first half video segment uses k-means clustering algorithm, and extracts key frame for frame of video immediate with cluster centre; Later half video segment then will be proceeded change point detection; When whole detection process terminates, extract all key frames composition set F and arrange chronologically, being the video frequency abstract ultimately produced.

Present invention have an advantage that

Extracting method of the present invention is under increment sliding window (incrementalsliding-window) technological frame, first the timing variations point of video sequence is detected, and utilize the change point detected that Video segmentation becomes the video segment comprising different content, and then clustering method is utilized to complete the extraction of key frame in the video segment obtained. The method does not need any artificial parameter set in advance, it may be achieved complete unsupervised monitor video key-frame extraction.

Accompanying drawing explanation

Fig. 1 is the flow chart of extracting method of the present invention.

Detailed description of the invention

Below in conjunction with embodiment and Figure of description, the present invention is described in detail, but is not limited to this.

As described in Figure 1.

Embodiment,

A kind of monitor video characteristic frame On-line testing method, comprises the following steps that and first video sequence carries out timing variations point detection; And utilize the change point detected that Video segmentation becomes the video segment comprising different content, and then clustering method is utilized to complete the extraction of key frame in the video segment obtained. Wherein said video is the video of N frame: F={f₁,f₂,...,f_N},f_iRepresent the frame of video at time i place, extract the key frame F={f in described video_r1,f_r2,...,f_rk, and carry out arrangement chronologically and obtain.

According to currently preferred, the described method that video sequence carries out timing variations point detection comprises the steps:

Step (1-1): set up sliding window model

Initialize the start frame n of video change point detection₁=1 and the frame length L of corresponding sliding window₁=L₀;

Step (1-2)

A detection it is changed in the video sliding window set up in described step (1-1);

Step (1-3)

If sequential change point η having been detected in video sequence window, then the start frame that detects with time point η for next round also reinitializes sliding window frame length for L₀, i.e. n_i+1=η and L₁=L₀, subsequent video is carried out next round change point detection; If being not detected by timing variations point in video sequence window, then still with initialized n_iFor detection start frame, it may be assumed that n_i+1=n_i, and sliding window length is updated to L_i+1=L_i+ Δ L, Δ L are sliding window length increment step-length:Proceed change point detection;

Step (1-4)

Whole change point detection process is until all sequence of frames of video have all detected or arrived T preassigned deadline₀Terminate, i.e. L > N or i > T₀, wherein N is given complete monitor video totalframes, T₀For preassigned deadline; Otherwise, i=i+1 return step (1-2).

Video segmentation is become the method for video segment comprising different content to be by the change point that described utilization detects: realizing the sequential to video split by detecting the timing variations point of video sequence in each window, wherein the timing variations point of video sequence detects and includes:

Step (2-1): video feature extraction

In hsv color space, frame of video is carried out the feature extraction based on color histogram, and adopts quantization method to form and aspect, saturation, lightness dimensionality reduction respectively to 16 dimensions, 8 dimensions, 8 dimensions, finally give the video time sequence feature of 32 dimensions, be designated as equally

Step (2-2): diversity detects

Assume Y_iIt is that in given video F, time span is one section of video sequence of LFrom time i, terminate to time i+L-1, for each reference change point η ∈ Y_i, similarity measurement formula is:

$L\left\{Y_i\right\}=\underset{{\mathrm{\&eta;}}^{\&prime;}\&Element;\left\{{\mathrm{\&eta;}}_i\right\}}{\sup }\left(L\right\{Y_i\left|{\mathrm{\&eta;}}^{\&prime;}\right\})=\underset{{\mathrm{\&eta;}}^{\&prime;}\&Element;\left\{{\mathrm{\&eta;}}_i\right\}}{\sup }(\frac{1}{m^2}\underset{i,j=1}{\overset{m}{\&Sigma;}}k(x_i,y_i)-\frac{2}{mn}\underset{i,j=1}{\overset{m,n}{\&Sigma;}}k(x_i,y_j)+\frac{1}{n^2}k(x_i,y_i))$

WhereinIt is by sample point (x_i, y_j) be mapped in gaussian kernel; The sample point making former sample point different classes of in new space by such mapping has bigger separation property, and makes the sample point in new space that former data are had better descriptive power;

Step (2-3): hypothesis testing

Based on it is assumed hereinafter that inspection carries out timing variations point detection:

H₀: L{Y_i| η ' } < λ_i

H_A: L{Y_i|η′}≥λ_i

Wherein λ_iIt is a setting threshold value, self adaptation can obtain in algorithm performs, if H₀Set up, be then not changed in a little; Otherwise, H_AFor true time, there is timing variations point at η ' place, and at η ' place to Y_iSplit.

The described extracting method utilizing clustering method to complete key frame in the video segment obtained, including following content:

Obtaining change point η ' and to Y_iAfter splitting, first half video segment uses k-means clustering algorithm, and extracts key frame for frame of video immediate with cluster centre; Later half video segment then will be proceeded change point detection; When whole detection process terminates, extract all key frames composition set F and arrange chronologically, being the video frequency abstract ultimately produced.

Claims (4)

1. a monitor video characteristic frame On-line testing method, it is characterised in that described extracting method comprises the following steps that and first video sequence carries out timing variations point detection; And utilize the change point detected that Video segmentation becomes the video segment comprising different content, and then clustering method is utilized to complete the extraction of key frame in the video segment obtained.

2. a kind of monitor video characteristic frame On-line testing method according to claim 1, it is characterised in that the described method that video sequence carries out timing variations point detection comprises the steps:

Step (1-1): set up sliding window model

Initialize the start frame n of video change point detection₁=1 and the frame length L of corresponding sliding window₁=L₀;

Step (1-2)

A detection it is changed in the video sliding window set up in described step (1-1);

Step (1-3)

If sequential change point η having been detected in video sequence window, then the start frame that detects with time point η for next round also reinitializes sliding window frame length for L₀, i.e. n_i+1=η and L₁=L₀, subsequent video is carried out next round change point detection; If being not detected by timing variations point in video sequence window, then still with initialized n_iFor detection start frame, it may be assumed that n_i+1=n_i, and sliding window length is updated to L_i+1=L_i+ Δ L, Δ L are sliding window length increment step-length:Proceed change point detection;

Step (1-4)

Whole change point detection process is until all sequence of frames of video have all detected or arrived T preassigned deadline₀Terminate, namely

L > N or i > T₀, wherein N is given complete monitor video totalframes, T₀For preassigned deadline; Otherwise, i=i+1 return step (1-2).

3. a kind of monitor video characteristic frame On-line testing method according to claim 1, it is characterized in that, Video segmentation is become the method for video segment comprising different content to be by the change point that described utilization detects: realizing the sequential to video split by detecting the timing variations point of video sequence in each window, wherein the timing variations point of video sequence detects and includes:

Step (2-1): video feature extraction

In hsv color space, frame of video is carried out the feature extraction based on color histogram, and adopts quantization method to form and aspect, saturation, lightness dimensionality reduction respectively to 16 dimensions, 8 dimensions, 8 dimensions, finally give the video time sequence feature of 32 dimensions, be designated as F={f equally₁, f₂..., f_N∈ 32 × N;

Step (2-2): diversity detects

Assume Y_iIt is that in given video F, time span is one section of video sequence of LFrom time i, terminate to time i+L-1, for each reference change point η ∈ Y_i, similarity measurement formula is:

$L\left\{Y_i\right\}=\underset{{\mathrm{\&eta;}}^{\&prime;}\&Element;\left\{{\mathrm{\&eta;}}_i\right\}}{\sup }\left(L\left\{Y_i|{\mathrm{\&eta;}}^{\&prime;}\right\}\right)=\underset{{\mathrm{\&eta;}}^{\&prime;}\&Element;\left\{{\mathrm{\&eta;}}_i\right\}}{\sup }\left(\frac{1}{m^2}\underset{i,j=1}{\overset{m}{\&Sigma;}}k\left(x_i,y_i\right)-\frac{2}{mn}\underset{i,j=1}{\overset{m,n}{\&Sigma;}}k\left(x_i,y_j\right)+\frac{1}{n^2}k\left(x_i,y_i\right)\right)$

WhereinIt is by sample point (x_i, y_j) be mapped in gaussian kernel;

Step (2-3): hypothesis testing

Based on it is assumed hereinafter that inspection carries out timing variations point detection:

H₀: L{Y_i| η ' } < λ_i

H_A: L{Y_i|η′}≥λ_i

Wherein λ_iIt is a setting threshold value, if H₀Set up, be then not changed in a little; Otherwise, H_AFor true time, there is timing variations point at η ' place, and at η ' place to Y_iSplit.

4. a kind of monitor video characteristic frame On-line testing method according to claim 1, it is characterised in that the described extracting method utilizing clustering method to complete key frame in the video segment obtained, including following content:

Obtaining change point η ' and to Y_iAfter splitting, first half video segment uses k-means clustering algorithm, and extracts key frame for frame of video immediate with cluster centre; Later half video segment then will be proceeded change point detection; When whole detection process terminates, extract all key frames composition set F and arrange chronologically, being the video frequency abstract ultimately produced.