CN107223344A - The generation method and device of a kind of static video frequency abstract - Google Patents
The generation method and device of a kind of static video frequency abstract Download PDFInfo
- Publication number
- CN107223344A CN107223344A CN201780000556.2A CN201780000556A CN107223344A CN 107223344 A CN107223344 A CN 107223344A CN 201780000556 A CN201780000556 A CN 201780000556A CN 107223344 A CN107223344 A CN 107223344A
- Authority
- CN
- China
- Prior art keywords
- cluster
- frame
- video
- candidate
- value
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
- H04N21/85—Assembly of content; Generation of multimedia applications
- H04N21/854—Content authoring
- H04N21/8549—Creating video summaries, e.g. movie trailer
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
- G06V10/75—Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries
- G06V10/758—Involving statistics of pixels or of feature values, e.g. histogram matching
Landscapes
- Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Computer Security & Cryptography (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Image Analysis (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The present invention is applicable field of computer technology there is provided a kind of generation method of static video frequency abstract and device, and methods described includes：Receive the pending video of user's input；Pre-sampling is carried out to pending video by singular value decomposition algorithm, to extract the candidate frame of pending video；According to bag of words algorithm, generation respectively is used for representing the histogram of each candidate frame；All histograms are clustered by the high density peak search algorithm based on representation of video shot, and obtain the cluster central point after cluster；According to each cluster central point, the static video frequency abstract of pending video is generated.So as to be represented by the generation and histogram of candidate frame, the frame of redundancy is removed deeper into ground, and Cu Lei centers are adaptively generated in cluster process, the quantity of cluster need not be pre-set, without iterative process, it is effectively improved the Stability and adaptability of cluster, reduces the time complexity of cluster, and then is effectively improved the formation efficiency and quality of static video frequency abstract.
Description
Technical field
The invention belongs to the generation method and device of field of computer technology, more particularly to a kind of static video frequency abstract.
Background technology
In recent years, with the development of multimedia technology, the video oneself liked is watched on network and has become majority
An indispensable part in daily life, but how to help that people's quick obtaining oneself from substantial amounts of video likes, sense
The problem of video of interest is still technically one challenging.Static Video summary is a solution effectively, classical
The certainly method of the problem, this method is by removing the redundant frame in video, and the static state for obtaining briefly expressing video content is regarded
Frequency is made a summary.User is just recognized that the approximate contents of video by watching video frequency abstract, and judges whether that interesting viewing is whole
Section video.
At present, related researcher has been presented for a variety of methods of static video frequency abstract, wherein, a kind of method divides video
Into multiple camera lenses, and based on color histogram feature, the frame of each camera lens is grouped using k- averages (k-means) clustering algorithm
Cluster (quantity for pre-setting cluster), static video summary results are set to by the cluster centre of each camera lens；Another side
Method proposes three steps of static video frequency abstract, first, and border detection is carried out to camera lens based on color and marginal information, its
Secondary, type of sports and scene in cluster process in camera lens are classified to camera lens, finally, using the important filtering of camera lens
Device, the importance of each camera lens is determined by calculating kinergety and color change, selects each during lens shooting
The important camera lens of cluster；There is a method in which, first pass through some insignificant frames in elimination video and obtain candidate frame, then using k-
All candidate frames are divided into cluster (quantity of cluster has the change decision of vision content between consecutive frame) by means clustering methods,
Some similar frames are finally filtered in these clusters, remaining frame is considered as the result of static video frequency abstract after filtering.
In above-mentioned existing method, because similar camera lens is likely to occur repeatedly, so first method in video
With second method using based on camera lens by the way of there is cluster that cluster is pre-set in redundancy, and first method
Quantity influences whether the generation of best video summary result, and de-redundancy work of the third method before cluster is not deep enough,
Simply just eliminate some simple, insignificant frame of video.
The content of the invention
It is an object of the invention to provide a kind of generation method of static video frequency abstract and device, it is intended to solves due to existing
Technology can not provide a kind of effective ways of static video frequency abstract generation, the redundant frame in generation static video frequency abstract in video
Removal degree is relatively low, the quantity of cluster after cluster need to be manually specified, and causes the static state that static video frequency abstract formation efficiency is relatively low, generate
The problem of video frequency abstract quality is unstable.
On the one hand, the invention provides a kind of generation method of static video frequency abstract, methods described comprises the steps：
Receive the pending video of user's input；
Pre-sampling is carried out to the pending video by singular value decomposition algorithm, to extract the time of the pending video
Select frame；
According to bag of words algorithm, the histogram of all candidate frames is generated respectively；
All histograms are clustered by the high density peak search algorithm based on representation of video shot, and obtain poly-
Cluster central point after class；
According to each cluster central point, the static video frequency abstract of the pending video is generated.
On the other hand, the invention provides a kind of generating means of static video frequency abstract, described device includes：
Video reception module, the pending video for receiving user's input；
Candidate's frame extraction module, for carrying out pre-sampling to the pending video by singular value decomposition algorithm, to carry
Take the candidate frame of the pending video；
Histogram representation module, for according to bag of words algorithm, the histogram of all candidate frames to be generated respectively；
Cluster computing module, for by the high density peak search algorithm based on representation of video shot to all histograms
Clustered, and obtain the cluster central point after cluster；And
Video summary generation module, for according to each cluster central point, the static state for generating the pending video to be regarded
Frequency is made a summary.
The present invention first uses singular value decomposition algorithm, carries out pre-sampling to pending video, obtains the time of pending video
Frame is selected, then using bag of words, generation is used for representing the histogram of these candidate frames, then, using the height based on representation of video shot
Density peaks searching algorithm, is clustered to all histograms, finally according to each cluster central point after cluster, is generated and is waited to locate
The static video frequency abstract of video is managed, so that the de-redundancy effect of frame in video is not only effectively improved, and in cluster process
It is not required to pre-set the quantity at Cu Lei centers, can be generated according to the content-adaptive of video in a number of cluster class
The heart, is effectively improved the Stability and adaptability of cluster, reduces the time complexity of cluster, and then is effectively improved quiet
The formation efficiency and quality of state video frequency abstract.
Brief description of the drawings
Fig. 1 is the implementation process figure of the generation method for the static video frequency abstract that the embodiment of the present invention one is provided；
Fig. 2 is the structural representation of the generating means for the static video frequency abstract that the embodiment of the present invention two is provided；
Fig. 3 be the embodiment of the present invention two provide static video frequency abstract generating means in candidate's frame extraction module structure
Schematic diagram；
Fig. 4 be the embodiment of the present invention two provide static video frequency abstract generating means in histogram representation module structure
Schematic diagram；
Fig. 5 be the embodiment of the present invention two provide static video frequency abstract generating means in cluster computing module structure show
It is intended to；And
Fig. 6 be the embodiment of the present invention two provide static video frequency abstract generating means in video summary generation module
Structural representation.
Embodiment
In order to make the purpose , technical scheme and advantage of the present invention be clearer, it is right below in conjunction with drawings and Examples
The present invention is further elaborated.It should be appreciated that the specific embodiments described herein are merely illustrative of the present invention, and
It is not used in the restriction present invention.
Implementing for the present invention is described in detail below in conjunction with specific embodiment：
Embodiment one：
Fig. 1 shows the implementation process of the generation method for the static video frequency abstract that the embodiment of the present invention one is provided, in order to just
In explanation, the part related to the embodiment of the present invention is illustrate only, details are as follows：
In step S101, the pending video of user's input is received.
The embodiment of the present invention is applied to that the platform or smart machine of Video processing can be carried out.When user needs one section of extraction to regard
During the static video frequency abstract of frequency, can using this section of video as pending video, input can currently carry out Video processing platform or
Smart machine.
In step s 102, pre-sampling is carried out to pending video by singular value decomposition algorithm, to extract pending regard
The candidate frame of frequency.
In embodiments of the present invention, there can be considerable duplicate message between the image of different frame in one section of video.It is logical
Cross and pre-sampling is carried out to all input frames in video, can remove some frames for repeating (or redundancy), obtain multiple candidate frames.This
A little candidate frames can as follow-up cluster operation object.
By singular value decomposition algorithm, the singular value and order of the matrix that can obtain being decomposed.Specifically, singular value decomposition is passed through
Algorithm carries out pre-sampling to pending video, with the process for the candidate frame for extracting pending video, can be realized by following step：
(1) the time varying characteristic vector of each input frame in pending video is generated.
In embodiments of the present invention, input frame is all frame of video of pending video.Can be full in form and aspect by input frame
With three Color Channels of angle value (HSV) color space, the corresponding time varying characteristic vector of the input frame is generated.Specifically, time-varying
Characteristic vector is row vector.
As illustratively, in pending video, the corresponding time varying characteristic vector of input frame that the time is t is xt=
[hHhShV].Wherein, hH、hSAnd hVRespectively three Color Channels of color saturation value (HSV) color space, are respectively this
Three Color Channels set up three histograms, and length is lH、lSAnd lV, so the length of time varying characteristic vector is L=lH+lS+
lV。
(2) according to time varying characteristic vector, all input frame construction feature matrixes are followed successively by, each eigenmatrix is comprising default
The time varying characteristic vector of window size, continuous input frame.
In embodiments of the present invention, window size is equal to the quantity of frame in the window.Can be individual, continuous defeated by window size
Enter the time varying characteristic vector corresponding to frame, constitute an eigenmatrix.
As illustratively, in pending video, the corresponding eigenmatrix of input frame that the time is t ist
=N, N+1 ..., T-1, T, the size of eigenmatrix is N × L.Wherein, N is window size, and T is all defeated in pending video
Enter the quantity of frame.
As illustratively, eigenmatrix XNBy window size, continuous time varying characteristic vector x1,x2,...,xNConstitute,
The eigenmatrix X adjacent with this feature matrixN+1By window size, continuous time varying characteristic vector x2,x3,...,xN+1Constitute.
(3) singular value decomposition is carried out to all eigenmatrixes, to obtain the corresponding singular value matrix of each eigenmatrix, and
According to singular value matrix, it is determined that the order of each eigenmatrix.
In embodiments of the present invention, can be to the formula that eigenmatrix carries out singular value decomposition：
X=U Σ VT, wherein, X is characterized matrix, and U is the matrix of one group of orthogonal singular vector of output, VTFor one group of input just
Singular vector matrix is handed over, Σ is singular value matrix.Eigenmatrix X can obtain singular value matrix Σ, singular value after unusual decomposition
Matrix is diagonal matrix, and the diagonal element of singular value matrix is singular value, and these singular values are arranged according to order from big to small
Row.As illustratively, when the diagonal element of singular value matrix is respectively q1,q2,…,qNWhen, q1,q2,…,qNAll it is singular value,
And q1It is maximum of which singular value.
The order of corresponding eigenmatrix is can determine that by singular value matrix, specifically, a threshold value is pre-set, successively will
Singular value in singular value matrix is compared with the threshold value, and counts the singular value quantity more than the threshold value, and this quantity is
The order of eigenmatrix corresponding to this singular value matrix.
(4) order of adjacent eigenmatrix is compared successively, when the order of second characteristic matrix is more than fisrt feature square
During rank of matrix, last input frame corresponding to second characteristic matrix is set to candidate frame.
In embodiments of the present invention, when the order of second characteristic matrix exceedes fisrt feature rank of matrix, it is believed that second
The corresponding input frame of the time varying characteristic of last in eigenmatrix vector, is different from previously input frame, institute on vision content
So that the corresponding input frame of the time varying characteristic of last in second characteristic matrix vector is set into candidate frame.By all adjacent spies
Levy after rank of matrix compared one by one, can obtain multiple candidate frames.
Specifically, fisrt feature matrix is any feature matrix in all eigenmatrixes, and second characteristic matrix is in institute
There is next eigenmatrix adjacent with fisrt feature matrix in eigenmatrix, i.e., when fisrt feature matrix is current adjacent feature
During first eigenmatrix in matrix, second characteristic matrix is second eigenmatrix in current adjacency matrix.
In step s 103, according to bag of words algorithm, the histogram of all candidate frames is generated respectively.
In embodiments of the present invention, bag of words are used for the expression of candidate frame, the superfluous of frame in video can be efficiently reduced
It is remaining.
Specifically, by bag of words, the histogram of all candidate frames is generated respectively, can be realized by following steps：
(1) characteristics of image of all candidate frames is extracted.
Specifically, by image characteristics extraction algorithm, the characteristics of image of candidate frame is extracted.Preferably, image characteristics extraction
Algorithm uses Scale invariant features transform (SIFT) feature extraction algorithm, and the algorithm can efficiently extract out a large amount of in candidate frame
SIFT descriptors.
(2) according to all characteristics of image, the feature code book of each candidate frame is generated by clustering.
Specifically, by clustering algorithm, all characteristics of image in all candidate frames are clustered, to select with generation
The characteristics of image of table, and these representative characteristics of image are set to feature code book.Alternatively, clustering algorithm is used
Conventional k-means clustering algorithms.
(3) feature distribution in all feature code books, generates the histogram for representing each candidate frame.
Specifically, according to the feature distribution situation on feature code book, histogram can be generated for each candidate frame, to pass through
Corresponding histogram represents each candidate frame.
In step S104, all histograms are gathered by the high density peak search algorithm based on representation of video shot
Class, and obtain the cluster central point after cluster.
In embodiments of the present invention, it is proposed that the high density peak search algorithm based on representation of video shot, the algorithm is more suitable for
Handle the cluster task of frame in video frequency abstract generating process.
Wherein, all histograms are clustered by the high density peak search algorithm based on representation of video shot, and obtained
Cluster central point after cluster, can be realized by following step：
(1) according to all histograms, the distance between each two candidate frame in all candidate frames is calculated.
Specifically, histogram can regard the distance between data point, each two candidate frame as, i.e., corresponding to two candidate frames
Euclidean distance between histogram.
(2) according to the distance between each two candidate frame and default cut-off distance, the corresponding office of each candidate frame is calculated
Portion's density.
Specifically, the calculation formula of local density is：Work as dij-dc<When 0, χ (dij-dc)
=1, otherwise χ (dij-dc)=0.Wherein, ρiFor the local density of i-th of candidate frame, dijFor i-th of candidate frame and j-th candidates
The distance between frame, dcFor default cut-off distance.It can be seen that, the local density ρ of candidate frameiCut to be less than with candidate frame distance
Only apart from dcCandidate's number of frames.
(3) according to all local densities, the corresponding high density point distance of each candidate frame is calculated.
Specifically, the high density point distance of candidate frame, the i.e. candidate frame and candidate's interframe with higher local density
Distance.The calculation formula of the high density point distance of i-th of candidate frame is：
Wherein, δiFor the high density point distance of i-th of candidate frame, dijFor
The distance between i-th of candidate frame and j-th candidates frame.
Specifically, as the local density ρ of i-th of candidate framei(now i-th of candidate frame is most during for highest local density
High local density's point, the numerical value of the local density is maximum), calculate between i-th of candidate frame and remaining candidate frame it is maximum away from
From the ultimate range is set into the high density point of i-th of candidate frame apart from δi。
As the local density ρ of i-th of candidate frameiWhen being not highest local density, than i-th candidate of local density is obtained big
Candidate frame, calculate the minimum range between i-th of candidate frame and these candidate frames, and this minimum range is set to i-th
The high density point of candidate frame is apart from δi。
(4) according to the corresponding local density of each candidate frame and high density point distance, cluster central point is obtained.
Specifically, in the high density peak search algorithm based on representation of video shot, we have proposed a kind of new strategy, come
Realize the generation of cluster central point so that clustering algorithm can more capture the essence of video content.This new strategy is i.e. based on weighting
Peak value searching strategy, specific formula is：
γ=α * (ρ * δ)+(1- α) * δ, wherein, α is parameter preset, and the span of the parameter is that 0~0.5, ρ is local
Distance is put centered on density, δ, γ is cluster value.
In the acquisition process of video frequency abstract, the time with relatively low local density and larger high density point distance
Select frame even more important.This new strategy just causes this kind of candidate frame, is more appulsively considered the cluster center of video frequency abstract
Point.
In step S105, according to each cluster central point, the static video frequency abstract of pending video is generated.
In embodiments of the present invention, it is not that each cluster central point can be made in multiple cluster central points that cluster is obtained
For the frame in static video frequency abstract, so, these cluster central points are screened.
Specifically, the cluster value of each cluster central point is arranged, obtains the scatter diagram of all cluster values.From the scatterplot
The cluster value that increasing degree or slope significantly increase suddenly is obtained in figure, and this cluster value is set to threshold value.Again by institute
The cluster value for having cluster central point is compared one by one with the threshold value, when cluster value exceedes the threshold value, by this cluster value correspondence cluster
The candidate frame of central point is left a frame of static video frequency abstract.Finally, complete static video frequency abstract is generated.
In the present invention is implemented, the candidate frame of pending video is extracted using a singular value decomposition algorithm first, is passed through
Bag of words generation is used for representing the histogram of these candidate frames, significantly reduces the redundancy of frame in video.Then, using base
In the high density peak value searching clustering algorithm of representation of video shot, all candidate frames are clustered, with the Nogata according to frame of video
Figure adaptively produces a number of Cu Lei centers, it is to avoid the quantity at Cu Lei centers is pre-set before cluster, and need not be held
Row iteration process, is effectively improved the Stability and adaptability of cluster, reduces the time complexity of cluster.Finally, use
Pre-set strategy carries out the screening of cluster centre, generates more representational static video frequency abstract.So as to effectively carry
The formation efficiency and generation quality of high static video frequency abstract.
Can be with one of ordinary skill in the art will appreciate that realizing that all or part of step in above-described embodiment method is
The hardware of correlation is instructed to complete by program, described program can be stored in a computer read/write memory medium,
Described storage medium, such as ROM/RAM, disk, CD.
Embodiment two：
Fig. 2 shows the structure of the generating means for the static video frequency abstract that the embodiment of the present invention two is provided, for the ease of saying
It is bright, the part related to the embodiment of the present invention is illustrate only, including：
Video reception module 21, the pending video for receiving user's input；
Candidate's frame extraction module 22, for carrying out pre-sampling to pending video by singular value decomposition algorithm, to extract
The candidate frame of pending video；
Histogram representation module 23, for according to bag of words algorithm, the histogram of all candidate frames to be generated respectively；
Computing module 24 is clustered, for entering by the high density peak search algorithm based on representation of video shot to all histograms
Row cluster, and obtain the cluster central point after cluster；And
Video summary generation module 25, for according to each cluster central point, generating the static video frequency abstract of pending video.
Preferably, as shown in figure 3, candidate's frame extraction module 22 also includes vector generation module 321, eigenmatrix builds mould
Block 322, singular value decomposition module 323 and candidate frame determining module 324, wherein：
Vector generation module 321, the time varying characteristic vector for generating each input frame in pending video；
Eigenmatrix builds module 322, for according to time varying characteristic vector, being followed successively by all input frame construction feature squares
Battle array, time varying characteristic vector of each eigenmatrix comprising preset window size, continuous input frame；
Singular value decomposition module 323, for carrying out singular value decomposition to all eigenmatrixes, to obtain after singular value decomposition
Singular value matrix, and according to singular value matrix, it is determined that the order of each eigenmatrix；And
Candidate frame determining module 324, for being successively compared the order of adjacent eigenmatrix, works as second characteristic matrix
Order be more than fisrt feature rank of matrix when, last input frame corresponding to second characteristic matrix is set to candidate frame,
Fisrt feature matrix is any feature matrix in all eigenmatrixes, and second characteristic matrix is with the in all eigenmatrixes
The adjacent next eigenmatrix of one eigenmatrix.
Preferably, as shown in figure 4, histogram representation module 23 also includes characteristic extracting module 431, code book generation module
432 and histogram generation module 433, wherein：
Characteristic extracting module 431, the characteristics of image for extracting all candidate frames；
Code book generation module 432, for according to all characteristics of image, the condition code of each candidate frame to be generated by clustering
This；And
Histogram generation module 433, for the feature distribution in all feature code books, generation is used for representing each time
Select the histogram of frame.
Preferably, as shown in figure 5, cluster computing module 24 also includes candidate frame distance calculation module 541, local density's meter
Module 542, high density point distance calculation module 543 and cluster central point acquisition module 544 are calculated, wherein：
Candidate frame distance calculation module 541, for according to all histograms, calculating each two candidate frame in all candidate frames
The distance between；
Local density's computing module 542, for according to the distance between each two candidate frame and default cut-off distance, meter
Calculate the local density of each candidate frame；
High density point distance calculation module 543, for according to all local densities, calculating the high density point of each candidate frame
Distance；And
Cluster central point acquisition module 544, for the local density according to each candidate frame and high density point distance, obtains cluster
Central point.
Preferably, cluster central point acquisition module 544 also includes cluster value computing module 5441, wherein：
Cluster value computing module 5441, for the local density according to each candidate frame and high density point distance, using base
Strategy is clustered in the peak value searching of weighting, the corresponding cluster value of each candidate frame is calculated, the peak value searching cluster plan based on weighting
Slightly formula be:
γ=α * (ρ * δ)+(1- α) * δ, wherein, γ is cluster value, and α is parameter preset, and ρ is local density, and δ is high density
Point distance.
Preferably, as shown in fig. 6, video summary generation module 25 also includes
Threshold setting module 651, for the cluster value of each cluster central point to be arranged, obtains in all cluster values and increases
The cluster value that long amplitude or slope significantly increase suddenly, and by increasing degree or slope significantly increase suddenly it is poly-
Class value is set to threshold value；And
Video frequency abstract frame setup module 652, for each cluster value to be compared with threshold value, when cluster value exceedes threshold value
When, the candidate frame of cluster central point corresponding to cluster value is set to the frame of video in static video frequency abstract.
In embodiments of the present invention, first using a singular value decomposition algorithm, the candidate frame of pending video is extracted, then
By bag of words, generation is used for representing the histogram of these candidate frames, then using the high density peak value based on representation of video shot
Clustering algorithm is searched for, all frame of video are clustered, and pre-set strategy is used in cluster in cluster process
The heart is selected, to generate more representational static video frequency abstract, so that the redundancy of frame in video is not only significantly reduced,
And a number of Cu Lei centers can adaptively be produced according to the histogram of frame of video in cluster, it is not required to pre-set cluster class
The quantity at center, no iterative process, the Stability and adaptability for being effectively improved cluster, the time for reducing cluster are complicated
Degree, and then so as to be effectively improved the formation efficiency and quality of static video frequency abstract.
In embodiments of the present invention, each module of the generating means of static video frequency abstract can be by corresponding hardware or software mould
Block realizes that each module can be independent soft and hardware module, a soft and hardware module can also be integrated into, herein not to limit
The system present invention.The embodiment of each module refers to the description of each step in previous embodiment one in the embodiment of the present invention,
It will not be repeated here.
The foregoing is merely illustrative of the preferred embodiments of the present invention, is not intended to limit the invention, all essences in the present invention
Any modifications, equivalent substitutions and improvements made within refreshing and principle etc., should be included in the scope of the protection.
Claims (12)
1. a kind of generation method of static video frequency abstract, it is characterised in that methods described comprises the steps：
Receive the pending video of user's input；
Pre-sampling is carried out to the pending video by singular value decomposition algorithm, to extract the candidate of the pending video
Frame；
According to bag of words algorithm, the histogram of all candidate frames is generated respectively；
All histograms are clustered by the high density peak search algorithm based on representation of video shot, and obtained after cluster
Cluster central point；
According to each cluster central point, the static video frequency abstract of the pending video is generated.
2. the method as described in claim 1, it is characterised in that carried out by singular value decomposition algorithm to the pending video
The step of pre-sampling, candidate frame to extract the pending video, including：
Generate the time varying characteristic vector of each input frame in the pending video；
According to time varying characteristic vector, all input frame construction feature matrixes, each eigenmatrix bag are followed successively by
The time varying characteristic vector of size containing preset window, continuous input frame；
Singular value decomposition is carried out to all eigenmatrixes, to obtain the corresponding singular value matrix of each eigenmatrix,
And according to the singular value matrix, determine the order of each eigenmatrix；
The order of adjacent eigenmatrix is compared successively, when the order of second characteristic matrix is more than fisrt feature rank of matrix
When, last input frame corresponding to the second characteristic matrix is set to candidate frame, the fisrt feature matrix is institute
State any feature matrix in all eigenmatrixes, the second characteristic matrix is with described the in all eigenmatrixes
The adjacent next eigenmatrix of one eigenmatrix.
3. the method as described in claim 1, it is characterised in that according to bag of words algorithm, generates all candidates respectively
The histogrammic step of frame, including：
Extract the characteristics of image of all candidate frames；
According to all characteristics of image, the feature code book of each candidate frame is generated by clustering；
According to the feature distribution in all feature code books, the histogram for representing each candidate frame is generated.
4. the method as described in claim 1, it is characterised in that pass through the high density peak search algorithm pair based on representation of video shot
All histograms are clustered, and obtain cluster after cluster central point the step of, including：
According to all histograms, the distance between each two candidate frame in all candidate frames is calculated；
According to the distance between described each two candidate frame and default cut-off distance, the corresponding office of each candidate frame is calculated
Portion's density；
According to all local densities, the corresponding high density point distance of each candidate frame is calculated；
According to the corresponding local density of each candidate frame and high density point distance, the cluster central point is obtained.
5. method as claimed in claim 4, it is characterised in that according to the corresponding local density of each candidate frame and highly dense
Degree point distance, the step of obtaining the cluster central point, including：
According to the local density of each candidate frame and high density point distance, plan is clustered using the peak value searching based on weighting
Slightly, the corresponding cluster value of each candidate frame is calculated, the formula of the peak value searching cluster strategy based on weighting is：
γ=α * (ρ * δ)+(1- α) * δ, wherein, γ is the cluster value, and α is parameter preset, and ρ is the local density, and δ is institute
State high density point distance.
6. the method as described in claim 1, it is characterised in that according to each cluster central point, described pending regard is generated
The step of static video frequency abstract of frequency, including：
The cluster value of each cluster central point is arranged, increasing degree or slope in all cluster values is obtained prominent
The cluster value so significantly increased, and the cluster value that the increasing degree or slope significantly increase suddenly is set to threshold value；
Each cluster value is compared with the threshold value, when the cluster value exceedes the threshold value, by the cluster
The candidate frame of cluster central point is set to the frame of video in the static video frequency abstract corresponding to value.
7. a kind of generating means of static video frequency abstract, it is characterised in that described device includes：
Video reception module, the pending video for receiving user's input；
Candidate's frame extraction module, for carrying out pre-sampling to the pending video by singular value decomposition algorithm, to extract
State the candidate frame of pending video；
Histogram representation module, for according to bag of words algorithm, the histogram of all candidate frames to be generated respectively；
Computing module is clustered, for being carried out by the high density peak search algorithm based on representation of video shot to all histograms
Cluster, and obtain the cluster central point after cluster；And
Video summary generation module, for according to each cluster central point, the static video for generating the pending video to be plucked
Will.
8. device as claimed in claim 7, it is characterised in that candidate's frame extraction module includes：
Vector generation module, the time varying characteristic vector for generating each input frame in the pending video；
Eigenmatrix builds module, for according to time varying characteristic vector, being followed successively by all input frame construction feature squares
Battle array, time varying characteristic vector of each eigenmatrix comprising preset window size, continuous input frame；
Singular value decomposition module, for carrying out singular value decomposition to all eigenmatrixes, to obtain each feature square
The corresponding singular value matrix of battle array, and according to the singular value matrix, determine the order of each eigenmatrix；And
Candidate frame determining module, for being successively compared the order of adjacent eigenmatrix, when the order of second characteristic matrix is big
When fisrt feature rank of matrix, last input frame corresponding to the second characteristic matrix is set to candidate frame, institute
It is any feature matrix in all eigenmatrixes to state fisrt feature matrix, and the second characteristic matrix is described all
The next eigenmatrix adjacent with the fisrt feature matrix in eigenmatrix.
9. device as claimed in claim 7, it is characterised in that the histogram representation module includes：
Characteristic extracting module, the characteristics of image for extracting all candidate frames；
Code book generation module, for according to all characteristics of image, the condition code of each candidate frame to be generated by clustering
This；And
Histogram generation module, for the feature distribution in all feature code books, is generated for representing described each
The histogram of candidate frame.
10. device as claimed in claim 7, it is characterised in that the cluster computing module includes：
Candidate frame distance calculation module, for according to all histograms, calculating each two candidate in all candidate frames
The distance between frame；
Local density's computing module, for according to the distance between described each two candidate frame and default cut-off distance, calculating
The local density of each candidate frame；
High density point distance calculation module, for according to all local densities, calculating the corresponding high density of each candidate frame
Point distance；And
Cluster central point acquisition module, for the local density according to each candidate frame and high density point distance, obtains described
Cluster central point.
11. device as claimed in claim 10, it is characterised in that the cluster central point acquisition module includes：
Cluster value computing module, for the local density according to each candidate frame and high density point distance, using based on adding
The peak value searching cluster strategy of power, calculates the corresponding cluster value of each candidate frame, and the peak value searching based on weighting gathers
The formula of class strategy is：
γ=α * (ρ * δ)+(1- α) * δ, wherein, γ is the cluster value, and α is parameter preset, and ρ is the local density, and δ is institute
State high density point distance.
12. device as claimed in claim 7, it is characterised in that the video summary generation module includes：
Threshold setting module, for the cluster value of each cluster central point to be arranged, is obtained in all cluster values
The cluster value that increasing degree or slope significantly increase suddenly, and the increasing degree or slope are significantly increased suddenly
Big cluster value is set to threshold value；And
Video frequency abstract frame setup module, for each cluster value to be compared with the threshold value, when the cluster value is super
When crossing the threshold value, the candidate frame of cluster central point corresponding to the cluster value is set to the video in the static video frequency abstract
Frame.
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| PCT/CN2017/072416 WO2018137126A1 (en) | 2017-01-24 | 2017-01-24 | Method and device for generating static video abstract |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN107223344A true CN107223344A (en) | 2017-09-29 |
Family
ID=59955073
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201780000556.2A Pending CN107223344A (en) | 2017-01-24 | 2017-01-24 | The generation method and device of a kind of static video frequency abstract |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN107223344A (en) |
| WO (1) | WO2018137126A1 (en) |
Cited By (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108848422A (en) * | 2018-04-19 | 2018-11-20 | 清华大学 | A kind of video abstraction generating method based on target detection |
| CN109819338A (en) * | 2019-02-22 | 2019-05-28 | 深圳岚锋创视网络科技有限公司 | A kind of automatic editing method, apparatus of video and portable terminal |
| CN109934142A (en) * | 2019-03-04 | 2019-06-25 | 北京字节跳动网络技术有限公司 | Method and apparatus for generating the feature vector of video |
| CN110223380A (en) * | 2019-06-11 | 2019-09-10 | 中国科学院自动化研究所 | Fusion is taken photo by plane and the scene modeling method of ground multi-view image, system, device |
| CN110650379A (en) * | 2019-09-26 | 2020-01-03 | 北京达佳互联信息技术有限公司 | Video abstract generation method and device, electronic equipment and storage medium |
| CN110996183A (en) * | 2019-07-12 | 2020-04-10 | 北京达佳互联信息技术有限公司 | Video abstract generation method, device, terminal and storage medium |
| CN111510724A (en) * | 2019-01-31 | 2020-08-07 | 北京小犀智能科技中心(有限合伙) | Equivalent video compression storage method and system based on image feature extraction |
| CN112579823A (en) * | 2020-12-28 | 2021-03-30 | 山东师范大学 | Video abstract generation method and system based on feature fusion and incremental sliding window |
| CN112861852A (en) * | 2021-01-19 | 2021-05-28 | 北京金山云网络技术有限公司 | Sample data screening method and device, electronic equipment and storage medium |
| CN112883782A (en) * | 2021-01-12 | 2021-06-01 | 上海肯汀通讯科技有限公司 | Method, device, equipment and storage medium for identifying putting behaviors |
| CN113038142A (en) * | 2021-03-25 | 2021-06-25 | 北京金山云网络技术有限公司 | Video data screening method and device and electronic equipment |
| CN114786039A (en) * | 2022-04-25 | 2022-07-22 | 海信电子科技(武汉)有限公司 | Server and video preview image making method |
| CN116233569A (en) * | 2023-05-06 | 2023-06-06 | 石家庄铁道大学 | Video abstract generation method based on motion information assistance |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112528025A (en) * | 2020-12-16 | 2021-03-19 | 平安科技（深圳）有限公司 | Text clustering method, device and equipment based on density and storage medium |
| CN112580563B (en) * | 2020-12-25 | 2024-02-06 | 北京百度网讯科技有限公司 | Video abstract generation method and device, electronic equipment and storage medium |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7257261B2 (en) * | 2001-12-28 | 2007-08-14 | Lg Electronics Inc. | Apparatus and method for generating thumbnail images |
| CN101404030A (en) * | 2008-11-05 | 2009-04-08 | 中国科学院计算技术研究所 | Method and system for periodic structure fragment detection in video |
| CN102098449A (en) * | 2010-12-06 | 2011-06-15 | 北京邮电大学 | Method for realizing automatic inside segmentation of TV programs by utilizing mark detection |
| CN103150373A (en) * | 2013-03-08 | 2013-06-12 | 北京理工大学 | Generation method of high-satisfaction video summary |
| CN104063883A (en) * | 2014-07-07 | 2014-09-24 | 杭州银江智慧医疗集团有限公司 | Surveillance video abstract generating method based on combination of object and key frames |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| FR2845186B1 (en) * | 2002-09-27 | 2004-11-05 | Thomson Licensing Sa | METHOD AND DEVICE FOR MEASURING SIMILARITY BETWEEN IMAGES |
-
2017
- 2017-01-24 CN CN201780000556.2A patent/CN107223344A/en active Pending
- 2017-01-24 WO PCT/CN2017/072416 patent/WO2018137126A1/en active Application Filing
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7257261B2 (en) * | 2001-12-28 | 2007-08-14 | Lg Electronics Inc. | Apparatus and method for generating thumbnail images |
| CN101404030A (en) * | 2008-11-05 | 2009-04-08 | 中国科学院计算技术研究所 | Method and system for periodic structure fragment detection in video |
| CN102098449A (en) * | 2010-12-06 | 2011-06-15 | 北京邮电大学 | Method for realizing automatic inside segmentation of TV programs by utilizing mark detection |
| CN103150373A (en) * | 2013-03-08 | 2013-06-12 | 北京理工大学 | Generation method of high-satisfaction video summary |
| CN104063883A (en) * | 2014-07-07 | 2014-09-24 | 杭州银江智慧医疗集团有限公司 | Surveillance video abstract generating method based on combination of object and key frames |
Non-Patent Citations (2)
| Title |
|---|
| ALEX, RODRIGUEZ，ALESSANDRO, LAIO: "Clustering by fast search and find of density peaks", 《SCIENCE (NEW YORK, N.Y.)》 * |
| 江健生，朱会萍: "基于直方图的K-means聚类中心自动检测方法", 《廊坊师范学院学报(自然科学版)》 * |
Cited By (22)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108848422B (en) * | 2018-04-19 | 2020-06-02 | 清华大学 | Video abstract generation method based on target detection |
| CN108848422A (en) * | 2018-04-19 | 2018-11-20 | 清华大学 | A kind of video abstraction generating method based on target detection |
| CN111510724A (en) * | 2019-01-31 | 2020-08-07 | 北京小犀智能科技中心(有限合伙) | Equivalent video compression storage method and system based on image feature extraction |
| US11955143B2 (en) | 2019-02-22 | 2024-04-09 | Arashi Vision Inc. | Automatic video editing method and portable terminal |
| CN109819338A (en) * | 2019-02-22 | 2019-05-28 | 深圳岚锋创视网络科技有限公司 | A kind of automatic editing method, apparatus of video and portable terminal |
| CN109819338B (en) * | 2019-02-22 | 2021-09-14 | 影石创新科技股份有限公司 | Automatic video editing method and device and portable terminal |
| CN109934142A (en) * | 2019-03-04 | 2019-06-25 | 北京字节跳动网络技术有限公司 | Method and apparatus for generating the feature vector of video |
| CN110223380A (en) * | 2019-06-11 | 2019-09-10 | 中国科学院自动化研究所 | Fusion is taken photo by plane and the scene modeling method of ground multi-view image, system, device |
| CN110996183A (en) * | 2019-07-12 | 2020-04-10 | 北京达佳互联信息技术有限公司 | Video abstract generation method, device, terminal and storage medium |
| CN110650379A (en) * | 2019-09-26 | 2020-01-03 | 北京达佳互联信息技术有限公司 | Video abstract generation method and device, electronic equipment and storage medium |
| CN110650379B (en) * | 2019-09-26 | 2022-04-01 | 北京达佳互联信息技术有限公司 | Video abstract generation method and device, electronic equipment and storage medium |
| CN112579823A (en) * | 2020-12-28 | 2021-03-30 | 山东师范大学 | Video abstract generation method and system based on feature fusion and incremental sliding window |
| CN112579823B (en) * | 2020-12-28 | 2022-06-24 | 山东师范大学 | Video abstract generation method and system based on feature fusion and incremental sliding window |
| CN112883782A (en) * | 2021-01-12 | 2021-06-01 | 上海肯汀通讯科技有限公司 | Method, device, equipment and storage medium for identifying putting behaviors |
| CN112883782B (en) * | 2021-01-12 | 2023-03-24 | 上海肯汀通讯科技有限公司 | Method, device, equipment and storage medium for identifying putting behaviors |
| CN112861852A (en) * | 2021-01-19 | 2021-05-28 | 北京金山云网络技术有限公司 | Sample data screening method and device, electronic equipment and storage medium |
| CN113038142A (en) * | 2021-03-25 | 2021-06-25 | 北京金山云网络技术有限公司 | Video data screening method and device and electronic equipment |
| CN113038142B (en) * | 2021-03-25 | 2022-11-01 | 北京金山云网络技术有限公司 | Video data screening method and device and electronic equipment |
| CN114786039A (en) * | 2022-04-25 | 2022-07-22 | 海信电子科技(武汉)有限公司 | Server and video preview image making method |
| CN114786039B (en) * | 2022-04-25 | 2024-03-26 | 海信电子科技(武汉)有限公司 | Server and video preview drawing manufacturing method |
| CN116233569A (en) * | 2023-05-06 | 2023-06-06 | 石家庄铁道大学 | Video abstract generation method based on motion information assistance |
| CN116233569B (en) * | 2023-05-06 | 2023-07-11 | 石家庄铁道大学 | Video abstract generation method based on motion information assistance |
Also Published As
| Publication number | Publication date |
|---|---|
| WO2018137126A1 (en) | 2018-08-02 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107223344A (en) | The generation method and device of a kind of static video frequency abstract | |
| Tran et al. | Learning spatiotemporal features with 3d convolutional networks | |
| Zhang et al. | Filtered channel features for pedestrian detection. | |
| US8467610B2 (en) | Video summarization using sparse basis function combination | |
| CN108335307A (en) | Adaptive tobacco leaf picture segmentation method and system based on dark primary | |
| Duta et al. | Histograms of motion gradients for real-time video classification | |
| WO2014022254A2 (en) | Identifying key frames using group sparsity analysis | |
| Zou et al. | Crowd counting via hierarchical scale recalibration network | |
| CN110147469A (en) | A kind of data processing method, equipment and storage medium | |
| Wang et al. | RGB stream is enough for temporal action detection | |
| Mahmoud et al. | Unsupervised video summarization via dynamic modeling-based hierarchical clustering | |
| Fu et al. | Image aesthetics assessment using composite features from off-the-shelf deep models | |
| CN107886109B (en) | Video abstraction method based on supervised video segmentation | |
| Korzh et al. | Stacking approach for CNN transfer learning ensemble for remote sensing imagery | |
| CN101504723B (en) | Projection space establishing method and apparatus | |
| Sidiropoulos et al. | Video tomographs and a base detector selection strategy for improving large-scale video concept detection | |
| Gorisse et al. | IRIM at TRECVID 2010: semantic indexing and instance search | |
| Yu et al. | Cascaded temporal spatial features for video action recognition | |
| Varol et al. | Extreme learning machine for large-scale action recognition | |
| Yan et al. | A comparison of l_1 norm and l_2 norm multiple kernel SVMs in image and video classification | |
| CN109409381A (en) | The classification method and system of furniture top view based on artificial intelligence | |
| Zhong et al. | Prediction system for activity recognition with compressed video | |
| Khan et al. | Semantic analysis of news based on the deep convolution neural network | |
| Shu et al. | Understanding deep representations through random weights | |
| Martins et al. | Temporal-and spatial-driven video summarization using optimum-path forest |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20170929 |
|
| RJ01 | Rejection of invention patent application after publication |