CN102129707A - Heterogeneous feature dimension reduction-based two-dimensional role cartoon generation method - Google Patents

Abstract

The invention discloses a heterogeneous feature dimension reduction-based two-dimensional role cartoon generation method. In the method, the functions of indexing and editing similar pose-containing cartoon pictures in a database are realized based on real video pictures by utilizing knowledge obtained by machine learning and feature extraction. The method comprises the following steps of: inputting sequence key frames of an action performed by real people; extracting feature expressions of the sequence key frames by using a system; computing the Euclidean distance between each feature expression and a cartoon neighboring node in projection space according the feature expressions and a projection matrix trained by the method; returning a closest cartoon neighboring node serving as a search result; and editing on an obtained cartoon key frame sequence by a user. In the method, implementation difficulties caused by taking the cartoon pictures as indexes in a conventional method are reduced by taking actually shot video pictures as the indexes; simultaneously the problem of narrow applicability caused by the conventional method which can only aim at isomorphism feature dimension reduction is solved by disclosing a heterogeneous feature dimensional reduction method; therefore, the accuracy is improved and the application range is enlarged.

Description

Two-dimensional character cartoon generation method based on the heterogeneous characteristic dimensionality reduction

Technical field

The present invention relates to a kind of two-dimensional character cartoon generation method, be specifically related to a kind of two-dimensional cartoon character generation method of handling and retrieving based on heterogeneous characteristic, belong to the general field of computer animation and computer machine study based on the heterogeneous characteristic dimensionality reduction.

Background technology

The extensive application in industry and entertainment field along with computer two-dimensional animation and computer machine study, the two-dimensional cartoon generation method of reusing based on existing two-dimensional cartoon character video data becomes an important research focus gradually.Although still be in the stage of exploration at present about the research in this field, produced some more representational methods.

The researcher has developed some two-dimensional cartoon synthetic methods of reusing based on multi-medium data.Such as, be published in the paper " Video Textures " on the meeting SIGGRAPH in 2000, extracted and at first analyzed the cartoon video fragment, and then put in order the method for synthesizing new cartoon video by changing original frame of video.Be published in the paper " Motion texture:a two-level statistical model for character motion synthesis " on the meeting SIGGRAPH in 2002, the researcher successfully obtains new three-dimensional cartoon motion by reusing the three-dimensional motion data.Above method has excited us to develop a kind of generation method based on two-dimensional character cartoon data reusing.

The method that several comparative maturities have been arranged at present based on the two-dimensional cartoon character data reusing, the synthetic technology that on behalf of present popular content-based analysis, these methods retrieve.Such as, be published in the paper " Cartoon Textures " on the meeting SCA in 2004, need manual initial frame and the terminal frame that needs in the synthetic cartoon video frame of specifying of user, improved synthetic efficient and practicality.But the problem that causes thus is that the result who is synthesized can be limited to a great extent by user's appointment, and uncontrollable synthetic direction of user and result.Because method itself can not be carried out iteration or repeatedly, if the unsatisfied result of user, can't make amendment and feedback processing, and also exist in the building-up process for the concrete implementation detail of how to specify cartoon role to combine and explain unclear problem with background frames.Be published in the paper " Perspective-aware Cartoon Clips Synthesis " on the periodical CAVW in 2008, reusing of two-dimensional cartoon resource concentrates on this aspect of cartoon video section, really be not deep into the level of frame, synthesize new cartoon video by splicing existing two-dimensional cartoon fragment.Problem is that in the new synthetic cartoon video, the order of original fragment frame interior was not made amendment, and existed the situation of the uncontrollable building-up process of user yet.Based on the defective of above-mentioned two kinds of methods, we wish to develop a kind of content-based analysis retrieval and can realize the two-dimensional character cartoon generation method that omnidistance user is controlled.

The key point of method that we propose, except the top cartoon retrieval of analyzing that how to design content-based analysis, also be how to finish under the heterogeneous characteristic situation based on the cartoon of gesture recognition synthetic, such as similarity how to weigh attitude between real video key frame and the cartoon video key frame quantitatively, and the research of this respect is fewer at present.In order to realize higher gesture recognition accuracy rate, we have adopted different feature extracting methods.Be published in the paper " Silhouette representation and matching for 3D pose discrimination-A comparative study " on the periodical IVC in 2010, several feature extracting methods relatively commonly used in the gesture recognition field have been proposed, and at different towards having done careful deep comparison test.In our method,, finally utilized directivity histogram of gradients (HOG) feature extracting method of being mentioned in this paper to obtain the feature representation matrix of cartoon character by the contrast experiment; Occupy the feature representation matrix that figure (OM) feature extracting method obtains true personage.

Summary of the invention

The objective of the invention is limitation, a kind of two-dimensional cartoon generation method of finishing the cartoon gesture recognition based on heterogeneous characteristic analysis and dimensionality reduction is provided for the two-dimensional cartoon synthetic method that overcomes present content-based analysis retrieval.

Step based on the two-dimensional character cartoon generation method of heterogeneous characteristic dimensionality reduction is as follows:

1) from the two-dimensional cartoon video, extracts the video-frequency band that comprises the complete action of role's cartoon, and according to the movement content of role's cartoon and towards, take corresponding action video by true personage's performance, after the action video process video image technical finesse of the action video that the diagonal angle colour atla is logical and true personage's performance, utilize self-defining extraction method of key frame to extract the key frame of the action video and the action video that true personage performs of role's cartoon, obtain cartoon key frame and true key frame, and cartoon key frame and true key frame are carried out normalization and centralization processing; The cartoon key frame is utilized different feature extracting methods with true key frame, obtain cartoon key frame feature representation matrix and true key frame feature representation matrix, and the feature representation matrix classified according to self-defining action classification, set up " cartoon-true " role data storehouse;

2) from the action video that the true personage who takes performs, utilize self-defining extraction method of key frame to extract the key frame of the action video of true personage's performance, obtain retrieving true key frame, and true key frame carries out normalization and centralization is handled to retrieving; Utilize feature extracting method to obtain the true key frame feature representation matrix of retrieval to retrieving true key frame;

3) cartoon key frame feature representation matrix and true key frame feature representation matrix be by self-defining heterogeneous characteristic dimensionality reduction algorithm, obtains the dimensionality reduction projecting direction matrix of cartoon key frame feature representation matrix of cartoon key frame feature representation matrix and true key frame feature representation matrix correspondence and the dimensionality reduction projecting direction matrix of true key frame feature representation matrix through training; Retrieve true key frame sequence signature and express matrix, express the true key frame sequence dimensionality reduction feature representation matrix of dimensionality reduction projecting direction matrix acquisition retrieval that matrix multiply by true key frame feature representation matrix with the true key frame sequence signature of retrieval, and in projector space, calculate in " cartoon-true " the role data storehouse that obtains and the nearest cartoon key frame sequence of the true key frame sequence dimensionality reduction feature representation matrix of retrieval, at last the cartoon key frame sequence that calculates is returned; The user obtains final cartoon effect video in enterprising edlin of cartoon key frame sequence and the interpolation returned.

Described step 1) comprises:

From the two-dimensional cartoon character video, extract the cartoon video V that comprises complete cartoon character action fragment _Cart, the true personage of cause is according to V _CartIn cartoon character movement content and towards before monocular-camera, imitating performance, obtain to include the real video V of complete true figure action fragment _Real

To from V _CartAnd V _RealIn the frame of video playing up out, utilize Hausdorff distance algorithm feature to obtain the cartoon distance matrix

With the actual distance matrix N wherein ₁And n ₂Be respectively the number of frames that cartoon and real video are played up out,

Hausdorff distance in the expression cartoon video frame between i frame and the j frame,

Hausdorff distance in the expression real video frame between i frame and the j frame, matrix M _CartAnd M _RealIn each multiply by coefficient respectively

With Finish normalization, d wherein _{Cart_max}And d _{Real_max}Be respectively matrix M _CartAnd M _RealIn maximal value, obtaining through normalized distance matrix M _CartAnd M _RealAfterwards, according to preset threshold

With

Come the diagonal values in the filtered matrix, will obtain respectively

With

Pairing i frame obtains the cartoon key frame thus as key frame With true key frame

Wherein n is both quantity;

At the cartoon key frame

Obtain cartoon key frame feature representation matrix according to directivity histogram of gradients feature extracting method

D wherein ₁For each key frame characteristic of correspondence is expressed vector

Dimension; Similarly, at true key frame

Obtain true key frame feature representation matrix according to occupying the figure feature extracting method

D wherein ₂For each key frame characteristic of correspondence is expressed vector

Dimension; With X ₁And X ₂In all key frames all be divided into the r class according to the difference of movement content, thereby make all have in each class the cartoon of equal number and true key frame to form X ₁And X ₂Correspondence one by one on the classification aspect;

To the X that is obtained ₁And X ₂, respectively through obtain the matrix of centralization as down conversion:

$X_1=-\frac{1}{2}{\mathrm{HX}}_{1}H,$ $X_2=-\frac{1}{2}{\mathrm{HX}}_{2}H---1$

Wherein And N is X ₁With X ₂Sample size n, so far finish normalization and centralization operation, thereby set up " cartoon-true " role data storehouse.

Described step 3) comprises:

The cartoon of normalization and centralization and real features are expressed matrix X ₁And X ₂, calculate the dimensionality reduction projecting direction matrix W that obtains cartoon key frame feature representation matrix according to following objective function algorithm ₁Dimensionality reduction projecting direction matrix W with true key frame feature representation matrix ₂:

$\underset{F,W_1,W_2}{\min }\mathrm{tr}\left(F^T{L}_{\mathrm{syn}}F\right)-\mathrm{\α}{\left|\right|X_1{W}_1-F\left|\right|}_F^2+{\mathrm{\γ}}_1{\left|\right|W_1\left|\right|}_F^2+\mathrm{\β}{\left|\right|X_2{W}_2-F\left|\right|}_F^2+{\mathrm{\γ}}_2{\left|\right|W_2\left|\right|}_F^2+$

$\mathrm{\δ}{\left|\right|F-Y\left|\right|}_F^2---2$

Alpha, gamma ₁, beta, gamma ₂, δ is a heterogeneous equilibrium degree coefficient, matrix Y=[y ₁, y ₂..., y _n] ∈ 0,1} ^{N * r}, wherein work as cartoon samples And authentic specimen

When all belonging to k classification, Y then _Ik=1; Otherwise Y _Ik=0;

With

Belong to same classification, form the Y matrix of full rank; Tr (.) is the mark operational character; The Lagrangian matrix L of cartoon key frame feature representation matrix and cartoon key frame feature representation matrix ¹=D ¹-A ¹, L ²=D ²-A ²

L _syn＝[((L ¹+L ²)/2)′+((L ¹+L ²)/2)]/2 3

So L _Syn=L ' _SynWherein

Represent not this normal form of Luo Beini crow, and

Satisfy all matrix Z; Through differentiate, can obtain at last:

W ₁＝δB ₁(αU+βV+E) ^-1Y 4

W ₂＝δB ₂(αU+βV+E) ^-1Y 5

Wherein:

$U=B_1^T{X}_1^T{X}_1{B}_1-X_1{B}_1-B_1^T{X}_1^T+1---6$

$V=B_2^T{X}_2^T{X}_2{B}_2-X_2{B}_2-B_2^T{X}_2^T+1---7$

$E=L_{\mathrm{syn}}+{\mathrm{\γ}}_1{B}_1^T{B}_1+{\mathrm{\γ}}_2{B}_2^T{B}_2+\mathrm{\δI}---8$

Wherein:

$B_1={({\mathrm{\αX}}_1^T{X}_1+{\mathrm{\γ}}_{1}I)}^{-1}{\mathrm{\αX}}_1^T---9$

$B_2={({\mathrm{\βX}}_2^T{X}_2+{\mathrm{\γ}}_{2}I)}^{-1}{\mathrm{\βX}}_2^T---10$

So far, obtained the dimensionality reduction projecting direction matrix W of cartoon key frame feature representation matrix ₁Dimensionality reduction projecting direction matrix W with true key frame feature representation matrix ₂

When the true key frame of input retrieval, obtain the corresponding true key frame feature representation of retrieval matrix according to occupying the figure feature extracting method

Wherein m is the true quantity of key frames of retrieval; Dimensionality reduction projecting direction matrix W according to the true key frame feature representation matrix that obtains ₂, calculate the retrieval dimensionality reduction real features expression formula in the space of the true key frame of retrieval behind dimensionality reduction

Dimensionality reduction projecting direction matrix W according to the cartoon key frame feature representation matrix that obtains ₁, calculate the dimensionality reduction cartoon feature representation formula X ' in the space of cartoon key frame feature representation matrix behind dimensionality reduction in " cartoon-true " role data storehouse ₁=X ₁W ₁∈ R ^{N * r}, wherein r is the dimension in dimensionality reduction space;

In the dimensionality reduction space, with retrieval dimensionality reduction real features expression formula

In each element as index separately, calculate and dimensionality reduction cartoon feature representation formula X ₁' in the Euclidean distance of each element, and several of layback minimum are as cartoon indexed results X _Result, finish the process of returning the cartoon key frame that obtains with true key frame as index;

The user is at cartoon indexed results X _ResultOn carry out the interest that operations such as deformation, stretching, replacement strengthen the cartoon key frame, obtain final cartoon video by approach based on linear interpolation at last.

The present invention by with true capture video picture as index, reduced the performance difficulty that classic method is brought as index with the cartoon picture; Simultaneously by proposing the heterogeneous characteristic dimension reduction method, solved the narrow problem of applicability that classic method can only be brought at isomorphism feature dimensionality reduction, improve accuracy, enlarged the scope of using.

Description of drawings

The present invention is further illustrated below in conjunction with the drawings and specific embodiments.

Fig. 1 is a method algorithm flow chart of the present invention.

Embodiment

Step based on the two-dimensional character cartoon generation method of heterogeneous characteristic dimensionality reduction is as follows:

1) from the two-dimensional cartoon video, extracts the video-frequency band that comprises the complete action of role's cartoon, and according to the movement content of role's cartoon and towards, take corresponding action video by true personage's performance, after the action video process video image technical finesse of the action video that the diagonal angle colour atla is logical and true personage's performance, utilize self-defining extraction method of key frame to extract the key frame of the action video and the action video that true personage performs of role's cartoon, obtain cartoon key frame and true key frame, and cartoon key frame and true key frame are carried out normalization and centralization processing; The cartoon key frame is utilized different feature extracting methods with true key frame, obtain cartoon key frame feature representation matrix and true key frame feature representation matrix, and the feature representation matrix classified according to self-defining action classification, set up " cartoon-true " role data storehouse;

2) from the action video that the true personage who takes performs, utilize self-defining extraction method of key frame to extract the key frame of the action video of true personage's performance, obtain retrieving true key frame, and true key frame carries out normalization and centralization is handled to retrieving; Utilize feature extracting method to obtain the true key frame feature representation matrix of retrieval to retrieving true key frame;

3) cartoon key frame feature representation matrix and true key frame feature representation matrix be by self-defining heterogeneous characteristic dimensionality reduction algorithm, obtains the dimensionality reduction projecting direction matrix of cartoon key frame feature representation matrix of cartoon key frame feature representation matrix and true key frame feature representation matrix correspondence and the dimensionality reduction projecting direction matrix of true key frame feature representation matrix through training; Retrieve true key frame sequence signature and express matrix, express the true key frame sequence dimensionality reduction feature representation matrix of dimensionality reduction projecting direction matrix acquisition retrieval that matrix multiply by true key frame feature representation matrix with the true key frame sequence signature of retrieval, and in projector space, calculate in " cartoon-true " the role data storehouse that obtains and the nearest cartoon key frame sequence of the true key frame sequence dimensionality reduction feature representation matrix of retrieval, at last the cartoon key frame sequence that calculates is returned; The user obtains final cartoon effect video in enterprising edlin of cartoon key frame sequence and the interpolation returned.

Described step 1) comprises:

From the two-dimensional cartoon character video, extract the cartoon video V that comprises complete cartoon character action fragment _Cart, the true personage of cause is according to V _CartIn cartoon character movement content and towards before monocular-camera, imitating performance, obtain to include the real video V of complete true figure action fragment _Real

To from V _CartAnd V _RealIn the frame of video playing up out, utilize Hausdorff distance algorithm feature to obtain the cartoon distance matrix

With the actual distance matrix

N wherein ₁And n ₂Be respectively the number of frames that cartoon and real video are played up out,

Hausdorff distance in the expression cartoon video frame between i frame and the j frame,

Hausdorff distance in the expression real video frame between i frame and the j frame, matrix M _CartAnd M _RealIn each multiply by coefficient respectively

With

Finish normalization, d wherein _{Cart_max}And d _{Real_max}Be respectively matrix M _CartAnd M _RealIn maximal value, obtaining through normalized distance matrix M _CartAnd M _RealAfterwards, according to preset threshold

With Come the diagonal values in the filtered matrix, will obtain respectively

With Pairing i frame obtains the cartoon key frame thus as key frame With true key frame

Wherein n is both quantity;

At the cartoon key frame

Obtain cartoon key frame feature representation matrix according to directivity histogram of gradients feature extracting method

D wherein ₁For each key frame characteristic of correspondence is expressed vector

Dimension; Similarly, at true key frame Obtain true key frame feature representation matrix according to occupying the figure feature extracting method

D wherein ₂For each key frame characteristic of correspondence is expressed vector

Dimension; With X ₁And X ₂In all key frames all be divided into the r class according to the difference of movement content, thereby make all have in each class the cartoon of equal number and true key frame to form X ₁And X ₂Correspondence one by one on the classification aspect;

To the X that is obtained ₁And X ₂, respectively through obtain the matrix of centralization as down conversion:

$X_1=-\frac{1}{2}{\mathrm{HX}}_{1}H,$ $X_2=-\frac{1}{2}{\mathrm{HX}}_{2}H---1$

Wherein

And N is X ₁With X ₂Sample size n, so far finish normalization and centralization operation, thereby set up " cartoon-true " role data storehouse.

Described step 3) comprises:

The cartoon of normalization and centralization and real features are expressed matrix X ₁And X ₂, calculate the dimensionality reduction projecting direction matrix W that obtains cartoon key frame feature representation matrix according to following objective function algorithm ₁Dimensionality reduction projecting direction matrix W with true key frame feature representation matrix ₂:

$\underset{F,W_1,W_2}{\min }\mathrm{tr}\left(F^T{L}_{\mathrm{syn}}F\right)-\mathrm{\α}{\left|\right|X_1{W}_1-F\left|\right|}_F^2+{\mathrm{\γ}}_1{\left|\right|W_1\left|\right|}_F^2+\mathrm{\β}{\left|\right|X_2{W}_2-F\left|\right|}_F^2+{\mathrm{\γ}}_2{\left|\right|W_2\left|\right|}_F^2+$

$\mathrm{\δ}{\left|\right|F-Y\left|\right|}_F^2---2$

Alpha, gamma ₁, beta, gamma ₂, δ is a heterogeneous equilibrium degree coefficient, matrix Y=[y ₁, y ₂..., y _n] ∈ 0,1} ^{N * r}, wherein work as cartoon samples

And authentic specimen

When all belonging to k classification, Y then _Ik=1; Otherwise Y _Ik=0;

With

Belong to same classification, form the Y matrix of full rank; Tr (.) is the mark operational character; The Lagrangian matrix L of cartoon key frame feature representation matrix and cartoon key frame feature representation matrix ¹=D ¹-A ¹, L ²=D ²-A ²

L _syn＝[((L ¹+L ²)/2)′+((L ¹+L ²)/2)]/2 3

So L _Syn=L ' _SynWherein

Represent not this normal form of Luo Beini crow, and Satisfy all matrix Z; Through differentiate, can obtain at last:

W ₁＝δB ₁(αU+βV+E) ^-1Y 4

W ₂＝δB ₂(αU+βV+E) ^-1Y 5

Wherein:

$U=B_1^T{X}_1^T{X}_1{B}_1-X_1{B}_1-B_1^T{X}_1^T+1---6$

$V=B_2^T{X}_2^T{X}_2{B}_2-X_2{B}_2-B_2^T{X}_2^T+1---7$

$E=L_{\mathrm{syn}}+{\mathrm{\γ}}_1{B}_1^T{B}_1+{\mathrm{\γ}}_2{B}_2^T{B}_2+\mathrm{\δI}---8$

Wherein:

$B_1={({\mathrm{\αX}}_1^T{X}_1+{\mathrm{\γ}}_{1}I)}^{-1}{\mathrm{\αX}}_1^T---9$

$B_2={({\mathrm{\βX}}_2^T{X}_2+{\mathrm{\γ}}_{2}I)}^{-1}{\mathrm{\βX}}_2^T---10$

So far, obtained the dimensionality reduction projecting direction matrix W of cartoon key frame feature representation matrix ₁Dimensionality reduction projecting direction matrix W with true key frame feature representation matrix ₂

When the true key frame of input retrieval, obtain the corresponding true key frame feature representation of retrieval matrix according to occupying the figure feature extracting method

Wherein m is the true quantity of key frames of retrieval; Dimensionality reduction projecting direction matrix W according to the true key frame feature representation matrix that obtains ₂, calculate the retrieval dimensionality reduction real features expression formula in the space of the true key frame of retrieval behind dimensionality reduction Dimensionality reduction projecting direction matrix W according to the cartoon key frame feature representation matrix that obtains ₁, calculate the dimensionality reduction cartoon feature representation formula X ' in the space of cartoon key frame feature representation matrix behind dimensionality reduction in " cartoon-true " role data storehouse ₁=X ₁W ₁∈ R ^{N * r}, wherein r is the dimension in dimensionality reduction space;

In the dimensionality reduction space, with retrieval dimensionality reduction real features expression formula

In each element as index separately, calculate and dimensionality reduction cartoon feature representation formula X ₁' in the Euclidean distance of each element, and several of layback minimum are as cartoon indexed results X _Result, finish the process of returning the cartoon key frame that obtains with true key frame as index;

The user is at cartoon indexed results X _ResultOn carry out the interest that operations such as deformation, stretching, replacement strengthen the cartoon key frame, obtain final cartoon video by approach based on linear interpolation at last.

Embodiment

1) entire flow of method as shown in Figure 1.At first from traditional two-dimensional cartoon character video,, manually extract the cartoon video V that plurality of sections includes complete action fragment according to the integrality and the representativeness of action _Cart, in the present embodiment, from manually having extracted about cartoon video that contained complete action fragment in 29 minutes 102 video segments altogether; According to V _CartIn cartoon character movement content and towards, the true personage of cause is according to V _CartIn action content and towards before monocular-camera, imitating performance, obtain to include the real video V of complete action fragment _Real

To V _CartAnd V _Real, all frame of video are played up out, and then utilize Hausdorff distance algorithm feature to obtain the distance matrix of all frames

With

N wherein ₁And n ₂Be respectively the number of frames that cartoon and real video are played up out, Hausdorff distance in the expression cartoon video between i frame and the j frame, corresponding

Hausdorff distance in the expression real video between i frame and the j frame.The while matrix M _CartAnd M _RealEach multiply by coefficient respectively

With Finish normalization, d wherein _{Cart_max}And d _{Real_max}Be respectively two maximal values in the matrix.Obtaining through normalized distance matrix M _CartAnd M _RealAfterwards, according to preset threshold

With

Come the diagonal values in the filtered matrix, in the present embodiment we compare by experiment with

Value be made as 3.2, will

Value be made as 2.7, with what obtain

With

Pairing i frame obtains cartoon key frame sequence thus as key frame

With true key frame sequence

Wherein n is both quantity, will guarantee the quantity unanimity in the method, and the quantity of n is set at 926 in the present embodiment, and the cartoon key frame of 926 frames and the true key frame of 926 frames are promptly arranged;

At cartoon key frame sequence

Obtain characteristic of correspondence according to directivity histogram of gradients (HOG) feature extracting method and express matrix

D wherein ₁For each key frame characteristic of correspondence is expressed vector

Dimension, obtain best dimension 620 dimensions through overtesting comparison in the present embodiment; Similarly, at true key frame sequence

Obtain characteristic of correspondence expression matrix according to occupying figure (OM) feature extracting method

D wherein ₂For each key frame characteristic of correspondence is expressed vector

Dimension, obtain best dimension 900 dimensions through overtesting comparison in the present embodiment; According to self-defining action classification, with X ₁And X ₂In all key frames all be divided into the r class, and to guarantee all will to have in each class the cartoon and the true key frame quantity of equal number, so far guaranteed X ₁And X ₂Correspondence one by one on the classification aspect; In the present embodiment, we have defined the action of 71 classes based on natural semantic classification altogether, and distribute unique, different tag along sorts for each class action, and the result of Xing Chenging is X like this ₁And X ₂In sample all be divided into 71 classes, and each class all includes the cartoon and the authentic specimen of equal number;

To the X that is obtained ₁And X ₂, respectively through obtain the matrix of centralization as down conversion:

$X_1=-\frac{1}{2}{\mathrm{HX}}_{1}H,$ $X_2=-\frac{1}{2}{\mathrm{HX}}_{2}H---1$

Wherein

And N is X ₁With X ₂Sample size n, so far finish normalization and centralization the operation;

2) cartoon and the real features through normalization and centralization that obtains based on step 1) expressed matrix X ₁And X ₂, train the dimensionality reduction projecting direction matrix W that obtains correspondence according to following objective function algorithm ₁And W ₂:

$\underset{F,W_1,W_2}{\min }\mathrm{tr}\left(F^T{L}_{\mathrm{syn}}F\right)-\mathrm{\α}{\left|\right|X_1{W}_1-F\left|\right|}_F^2+{\mathrm{\γ}}_1{\left|\right|W_1\left|\right|}_F^2+\mathrm{\β}{\left|\right|X_2{W}_2-F\left|\right|}_F^2+{\mathrm{\γ}}_2{\left|\right|W_2\left|\right|}_F^2+$

$\mathrm{\δ}{\left|\right|F-Y\left|\right|}_F^2---2$

Wherein R (F) is the regularization term of the imitation exercise data structure introduced, and the thought of utilization is the Gauss map model; Regularization term wherein

With

Be in order to control the complexity of study, every coefficient is for the complexity and the error rate of balance study, in the present embodiment, makes the coefficient sets of error rate optimum be combined into α=10 ^-2, γ ₁=10 ^-3, β=10 ⁰, γ ₂=10 ^-1, δ=10 ¹Matrix Y wherein is a validity score category information matrix, and Y=[y ₁, y ₂..., y _n] ∈ 0,1} ^{N * r}, wherein work as cartoon samples

And authentic specimen

When all belonging to k classification, Y then _Ik=1; Otherwise Y _Ik=0; In the method, in the training process cartoon samples and authentic specimen are selected, guaranteed

With

All, form the Y matrix of full rank with this from same classification;

R (F) provides the simulation mechanism of a kind of introducing about the training data distributed model, F ∈ R wherein ^{N * r}Represented prediction matrix based on mark.For the dimensionality reduction problem, at first hypothesis is still keeping original syntople through after the dimensionality reduction between the data.So we have introduced arest neighbors illustraton of model G in the method ^p, p=1 wherein, 2 have represented the adjacent map of cartoon and True Data respectively.The affine matrix of adjacent map correspondence is A ^p, p=1 wherein, 2 have represented the affine matrix of cartoon and True Data respectively, item wherein

Reflected sample

With

Syntople:

R (F) just can derive and puts in order according to following formula so:

$R\left(F\right)={\mathrm{\Σ}}_{k=1}^r{\mathrm{\Σ}}_{i,j=1}^n{(F_{\mathrm{ik}}-F_{\mathrm{jk}})}^2{A}_{\mathrm{ij}}^p$

$={\mathrm{\Σ}}_{i,j=1}^n{A}_{\mathrm{ij}}^p(F_i^T{F}_i+F_j^T{F}_j-2F_i^T{F}_j)---4$

$=2\mathrm{tr}\left(F^T(D^p-A^p)F\right)=2\mathrm{tr}\left(F^T{L}^{p}F\right)$

Tr (.) wherein is the mark operational character, D ^pBe diagonal matrix diagonal angle item wherein

P=1 wherein, 2.By formula (4), we can obtain cartoon and real Lagrangian matrix L ¹=D ¹-A ¹, L ²=D ²-A ²

In order to simplify solution, we are with L ¹And L ²Merge by following formula:

L _syn＝[((L ¹+L ²)/2)′+((L ¹+L ²)/2)]/2 5

So L _Syn=L ' _SynSo far, formula (2) can be organized into following form:

$\underset{F,W_1,W_2}{\min }\mathrm{tr}\left(F^T{L}_{\mathrm{syn}}F\right)-\mathrm{\α}{\left|\right|X_1{W}_1-F\left|\right|}_F^2+{\mathrm{\γ}}_1{\left|\right|W_1\left|\right|}_F^2+\mathrm{\β}{\left|\right|X_2{W}_2-F\left|\right|}_F^2+{\mathrm{\γ}}_2{\left|\right|W_2\left|\right|}_F^2+$

$\mathrm{\δ}{\left|\right|F-Y\left|\right|}_F^2---6$

Wherein Represent not this normal form of Luo Beini crow, and

Satisfy all matrix Z; Through differentiate, can obtain at last:

W ₁＝δB ₁(αU+βV+E) ^-1Y 7

W ₂＝δB ₂(αU+βV+E) ^-1Y 8

Wherein:

$U=B_1^T{X}_1^T{X}_1{B}_1-X_1{B}_1-B_1^T{X}_1^T+1---9$

$V=B_2^T{X}_2^T{X}_2{B}_2-X_2{B}_2-B_2^T{X}_2^T+1---10$

$E=L_{\mathrm{syn}}+{\mathrm{\γ}}_1{B}_1^T{B}_1+{\mathrm{\γ}}_2{B}_2^T{B}_2+\mathrm{\δI}---11$

Wherein:

$B_1={({\mathrm{\αX}}_1^T{X}_1+{\mathrm{\γ}}_{1}I)}^{-1}{\mathrm{\αX}}_1^T---12$

$B_2={({\mathrm{\βX}}_2^T{X}_2+{\mathrm{\γ}}_{2}I)}^{-1}{\mathrm{\βX}}_2^T---13$

So far, cartoon and true corresponding dimensionality reduction projecting direction matrix W have been obtained ₁And W ₂

When importing new true key frame sequence, obtain characteristic of correspondence expression matrix according to occupying the figure feature extracting method as index

Wherein m is the key frame quantity of index, m=12 in the present embodiment, and promptly importing length is the true key frame sequence of 12 frames.Obtain the new feature representation formula in the space of true key frame behind dimensionality reduction thus

In like manner, calculate based on the new feature representation formula X ' in the space of the true key frame of cartoon behind dimensionality reduction in " cartoon-true " role data storehouse of step 1) ₁=X ₁W ₁∈ R ^{N * r}

In the dimensionality reduction space, with

In each element as index separately, calculate itself and X ₁' in the Euclidean distance of each element, be in the present embodiment 5 of layback minimum as indexed results as the candidate, the user can be from wherein selecting the most suitable key frame; Obtained thus Corresponding

Promptly return the cartoon key frame that obtains as index with true key frame;

The user can be at the cartoon key frame

On carry out the interest that operations such as deformation, stretching, replacement strengthen the cartoon key frame, in the present embodiment, we provide based on the maintenance rigidity deformation method of tri patch and have finished deformation for the cartoon character health; And by introducing affine matrix, control 6 parameters are wherein finished the affined transformation for cartoon character health any direction on the coordinate transverse axis and the longitudinal axis; Finish replacement for the cartoon character body part by the method for shearing at last, the user can replace on the cartoon key frame accordingly with certain part of the interested jpeg format picture of own institute.

Claims (3)

1. two-dimensional character cartoon generation method based on the heterogeneous characteristic dimensionality reduction is characterized in that its step is as follows:

1) from the two-dimensional cartoon video, extracts the video-frequency band that comprises the complete action of role's cartoon, and according to the movement content of role's cartoon and towards, take corresponding action video by true personage's performance, after the action video process video image technical finesse of the action video that the diagonal angle colour atla is logical and true personage's performance, utilize self-defining extraction method of key frame to extract the key frame of the action video and the action video that true personage performs of role's cartoon, obtain cartoon key frame and true key frame, and cartoon key frame and true key frame are carried out normalization and centralization processing; The cartoon key frame is utilized different feature extracting methods with true key frame, obtain cartoon key frame feature representation matrix and true key frame feature representation matrix, and the feature representation matrix classified according to self-defining action classification, set up " cartoon-true " role data storehouse;

2) from the action video that the true personage who takes performs, utilize self-defining extraction method of key frame to extract the key frame of the action video of true personage's performance, obtain retrieving true key frame, and true key frame carries out normalization and centralization is handled to retrieving; Utilize feature extracting method to obtain the true key frame feature representation matrix of retrieval to retrieving true key frame;

3) cartoon key frame feature representation matrix and true key frame feature representation matrix be by self-defining heterogeneous characteristic dimensionality reduction algorithm, obtains the dimensionality reduction projecting direction matrix of cartoon key frame feature representation matrix of cartoon key frame feature representation matrix and true key frame feature representation matrix correspondence and the dimensionality reduction projecting direction matrix of true key frame feature representation matrix through training; Retrieve true key frame sequence signature and express matrix, express the true key frame sequence dimensionality reduction feature representation matrix of dimensionality reduction projecting direction matrix acquisition retrieval that matrix multiply by true key frame feature representation matrix with the true key frame sequence signature of retrieval, and in projector space, calculate in " cartoon-true " the role data storehouse that obtains and the nearest cartoon key frame sequence of the true key frame sequence dimensionality reduction feature representation matrix of retrieval, at last the cartoon key frame sequence that calculates is returned; The user obtains final cartoon effect video in enterprising edlin of cartoon key frame sequence and the interpolation returned.

2. a kind of two-dimensional character cartoon generation method according to claim 1 based on the heterogeneous characteristic dimensionality reduction, it is characterized in that: described step 1) comprises:

From the two-dimensional cartoon character video, extract the cartoon video V that comprises complete cartoon character action fragment _Cart, the true personage of cause is according to V _CartIn cartoon character movement content and towards before monocular-camera, imitating performance, obtain to include the real video V of complete true figure action fragment _Real

To from V _CartAnd V _RealIn the frame of video playing up out, utilize Hausdorff distance algorithm feature to obtain the cartoon distance matrix

With the actual distance matrix

N wherein ₁And n ₂Be respectively the number of frames that cartoon and real video are played up out,

Hausdorff distance in the expression cartoon video frame between i frame and the j frame,

Hausdorff distance in the expression real video frame between i frame and the j frame, matrix M _CartAnd M _RealIn each multiply by coefficient respectively With

Finish normalization, d wherein _{Cart_max}And d _{Real_max}Be respectively matrix M _CartAnd M _RealIn maximal value, obtaining through normalized distance matrix M _CartAnd M _RealAfterwards, according to preset threshold

With

Come the diagonal values in the filtered matrix, will obtain respectively

With Pairing i frame obtains the cartoon key frame thus as key frame

With true key frame

Wherein n is both quantity;

At the cartoon key frame

Obtain cartoon key frame feature representation matrix according to directivity histogram of gradients feature extracting method

D wherein ₁For each key frame characteristic of correspondence is expressed vector

Dimension; Similarly, at true key frame

Obtain true key frame feature representation matrix according to occupying the figure feature extracting method

D wherein ₂For each key frame characteristic of correspondence is expressed vector

Dimension; With X ₁And X ₂In all key frames all be divided into the r class according to the difference of movement content, thereby make all have in each class the cartoon of equal number and true key frame to form X ₁And X ₂Correspondence one by one on the classification aspect;

To the X that is obtained ₁And X ₂, respectively through obtain the matrix of centralization as down conversion:

$X_1=-\frac{1}{2}{\mathrm{HX}}_{1}H,$ $X_2=-\frac{1}{2}{\mathrm{HX}}_{2}H---1$

Wherein

And N is X ₁With X ₂Sample size n, so far finish normalization and centralization operation, thereby set up " cartoon-true " role data storehouse.

3. a kind of two-dimensional character cartoon generation method according to claim 1 based on the heterogeneous characteristic dimensionality reduction, it is characterized in that: described step 3) comprises:

The cartoon of normalization and centralization and real features are expressed matrix X ₁And X ₂, calculate the dimensionality reduction projecting direction matrix W that obtains cartoon key frame feature representation matrix according to following objective function algorithm ₁Dimensionality reduction projecting direction matrix W with true key frame feature representation matrix ₂:

$\underset{F,W_1,W_2}{\min }\mathrm{tr}\left(F^T{L}_{\mathrm{syn}}F\right)-\mathrm{\α}{\left|\right|X_1{W}_1-F\left|\right|}_F^2+{\mathrm{\γ}}_1{\left|\right|W_1\left|\right|}_F^2+\mathrm{\β}{\left|\right|X_2{W}_2-F\left|\right|}_F^2+{\mathrm{\γ}}_2{\left|\right|W_2\left|\right|}_F^2+$

$\mathrm{\δ}{\left|\right|F-Y\left|\right|}_F^2---2$

Alpha, gamma ₁, beta, gamma ₂, δ is a heterogeneous equilibrium degree coefficient, matrix Y=[y ₁, y ₂..., y _n] ∈ 0,1} ^{N * r}, wherein work as cartoon samples

And authentic specimen

When all belonging to k classification, Y then _Ik=1; Otherwise Y _Ik=0;

With

Belong to same classification, form the Y matrix of full rank; Tr (.) is the mark operational character; The Lagrangian matrix L of cartoon key frame feature representation matrix and cartoon key frame feature representation matrix ¹=D ¹-A ¹, L ²=D ²-A ²

L _syn＝[((L ¹+L ²)/ ²)′+((L ¹+L ²)/2)]/2 3

So L _Syn=L ' _SynWherein Represent not this normal form of Luo Beini crow, and

Satisfy all matrix Z; Through differentiate, can obtain at last:

W ₁＝δB ₁(αU+βV+E) ^-1Y 4

W ₂＝δB ₂(αU+βV+E) ^-1Y 5

Wherein:

$U=B_1^T{X}_1^T{X}_1{B}_1-X_1{B}_1-B_1^T{X}_1^T+1---6$

$V=B_2^T{X}_2^T{X}_2{B}_2-X_2{B}_2-B_2^T{X}_2^T+1---7$

$E=L_{\mathrm{syn}}+{\mathrm{\γ}}_1{B}_1^T{B}_1+{\mathrm{\γ}}_2{B}_2^T{B}_2+\mathrm{\δI}---8$

Wherein:

$B_1={({\mathrm{\αX}}_1^T{X}_1+{\mathrm{\γ}}_{1}I)}^{-1}{\mathrm{\αX}}_1^T---9$

$B_2={({\mathrm{\βX}}_2^T{X}_2+{\mathrm{\γ}}_{2}I)}^{-1}{\mathrm{\βX}}_2^T---10$

So far, obtained the dimensionality reduction projecting direction matrix W of cartoon key frame feature representation matrix ₁Dimensionality reduction projecting direction matrix W with true key frame feature representation matrix ₂

When the true key frame of input retrieval, obtain the corresponding true key frame feature representation of retrieval matrix according to occupying the figure feature extracting method

Wherein m is the true quantity of key frames of retrieval; Dimensionality reduction projecting direction matrix W according to the true key frame feature representation matrix that obtains ₂, calculate the retrieval dimensionality reduction real features expression formula in the space of the true key frame of retrieval behind dimensionality reduction Dimensionality reduction projecting direction matrix W according to the cartoon key frame feature representation matrix that obtains ₁, calculate the dimensionality reduction cartoon feature representation formula X ' in the space of cartoon key frame feature representation matrix behind dimensionality reduction in " cartoon-true " role data storehouse ₁=X ₁W ₁∈ R ^{N * r}, wherein r is the dimension in dimensionality reduction space;

In the dimensionality reduction space, with retrieval dimensionality reduction real features expression formula

In each element as index separately, calculate and dimensionality reduction cartoon feature representation formula X ₁' in the Euclidean distance of each element, and several of layback minimum are as cartoon indexed results X _Result, finish the process of returning the cartoon key frame that obtains with true key frame as index;

The user is at cartoon indexed results X _ResultOn carry out the interest that operations such as deformation, stretching, replacement strengthen the cartoon key frame, obtain final cartoon video by approach based on linear interpolation at last.

Images