CN102129707A - Heterogeneous feature dimension reduction-based two-dimensional role cartoon generation method - Google Patents
Heterogeneous feature dimension reduction-based two-dimensional role cartoon generation method Download PDFInfo
- Publication number
- CN102129707A CN102129707A CN2011100525858A CN201110052585A CN102129707A CN 102129707 A CN102129707 A CN 102129707A CN 2011100525858 A CN2011100525858 A CN 2011100525858A CN 201110052585 A CN201110052585 A CN 201110052585A CN 102129707 A CN102129707 A CN 102129707A
- Authority
- CN
- China
- Prior art keywords
- cartoon
- key frame
- matrix
- true
- feature representation
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000009467 reduction Effects 0.000 title claims abstract description 81
- 238000000034 method Methods 0.000 title claims abstract description 69
- 239000011159 matrix material Substances 0.000 claims abstract description 169
- 230000009471 action Effects 0.000 claims abstract description 39
- 230000014509 gene expression Effects 0.000 claims abstract description 19
- 238000000605 extraction Methods 0.000 claims abstract description 7
- 238000010606 normalization Methods 0.000 claims description 18
- 239000012634 fragment Substances 0.000 claims description 12
- 230000000875 corresponding effect Effects 0.000 claims description 9
- 230000008569 process Effects 0.000 claims description 9
- 239000000284 extract Substances 0.000 claims description 7
- 238000012549 training Methods 0.000 claims description 5
- 238000006243 chemical reaction Methods 0.000 claims description 4
- 238000012545 processing Methods 0.000 claims description 4
- 241000532370 Atla Species 0.000 claims description 3
- 238000013459 approach Methods 0.000 claims description 3
- 230000000694 effects Effects 0.000 claims description 3
- 230000006870 function Effects 0.000 abstract description 4
- 238000007796 conventional method Methods 0.000 abstract 2
- 238000010801 machine learning Methods 0.000 abstract 1
- PXFBZOLANLWPMH-UHFFFAOYSA-N 16-Epiaffinine Natural products C1C(C2=CC=CC=C2N2)=C2C(=O)CC2C(=CC)CN(C)C1C2CO PXFBZOLANLWPMH-UHFFFAOYSA-N 0.000 description 3
- 238000011160 research Methods 0.000 description 3
- 230000015572 biosynthetic process Effects 0.000 description 2
- 230000000052 comparative effect Effects 0.000 description 2
- 238000002474 experimental method Methods 0.000 description 2
- 230000036541 health Effects 0.000 description 2
- 238000003786 synthesis reaction Methods 0.000 description 2
- 238000010189 synthetic method Methods 0.000 description 2
- HUTDUHSNJYTCAR-UHFFFAOYSA-N ancymidol Chemical compound C1=CC(OC)=CC=C1C(O)(C=1C=NC=NC=1)C1CC1 HUTDUHSNJYTCAR-UHFFFAOYSA-N 0.000 description 1
- 239000012141 concentrate Substances 0.000 description 1
- 230000002950 deficient Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 238000010008 shearing Methods 0.000 description 1
- 238000004088 simulation Methods 0.000 description 1
- 238000013179 statistical model Methods 0.000 description 1
- 230000002194 synthesizing effect Effects 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
Images
Landscapes
- Image Analysis (AREA)
Abstract
The invention discloses a heterogeneous feature dimension reduction-based two-dimensional role cartoon generation method. In the method, the functions of indexing and editing similar pose-containing cartoon pictures in a database are realized based on real video pictures by utilizing knowledge obtained by machine learning and feature extraction. The method comprises the following steps of: inputting sequence key frames of an action performed by real people; extracting feature expressions of the sequence key frames by using a system; computing the Euclidean distance between each feature expression and a cartoon neighboring node in projection space according the feature expressions and a projection matrix trained by the method; returning a closest cartoon neighboring node serving as a search result; and editing on an obtained cartoon key frame sequence by a user. In the method, implementation difficulties caused by taking the cartoon pictures as indexes in a conventional method are reduced by taking actually shot video pictures as the indexes; simultaneously the problem of narrow applicability caused by the conventional method which can only aim at isomorphism feature dimension reduction is solved by disclosing a heterogeneous feature dimensional reduction method; therefore, the accuracy is improved and the application range is enlarged.
Description
Technical field
The present invention relates to a kind of two-dimensional character cartoon generation method, be specifically related to a kind of two-dimensional cartoon character generation method of handling and retrieving based on heterogeneous characteristic, belong to the general field of computer animation and computer machine study based on the heterogeneous characteristic dimensionality reduction.
Background technology
The extensive application in industry and entertainment field along with computer two-dimensional animation and computer machine study, the two-dimensional cartoon generation method of reusing based on existing two-dimensional cartoon character video data becomes an important research focus gradually.Although still be in the stage of exploration at present about the research in this field, produced some more representational methods.
The researcher has developed some two-dimensional cartoon synthetic methods of reusing based on multi-medium data.Such as, be published in the paper " Video Textures " on the meeting SIGGRAPH in 2000, extracted and at first analyzed the cartoon video fragment, and then put in order the method for synthesizing new cartoon video by changing original frame of video.Be published in the paper " Motion texture:a two-level statistical model for character motion synthesis " on the meeting SIGGRAPH in 2002, the researcher successfully obtains new three-dimensional cartoon motion by reusing the three-dimensional motion data.Above method has excited us to develop a kind of generation method based on two-dimensional character cartoon data reusing.
The method that several comparative maturities have been arranged at present based on the two-dimensional cartoon character data reusing, the synthetic technology that on behalf of present popular content-based analysis, these methods retrieve.Such as, be published in the paper " Cartoon Textures " on the meeting SCA in 2004, need manual initial frame and the terminal frame that needs in the synthetic cartoon video frame of specifying of user, improved synthetic efficient and practicality.But the problem that causes thus is that the result who is synthesized can be limited to a great extent by user's appointment, and uncontrollable synthetic direction of user and result.Because method itself can not be carried out iteration or repeatedly, if the unsatisfied result of user, can't make amendment and feedback processing, and also exist in the building-up process for the concrete implementation detail of how to specify cartoon role to combine and explain unclear problem with background frames.Be published in the paper " Perspective-aware Cartoon Clips Synthesis " on the periodical CAVW in 2008, reusing of two-dimensional cartoon resource concentrates on this aspect of cartoon video section, really be not deep into the level of frame, synthesize new cartoon video by splicing existing two-dimensional cartoon fragment.Problem is that in the new synthetic cartoon video, the order of original fragment frame interior was not made amendment, and existed the situation of the uncontrollable building-up process of user yet.Based on the defective of above-mentioned two kinds of methods, we wish to develop a kind of content-based analysis retrieval and can realize the two-dimensional character cartoon generation method that omnidistance user is controlled.
The key point of method that we propose, except the top cartoon retrieval of analyzing that how to design content-based analysis, also be how to finish under the heterogeneous characteristic situation based on the cartoon of gesture recognition synthetic, such as similarity how to weigh attitude between real video key frame and the cartoon video key frame quantitatively, and the research of this respect is fewer at present.In order to realize higher gesture recognition accuracy rate, we have adopted different feature extracting methods.Be published in the paper " Silhouette representation and matching for 3D pose discrimination-A comparative study " on the periodical IVC in 2010, several feature extracting methods relatively commonly used in the gesture recognition field have been proposed, and at different towards having done careful deep comparison test.In our method,, finally utilized directivity histogram of gradients (HOG) feature extracting method of being mentioned in this paper to obtain the feature representation matrix of cartoon character by the contrast experiment; Occupy the feature representation matrix that figure (OM) feature extracting method obtains true personage.
Summary of the invention
The objective of the invention is limitation, a kind of two-dimensional cartoon generation method of finishing the cartoon gesture recognition based on heterogeneous characteristic analysis and dimensionality reduction is provided for the two-dimensional cartoon synthetic method that overcomes present content-based analysis retrieval.
Step based on the two-dimensional character cartoon generation method of heterogeneous characteristic dimensionality reduction is as follows:
1) from the two-dimensional cartoon video, extracts the video-frequency band that comprises the complete action of role's cartoon, and according to the movement content of role's cartoon and towards, take corresponding action video by true personage's performance, after the action video process video image technical finesse of the action video that the diagonal angle colour atla is logical and true personage's performance, utilize self-defining extraction method of key frame to extract the key frame of the action video and the action video that true personage performs of role's cartoon, obtain cartoon key frame and true key frame, and cartoon key frame and true key frame are carried out normalization and centralization processing; The cartoon key frame is utilized different feature extracting methods with true key frame, obtain cartoon key frame feature representation matrix and true key frame feature representation matrix, and the feature representation matrix classified according to self-defining action classification, set up " cartoon-true " role data storehouse;
2) from the action video that the true personage who takes performs, utilize self-defining extraction method of key frame to extract the key frame of the action video of true personage's performance, obtain retrieving true key frame, and true key frame carries out normalization and centralization is handled to retrieving; Utilize feature extracting method to obtain the true key frame feature representation matrix of retrieval to retrieving true key frame;
3) cartoon key frame feature representation matrix and true key frame feature representation matrix be by self-defining heterogeneous characteristic dimensionality reduction algorithm, obtains the dimensionality reduction projecting direction matrix of cartoon key frame feature representation matrix of cartoon key frame feature representation matrix and true key frame feature representation matrix correspondence and the dimensionality reduction projecting direction matrix of true key frame feature representation matrix through training; Retrieve true key frame sequence signature and express matrix, express the true key frame sequence dimensionality reduction feature representation matrix of dimensionality reduction projecting direction matrix acquisition retrieval that matrix multiply by true key frame feature representation matrix with the true key frame sequence signature of retrieval, and in projector space, calculate in " cartoon-true " the role data storehouse that obtains and the nearest cartoon key frame sequence of the true key frame sequence dimensionality reduction feature representation matrix of retrieval, at last the cartoon key frame sequence that calculates is returned; The user obtains final cartoon effect video in enterprising edlin of cartoon key frame sequence and the interpolation returned.
Described step 1) comprises:
From the two-dimensional cartoon character video, extract the cartoon video V that comprises complete cartoon character action fragment
Cart, the true personage of cause is according to V
CartIn cartoon character movement content and towards before monocular-camera, imitating performance, obtain to include the real video V of complete true figure action fragment
Real
To from V
CartAnd V
RealIn the frame of video playing up out, utilize Hausdorff distance algorithm feature to obtain the cartoon distance matrix
With the actual distance matrix
N wherein
1And n
2Be respectively the number of frames that cartoon and real video are played up out,
Hausdorff distance in the expression cartoon video frame between i frame and the j frame,
Hausdorff distance in the expression real video frame between i frame and the j frame, matrix M
CartAnd M
RealIn each multiply by coefficient respectively
With
Finish normalization, d wherein
Cart_maxAnd d
Real_maxBe respectively matrix M
CartAnd M
RealIn maximal value, obtaining through normalized distance matrix M
CartAnd M
RealAfterwards, according to preset threshold
With
Come the diagonal values in the filtered matrix, will obtain respectively
With
Pairing i frame obtains the cartoon key frame thus as key frame
With true key frame
Wherein n is both quantity;
At the cartoon key frame
Obtain cartoon key frame feature representation matrix according to directivity histogram of gradients feature extracting method
D wherein
1For each key frame characteristic of correspondence is expressed vector
Dimension; Similarly, at true key frame
Obtain true key frame feature representation matrix according to occupying the figure feature extracting method
D wherein
2For each key frame characteristic of correspondence is expressed vector
Dimension; With X
1And X
2In all key frames all be divided into the r class according to the difference of movement content, thereby make all have in each class the cartoon of equal number and true key frame to form X
1And X
2Correspondence one by one on the classification aspect;
To the X that is obtained
1And X
2, respectively through obtain the matrix of centralization as down conversion:
Wherein
And N is X
1With X
2Sample size n, so far finish normalization and centralization operation, thereby set up " cartoon-true " role data storehouse.
Described step 3) comprises:
The cartoon of normalization and centralization and real features are expressed matrix X
1And X
2, calculate the dimensionality reduction projecting direction matrix W that obtains cartoon key frame feature representation matrix according to following objective function algorithm
1Dimensionality reduction projecting direction matrix W with true key frame feature representation matrix
2:
Alpha, gamma
1, beta, gamma
2, δ is a heterogeneous equilibrium degree coefficient, matrix Y=[y
1, y
2..., y
n] ∈ 0,1}
N * r, wherein work as cartoon samples
And authentic specimen
When all belonging to k classification, Y then
Ik=1; Otherwise Y
Ik=0;
With
Belong to same classification, form the Y matrix of full rank; Tr (.) is the mark operational character; The Lagrangian matrix L of cartoon key frame feature representation matrix and cartoon key frame feature representation matrix
1=D
1-A
1, L
2=D
2-A
2
L
syn=[((L
1+L
2)/2)′+((L
1+L
2)/2)]/2 3
So L
Syn=L '
SynWherein
Represent not this normal form of Luo Beini crow, and
Satisfy all matrix Z; Through differentiate, can obtain at last:
W
1=δB
1(αU+βV+E)
-1Y 4
W
2=δB
2(αU+βV+E)
-1Y 5
Wherein:
Wherein:
So far, obtained the dimensionality reduction projecting direction matrix W of cartoon key frame feature representation matrix
1Dimensionality reduction projecting direction matrix W with true key frame feature representation matrix
2
When the true key frame of input retrieval, obtain the corresponding true key frame feature representation of retrieval matrix according to occupying the figure feature extracting method
Wherein m is the true quantity of key frames of retrieval; Dimensionality reduction projecting direction matrix W according to the true key frame feature representation matrix that obtains
2, calculate the retrieval dimensionality reduction real features expression formula in the space of the true key frame of retrieval behind dimensionality reduction
Dimensionality reduction projecting direction matrix W according to the cartoon key frame feature representation matrix that obtains
1, calculate the dimensionality reduction cartoon feature representation formula X ' in the space of cartoon key frame feature representation matrix behind dimensionality reduction in " cartoon-true " role data storehouse
1=X
1W
1∈ R
N * r, wherein r is the dimension in dimensionality reduction space;
In the dimensionality reduction space, with retrieval dimensionality reduction real features expression formula
In each element as index separately, calculate and dimensionality reduction cartoon feature representation formula X
1' in the Euclidean distance of each element, and several of layback minimum are as cartoon indexed results X
Result, finish the process of returning the cartoon key frame that obtains with true key frame as index;
The user is at cartoon indexed results X
ResultOn carry out the interest that operations such as deformation, stretching, replacement strengthen the cartoon key frame, obtain final cartoon video by approach based on linear interpolation at last.
The present invention by with true capture video picture as index, reduced the performance difficulty that classic method is brought as index with the cartoon picture; Simultaneously by proposing the heterogeneous characteristic dimension reduction method, solved the narrow problem of applicability that classic method can only be brought at isomorphism feature dimensionality reduction, improve accuracy, enlarged the scope of using.
Description of drawings
The present invention is further illustrated below in conjunction with the drawings and specific embodiments.
Fig. 1 is a method algorithm flow chart of the present invention.
Embodiment
Step based on the two-dimensional character cartoon generation method of heterogeneous characteristic dimensionality reduction is as follows:
1) from the two-dimensional cartoon video, extracts the video-frequency band that comprises the complete action of role's cartoon, and according to the movement content of role's cartoon and towards, take corresponding action video by true personage's performance, after the action video process video image technical finesse of the action video that the diagonal angle colour atla is logical and true personage's performance, utilize self-defining extraction method of key frame to extract the key frame of the action video and the action video that true personage performs of role's cartoon, obtain cartoon key frame and true key frame, and cartoon key frame and true key frame are carried out normalization and centralization processing; The cartoon key frame is utilized different feature extracting methods with true key frame, obtain cartoon key frame feature representation matrix and true key frame feature representation matrix, and the feature representation matrix classified according to self-defining action classification, set up " cartoon-true " role data storehouse;
2) from the action video that the true personage who takes performs, utilize self-defining extraction method of key frame to extract the key frame of the action video of true personage's performance, obtain retrieving true key frame, and true key frame carries out normalization and centralization is handled to retrieving; Utilize feature extracting method to obtain the true key frame feature representation matrix of retrieval to retrieving true key frame;
3) cartoon key frame feature representation matrix and true key frame feature representation matrix be by self-defining heterogeneous characteristic dimensionality reduction algorithm, obtains the dimensionality reduction projecting direction matrix of cartoon key frame feature representation matrix of cartoon key frame feature representation matrix and true key frame feature representation matrix correspondence and the dimensionality reduction projecting direction matrix of true key frame feature representation matrix through training; Retrieve true key frame sequence signature and express matrix, express the true key frame sequence dimensionality reduction feature representation matrix of dimensionality reduction projecting direction matrix acquisition retrieval that matrix multiply by true key frame feature representation matrix with the true key frame sequence signature of retrieval, and in projector space, calculate in " cartoon-true " the role data storehouse that obtains and the nearest cartoon key frame sequence of the true key frame sequence dimensionality reduction feature representation matrix of retrieval, at last the cartoon key frame sequence that calculates is returned; The user obtains final cartoon effect video in enterprising edlin of cartoon key frame sequence and the interpolation returned.
Described step 1) comprises:
From the two-dimensional cartoon character video, extract the cartoon video V that comprises complete cartoon character action fragment
Cart, the true personage of cause is according to V
CartIn cartoon character movement content and towards before monocular-camera, imitating performance, obtain to include the real video V of complete true figure action fragment
Real
To from V
CartAnd V
RealIn the frame of video playing up out, utilize Hausdorff distance algorithm feature to obtain the cartoon distance matrix
With the actual distance matrix
N wherein
1And n
2Be respectively the number of frames that cartoon and real video are played up out,
Hausdorff distance in the expression cartoon video frame between i frame and the j frame,
Hausdorff distance in the expression real video frame between i frame and the j frame, matrix M
CartAnd M
RealIn each multiply by coefficient respectively
With
Finish normalization, d wherein
Cart_maxAnd d
Real_maxBe respectively matrix M
CartAnd M
RealIn maximal value, obtaining through normalized distance matrix M
CartAnd M
RealAfterwards, according to preset threshold
With
Come the diagonal values in the filtered matrix, will obtain respectively
With
Pairing i frame obtains the cartoon key frame thus as key frame
With true key frame
Wherein n is both quantity;
At the cartoon key frame
Obtain cartoon key frame feature representation matrix according to directivity histogram of gradients feature extracting method
D wherein
1For each key frame characteristic of correspondence is expressed vector
Dimension; Similarly, at true key frame
Obtain true key frame feature representation matrix according to occupying the figure feature extracting method
D wherein
2For each key frame characteristic of correspondence is expressed vector
Dimension; With X
1And X
2In all key frames all be divided into the r class according to the difference of movement content, thereby make all have in each class the cartoon of equal number and true key frame to form X
1And X
2Correspondence one by one on the classification aspect;
To the X that is obtained
1And X
2, respectively through obtain the matrix of centralization as down conversion:
Wherein
And N is X
1With X
2Sample size n, so far finish normalization and centralization operation, thereby set up " cartoon-true " role data storehouse.
Described step 3) comprises:
The cartoon of normalization and centralization and real features are expressed matrix X
1And X
2, calculate the dimensionality reduction projecting direction matrix W that obtains cartoon key frame feature representation matrix according to following objective function algorithm
1Dimensionality reduction projecting direction matrix W with true key frame feature representation matrix
2:
Alpha, gamma
1, beta, gamma
2, δ is a heterogeneous equilibrium degree coefficient, matrix Y=[y
1, y
2..., y
n] ∈ 0,1}
N * r, wherein work as cartoon samples
And authentic specimen
When all belonging to k classification, Y then
Ik=1; Otherwise Y
Ik=0;
With
Belong to same classification, form the Y matrix of full rank; Tr (.) is the mark operational character; The Lagrangian matrix L of cartoon key frame feature representation matrix and cartoon key frame feature representation matrix
1=D
1-A
1, L
2=D
2-A
2
L
syn=[((L
1+L
2)/2)′+((L
1+L
2)/2)]/2 3
So L
Syn=L '
SynWherein
Represent not this normal form of Luo Beini crow, and
Satisfy all matrix Z; Through differentiate, can obtain at last:
W
1=δB
1(αU+βV+E)
-1Y 4
W
2=δB
2(αU+βV+E)
-1Y 5
Wherein:
Wherein:
So far, obtained the dimensionality reduction projecting direction matrix W of cartoon key frame feature representation matrix
1Dimensionality reduction projecting direction matrix W with true key frame feature representation matrix
2
When the true key frame of input retrieval, obtain the corresponding true key frame feature representation of retrieval matrix according to occupying the figure feature extracting method
Wherein m is the true quantity of key frames of retrieval; Dimensionality reduction projecting direction matrix W according to the true key frame feature representation matrix that obtains
2, calculate the retrieval dimensionality reduction real features expression formula in the space of the true key frame of retrieval behind dimensionality reduction
Dimensionality reduction projecting direction matrix W according to the cartoon key frame feature representation matrix that obtains
1, calculate the dimensionality reduction cartoon feature representation formula X ' in the space of cartoon key frame feature representation matrix behind dimensionality reduction in " cartoon-true " role data storehouse
1=X
1W
1∈ R
N * r, wherein r is the dimension in dimensionality reduction space;
In the dimensionality reduction space, with retrieval dimensionality reduction real features expression formula
In each element as index separately, calculate and dimensionality reduction cartoon feature representation formula X
1' in the Euclidean distance of each element, and several of layback minimum are as cartoon indexed results X
Result, finish the process of returning the cartoon key frame that obtains with true key frame as index;
The user is at cartoon indexed results X
ResultOn carry out the interest that operations such as deformation, stretching, replacement strengthen the cartoon key frame, obtain final cartoon video by approach based on linear interpolation at last.
Embodiment
1) entire flow of method as shown in Figure 1.At first from traditional two-dimensional cartoon character video,, manually extract the cartoon video V that plurality of sections includes complete action fragment according to the integrality and the representativeness of action
Cart, in the present embodiment, from manually having extracted about cartoon video that contained complete action fragment in 29 minutes 102 video segments altogether; According to V
CartIn cartoon character movement content and towards, the true personage of cause is according to V
CartIn action content and towards before monocular-camera, imitating performance, obtain to include the real video V of complete action fragment
Real
To V
CartAnd V
Real, all frame of video are played up out, and then utilize Hausdorff distance algorithm feature to obtain the distance matrix of all frames
With
N wherein
1And n
2Be respectively the number of frames that cartoon and real video are played up out,
Hausdorff distance in the expression cartoon video between i frame and the j frame, corresponding
Hausdorff distance in the expression real video between i frame and the j frame.The while matrix M
CartAnd M
RealEach multiply by coefficient respectively
With
Finish normalization, d wherein
Cart_maxAnd d
Real_maxBe respectively two maximal values in the matrix.Obtaining through normalized distance matrix M
CartAnd M
RealAfterwards, according to preset threshold
With
Come the diagonal values in the filtered matrix, in the present embodiment we compare by experiment with
Value be made as 3.2, will
Value be made as 2.7, with what obtain
With
Pairing i frame obtains cartoon key frame sequence thus as key frame
With true key frame sequence
Wherein n is both quantity, will guarantee the quantity unanimity in the method, and the quantity of n is set at 926 in the present embodiment, and the cartoon key frame of 926 frames and the true key frame of 926 frames are promptly arranged;
At cartoon key frame sequence
Obtain characteristic of correspondence according to directivity histogram of gradients (HOG) feature extracting method and express matrix
D wherein
1For each key frame characteristic of correspondence is expressed vector
Dimension, obtain best dimension 620 dimensions through overtesting comparison in the present embodiment; Similarly, at true key frame sequence
Obtain characteristic of correspondence expression matrix according to occupying figure (OM) feature extracting method
D wherein
2For each key frame characteristic of correspondence is expressed vector
Dimension, obtain best dimension 900 dimensions through overtesting comparison in the present embodiment; According to self-defining action classification, with X
1And X
2In all key frames all be divided into the r class, and to guarantee all will to have in each class the cartoon and the true key frame quantity of equal number, so far guaranteed X
1And X
2Correspondence one by one on the classification aspect; In the present embodiment, we have defined the action of 71 classes based on natural semantic classification altogether, and distribute unique, different tag along sorts for each class action, and the result of Xing Chenging is X like this
1And X
2In sample all be divided into 71 classes, and each class all includes the cartoon and the authentic specimen of equal number;
To the X that is obtained
1And X
2, respectively through obtain the matrix of centralization as down conversion:
Wherein
And N is X
1With X
2Sample size n, so far finish normalization and centralization the operation;
2) cartoon and the real features through normalization and centralization that obtains based on step 1) expressed matrix X
1And X
2, train the dimensionality reduction projecting direction matrix W that obtains correspondence according to following objective function algorithm
1And W
2:
Wherein R (F) is the regularization term of the imitation exercise data structure introduced, and the thought of utilization is the Gauss map model; Regularization term wherein
With
Be in order to control the complexity of study, every coefficient is for the complexity and the error rate of balance study, in the present embodiment, makes the coefficient sets of error rate optimum be combined into α=10
-2, γ
1=10
-3, β=10
0, γ
2=10
-1, δ=10
1Matrix Y wherein is a validity score category information matrix, and Y=[y
1, y
2..., y
n] ∈ 0,1}
N * r, wherein work as cartoon samples
And authentic specimen
When all belonging to k classification, Y then
Ik=1; Otherwise Y
Ik=0; In the method, in the training process cartoon samples and authentic specimen are selected, guaranteed
With
All, form the Y matrix of full rank with this from same classification;
R (F) provides the simulation mechanism of a kind of introducing about the training data distributed model, F ∈ R wherein
N * rRepresented prediction matrix based on mark.For the dimensionality reduction problem, at first hypothesis is still keeping original syntople through after the dimensionality reduction between the data.So we have introduced arest neighbors illustraton of model G in the method
p, p=1 wherein, 2 have represented the adjacent map of cartoon and True Data respectively.The affine matrix of adjacent map correspondence is A
p, p=1 wherein, 2 have represented the affine matrix of cartoon and True Data respectively, item wherein
Reflected sample
With
Syntople:
R (F) just can derive and puts in order according to following formula so:
Tr (.) wherein is the mark operational character, D
pBe diagonal matrix diagonal angle item wherein
P=1 wherein, 2.By formula (4), we can obtain cartoon and real Lagrangian matrix L
1=D
1-A
1, L
2=D
2-A
2
In order to simplify solution, we are with L
1And L
2Merge by following formula:
L
syn=[((L
1+L
2)/2)′+((L
1+L
2)/2)]/2 5
So L
Syn=L '
SynSo far, formula (2) can be organized into following form:
Wherein
Represent not this normal form of Luo Beini crow, and
Satisfy all matrix Z; Through differentiate, can obtain at last:
W
1=δB
1(αU+βV+E)
-1Y 7
W
2=δB
2(αU+βV+E)
-1Y 8
Wherein:
Wherein:
So far, cartoon and true corresponding dimensionality reduction projecting direction matrix W have been obtained
1And W
2
When importing new true key frame sequence, obtain characteristic of correspondence expression matrix according to occupying the figure feature extracting method as index
Wherein m is the key frame quantity of index, m=12 in the present embodiment, and promptly importing length is the true key frame sequence of 12 frames.Obtain the new feature representation formula in the space of true key frame behind dimensionality reduction thus
In like manner, calculate based on the new feature representation formula X ' in the space of the true key frame of cartoon behind dimensionality reduction in " cartoon-true " role data storehouse of step 1)
1=X
1W
1∈ R
N * r
In the dimensionality reduction space, with
In each element as index separately, calculate itself and X
1' in the Euclidean distance of each element, be in the present embodiment 5 of layback minimum as indexed results as the candidate, the user can be from wherein selecting the most suitable key frame; Obtained thus
Corresponding
Promptly return the cartoon key frame that obtains as index with true key frame;
The user can be at the cartoon key frame
On carry out the interest that operations such as deformation, stretching, replacement strengthen the cartoon key frame, in the present embodiment, we provide based on the maintenance rigidity deformation method of tri patch and have finished deformation for the cartoon character health; And by introducing affine matrix, control 6 parameters are wherein finished the affined transformation for cartoon character health any direction on the coordinate transverse axis and the longitudinal axis; Finish replacement for the cartoon character body part by the method for shearing at last, the user can replace on the cartoon key frame accordingly with certain part of the interested jpeg format picture of own institute.
Claims (3)
1. two-dimensional character cartoon generation method based on the heterogeneous characteristic dimensionality reduction is characterized in that its step is as follows:
1) from the two-dimensional cartoon video, extracts the video-frequency band that comprises the complete action of role's cartoon, and according to the movement content of role's cartoon and towards, take corresponding action video by true personage's performance, after the action video process video image technical finesse of the action video that the diagonal angle colour atla is logical and true personage's performance, utilize self-defining extraction method of key frame to extract the key frame of the action video and the action video that true personage performs of role's cartoon, obtain cartoon key frame and true key frame, and cartoon key frame and true key frame are carried out normalization and centralization processing; The cartoon key frame is utilized different feature extracting methods with true key frame, obtain cartoon key frame feature representation matrix and true key frame feature representation matrix, and the feature representation matrix classified according to self-defining action classification, set up " cartoon-true " role data storehouse;
2) from the action video that the true personage who takes performs, utilize self-defining extraction method of key frame to extract the key frame of the action video of true personage's performance, obtain retrieving true key frame, and true key frame carries out normalization and centralization is handled to retrieving; Utilize feature extracting method to obtain the true key frame feature representation matrix of retrieval to retrieving true key frame;
3) cartoon key frame feature representation matrix and true key frame feature representation matrix be by self-defining heterogeneous characteristic dimensionality reduction algorithm, obtains the dimensionality reduction projecting direction matrix of cartoon key frame feature representation matrix of cartoon key frame feature representation matrix and true key frame feature representation matrix correspondence and the dimensionality reduction projecting direction matrix of true key frame feature representation matrix through training; Retrieve true key frame sequence signature and express matrix, express the true key frame sequence dimensionality reduction feature representation matrix of dimensionality reduction projecting direction matrix acquisition retrieval that matrix multiply by true key frame feature representation matrix with the true key frame sequence signature of retrieval, and in projector space, calculate in " cartoon-true " the role data storehouse that obtains and the nearest cartoon key frame sequence of the true key frame sequence dimensionality reduction feature representation matrix of retrieval, at last the cartoon key frame sequence that calculates is returned; The user obtains final cartoon effect video in enterprising edlin of cartoon key frame sequence and the interpolation returned.
2. a kind of two-dimensional character cartoon generation method according to claim 1 based on the heterogeneous characteristic dimensionality reduction, it is characterized in that: described step 1) comprises:
From the two-dimensional cartoon character video, extract the cartoon video V that comprises complete cartoon character action fragment
Cart, the true personage of cause is according to V
CartIn cartoon character movement content and towards before monocular-camera, imitating performance, obtain to include the real video V of complete true figure action fragment
Real
To from V
CartAnd V
RealIn the frame of video playing up out, utilize Hausdorff distance algorithm feature to obtain the cartoon distance matrix
With the actual distance matrix
N wherein
1And n
2Be respectively the number of frames that cartoon and real video are played up out,
Hausdorff distance in the expression cartoon video frame between i frame and the j frame,
Hausdorff distance in the expression real video frame between i frame and the j frame, matrix M
CartAnd M
RealIn each multiply by coefficient respectively
With
Finish normalization, d wherein
Cart_maxAnd d
Real_maxBe respectively matrix M
CartAnd M
RealIn maximal value, obtaining through normalized distance matrix M
CartAnd M
RealAfterwards, according to preset threshold
With
Come the diagonal values in the filtered matrix, will obtain respectively
With
Pairing i frame obtains the cartoon key frame thus as key frame
With true key frame
Wherein n is both quantity;
At the cartoon key frame
Obtain cartoon key frame feature representation matrix according to directivity histogram of gradients feature extracting method
D wherein
1For each key frame characteristic of correspondence is expressed vector
Dimension; Similarly, at true key frame
Obtain true key frame feature representation matrix according to occupying the figure feature extracting method
D wherein
2For each key frame characteristic of correspondence is expressed vector
Dimension; With X
1And X
2In all key frames all be divided into the r class according to the difference of movement content, thereby make all have in each class the cartoon of equal number and true key frame to form X
1And X
2Correspondence one by one on the classification aspect;
To the X that is obtained
1And X
2, respectively through obtain the matrix of centralization as down conversion:
3. a kind of two-dimensional character cartoon generation method according to claim 1 based on the heterogeneous characteristic dimensionality reduction, it is characterized in that: described step 3) comprises:
The cartoon of normalization and centralization and real features are expressed matrix X
1And X
2, calculate the dimensionality reduction projecting direction matrix W that obtains cartoon key frame feature representation matrix according to following objective function algorithm
1Dimensionality reduction projecting direction matrix W with true key frame feature representation matrix
2:
Alpha, gamma
1, beta, gamma
2, δ is a heterogeneous equilibrium degree coefficient, matrix Y=[y
1, y
2..., y
n] ∈ 0,1}
N * r, wherein work as cartoon samples
And authentic specimen
When all belonging to k classification, Y then
Ik=1; Otherwise Y
Ik=0;
With
Belong to same classification, form the Y matrix of full rank; Tr (.) is the mark operational character; The Lagrangian matrix L of cartoon key frame feature representation matrix and cartoon key frame feature representation matrix
1=D
1-A
1, L
2=D
2-A
2
L
syn=[((L
1+L
2)/
2)′+((L
1+L
2)/2)]/2 3
So L
Syn=L '
SynWherein
Represent not this normal form of Luo Beini crow, and
Satisfy all matrix Z; Through differentiate, can obtain at last:
W
1=δB
1(αU+βV+E)
-1Y 4
W
2=δB
2(αU+βV+E)
-1Y 5
Wherein:
Wherein:
So far, obtained the dimensionality reduction projecting direction matrix W of cartoon key frame feature representation matrix
1Dimensionality reduction projecting direction matrix W with true key frame feature representation matrix
2
When the true key frame of input retrieval, obtain the corresponding true key frame feature representation of retrieval matrix according to occupying the figure feature extracting method
Wherein m is the true quantity of key frames of retrieval; Dimensionality reduction projecting direction matrix W according to the true key frame feature representation matrix that obtains
2, calculate the retrieval dimensionality reduction real features expression formula in the space of the true key frame of retrieval behind dimensionality reduction
Dimensionality reduction projecting direction matrix W according to the cartoon key frame feature representation matrix that obtains
1, calculate the dimensionality reduction cartoon feature representation formula X ' in the space of cartoon key frame feature representation matrix behind dimensionality reduction in " cartoon-true " role data storehouse
1=X
1W
1∈ R
N * r, wherein r is the dimension in dimensionality reduction space;
In the dimensionality reduction space, with retrieval dimensionality reduction real features expression formula
In each element as index separately, calculate and dimensionality reduction cartoon feature representation formula X
1' in the Euclidean distance of each element, and several of layback minimum are as cartoon indexed results X
Result, finish the process of returning the cartoon key frame that obtains with true key frame as index;
The user is at cartoon indexed results X
ResultOn carry out the interest that operations such as deformation, stretching, replacement strengthen the cartoon key frame, obtain final cartoon video by approach based on linear interpolation at last.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN2011100525858A CN102129707A (en) | 2011-02-28 | 2011-02-28 | Heterogeneous feature dimension reduction-based two-dimensional role cartoon generation method |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN2011100525858A CN102129707A (en) | 2011-02-28 | 2011-02-28 | Heterogeneous feature dimension reduction-based two-dimensional role cartoon generation method |
Publications (1)
Publication Number | Publication Date |
---|---|
CN102129707A true CN102129707A (en) | 2011-07-20 |
Family
ID=44267781
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN2011100525858A Pending CN102129707A (en) | 2011-02-28 | 2011-02-28 | Heterogeneous feature dimension reduction-based two-dimensional role cartoon generation method |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN102129707A (en) |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN111079588B (en) * | 2019-12-03 | 2021-09-10 | 北京字节跳动网络技术有限公司 | Image processing method, device and storage medium |
Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN1851710A (en) * | 2006-05-25 | 2006-10-25 | 浙江大学 | Embedded multimedia key frame based video search realizing method |
CN101079154A (en) * | 2007-03-02 | 2007-11-28 | 腾讯科技(深圳)有限公司 | Role animation realization method and system |
CN101216948A (en) * | 2008-01-14 | 2008-07-09 | 浙江大学 | Cartoon animation fabrication method based on video extracting and reusing |
CN101360184A (en) * | 2008-09-22 | 2009-02-04 | 腾讯科技(深圳)有限公司 | System and method for extracting key frame of video |
CN101394522A (en) * | 2007-09-19 | 2009-03-25 | 中国科学院计算技术研究所 | Detection method and system for video copy |
-
2011
- 2011-02-28 CN CN2011100525858A patent/CN102129707A/en active Pending
Patent Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN1851710A (en) * | 2006-05-25 | 2006-10-25 | 浙江大学 | Embedded multimedia key frame based video search realizing method |
CN101079154A (en) * | 2007-03-02 | 2007-11-28 | 腾讯科技(深圳)有限公司 | Role animation realization method and system |
CN101394522A (en) * | 2007-09-19 | 2009-03-25 | 中国科学院计算技术研究所 | Detection method and system for video copy |
CN101216948A (en) * | 2008-01-14 | 2008-07-09 | 浙江大学 | Cartoon animation fabrication method based on video extracting and reusing |
CN101360184A (en) * | 2008-09-22 | 2009-02-04 | 腾讯科技(深圳)有限公司 | System and method for extracting key frame of video |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN111079588B (en) * | 2019-12-03 | 2021-09-10 | 北京字节跳动网络技术有限公司 | Image processing method, device and storage medium |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN109948425B (en) | Pedestrian searching method and device for structure-aware self-attention and online instance aggregation matching | |
CN110837836B (en) | Semi-supervised semantic segmentation method based on maximized confidence | |
Elasri et al. | Image generation: A review | |
CN102054178B (en) | A kind of image of Chinese Painting recognition methods based on local semantic concept | |
Li et al. | A comparison of 3D shape retrieval methods based on a large-scale benchmark supporting multimodal queries | |
US9558268B2 (en) | Method for semantically labeling an image of a scene using recursive context propagation | |
Prosser et al. | Person re-identification by support vector ranking. | |
CN108765383B (en) | Video description method based on deep migration learning | |
CN106096542B (en) | Image video scene recognition method based on distance prediction information | |
CN108509978A (en) | The multi-class targets detection method and model of multi-stage characteristics fusion based on CNN | |
CN106845430A (en) | Pedestrian detection and tracking based on acceleration region convolutional neural networks | |
CN106126581A (en) | Cartographical sketching image search method based on degree of depth study | |
CN105574063A (en) | Image retrieval method based on visual saliency | |
Su et al. | 3D-assisted feature synthesis for novel views of an object | |
CN104572804A (en) | Video object retrieval system and method | |
CN108846047A (en) | A kind of picture retrieval method and system based on convolution feature | |
CN105701225B (en) | A kind of cross-media retrieval method based on unified association hypergraph specification | |
Veeravasarapu et al. | Adversarially tuned scene generation | |
CN109213853A (en) | A kind of Chinese community's question and answer cross-module state search method based on CCA algorithm | |
Ye et al. | Embedding sequential information into spatiotemporal features for action recognition | |
CN110334724A (en) | Remote sensing object natural language description and multiple dimensioned antidote based on LSTM | |
CN104616005A (en) | Domain-self-adaptive facial expression analysis method | |
CN115690549A (en) | Target detection method for realizing multi-dimensional feature fusion based on parallel interaction architecture model | |
CN112364791A (en) | Pedestrian re-identification method and system based on generation of confrontation network | |
Wu et al. | Joint analysis of shapes and images via deep domain adaptation |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
C06 | Publication | ||
PB01 | Publication | ||
C10 | Entry into substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
C02 | Deemed withdrawal of patent application after publication (patent law 2001) | ||
WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20110720 |