CN101853239A - Nonnegative matrix factorization-based dimensionality reducing method used for clustering - Google Patents
Nonnegative matrix factorization-based dimensionality reducing method used for clustering Download PDFInfo
- Publication number
- CN101853239A CN101853239A CN201010167504.4A CN201010167504A CN101853239A CN 101853239 A CN101853239 A CN 101853239A CN 201010167504 A CN201010167504 A CN 201010167504A CN 101853239 A CN101853239 A CN 101853239A
- Authority
- CN
- China
- Prior art keywords
- matrix
- data
- sigma
- dimension
- dimensional
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 239000011159 matrix material Substances 0.000 title claims abstract description 89
- 238000000034 method Methods 0.000 title claims abstract description 62
- 238000013507 mapping Methods 0.000 claims abstract description 27
- 238000010606 normalization Methods 0.000 claims abstract description 17
- 238000007405 data analysis Methods 0.000 claims abstract description 3
- 238000013144 data compression Methods 0.000 claims abstract description 3
- 230000009467 reduction Effects 0.000 claims description 19
- 239000013598 vector Substances 0.000 claims description 12
- 238000009833 condensation Methods 0.000 claims description 7
- 230000005494 condensation Effects 0.000 claims description 7
- 238000004364 calculation method Methods 0.000 claims description 2
- 238000013480 data collection Methods 0.000 claims description 2
- 238000004422 calculation algorithm Methods 0.000 abstract description 12
- 230000006870 function Effects 0.000 abstract description 8
- 238000010801 machine learning Methods 0.000 abstract description 4
- 238000013179 statistical model Methods 0.000 abstract description 2
- 238000000354 decomposition reaction Methods 0.000 description 13
- 230000008569 process Effects 0.000 description 6
- 230000005021 gait Effects 0.000 description 5
- 238000004458 analytical method Methods 0.000 description 4
- 238000007906 compression Methods 0.000 description 4
- 230000006835 compression Effects 0.000 description 4
- 238000012545 processing Methods 0.000 description 4
- 238000000513 principal component analysis Methods 0.000 description 3
- 230000017105 transposition Effects 0.000 description 3
- 241001269238 Data Species 0.000 description 2
- 238000013528 artificial neural network Methods 0.000 description 2
- 230000008901 benefit Effects 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000012549 training Methods 0.000 description 2
- 241000271566 Aves Species 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 238000007621 cluster analysis Methods 0.000 description 1
- 238000012217 deletion Methods 0.000 description 1
- 230000037430 deletion Effects 0.000 description 1
- 238000000151 deposition Methods 0.000 description 1
- 238000009795 derivation Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 230000008030 elimination Effects 0.000 description 1
- 238000003379 elimination reaction Methods 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 210000003746 feather Anatomy 0.000 description 1
- 239000012467 final product Substances 0.000 description 1
- 244000144992 flock Species 0.000 description 1
- 238000012804 iterative process Methods 0.000 description 1
- 238000003064 k means clustering Methods 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 210000000653 nervous system Anatomy 0.000 description 1
- 238000003909 pattern recognition Methods 0.000 description 1
- 239000000047 product Substances 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000000926 separation method Methods 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 230000003595 spectral effect Effects 0.000 description 1
- 238000010183 spectrum analysis Methods 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/213—Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods
Landscapes
- Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Data Mining & Analysis (AREA)
- Theoretical Computer Science (AREA)
- Bioinformatics & Computational Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention belongs to the technical field of statistical model identification and machine learning, and in particular discloses a nonnegative matrix factorization-based dimensionality reducing method used for clustering. The method comprises the following steps of: adopting a KL distance; adding a data normalization constraint; directly discovering the internal relation among data dimensionalities by minimizing an object error function between data compression and reconstruction; obtaining a mapping matrix; and projecting high-dimensional data to a low-dimensional subspace by using the mapping matrix so as to perform effective data analysis such as clustering and the like. A simpler iterative formula compared with an original factorization method is obtained by the method; and normalization can be maintained naturally in each iterative updating. The normalization makes a final mapping matrix have higher sparsity compared with the original factorization method. In the obtained low-dimensional space, a clustering result shows that a more effective low-dimensional data characteristic can be obtained by the method; and an algorithm is simple and effective.
Description
Technical field
The invention belongs to statistical model identification and machine learning techniques field, be specifically related to a kind of dimension reduction method that is used for cluster based on the nonnegative matrix decomposition.
Background technology
Cluster is one of the most basic research task in machine learning field.In actual applications, each venn diagram of data shows a relevant feature.Generally, be difficult to judge simply which feature helps cluster, method commonly used is exactly an image data feature as much as possible, carries out cluster then.Therefore, data characteristics generally is a higher-dimension, and the data characteristics of higher-dimension is brought the problem of two aspects usually: 1) storage and calculation cost are higher, and 2) the dimension disaster problem.In actual applications, the dimension disaster problem is one of subject matter of facing of many mode identification methods, as Gait Recognition, image classification and text-processing etc.During especially in the face of the limited sample of higher-dimension, dimension disaster is more outstanding, and it directly causes the cluster performance decrease.How dimensionality reduction is studied exactly the high dimensional data compressing mapping in low n-dimensional subspace n, thereby more effectively finishes task such as cluster.This mapping can be linear or nonlinear.Because linear dimension reduction method is simply effective, it is widely used in the every field of machine learning and pattern-recognition.
Current, except above-mentioned linear or nonlinear dimension reduction method,, can be divided into following a few class again from different angles: as, based on whether using the class label information, can be divided into not having and supervise semi-supervised and measure of supervision.What the present invention considered is the nonnegativity of data.Many methods have symbol arbitrarily, but the decomposition of picture nonnegative matrix (nonnegative matrix factorization, NNMF) method can keep the nonnegativity of data symbol, and this has reflected text, the essential characteristic of data such as image.
(principal component analysis, PCA) to supervise linear dimension reduction method be [1] to Jing Dian nothing to principal component analysis.One of basic problem that dimensionality reduction relates to is how to select suitable dimension r, and the PCA method is easy to obtain the r value by the analytical characteristic value.Finally, all former data can be represented best according to the least squares error principle in the r dimensional linear subspace of former feature space.Spectral analysis method has the similar method of choosing the low-dimensional number [2].Spectral method has solid theory and uses widely and be easy to and carry out.
Another kind of popular subspace dimension reduction method be linear discriminant analysis (Fisher ' s linear discriminant analysis, LDA) [3].This method can keep the dependency structure of category information in the low-dimensional projector space.When training sample is insufficient, gets and to adopt semi-supervised method [4].Contrast, PCA and LDA are linear dimension reduction methods, LDA carries out modeling to the class differences clearly, and PCA does not consider category information.It is optimum that the PCA method is rebuild (reconstruction) for data, but it is not suitable for the separation and the identification of class.
Considered linear case above, when data set can not be represented with sample average and covariance matrix effectively, perhaps, and when comprising complex characteristic in the data, the linear method poor efficiency that becomes.In this case, can utilize nuclear skill (kernel trick), as principal component analysis (kernel PCA) [5] based on nuclear.(locally linearembedding, LLE) (artificial neural network, ANN) [7] also are the methods of using always to other nonlinear technology for [6] and artificial neural network as local linear embedding.The LLE method has kept the neighbor relationships before and after the dimensionality reduction.But the ANN method simulated the nervous system retrospective (as, study) mechanism.But the training of ANN model generally is consuming time.
The present invention focuses on nonnegative matrix decomposition method [8].The NNMF method can be data matrix X
N * mBe decomposed into C
N * r* M
R * m, when n and m represented dimension and sample number respectively, C was a basis matrix and M is a matrix of coefficients.Popular its simple and practical property that is of NNMF method.In order to solve the problem in the application, different constraints or parameter have been added among the standard N NMF.As, Li etc. have proposed local NNMF by three new constraints of adding on base and matrix of coefficients, and (localNNMF LNMF), thereby realizes useful local visual pattern [9]; Cichocki etc. have investigated (the method for [10] of α-divergence) (brief note is α NMF) based on α " distance "; Shahnaz etc. have considered the sparse property of matrix of coefficients, have proposed GD-CLS algorithm [11].Basis matrix can be considered projection matrix, so NNMF can be used for dimensionality reduction [12,13].Contrast, except that the NNMF method, above-mentioned method all has symbol arbitrarily.But these NNMF methods based on KL " distance " or Euclidean distance also have some common problems.At first, former NNMF method that they are complicated to some extent is so final update rule needs more computing time than former method.Secondly, iteration update method explanation Matrix C and M are the functions of self, so C and M need be initialised simultaneously, and too much initially may not necessarily produce more effective matrix decomposition.The present invention has provided a kind of normalization compression method (normalized compressionusing NNMF is designated as NCMF) that decomposes based on nonnegative matrix, and NCMF has finally obtained simple and effective alternative manner by the normalization data dimension.
Summary of the invention
The object of the present invention is to provide does not a kind ofly increase original NNMF complexity and fast convergence rate, the nonnegative matrix dimension reduction method that economize computing time.
Dimension reduction method provided by the invention adopts KL " distance ", and the target error function between minimise data compression and the reconstruction finally obtains mapping matrix; Utilize this mapping matrix again, high dimensional data is projected in the low n-dimensional subspace n, thereby carry out the active data analysis, as cluster etc.In minimizing the process of above-mentioned error, on the basis of the non-negative requirement of data, add the data normalization constraint, the L1 normal form of promptly decomposing two every row of matrix that obtain is always 1.This normalization constraint does not have complicated former resolution problem, on the contrary, has obtained the result more terse than methods such as former NNMF.The NCMF algorithm that the present invention proposes can keep normalizing naturally in each iteration is upgraded.These characteristics make final mapping matrix compare with methods such as former NNMF to have better sparse property.In the lower dimensional space that obtains, shared class k means clustering algorithm shows that NCMF has obtained more effectively low-dimensional data characteristics.Compare with existing related algorithm, great advantage of the present invention is simple and effective.
Among the present invention, when among the data matrix X during sample of every line display, i.e. X
N * mBe decomposed into C
N * r* M
R * mThe time, different with above-mentioned matrix decomposition, be equivalent to the X data matrix has been carried out transposition, make n and m represent sample number and dimension respectively.The internal relation between the different dimensional can be directly realized in this processing, thereby reaches effective dimensionality reduction purpose.In many application, the sample column vector among the data matrix X is also by normalization, and the present invention has carried out transpose process to X, so all dimensional vectors of normalization.This disposal route makes Matrix C and M tool be of fresh significance: C has write down the compression result of high dimensional data among the X, then can be considered condensation matrix, and M has reflected the mapping relations between the low-dimensional (column vector) among higher-dimension among the X (column vector) and the C, and then M is a mapping matrix.The present invention's distance metric method of KL " distance " as matrix X and condensation matrix C and mapping matrix M product.Corresponding objective function is:
Wherein, ∑
iX
Ij=1, (row of representing matrix C of 1≤k≤r) and the row of matrix M, μ and v are positive parameters to k.The update rule that obtains is,
Wherein, the transposition of T representing matrix, i.e. M
TThe transposition of representing matrix M.
NCMF is resolving into from X the process of C and M, constantly rebuilds X by CM, and the .NCMF method has simplicity as shown in Figure 1, normalizing and sparse property.
1, simplicity
Compare the decomposition method based on KL " distance " (brief note is KL_NMF) that Lee proposes in [8], we have oversimplified the renewal to Matrix C.This simple renewal is from following observation.Pushed away to get XMT=C (MMT) and C=XMT (Z)-1 by X=CM, wherein Z=MMT is the matrix of an order r.If Z is a unit matrix, can delete safely.Following application example 1 (with reference to equation 8) illustrates that the Z of this moment is the matrix of a diagonal dominance elder generation, and then Z is an approximate diagonal matrix.When higher-dimension (row among the X) balancedly corresponded to low-dimensional (row among the C), Z had become an approximate numbers matrix (scalar matrix).This moment, Z can promptly upgrade getting final product by the safety deletion and being reverse into simple division equally as the rule (2) of C.Done two advantages like this: simple, eliminated contrary calculating; Avoid introducing negative.In fact, even higher-dimension (row among the X) corresponds to lack of balance in the low-dimensional (row among the C), above-mentioned formula is suitable equally, as the Occam razor: simple and practicality is best [14].The above-mentioned renewal of C also has another kind of the explanation.The renewal that is listed as among the C is equivalent to the equal vector of some row (dimension) corresponding among the X, and above-mentioned update rule just in time embodies these characteristics.This simplicity can be quickened iterative process, thereby makes the NCMF method have higher operational efficiency than methods such as NNMF.
2, normalizing
Based on above-mentioned two more new formula can further derive following conclusion: C and M is keeping normalizing in each iteration is upgraded.More new formula and the normalization of X in application based on C can get,
Similarly,
Above-mentioned derivation has proved NCMF in decomposable process, needs only data matrix X at first by normalization, and Matrix C and M just have the normalization characteristics.Same analysis explanation NNMF does not have this normalization characteristics.
We so can analyze the convergence of NCMF.Update rule (2) illustrates that C is the function of M, and then objective function (1) can be rewritten as the only function of variable M.Therefore, it is important that the convergence of M causes the pass.And article [15] illustrates that M can converge to local static point.So the renewal of M and C can reduce the value of objective function (1), NCMF is a convergent.
3, sparse property
Formula (3) illustrates that the renewal of M does not need the similarity between the computing dimension, but the correlativity between the row among each the peacekeeping C among the calculating X.In whole decomposable process, element Xij rebuilds with (CM) ij.And then, ω
Ij=(X
Ij)/((CM)
Ij) initial value and its magnitude relationship between rebuilding has been described.If (X
Ij)>(CM)
Ij, illustrate that reconstructed value is not enough to estimate initial value, so ω
Ij>1 amplifies C
IkOn the contrary, (X
Ij)<(CM)
IjReconstructed value over-fitting (overfitting) is described, then ω
Ij<1 can reduce C
IkIf make γ
Kj=∑
iω
IjC
Ik, M then
Kj=r
KjM
KjM is described
KjBy coefficient r
KjUpgrade.When mapping matrix M make j dimension Xvj (v=[1,2 ..., n] T) mainly be under the jurisdiction of M
KjThe time, we can illustrate r
KjAlways be not less than 1: upgrade the step in each iteration, formula (5) illustrates ∑
kM
KjAlways be 1.When Xvj mainly is under the jurisdiction of M
KjThe time, Xvj is under the jurisdiction of M
K ' j(1≤k '≤r ﹠amp; The value of k ' ≠ k) has less degree of membership.And this whole renewal is a convergent.Therefore, the dimension of each among the X can forward to the k from k ' the degree of membership of row (dimension) among the C.So, r
KjAlways be not less than 1.This degree of membership that each dimension is described is along with the renewal of M highlights gradually.This has also just in time illustrated the characteristics of the sparse property of M.
The sparse property characteristics of mapping matrix M also can obtain by analyzing experimental result.The present invention weighs sparse property with information entropy (entropy) [16].Entropy is to probabilistic a kind of tolerance, and when a non-negative matrix was only formed by 0 and 1, the entropy minimum was 0; When these nonnegative numbers all equate, the entropy maximum.So sparse property can be measured with entropy: entropy is more little, sparse property is good more.For edge distribution q ∈ R
1 * m, the entropy of q may be defined as,
If q matrix is considered as a vector, do same processing.From the angle of fuzzy clustering, the entropy of M has illustrated its ambiguity.At first should become joint distribution (joint distribution) to M is ∑
K, jM '
Kj=1 form.If M only forms by 0 and 1, then M ' has only the 1/m of m individual non-0, and this moment, entropy was
Then a method of reasonably measuring the sparse property of M may be defined as
g(M)=H(M’)-log(m).????????????????????(7)
As g (M) when equaling 0, reflected the most sparse result of desirable many-one mapping.
According to above content, corresponding algorithm is summarized as follows:
1, given data set X (depositing sample), low n-dimensional subspace n dimension r and iterations l by row.
2, the sample number n of computational data matrix X and dimension m.
3, normalization X makes ∑
iX
Ij=1.
4, initialization mapping matrix M ∈ R
R * m
5,1 step 6 of iteration and 7.
6, use
Upgrade C.
8, return condensation matrix C and mapping matrix M.
Description of drawings
The decomposition and reconstruction process of Fig. 1 NCMF.
Frame on Fig. 2 gait data collection is removed the image of the figure viewed from behind.
Embodiment
Embodiment 1 can illustrate matrix X, the characteristics of C and M more intuitively.Matrix
It is a simple data matrix.For text data, the word that each row is corresponding different, the statistics vector of various words occurrence number in a certain text of each line display.Our task is in the lower dimensional space of compression similar text to be got together.In order to simplify the problems referred to above, the data among the data matrix A can at first be carried out the binarization processing, that is,
Based on this, NCMF can obtain following decomposition,
Three matrixes of above-mentioned decomposition correspondence are used X respectively, when C and M represent, obtain X=CM.Matrix X is made up of 4 capable vectors of sample, and each sample has 5 dimensions (row), and the condensation matrix C after decomposing has dropped to 2 from 5 dimensions of X and ties up.Mapping matrix M has write down the correlativity between different higher-dimensions and the low-dimensional, its row vector has illustrated that the higher dimensional space among the original X is compressed together more similar dimension in current lower dimensional space, and the column vector of M has illustrated among the X mapping relations between the low-dimensional in a certain original higher-dimension and C.This is a special case, and in actual applications, M becomes with 1 array by approaching 0, and equation (5) illustrates that it is 1 that M can keep each row L1 normal form.
The experimental result of embodiment 2 explanations on gait (gait) data set [17].The raw data of this data set is some video datas, from all frame data such as the accompanying drawing 2 of the elimination background wherein extracted.In order to use these gait datas, can further extract data message, form the data matrix that each ID number (corresponding people) has some sample characteristics vectors.For the dimensionality reduction characteristics of NCMF algorithm are described, the present invention has adopted in the galData data from maximum preceding 6 of sample and ID number right data has been carried out cluster analysis.To relevant result, we adopt three kinds of measures, and working time, the precision of sparse property and cluster is used second respectively, and information entropy and purity (purity) are measured.
The class k mean algorithm SIB[18 that The data behind the dimensionality reduction is same] carry out cluster, and adopt purity tolerance clustering precision.If cluster result matrix S record, wherein Shk represents that h birdss of the same feather flock together to k bunch ratio (1≤h, k≤r).Then purity may be defined as,
Finally, we can use average purity
The tolerance clustering precision.
Each decomposition algorithm all uses row to represent data sample in the experiment.Subordinate list 1 has shown concrete experimental result.NCMF is minimum working time; The entropy minimum of mapping matrix M, this explanation matrix M is the most sparse, and the reduction data that NCMF obtains have obtained maximum precision.
NCMF is more more effective on the low-dimensional data characteristics of working time and acquisition than other method in these experimental result explanations.Although NCMF has added more more constraints than NNMF, the result has obtained more terse update rule.This simplicity makes NCMF more efficient than former NNMF on the one hand, simultaneously also than making M only approach identical decomposition more by 0 and 1 expression.Above-mentioned sparse property and normalized Analysis explanation NCMF have obtained sparse mapping matrix M naturally.The more new formula (2) of NCMF and (3) illustrate and account for 2/3 of all working times computing time of mapping matrix that this explanation NCMF has emphasized the renewal of matrix M.In case obtain this matrix, can calculate condensation matrix C apace.In a word, this simplicity, normalizing and sparse property make that the NCMF algorithm is more effective.
Table 1 has illustrated the execution time of the relevant decomposition algorithm with other of NCMF dimensionality reduction in 6 dimension spaces of compression, the experimental result of clustering precision three aspects of the sparse property of mapping matrix and low dimension data.
Table 1
Algorithm | ??NCMF | ??KL_NMF | ??LNMF | ??αNMF | ??GD-CLS |
Working time (second) | ??46.68 | ??57.73 | ??66.25 | ??76.71 | ??329.7 |
Degree of rarefication (entropy) | ??0.6465 | ??1.3876 | ??1.5346 | ??1.2147 | ??1.2678 |
Precision (purity) | ??0.7443 | ??0.7192 | ??0.7147 | ??0.7030 | ??0.7239 |
[0071]List of references:
[1]K.Fukunaga,Introduction?to?Statistical?pattern?recognition,Academic?Press,2ndedition,1991.
[2]J.Shi,J.Malik,Normalized?cuts?and?image?segmentation,IEEE?Transactions?on?PatternAnalysis?and?Machine?Intelligence?22(8)(2000)888-905.
[3]R.A.Fisher,The?use?of?multiple?measurements?in?taxonomic?problems,Annals?ofEugenics?7(1936)179-188.
[4]D.Cai,X.F.He,J.W.Han,Semi-supervised?discriminant?analysis,in:IEEE?11thInternational?Conference?on?Computer?Vision(ICCV),2007,pp.1-7.
[5]S.Bernhard,S.Alexander,M.Klaus-Robert,Kernel?principal?component?analysis,1999.
[6]S.Roweis,L.Saul,Nonlinear?dimensionality?reduction?by?locally?linear?embedding.Science?290(5500)(2000)2223-2326.
[7]R.O.Duda,P.E.Hart,D.G.Stork,Pattern?classification,2nd?edition,Wiley,2001.
[8]D.D.Lee,H.S.Seung,Learning?the?parts?of?objects?by?nonnegative?matrix?factorization,Nature?401(6755)(1999)788-791.
[9]S.Z.Li,X.W.Hou,H.J.Zhang,Learning?spatially?localized,parts-based?representation,in:IEEE?Conference?on?Computer?Vision?and?Pattern?Recognition(CVPR),vol.1,2001,pp.207-212.
[10]A.Cichocki,H.Lee,Y.D.Kim,S.Choi,Non-negative?matrix?factorization?with_-divergence,Pattern?Recognition?Letters?29(9)(2008)1433-1440.
[11]F.Shahnaz,M.W.Berry,V.P.Pauca,R.J.Plemmons,Document?clustering?usingnonegative?matrix?factorization,Information?Processing?and?Management?42(2006)(2005)373-386.
[12]W.X.Liu,K.H.Yuan,D.Yea,Reducing?microarray?data?via?nonnegative?matrixfactorization?for?visualization?and?clustering?analysis,Journal?of?Biomedical?Informatics?41(4)(2008)602-606.
[13]S.Tsuge,M.Shishibori,S.Kuroiwa,K.Kita,Dimensionality?reduction?usingnon-negative?matrix?factorization?for?information?retrieval,in:2001?IEEE?InternationalConference?of?Systems,Man,and?Cybernetics,vol.2,2001,pp.960-965.
[14]Occam?razor.http://en.wikipedia.org/wiki/Occam%27s_razor.
[15]D.D.Lee,H.S.Seung,Algorithms?for?non-negative?matrix?factorization,in:Proceedings?of?the?13th?Annual?Conference?on?Neural?Information?Processing?Systems(NIPS),vol.13,2000,pp.556-562.
[16]T.Cover,J.Thomas,Elements?of?Information?Theory,JohnWiley?&?Sons,New?York,USA,1991.
[17]Gait?data?set.http://marathon.csee.usf.edu/GaitBaseline/.
[18]N.Slonim,N.Friedman,N.Tishby,Unsupervised?document?classification?usingsequential?information?maximization,in:Proceeding?of?25th?ACM?intermational?Conference?onResearch?and?Development?in?Information?Retrieval(SIGIR),2002,pp.129-136.。
Claims (3)
1. dimension reduction method that decomposes based on nonnegative matrix that is used for cluster, it is characterized in that adopting KL " distance ", on the basis of the non-negative requirement of data, add the data normalization constraint, by the target error function between minimise data compression and the reconstruction, directly seek the internal relation between the data dimension, finally obtain mapping matrix; Utilize this mapping matrix again, high dimensional data is projected in the low n-dimensional subspace n, thereby carry out corresponding data analysis.
2. method according to claim 1 is characterized in that sample of every line display among the data matrix X, and X
N * mBe decomposed into C
N * r* M
R * m, by the dimensional vector of normalization data matrix X, finally obtaining condensation matrix C and mapping matrix M, corresponding objective function is:
Wherein, ∑
iX
Ij=1, n is a sample number, and m is a dimension, and r is the dimension of lower dimensional space, k (row of representing matrix C of 1≤k≤r) and the row of matrix M, μ and v are positive parameters, the update rule that obtains is:
The subscript T of matrix represents this transpose of a matrix.
3. method according to claim 1 is characterized in that concrete calculation procedure is as follows:
1), given by the capable sample data collection X that deposits, low n-dimensional subspace n dimension r and iterations l;
2), the sample number n of computational data matrix X and dimension m;
3), normalization X makes ∑
iX
Ij=1;
4), initialization mapping matrix M ∈ R
R * m
5), 1 step 6) of iteration and 7);
6), use
Upgrade C;
7), use
Upgrade M;
8), return condensation matrix C and mapping matrix M.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201010167504.4A CN101853239A (en) | 2010-05-06 | 2010-05-06 | Nonnegative matrix factorization-based dimensionality reducing method used for clustering |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201010167504.4A CN101853239A (en) | 2010-05-06 | 2010-05-06 | Nonnegative matrix factorization-based dimensionality reducing method used for clustering |
Publications (1)
Publication Number | Publication Date |
---|---|
CN101853239A true CN101853239A (en) | 2010-10-06 |
Family
ID=42804737
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN201010167504.4A Pending CN101853239A (en) | 2010-05-06 | 2010-05-06 | Nonnegative matrix factorization-based dimensionality reducing method used for clustering |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN101853239A (en) |
Cited By (20)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN102254328A (en) * | 2011-05-17 | 2011-11-23 | 西安电子科技大学 | Video motion characteristic extracting method based on local sparse constraint non-negative matrix factorization |
CN102779162A (en) * | 2012-06-14 | 2012-11-14 | 浙江大学 | Matrix concept decomposition method with local area limit |
CN102929894A (en) * | 2011-08-12 | 2013-02-13 | 中国人民解放军总参谋部第五十七研究所 | Online clustering visualization method of text |
CN103728135A (en) * | 2013-12-16 | 2014-04-16 | 西安交通大学 | Bearing fault feature extraction and diagnosis method of non-negative matrix factorization |
CN103824062A (en) * | 2014-03-06 | 2014-05-28 | 西安电子科技大学 | Motion identification method for human body by parts based on non-negative matrix factorization |
CN104834746A (en) * | 2015-05-23 | 2015-08-12 | 华东交通大学 | Heterogeneous feature time sequence data evolution and clustering method based on graphic processing unit |
CN105335528A (en) * | 2015-12-01 | 2016-02-17 | 中国计量学院 | Customized product similarity judgment method based on product structure |
CN105389291A (en) * | 2015-12-21 | 2016-03-09 | 西安电子科技大学 | Data processing method based on incremental locally convex non-negative matrix factorization |
CN107423717A (en) * | 2017-08-01 | 2017-12-01 | 山东大学 | A kind of matching and recognition method of the manual drawing electrical symbol decomposed based on joint nonnegative matrix two and standard electrical symbol |
CN107924484A (en) * | 2015-07-23 | 2018-04-17 | 丰田汽车欧洲股份有限公司 | For the system and computer based method in environmental background Imitating class people's controlling behavior |
CN108491863A (en) * | 2018-02-27 | 2018-09-04 | 南京信息工程大学 | Color image processing method based on Non-negative Matrix Factorization and convolutional neural networks |
CN108710502A (en) * | 2018-04-08 | 2018-10-26 | 华中科技大学 | The individual cultivation method and its system of digital control system |
WO2019014906A1 (en) * | 2017-07-20 | 2019-01-24 | 淮阴工学院 | Generalized multivariate singular spectrum analysis method for mode reconstruction and prediction |
CN109426861A (en) * | 2017-08-16 | 2019-03-05 | 阿里巴巴集团控股有限公司 | Data encryption, machine learning model training method, device and electronic equipment |
CN109960730A (en) * | 2019-04-19 | 2019-07-02 | 广东工业大学 | A kind of short text classification method, device and equipment based on feature extension |
CN110442911A (en) * | 2019-07-03 | 2019-11-12 | 中国农业大学 | A kind of higher-dimension complication system Uncertainty Analysis Method based on statistical machine learning |
CN110674755A (en) * | 2019-09-25 | 2020-01-10 | 中国计量大学 | Gait recognition method based on optimum gait flow pattern space |
CN111488900A (en) * | 2019-01-29 | 2020-08-04 | 大连理工大学 | Multi-view related feature learning model based on non-negative matrix factorization |
CN114140635A (en) * | 2021-08-10 | 2022-03-04 | 北京工业大学 | Non-negative matrix decomposition method for self-expression learning supervision |
CN117576493A (en) * | 2024-01-16 | 2024-02-20 | 武汉明炀大数据科技有限公司 | Cloud storage compression method and system for large sample data |
Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20030233369A1 (en) * | 2002-06-17 | 2003-12-18 | Fujitsu Limited | Data classifying device, and active learning method used by data classifying device and active learning program of data classifying device |
US20050027664A1 (en) * | 2003-07-31 | 2005-02-03 | Johnson David E. | Interactive machine learning system for automated annotation of information in text |
JP2008538041A (en) * | 2005-04-14 | 2008-10-02 | 本田技研工業株式会社 | Partially learned learning machine for data classification based on local neighborhood Laplace eigenmap |
-
2010
- 2010-05-06 CN CN201010167504.4A patent/CN101853239A/en active Pending
Patent Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20030233369A1 (en) * | 2002-06-17 | 2003-12-18 | Fujitsu Limited | Data classifying device, and active learning method used by data classifying device and active learning program of data classifying device |
US20050027664A1 (en) * | 2003-07-31 | 2005-02-03 | Johnson David E. | Interactive machine learning system for automated annotation of information in text |
JP2008538041A (en) * | 2005-04-14 | 2008-10-02 | 本田技研工業株式会社 | Partially learned learning machine for data classification based on local neighborhood Laplace eigenmap |
Non-Patent Citations (1)
Title |
---|
《Neurocomputing》 20100312 zhenfeng zhu et al "Normalized dimensionality reduction using nonnegative matrix factorization" 第1784页至第1787页 1-3 第73卷, 第10-12期 2 * |
Cited By (34)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN102254328A (en) * | 2011-05-17 | 2011-11-23 | 西安电子科技大学 | Video motion characteristic extracting method based on local sparse constraint non-negative matrix factorization |
CN102929894A (en) * | 2011-08-12 | 2013-02-13 | 中国人民解放军总参谋部第五十七研究所 | Online clustering visualization method of text |
CN102779162B (en) * | 2012-06-14 | 2014-09-17 | 浙江大学 | Matrix concept decomposition method with local area limit |
CN102779162A (en) * | 2012-06-14 | 2012-11-14 | 浙江大学 | Matrix concept decomposition method with local area limit |
CN103728135A (en) * | 2013-12-16 | 2014-04-16 | 西安交通大学 | Bearing fault feature extraction and diagnosis method of non-negative matrix factorization |
CN103824062B (en) * | 2014-03-06 | 2017-01-11 | 西安电子科技大学 | Motion identification method for human body by parts based on non-negative matrix factorization |
CN103824062A (en) * | 2014-03-06 | 2014-05-28 | 西安电子科技大学 | Motion identification method for human body by parts based on non-negative matrix factorization |
CN104834746A (en) * | 2015-05-23 | 2015-08-12 | 华东交通大学 | Heterogeneous feature time sequence data evolution and clustering method based on graphic processing unit |
CN104834746B (en) * | 2015-05-23 | 2017-12-12 | 华东交通大学 | Heterogeneous characteristic time series data evolution clustering method based on graphics processing unit |
US11379705B2 (en) | 2015-07-23 | 2022-07-05 | Toyota Motor Europe | System and computer-based method for simulating a human-like control behaviour in an environmental context |
CN107924484A (en) * | 2015-07-23 | 2018-04-17 | 丰田汽车欧洲股份有限公司 | For the system and computer based method in environmental background Imitating class people's controlling behavior |
CN105335528A (en) * | 2015-12-01 | 2016-02-17 | 中国计量学院 | Customized product similarity judgment method based on product structure |
CN105335528B (en) * | 2015-12-01 | 2019-03-19 | 中国计量学院 | A kind of customed product similarity judgment method based on product structure |
CN105389291A (en) * | 2015-12-21 | 2016-03-09 | 西安电子科技大学 | Data processing method based on incremental locally convex non-negative matrix factorization |
CN105389291B (en) * | 2015-12-21 | 2018-08-21 | 西安电子科技大学 | Data processing method based on the convex local Non-negative Matrix Factorization of increment type |
WO2019014906A1 (en) * | 2017-07-20 | 2019-01-24 | 淮阴工学院 | Generalized multivariate singular spectrum analysis method for mode reconstruction and prediction |
CN107423717A (en) * | 2017-08-01 | 2017-12-01 | 山东大学 | A kind of matching and recognition method of the manual drawing electrical symbol decomposed based on joint nonnegative matrix two and standard electrical symbol |
CN107423717B (en) * | 2017-08-01 | 2020-01-14 | 山东大学 | Matching identification method for manually drawn electrical symbols and standard electrical symbols based on joint non-negative matrix binary decomposition |
CN109426861A (en) * | 2017-08-16 | 2019-03-05 | 阿里巴巴集团控股有限公司 | Data encryption, machine learning model training method, device and electronic equipment |
CN108491863A (en) * | 2018-02-27 | 2018-09-04 | 南京信息工程大学 | Color image processing method based on Non-negative Matrix Factorization and convolutional neural networks |
CN108710502B (en) * | 2018-04-08 | 2020-09-29 | 华中科技大学 | Personalized configuration method and system of numerical control system |
CN108710502A (en) * | 2018-04-08 | 2018-10-26 | 华中科技大学 | The individual cultivation method and its system of digital control system |
CN111488900B (en) * | 2019-01-29 | 2023-08-04 | 大连理工大学 | Multi-view related feature learning method based on nonnegative matrix factorization |
CN111488900A (en) * | 2019-01-29 | 2020-08-04 | 大连理工大学 | Multi-view related feature learning model based on non-negative matrix factorization |
CN109960730B (en) * | 2019-04-19 | 2022-12-30 | 广东工业大学 | Short text classification method, device and equipment based on feature expansion |
CN109960730A (en) * | 2019-04-19 | 2019-07-02 | 广东工业大学 | A kind of short text classification method, device and equipment based on feature extension |
CN110442911A (en) * | 2019-07-03 | 2019-11-12 | 中国农业大学 | A kind of higher-dimension complication system Uncertainty Analysis Method based on statistical machine learning |
CN110442911B (en) * | 2019-07-03 | 2023-11-14 | 中国农业大学 | High-dimensional complex system uncertainty analysis method based on statistical machine learning |
CN110674755B (en) * | 2019-09-25 | 2022-02-11 | 中国计量大学 | Gait recognition method based on optimum gait flow pattern space |
CN110674755A (en) * | 2019-09-25 | 2020-01-10 | 中国计量大学 | Gait recognition method based on optimum gait flow pattern space |
CN114140635A (en) * | 2021-08-10 | 2022-03-04 | 北京工业大学 | Non-negative matrix decomposition method for self-expression learning supervision |
CN114140635B (en) * | 2021-08-10 | 2024-05-28 | 北京工业大学 | Non-negative matrix factorization method for self-expression learning supervision |
CN117576493A (en) * | 2024-01-16 | 2024-02-20 | 武汉明炀大数据科技有限公司 | Cloud storage compression method and system for large sample data |
CN117576493B (en) * | 2024-01-16 | 2024-04-02 | 武汉明炀大数据科技有限公司 | Cloud storage compression method and system for large sample data |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN101853239A (en) | Nonnegative matrix factorization-based dimensionality reducing method used for clustering | |
Zhao et al. | Incomplete multi-view clustering via deep semantic mapping | |
He et al. | A variance minimization criterion to feature selection using laplacian regularization | |
He et al. | Laplacian regularized Gaussian mixture model for data clustering | |
Nie et al. | Trace ratio criterion for feature selection. | |
Zhang et al. | Understanding bag-of-words model: a statistical framework | |
Cai et al. | Graph regularized nonnegative matrix factorization for data representation | |
Shen et al. | Non-negative matrix factorization clustering on multiple manifolds | |
Jin et al. | Low-rank matrix factorization with multiple hypergraph regularizer | |
Huang et al. | A kernel entropy manifold learning approach for financial data analysis | |
CN101615248B (en) | Age estimation method, equipment and face recognition system | |
Zhu et al. | Iterative Laplacian score for feature selection | |
Shao et al. | Deep canonical correlation analysis with progressive and hypergraph learning for cross-modal retrieval | |
Huang et al. | On nonlinear dimensionality reduction for face recognition | |
Du et al. | Multiple graph unsupervised feature selection | |
Shu et al. | Parameter-less auto-weighted multiple graph regularized nonnegative matrix factorization for data representation | |
CN102156878A (en) | Sparse embedding with manifold information-based human face identification method | |
Xing et al. | Discriminative semi-supervised non-negative matrix factorization for data clustering | |
CN109871880A (en) | Feature extracting method based on low-rank sparse matrix decomposition, local geometry holding and classification information maximum statistical correlation | |
Ilea et al. | Covariance matrices encoding based on the log-Euclidean and affine invariant Riemannian metrics | |
Chung et al. | Learning data manifolds with a cutting plane method | |
Mandal et al. | Unsupervised non-redundant feature selection: a graph-theoretic approach | |
Jin et al. | Accelerating infinite ensemble of clustering by pivot features | |
Shu et al. | Local and global regularized sparse coding for data representation | |
Bouguila et al. | On discrete data clustering |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
C06 | Publication | ||
PB01 | Publication | ||
C10 | Entry into substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
C02 | Deemed withdrawal of patent application after publication (patent law 2001) | ||
WD01 | Invention patent application deemed withdrawn after publication |
Open date: 20101006 |