CN105701506A - Improved method based on extreme learning machine (ELM) and sparse representation classification - Google Patents

Abstract

The invention discloses a kind of improved methods based on transfinite learning machine and rarefaction representation classification. Steps are as follows by the present invention: 1, hidden node parameter is randomly generated; 2, hidden node output matrix is calculated, 3, according to the size relation of L and N, the output weight of connection hidden node and output neuron is calculated using different formula 4, the output vector of inquiry picture y is calculated; 5, the difference of maximum ο f and second largest value ο s in ELM output vector ο are judged, if difference is greater than the set value, finding out the corresponding index of maximum value in output vector is inquiry picture generic; Otherwise 6 are entered step; 6, using training sample corresponding to k maximum value in output vector ο, constructor dictionary calculates the linear expression coefficient of picture y using coefficient restructing algorithm, calculates residual error and the classification according to corresponding to residual error determines the affiliated class of inquiry picture. Calculation amount of the present invention will greatly reduce, and realize higher discrimination, can also substantially reduce computation complexity.

Description

A kind of based on the learning machine improved method with rarefaction representation classification that transfinites

Technical field

The invention belongs to image classification field, particularly relate to a kind of based on the learning machine improved method with rarefaction representation classification that transfinites。

Background technology

Image is classified, and namely automatically input picture is integrated in a certain specific classification and has attracted increasingly to pay close attention to widely, especially because it is in security system, and medical diagnosis, the application in multiple fields such as man-machine interaction。At several years of the past, some technology developed from machine learning also created very big impact in image classification field。It is true that each method almost passing by propose has its merits and demerits。One inevitable problem is exactly the compromise of computation complexity and classification accuracy。In other words, one can not namely be designed in all application at efficiency and discrimination all the best ways。In order to solve this problem, the system of mixing is arisen at the historic moment, and is namely integrated with the advantage of various distinct methods thus forming a kind of significantly more efficient method。

One vital factor of successful image classification system is exactly grader。One design good grader will not because of some other factor, for instance different feature extracting methods and be affected。In the past few decades, artificial neural network is owing to can arbitrarily arrange input parameter and significantly benefited, and not only pace of learning is than very fast, and achieves better Generalization Capability。Among them, the learning machine (ExtremeLearningMachine, ELM) that transfinites is paid close attention to widely and is studied。Why so welcome the learning machine (ELM) that transfinites is, is because it and has quick pace of learning, the measurability of the real-time ability processed and neutral net。Except the learning machine that transfinites (ELM), another also enjoys research institution to be concerned with the classification (SparseRepresentationbasedClassification, SRC) based on rarefaction representation。Rarefaction representation classification (SRC) is initially to study the neuronic sparse performance of human vision most, finds that it was in recognition of face later, and the aspect such as machine vision and direction estimation also has good performance。Rarefaction representation classification (SRC) is to try to find out from the contact between of a sort samples pictures and the rarefaction representation coefficient being set up picture to be checked by linear regression。Although ELM and SRC is each with prominent advantage, but they there are still some drawbacks and limit they development in actual applications。But SRC experiments show that the pace of learning of ELM quickly, but noise can not be processed preferably, although can process noise preferably pay very big calculation cost。It is additionally noted that one is designed good grader and not only needs to show higher discrimination, and need recognition efficiency faster。Since transfiniting, learning machine (ELM) and rarefaction representation classification (SRC) are respectively arranged with advantage, then a kind of hybrid classifer of design is exactly rational。Experiments show that, ELM-SRC ratio in discrimination learning machine (ELM) that transfinites does very well, computation complexity also reduces than rarefaction representation classification (SRC), but owing to employing complete dictionary, the learning machine that transfinites remains significantly high with rarefaction representation classification (ELM-SRC) computation complexity。

Artificial neural network (ArtificialNeuralNetworks) is also referred to as neutral net (NNs), and it is a kind of imitation animal nerve network behavior feature, carries out the algorithm mathematics model of distributed parallel information processing。This network relies on the complexity of system, by adjusting interconnective relation between internal great deal of nodes, thus reaching the purpose of process information。Artificial neural network is that a kind of application is similar to the structure that cerebral nerve synapse couples and carries out the mathematical model of information processing。In engineering with academia also often directly referred to as neutral net。The each neuron constituting feedforward network accepts previous stage input, and exports next stage, feedback-less, it is possible to represent with a directed acyclic graph。The node of figure is divided into two classes, i.e. input node and computing unit。Each computing unit can have an arbitrarily input, but only one of which exports, and exports the input being alternatively coupled to other other nodes any number of。Feedforward neural network is generally divided into different layers, and the input of the i-th layer output with the i-th-1 layer is associated, and inputs with output node owing to can be connected with the external world, directly protected from environmental, is called visible layer, and other intermediate layer is then called hidden layer。Single hidden layer feedforward neural networks (Single-hiddenLayerFeedforwordneuralNetworks, SLFNs) is as the term suggests being exactly the hidden layer feedforward neural network that only has one layer。For N number of arbitrarily different sample (x_i,t_i), wherein x_i=[x_i1,x_i2,…x_in]^T, t_i=[t_i1,t_i2,…,t_in]^T, there is the standard Single hidden layer feedforward neural networks (SLFNs) of M hidden node, its mathematical model isWherein w_iIt is the weight of input node and hidden node, β_iIt is the weight between hidden node and output node, b_iBeing hidden layer deviation, g (x) is activation primitive。

The Single hidden layer feedforward neural networks (SLFNs) with M hidden node can zero error approach, it is meant that ${\mathrm{\Σ}}_{j=1}^{\stackrel{\‾}{N}}\left|\right|o_j-t_j\left|\right|=0,$ Namely $\underset{i=1}{\overset{\stackrel{\‾}{N}}{\Σ}}{\mathrm{\β}}_{i}g(w_i+x_j+b_i)=t_j,$ N number of equation above can also be write as H β=T, wherein

$H(w_1,...w_L,x_1,...,x_N,b_1,...b_L)=\left[\begin{array}{ccc}g(w_1\·x_1+b_1)& \mathrm{...}& g(w_L\·x_1+b_L)\\ .& & .\\ .& \mathrm{...}& .\\ .& & .\\ g(w_1\·x_n+b_1)& \mathrm{...}& g(w_L\·x_N+b_L)\end{array}\right].$

$\mathrm{\β}=\left[\begin{array}{c}{\mathrm{\β}}_1^T\\ .\\ .\\ .\\ {\mathrm{\β}}_L^T\end{array}\right],T=\left[\begin{array}{c}{t}_1^T\\ .\\ .\\ .\\ t_N^T\end{array}\right].$

H is hidden layer output matrix。Experiments show that, randomly select input weight and hidden layer deviation just can be accurately obtained N number of different observation。It turns out that, Single hidden layer feedforward neural networks (SLFNs) not only pace of learning is fast, and has good Generalization Capability。

Summary of the invention

It is an object of the invention to for Problems existing in existing method, it is provided that a kind of based on the learning machine improved method with rarefaction representation classification that transfinites。This method is a kind of based on learning machine and rarefaction representation classification (the ExtremeLearningMachine-SparseRepresentationbasedClassifi cation of transfiniting, ELM-SRC) self adaptation improved transfinites the algorithm of learning machine and rarefaction representation classification (ExtremeLearningMachineandAdaptiveSparseRepresentationbas edClassification, EA-SRC)。To achieve these goals, the present invention adopts following scheme。

The technical solution adopted for the present invention to solve the technical problems comprises the steps:

Step 1, randomly generate hidden node parameter (w_i,b_i), i=1,2 ..., L, wherein w_iIt is the input weight connecting i-th hidden node and input neuron, b_iBeing the deviation of i-th hidden node, L is hidden node number。

Step 2, calculate hidden node output matrix H (w₁,…w_L,x₁,…,x_N,b₁,…b_i,…,b_L), and $H(w_1,...w_L,x_1,...,x_N,b_1,...b_L)=\left[\begin{array}{ccc}g(w_1\·x_1+b_1)& \mathrm{...}& g(w_L\·x_1+b_L)\\ .& & .\\ .& \mathrm{...}& .\\ .& & .\\ g(w_1\·x_n+b_1)& \mathrm{...}& g(w_L\·x_N+b_L)\end{array}\right]$

Wherein w is the input weight connecting hidden node and input neuron, and x is training sample input, and N is training sample number, b_iBeing the deviation of i-th hidden node, g () represents activation primitive。

Step 3, magnitude relationship according to L and N, be respectively adopted different formula and calculate the output weight connecting hidden node and output neuron

Step 4, calculate inquiry picture y output vector

Step 5, to the maximum o in ELM output vector o_fWith second largest value o_sDifference judge, and obtain index corresponding to maximum in output vector be inquiry picture generic。

If the maximum o in ELM output vector o_fWith second largest value o_sDifference more than threshold value σ set in advance, i.e. o_f-o_s> σ, then the neutral net directly adopting the learning machine (ELM) that transfinites to train, obtain index corresponding to maximum in output vector and be inquiry picture generic。

If the maximum o in output vector_fWith second largest value o_sDifference less than us threshold value set in advance, i.e. o_f-o_s< σ, then it is assumed that the noise that picture comprises is higher adopts rarefaction representation sorting algorithm to classify。

In described step 3, if L and N's is sized to L≤N, namely hidden node number is less than or equal to sample number, then in order to improve computational efficiency, hidden node output matrix carries out singular value decomposition, concrete:

3-1. singular value decomposition H=UDV^T, wherein D=diag{d₁,…d_i,…d_NIt is diagonal matrix, d_iThe i-th singular value of matrix H, then HH^T=VD²V^T。Wherein U is n rank unitary matrice matrixes, and V is n rank unitary matrice。And UU^T=U^TU=I, VV^T=V^TV=I。

3-2. sets the upper limit λ adjusting parameter lambda_maxWith lower limit λ_min, at λ_i∈[λ_min,λ_max] in scope, every layer that calculates orthogonal intersection cast shadow matrix HAT respectively is decomposed square HAT_r, HAT_r=HV (D²+λ_nI)^-1V^TH^T, wherein HAT=HH₊=H (H^TH)_- ¹H^T。

3-3. is at λ_i∈[λ_min,λ_max] calculate difference in scope and adjust the mean square deviation that parameter lambda is correspondingComputing formula is:

${{\mathrm{MSE}}_i}^{PRESS}=\frac{1}{N}\underset{j=1}{\overset{N}{\&Sigma;}}{\left(\frac{t_j-o_j}{1-{\left({\mathrm{HAT}}_r\right)}_{jj}}\right)}^2;$

Wherein t_jIt is desired output, and o_jIt is actual output。

The Minimum Mean Square Error that 3-4. calculatesCorresponding λ, i.e. λ_opt。Now can obtain good Generalization Capability, and also be able to maximize classification boundaries。

3-5. calculates the output weight connecting hidden node and output neuron

$\widehat{\mathrm{\&beta;}}={(H^{T}H+{\mathrm{\&lambda;}}_{opt}I)}^{-1}{H}^{T}T=V{(D^2+{\mathrm{\&lambda;}}_{opt}I)}^{-1}{V}^T{H}^{T}T.$

In described step 3, if L and N's is sized to L > N, namely hidden node number is more than sample number, then in order to improve computational efficiency, hidden node output matrix carries out singular value decomposition, concrete:

3-6. singular value decomposition H=UDV^T, wherein D=diag{d₁,…d_i,…d_NIt is diagonal matrix, d_iThe i-th singular value of matrix H, then HH^T=UD²U^T, and UU^T=U^TU=I, VV^T=V^TV=I。

3-7. sets the upper limit λ adjusting parameter lambda_maxWith lower limit λ_min, at λ_i∈[λ_min,λ_max] in scope, every layer that calculates orthogonal intersection cast shadow matrix HAT respectively is decomposed square HAT_r=HH^TU(D²+λ_iΙ)^-1U^T, wherein HAT=HH⁺=H (H^TH)^-1H^T；

3-8. is at λ_i∈[λ_min,λ_max] in scope, calculate difference and adjust the mean square deviation that parameter lambda is correspondingComputing formula is:

${{\mathrm{MSE}}_i}^{PRESS}=\frac{1}{N}\underset{j=1}{\overset{N}{\&Sigma;}}{\left(\frac{t_j-o_j}{1-{\left({\mathrm{HAT}}_r\right)}_{jj}}\right)}^2;$

Wherein t_jIt is desired output, and o_jIt is actual output。

The Minimum Mean Square Error that 3-9. calculatesCorresponding λ, i.e. λ_opt。Now can obtain good Generalization Capability, and also be able to maximize classification boundaries。

3-10. calculates the output weight connecting hidden node and output neuron $\widehat{\mathrm{\&beta;}}=H^{T}U{(D^2+{\mathrm{\&lambda;}}_{opt}I)}^{-1}{U}^{T}T.$

Rarefaction representation sorting algorithm described in step 5 sets up the sub-dictionary of self adaptation first with k maximum front in output vector o, then the rarefaction representation coefficient of training sample is reconstructed, obtain corresponding residual error, finally find out index corresponding to minima in residual error and be the classification that picture belongs to, concrete:

First find out the classification corresponding to k maximum before in output vector o, then set up the sub-dictionary vector of self adaptation with the vector that k maximum front in output vector o is correspondingWherein m (i) ∈ 1,2 ... m}。

Then reconstruct rarefaction representation coefficient, formula is as follows,Wherein τ is regulation coefficient。

Finally calculate corresponding residual error $r_d\left(y\right)=\left|\right|y-A_d{\mathrm{\&delta;}}_d\left(\hat{x}\right)|{|}_2^2,1\&le;d<m.$

Wherein A_dIt it is the training sample of d class；It it is the rarefaction representation coefficient that d class sample is corresponding。

The present invention has the beneficial effect that:

The transfinite algorithm of learning machine and rarefaction representation classification (EA-SRC) of this self adaptation improved based on transfinite learning machine and rarefaction representation classification (ELM-SRC) not only has higher discrimination, and pace of learning faster, greatly reduce computation complexity。

Accompanying drawing explanation

Fig. 1 is schematic flow sheet of the present invention；

Fig. 2 is Single hidden layer feedforward neural networks schematic diagram of the present invention。

Detailed description of the invention

Verify the improvements of the present invention below in conjunction with concrete experiment, be described below and be only used as demonstration and explain, the present invention is not done any pro forma restriction。

As depicted in figs. 1 and 2, choose any one data base, first randomly generated L hidden node parameter (w by random function_i,b_i), i=1,2 ..., L, wherein w_iIt is the input weight connecting i-th hidden node and input neuron, b_iBeing the deviation of i-th hidden node, L is hidden node number。Calculate hidden layer output matrix

$H(w_1,...w_L,x_1,...,x_N,b_1,...b_L)=\left[\begin{array}{ccc}g(w_1\&CenterDot;x_1+b_1)& \mathrm{...}& g(w_L\&CenterDot;x_1+b_L)\\ .& & .\\ .& \mathrm{...}& .\\ .& & .\\ g(w_1\&CenterDot;x_n+b_1)& \mathrm{...}& g(w_L\&CenterDot;x_N+b_L)\end{array}\right].$

(1) if hidden node number is less than sample number, i.e. L≤N。In order to improve computational efficiency, hidden layer input matrix is carried out singular value decomposition, it is assumed that H=UDV^T, wherein D=diag{d₁,…d_i,…d_NIt is diagonal matrix, d_iThe i-th singular value of matrix H, then HH^T=VD²V^T, and VV^T=V^TV=I。Set the upper limit λ adjusting parameter lambda in advance_maxWith lower limit λ_min, at λ_i∈[λ_min,λ_max] in scope, every layer that calculates HAT matrix respectively is decomposed square HAT_r=HV (D²+λ_optI)^-1V^TH^T, wherein HAT=HH₊=H (H_TH)_-1H_T。At λ_i∈[λ_min,λ_max] in scope, calculate difference and adjust the mean square deviation that parameter lambda is correspondingComputing formulaWherein t_jIt is desired output, and o_jIt is actual output。Return λ, i.e. λ that the Minimum Mean Square Error calculated in above formula is corresponding_opt。Now can obtain good Generalization Capability, and classification boundaries can also be maximized。Then the output weight connecting hidden node and output neuron is calculatedComputing formula is as follows $\widehat{\mathrm{\&beta;}}={(H^{T}H+{\mathrm{\&lambda;}}_{opt}I)}^{-1}{H}^{T}T=V{(D^2+{\mathrm{\&lambda;}}_{opt}I)}^{-1}{V}^T{H}^{T}T.$

(2) if hidden node number is less than sample number, i.e. L >=N。Hidden layer input matrix is carried out singular value decomposition, it is assumed that H=UDV^T, wherein D=diag{d₁,…d_i,…d_NIt is diagonal matrix, d_iThe i-th singular value of matrix H, then HH^T=UD²U^T, and UU^T=U^TU=I。Set the upper limit λ adjusting parameter lambda in advance_maxWith lower limit λ_min, at λ_i∈[λ_min,λ_max] in scope, every layer that calculates HAT matrix respectively is decomposed square HAT_r=HH^TU(D²+λ_optΙ)^-1U^T, wherein HAT=HH⁺=H (H^TH)^-1H^T, then calculate the mean square deviation that different adjustment parameter lambda is corresponding, computing formulaWherein t_jIt is desired output, and o_jIt is actual output。It is then back to λ, i.e. λ that the Minimum Mean Square Error that calculates in above formula is corresponding_opt。Now can obtain good Generalization Capability, and classification boundaries can also be maximized。Calculate the output weight connecting hidden node and output neuron

Then output vector is calculatedIf the difference of maximum and second largest value is more than the threshold value set in advance, i.e. o in the output vector o of inquiry picture_f-o_s> σ, directly adopting the neutral net that the learning machine (ELM) that transfinites has trained, the index that in output vector, maximum is corresponding is inquiry picture generic。If the difference of maximum and second largest value is less than the threshold value set in advance, i.e. o in ELM output vector o_f-o_s< σ, index corresponding to k maximum before first finding out in output vector o, wherein d ∈ 1,2 ... m}, then set up the sub-dictionary of self adaptation with the vector that k maximum before in output vector o is corresponding vectorialThe rarefaction representation coefficient formulas of reconstruct is as follows,Wherein τ is regulation coefficient。For d=1, d, < m finds out corresponding A respectively_dWithCalculate corresponding residual errorFind out the classification y that residual error minima is corresponding, be picture generic。

Claims (4)

1. the improved method classified with rarefaction representation based on the learning machine that transfinites, it is characterised in that comprise the steps:

Step 1, randomly generate hidden node parameter (w_i,b_i), i=1,2 ..., L, wherein w_iIt is the input weight connecting i-th hidden node and input neuron, b_iBeing the deviation of i-th hidden node, L is hidden node number；

Step 2, calculate hidden node output matrix H (w₁,…w_L,x₁,…,x_N,b₁,…b_i,…,b_L), and $H(w_1,\mathrm{...}{w}_L,x_1,\mathrm{...},x_N,b_1,\mathrm{...}{b}_L)=\left[\begin{array}{ccc}g(w_1\&CenterDot;x_1+b_1)& \mathrm{...}& g(w_L\&CenterDot;x_1+b_L)\\ .& & .\\ .& \mathrm{...}& .\\ .& & .\\ g(w_1\&CenterDot;x_n+b_1)& \mathrm{...}& g(w_L\&CenterDot;x_N+b_L)\end{array}\right]$

Wherein w is the input weight connecting hidden node and input neuron, and x is training sample input, and N is training sample number, b_iBeing the deviation of i-th hidden node, g () represents activation primitive；

Step 3, magnitude relationship according to L and N, be respectively adopted different formula and calculate the output weight connecting hidden node and output neuron

Step 4, calculate inquiry picture y output vector

Step 5, to the maximum ο in ELM output vector ο_fWith second largest value ο_sDifference judge, and obtain index corresponding to maximum in output vector be inquiry picture generic；

If the maximum ο in ELM output vector ο_fWith second largest value ο_sDifference more than threshold value σ set in advance, i.e. ο_f-ο_s> σ, then the neutral net directly adopting the learning machine (ELM) that transfinites to train, obtain index corresponding to maximum in output vector and be inquiry picture generic；

If the maximum ο in output vector_fWith second largest value ο_sDifference less than us threshold value set in advance, i.e. ο_f-ο_s< σ, then it is assumed that the noise that picture comprises is higher adopts rarefaction representation sorting algorithm to classify。

2. a kind of based on the learning machine improved method with rarefaction representation classification that transfinites as claimed in claim 1, it is characterized in that in described step 3, if L and N is sized to L≤N, namely hidden node number is less than or equal to sample number, then in order to improve computational efficiency, hidden node output matrix is carried out singular value decomposition, concrete:

3-1. singular value decomposition H=UDV^T, wherein D=diag{d₁,…d_i,…d_NIt is diagonal matrix, d_iThe i-th singular value of matrix H, then HH^T=VD²V^T；Wherein U is n rank unitary matrice, and V is n rank unitary matrice；And UU^T=U^TU=I, VV^T=V^TV=I；

3-2. sets the upper limit λ adjusting parameter lambda_maxWith lower limit λ_min, at λ_i∈[λ_min,λ_max] in scope, every layer that calculates orthogonal intersection cast shadow matrix HAT respectively is decomposed square HAT_r, HAT_r=HV (D²+λ_nI)^-1V^TH^T, wherein HAT=HH₊=H (H^TH)^-1H^T；

3-3. is at λ_i∈[λ_min,λ_max] calculate difference in scope and adjust the statistics mean square error that parameter lambda is corresponding, computing formula is:

${{\mathrm{MSE}}_i}^{PRESS}=\frac{1}{N}\underset{j=1}{\overset{N}{\&Sigma;}}{\left(\frac{t_j-o_j}{1-{\left({\mathrm{HAT}}_r\right)}_{jj}}\right)}^2;$

Wherein t_jIt is desired output, and ο_jIt is actual output；

The minimum statistics mean square error that 3-4. calculatesCorresponding λ, i.e. λ_opt；Now can obtain good Generalization Capability, and also be able to maximize classification boundaries；

3-5. calculates the output weight connecting hidden node and output neuron

$\widehat{\mathrm{\&beta;}}={(H^{T}H+{\mathrm{\&lambda;}}_{opt}I)}^{-1}{H}^{T}T=V{(D^2+{\mathrm{\&lambda;}}_{opt}I)}^{-1}{V}^T{H}^{T}T.$

3. a kind of based on the learning machine improved method with rarefaction representation classification that transfinites as claimed in claim 1, it is characterized in that in described step 3, if L and N is sized to L > N, namely hidden node number is more than sample number, then in order to improve computational efficiency, hidden node output matrix is carried out singular value decomposition, concrete:

3-6. singular value decomposition H=UDV^T, wherein D=diag{d₁,…d_i,…d_NIt is diagonal matrix, d_iThe i-th singular value of matrix H, then HH^T=UD²U^T, and UU^T=U^TU=I, VV^T=V^TV=I；

3-7. sets the upper limit λ adjusting parameter lambda_maxWith lower limit λ_min, at λ_i∈[λ_min,λ_max] in scope, every layer that calculates orthogonal intersection cast shadow matrix HAT respectively is decomposed square HAT_r=HH^TU(D²+λ_iΙ)^-1U^T, wherein HAT=HH⁺=H (H^TH)^-1H^T；

3-8. is at λ_i∈[λ_min,λ_max] in scope, calculate difference and adjust the statistics mean square error that parameter lambda is correspondingComputing formula is:

${{\mathrm{MSE}}_i}^{PRESS}=\frac{1}{N}\underset{j=1}{\overset{N}{\&Sigma;}}{\left(\frac{t_j-o_j}{1-{\left({\mathrm{HAT}}_r\right)}_{jj}}\right)}^2;$

Wherein t_jIt is desired output, and ο_jIt is actual output；

The minimum statistics mean square error that 3-9. calculatesCorresponding λ, i.e. λ_opt；Now can obtain good Generalization Capability, and also be able to maximize classification boundaries；

3-10. calculates the output weight connecting hidden node and output neuron

$\widehat{\mathrm{\&beta;}}=H^{T}U{(D^2+{\mathrm{\&lambda;}}_{opt}I)}^{-1}{U}^{T}T.$

4. a kind of based on the learning machine improved method with rarefaction representation classification that transfinites as claimed in claim 1, it is characterized in that the rarefaction representation sorting algorithm described in step 5 sets up the sub-dictionary of self adaptation first with k maximum front in output vector ο, then the rarefaction representation coefficient of training sample is reconstructed, obtain corresponding residual error, finally find out the index that in residual error, minima is corresponding and be the classification that picture belongs to, concrete:

First find out the classification corresponding to k maximum before in output vector ο, then set up the sub-dictionary vector of self adaptation with the vector that k maximum front in output vector ο is corresponding

Wherein m (i) ∈ 1,2 ... m}；

Then reconstruct rarefaction representation coefficient, formula is as follows,Wherein τ is regulation coefficient；

Finally calculate corresponding residual error1≤d < m；

Wherein A_dIt it is the training sample of d class；It it is the rarefaction representation coefficient that d class sample is corresponding。