CN101515328B - Local projection preserving method for identification of statistical noncorrelation - Google Patents

Abstract

The invention relates to a local projection preserving method using for identification having statistical noncorrelation. The method comprises a weighting adjacent map building module, a matrix building module of a training sample, a projection matrix acquiring module and a data sorting module. The method introduces sort messages of training data during building the weighting adjacent map and can more accurately portray the relationship between data. The method can extract characteristics meeting the statistical noncorrelation, so that the extracted characteristics have the minimum redundancy when preserving the local message of the original data space. The method can improve identification performance if being applied to the identification problems. The method can be applied to various civilian and military systems, such as video monitoring systems, video conference systems, military target tracking and identifying systems and the like and has wide market prospect and application value.

Description

A kind ofly be used to differentiate that the part with statistics irrelevance keeps projecting method

Technical field

The invention belongs to mode identification technology, particularly a kind ofly be used to differentiate that the part with statistics irrelevance keeps projecting method.

Background technology

As the feature extracting method of one of pattern-recognition gordian technique, be the feature space that original high dimensional data is mapped to a low dimension, this has become a research focus of machine learning and pattern-recognition.Feature extracting method commonly used can be divided into two types: based on the analytical approach of global structure information with based on the analytical approach of partial structurtes information.In analytical approach based on global structure information; Principal component analytical method (PCA) is a kind of feature extraction and data representation technology of classics; It has kept the global structure of original data space, and the base vector of any two inequalities of projection matrix is that statistics is incoherent.Uncorrelated is very important characteristic in the pattern-recognition, and the uncorrelated data that can make have minimum redundancy.The local projecting method (LPP) that keeps is analyzed based on the partial structurtes of data, is the feature extracting method of a kind of linearity of latest developments, and algorithm is simple and be easy to realize; Its method is: the adjacent map of at first constructing raw data; Set up Laplce's matrix of figure, then with in two approaching data points of luv space middle distance, the distance after the projection in feature space more closely is a criterion; Try to achieve transformation matrix, obtain the partial structurtes information of data set.

Retrieval through to the prior art document is found; People such as X.He are at " IEEE Trans.on Pattern Analysis and Machine Intelligence " (pattern analysis and machine intelligence IEEE magazine; 2005, vol.27, no.3; Pp.328-340) in the article of delivering on " Face Recognition Using Laplacianfaces " (based on the face identification method of Laplce's face), local maintenance projection properties method for distilling has been proposed at first.Article is through description of test, and this method can access the recognition result that is superior to principal component analysis (PCA).But the base vector of the projection matrix of local maintenance projecting method is a statistical dependence, and the characteristic of therefore extracting contains redundancy, and the information of crossover can cause the actual distribution of characteristic to be distorted, and this shortcoming has had a strong impact on the performance of local maintenance projection algorithm.In addition, the local projecting method that keeps is not used classification information, be a kind of unsupervised feature extracting method, and for pattern recognition problem, classification information is generally all extremely important.Therefore seeking a kind of part with statistics irrelevance towards discriminating keeps projecting method to have great importance.Further do not finding as yet to keep projecting method in the retrieval towards the part of differentiating with statistics irrelevance.

Summary of the invention

Technical matters to be solved by this invention provides a kind of part with statistics irrelevance that is used to differentiate and keeps projecting method, makes it be used for pattern-recognition, can improve the precision of identification.

The technical solution adopted for the present invention to solve the technical problems is: provide a kind of part with statistics irrelevance that is used to differentiate to keep projecting method; Comprise: the matrix construction module of weighting adjacent map constructing module, training sample, projection matrix obtain module and data qualification module, wherein:

Weighting adjacent map constructing module as a summit, is set up a weighting adjacent map with each training sample, obtains the similarity weights between any two summits according to classification information, and the similarity weights are transferred to the matrix construction module of training sample;

The matrix construction module of training sample receives the similitude weights and based on nearest neighbouring rule; Make each summit all only be connected with several minimum summits of its similitude weights; Set up the similar matrix of training sample; Set up the degree matrix of training sample, Laplce's matrix of figure by the similar matrix of training sample again, and above-mentioned matrix is transferred to projection matrix acquisition module;

Projection matrix obtains Laplce's matrix of module reception degree matrix and figure, keeps projecting method based on the part again, adds the constraints of statistics irrelevance; Pass through iterative process; Separate eigenvalue problem, each iteration is chosen minimum characteristic value characteristic of correspondence vector, at last with these characteristic vectors as base vector; Formation keeps projection matrix towards the part of differentiating with statistics irrelevance, and projection matrix is transferred to the data sort module;

The data qualification module receives the training data and the test data of projection matrix, input; And training data and test data projected in the projection matrix; Acquisition training matrix of coefficients and test matrix of coefficients adopt minimum distance classifier, identify the affiliated classification of test data.

Said weighting adjacent map constructing module; It obtains the similarity weights between any two summits according to classification information; Be meant that establishing each training sample represents a summit xi, set up the similarity weights between any two summits, these similarity weights can be expressed as:

The structure of similarity weights has utilized the classification information of training data, has reflected the similarity degree between two data points that are connected preferably, and this two data points of the big more explanation of similarity weights is similar more, might belong to same classification more.

The matrix construction module of said training sample, it sets up the similar matrix of training sample, Laplce's matrix of degree matrix, figure, and is specific as follows:

The matrix construction module receives the similarity weights, in the similarity weights, according to nearest neighbouring rule, finds out k the neighbour summit on each summit, promptly finds out and summit x _iBetween preceding k minimum summit of similarity weights, make summit x _iOnly be connected with this k summit, set up the similar matrix W of training dataset, the element representation of W is:

Summit x _iDegree be: $d_i=\underset{j=1}{\overset{n}{\mathrm{\Σ}}}w(i,j),$ N is the number of training data point, and the degree matrix of setting up adjacent map is: D=diag (d ₁, d ₂..., d _n), Laplce's matrix of foundation figure is: L=D-W, promptly

Said projection matrix obtains module, and it separates eigenvalue problem through iterative process, obtains to keep projection matrix towards the part of differentiating with statistics irrelevance, is meant and establishes training sample set X={x ₁, x ₂..., x _n, S _T=E [(X-EX) (X-EX) ^T] be the covariance matrix of training sample, note S _L=XLX ^T, S _D=XDX ^TThe local projection matrix of projecting method that keeps can obtain through the proper vector of finding the solution following eigenvalue problem:

S _Lφ＝λS _Dφ (4)

Wherein, φ is an eigenvalue characteristic of correspondence vector.

Introduce the incoherent constraint condition of statistics then:

${\mathrm{\φ}}_k^T{S}_T{\mathrm{\φ}}_i=0$ (i＝1，2，…，k-1)

Adopt method of Lagrange multipliers, unite above condition and find the solution.

If { φ ₁, φ ₂..., φ _K-1Be preceding k-1 the projection base vector of having tried to achieve, note Φ _K-1=[φ ₁, φ ₂..., φ _K-1], then satisfy the projection vector φ that adds up irrelevance _k, can obtain according to the following steps iteration:

(a) matrix S _D ^-1S _LMinimal eigenvalue characteristic of correspondence vector as projection vector φ ₁

(b) find the solution the eigenwert of following secular equation, and get minimal eigenvalue characteristic of correspondence vector as incoherent projection vector φ _k

R ^(k)S _Lφ＝λS _Dφ (5)

Wherein,

$R^{\left(k\right)}=I-S_T{\mathrm{\Φ}}_{k-1}{\left({\mathrm{\Φ}}_{k-1}^T{S}_T{S}_D^{-1}{S}_T{\mathrm{\Φ}}_{k-1}\right)}^{-1}{\mathrm{\Φ}}_{k-1}^T{S}_T{S}_D^{-1}---\left(6\right)$

(c) repeat (b) step, until obtaining d vector { φ that satisfies the statistics irrelevance ₁, φ ₂..., φ _d.At last, obtain keeping projection square Φ=[φ towards the part of differentiating with statistics irrelevance ₁, φ ₂..., φ _d].

Beneficial effect

The present invention is when structure weighting adjacent map; Introduced the classification information of training data, can portray the relation between data more accurately, and the present invention can extract the characteristic that satisfies the statistics irrelevance; Thereby the characteristic of extracting can be in the local message that keeps original data space; Have minimum redundancy, be applied in the identification problem, can improve recognition performance.

The present invention can be applicable to have vast market prospect and using value in all kinds of civilian and military systems such as video monitoring system, video conferencing system, military target tracking and identifying system.

Description of drawings

Fig. 1 is the workflow of the inventive method.

The result that Fig. 2 discerns in the spacecraft image library for the inventive method with directly discern with moment characteristics, use linear discriminant analysis method (LDA) and the local comparison diagram as a result that keeps projecting method (LPP) to discern respectively.Wherein horizontal ordinate is the arest neighbors number, and ordinate is a discrimination.

Embodiment

Below in conjunction with specific embodiment, further set forth the present invention.Should be understood that these embodiment only to be used to the present invention is described and be not used in the restriction scope of the present invention.Should be understood that in addition those skilled in the art can do various changes or modification to the present invention after the content of having read the present invention's instruction, these equivalent form of values fall within the application's appended claims institute restricted portion equally.

Present embodiment is applied to the spacecraft model in the STK model bank is discerned; STK is the satellite simulation kit that U.S. AGI company produces; This kit not only can calculate track, attitude and the communication link of satellite, can also calculate the position and actual illumination of each celestial body according to real ephemeris information.In addition, comprise a lot of celestial body surface texture informations and spacecraft model in the kit, thereby can simulate the space what comes into a driver's.

In emulation, from the STK model bank, chosen 4 types of representational spacecraft models, utilize STK8.0 to calculate the spacecraft simulation image, every type of spacecraft model has 100 width of cloth images.The Hu invariant moments of calculating every width of cloth image experimentizes as observation data.

As shown in Figure 1, observation data is divided into training dataset and test data set, operate as follows then:

Step 1, structure weighting adjacent map:

Each summit x in the adjacent map _iCorresponding to the data points that the spacecraft training data is concentrated, set up the similarity weights between any two summits, these similarity weights can be expressed as:

The structure of similarity weights has utilized the classification information of spacecraft training data, has reflected the similarity degree between two data points that are connected preferably, and this two data points of the big more explanation of similarity weights is similar more, might belong to a kind of spacecraft more.

Step 2 according to the similarity weights between any two summits of the spacecraft training dataset that obtains in the step 1, is set up Laplce's matrix of similar matrix, degree matrix and figure, and is specific as follows:

In the similarity weights,, find out and summit x according to nearest neighbouring rule _iBetween the minimum preceding k of similarity weights (get k=1 in the experiment ... 40) individual summit makes summit x _iOnly be connected with this k summit, set up the similar matrix W of spacecraft training dataset, the element representation of W is:

Summit x _iDegree be: $d_i=\underset{j=1}{\overset{n}{\mathrm{\Σ}}}w(i,j),$ N is the number of spacecraft training data point, and the degree matrix of setting up adjacent map is: D=diag (d ₁, d ₂..., d _n), Laplce's matrix of foundation figure is: L=D-W, promptly

Step 3; According to degree matrix and the Laplce's matrix that step 2 obtains, will add up incoherent constraint condition and be incorporated into local the maintenance in the sciagraphy, through iterative process; Separate eigenvalue problem, obtain to keep projection matrix towards the part of differentiating with statistics irrelevance:

If training sample set X={x ₁, x ₂..., x _n, S _L=XLX ^T, S _D=XDX ^T, I=diag (1,1 ..., 1), covariance matrix S _T=E [(X-EX) (X-EX) ^T], Φ=[φ ₁, φ ₂..., φ _k] be projection matrix, and definition

Φ _k-1＝[φ ₁，φ ₂，...，φ _k-1] (10)

The local objective function of projecting method that keeps is:

$\min \underset{i,j}{\mathrm{\Σ}}{\left|\right|y_i-y_j\left|\right|}^{2}w(i,j)---\left(11\right)$

Wherein, y _iBe summit x _iProjection result corresponding to lower dimensional space.Through some simple geometric knowledge, above-mentioned objective function can turn to following minimization problem:

$\underset{{\mathrm{\Φ}}^T{\mathrm{XDX}}^T\mathrm{\Φ}=I}{\underset{\mathrm{\Φ}}{\arg \min }}\mathrm{trace}\left({\mathrm{\Φ}}^T{\mathrm{XLX}}^T\mathrm{\Φ}\right)---\left(12\right)$

Satisfy the projection matrix that minimizes objective function and can be converted into general eigenvalue problem:

XLX ^TΦ＝λXDX ^TΦ (13)

In order to obtain incoherent projection vector φ _k, on the basis of formula (13), increase the uncorrelated constraint of statistics:

${\mathrm{\φ}}_k^T{S}_T{\mathrm{\φ}}_i=0$ (i＝1，2，…，k-1) (14)

In order to eliminate φ _kArbitrariness, the local projecting method that keeps has increased a constraint again:

${\mathrm{\φ}}_k^T{S}_D{\mathrm{\φ}}_k=1---\left(15\right)$

Adopt method of Lagrange multipliers, association type (14), formula (15) are found the solution, and problem equivalent is in asking φ _kMake following function get maximal value:

$L\left({\mathrm{\φ}}_k\right)={\mathrm{\φ}}_k^T{S}_L{\mathrm{\φ}}_k-\mathrm{\λ}({\mathrm{\φ}}_k^T{S}_D{\mathrm{\φ}}_k-1)-\underset{i=1}{\overset{k-1}{\mathrm{\Σ}}}{\mathrm{\γ}}_i{\mathrm{\φ}}_k^T{S}_T{\mathrm{\φ}}_i---\left(16\right)$

About φ _kDifferentiate, and make that derivative is zero, can obtain:

$2S_L{\mathrm{\φ}}_k-2\mathrm{\λ}{S}_D{\mathrm{\φ}}_k-\underset{i=1}{\overset{k-1}{\mathrm{\Σ}}}{\mathrm{\γ}}_i{S}_T{\mathrm{\φ}}_i=0---\left(17\right)$

φ on the both sides premultiplication of formula (17) _k ^T, utilize the constraint of formula (14), can know that back two is zero, so can solve:

$\mathrm{\λ}=\frac{{\mathrm{\φ}}_k^T{S}_L{\mathrm{\φ}}_k}{{\mathrm{\φ}}_k^T{S}_D{\mathrm{\φ}}_k}---\left(18\right)$

Problem will make λ get maximal value exactly.

Distinguish φ on the premultiplication on the both sides of formula (17) again _j ^TS _TS _D ^-1, the arrangement of deriving can get:

$2{\mathrm{\φ}}_j^T{S}_T{S}_D^{-1}{S}_L{\mathrm{\φ}}_k-\underset{i=1}{\overset{k-1}{\mathrm{\Σ}}}{\mathrm{\γ}}_i{\mathrm{\φ}}_j^T{S}_T{S}_D^{-1}{S}_T{\mathrm{\φ}}_i=0---\left(19\right)$

Wherein, j=1,2 ..., k-1.

If γ=[γ ₁, γ ₂..., γ _K-1], then formula (19) can be expressed as:

$2{\mathrm{\Φ}}_{k-1}^T{S}_T{S}_D^{-1}{S}_L{\mathrm{\φ}}_k-{\mathrm{\Φ}}_{k-1}^T{S}_T{S}_D^{-1}{S}_T{\mathrm{\Φ}}_{k-1}\mathrm{\γ}=0---\left(20\right)$

Obtain γ by (20), and according to $\underset{i=1}{\overset{k-1}{\mathrm{\Σ}}}{\mathrm{\γ}}_i{S}_T{\mathrm{\φ}}_i-S_T{\mathrm{\Φ}}_{k-1}\mathrm{\γ},$ Formula (17) further is expressed as:

2S _Lφ _k-2λS _Dφ _k-S _TΦ _k-1γ＝0 (21)

With the substitution formula of separating (21) of γ, and through a series of derivations and arrangement, final, incoherent projection vector φ _kCan obtain according to the following steps iteration:

(a) matrix S _D ^-1S _LMinimal eigenvalue characteristic of correspondence vector as projection vector φ ₁

(b) find the solution the eigenwert of following secular equation, and get minimal eigenvalue characteristic of correspondence vector as incoherent projection vector φ _k

R ^(k)S _Lφ＝λS _Dφ (22)

Wherein

$R^{\left(k\right)}=I-S_T{\mathrm{\Φ}}_{k-1}{\left({\mathrm{\Φ}}_{k-1}^T{S}_T{S}_D^{-1}{S}_T{\mathrm{\Φ}}_{k-1}\right)}^{-1}{\mathrm{\Φ}}_{k-1}^T{S}_T{S}_D^{-1}---\left(23\right)$

At last, obtain keeping projection matrix Φ=[φ towards the part of differentiating with statistics irrelevance ₁, φ ₂..., φ _d].

Step 4 is carried out projective transformation, extracts characteristic, discerns

Projection process is following: x → y=Φ ^TX, then y is the d dimension expression of sample x.The projection matrix Φ that utilizes step 3 to generate respectively the training data and the test data of spacecraft projects in the feature space; Obtain training matrix of coefficients and test matrix of coefficients; Adopt minimum distance classifier; With the training matrix of coefficients is standard, can identify the affiliated classification of spacecraft test data.

Fig. 2 adopts result that present embodiment method (SULPP) discerns and directly discerns with moment characteristics in the spacecraft image library; And the result who uses linear discriminant analysis method (LDA), local maintenance projecting method (LPP) to discern respectively; Wherein horizontal ordinate is the arest neighbors number, and ordinate is a discrimination.Can find out that from figure the method that the present invention proposes obviously is superior to directly using the method for moment characteristics (Invmoments) and with the method for LDA, LPP extraction characteristic, can obtains to have more distinctive characteristic, can improve recognition performance.

Claims (1)

1. one kind is used to differentiate that the part with statistics irrelevance keeps projecting method, it is characterized in that, comprises following modules:

(1) weighting adjacent map constructing module: each training sample as a summit, is set up a weighting adjacent map, obtain the similarity weights between any two summits, and the similarity weights are transferred to the matrix construction module of training sample according to classification information;

Described weighting adjacent map constructing module, it obtains the similarity weights between any two summits according to classification information, is meant that establishing each training sample represents a summit x _i, set up the similarity weights between any two summits, these similarity weights can be expressed as:

(2) the matrix construction module of training sample: receive the similitude weights and based on nearest neighbouring rule; Make each summit all only with the minimum summit of its similitude weights be connected; Set up the similar matrix of training sample; Set up the degree matrix of training sample, Laplce's matrix of figure by the similar matrix of training sample again, and above-mentioned matrix is transferred to projection matrix acquisition module;

(3) projection matrix obtains module: Laplce's matrix of reception degree matrix and figure, keep projecting method based on the part again, and add the constraints of statistics irrelevance; Pass through iterative process; Separate eigenvalue problem, each iteration is chosen minimum characteristic value characteristic of correspondence vector, at last with these characteristic vectors as base vector; Formation keeps projection matrix towards the part of differentiating with statistics irrelevance, and projection matrix is transferred to the data sort module;

(4) data qualification module: the training data and the test data that receive projection matrix, input; And training data and test data projected in the projection matrix; Acquisition training matrix of coefficients and test matrix of coefficients adopt minimum distance classifier, identify the affiliated classification of test data.

Images