CN104050482B - A kind of manifold learning generalization algorithm based on local linear smoothing - Google Patents
A kind of manifold learning generalization algorithm based on local linear smoothing Download PDFInfo
- Publication number
- CN104050482B CN104050482B CN201410288959.XA CN201410288959A CN104050482B CN 104050482 B CN104050482 B CN 104050482B CN 201410288959 A CN201410288959 A CN 201410288959A CN 104050482 B CN104050482 B CN 104050482B
- Authority
- CN
- China
- Prior art keywords
- dimension
- manifold learning
- new
- matrix
- data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000009499 grossing Methods 0.000 title abstract 3
- 230000009467 reduction Effects 0.000 claims abstract description 35
- 239000011159 matrix material Substances 0.000 claims abstract description 31
- 238000012417 linear regression Methods 0.000 claims abstract description 15
- 238000000034 method Methods 0.000 claims abstract description 13
- 238000013507 mapping Methods 0.000 claims abstract description 7
- 238000012935 Averaging Methods 0.000 claims description 3
- 238000000354 decomposition reaction Methods 0.000 claims description 3
- 238000012163 sequencing technique Methods 0.000 claims description 2
- 238000001228 spectrum Methods 0.000 abstract description 6
- 230000008569 process Effects 0.000 abstract description 3
- 238000012360 testing method Methods 0.000 description 16
- 238000012549 training Methods 0.000 description 6
- 238000005070 sampling Methods 0.000 description 5
- 238000012545 processing Methods 0.000 description 4
- 238000002474 experimental method Methods 0.000 description 3
- 238000000513 principal component analysis Methods 0.000 description 2
- 235000008331 Pinus X rigitaeda Nutrition 0.000 description 1
- 235000011613 Pinus brutia Nutrition 0.000 description 1
- 241000018646 Pinus brutia Species 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000007405 data analysis Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 230000002349 favourable effect Effects 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000007781 pre-processing Methods 0.000 description 1
- 230000003595 spectral effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Landscapes
- Investigating Or Analysing Materials By Optical Means (AREA)
- Image Analysis (AREA)
- Image Processing (AREA)
Abstract
A kind of manifold learning generalization algorithm based on local linear smoothing, belongs to hyperspectral image data dimensionality reduction technology field.It is an object of the invention to propose it is a kind of suitable for any manifold learning arithmetic, can keep former manifold learning dimensionality reduction result, the extensive method of the manifold learning based on local linear smoothing, its step to be:First, neighborhood is found;2nd, projection matrix is calculated;3rd, linear regression coeffficient matrix is asked for;4th, new samples dimensionality reduction result is calculated.The present invention can realize the extensive of new samples on the basis of original manifold learning dimensionality reduction result is kept, construct a Linear Mapping from higher-dimension to low-dimensional, may be such that any manifold learning arithmetic for not possessing generalization ability such as LE, LLE, LTSA etc. have generalization ability, so that these time-consuming manifold learning arithmetics are applied to the dimension-reduction treatment process of high spectrum image.
Description
Technical Field
The invention belongs to the technical field of hyperspectral image data dimension reduction, and particularly relates to a generalization algorithm for manifold learning.
Background
The hyperspectral image can record rich spectral information of the ground objects, and the method is favorable for accurately and finely classifying and identifying the ground objects. However, the increase of the number of the hyperspectral data wave bands inevitably causes difficulty in information redundancy and data processing, dimension disaster is caused, and the phenomenon brings obstruction to hyperspectral data processing, so that the problem that the redundancy of the hyperspectral data is eliminated becomes necessary to be solved. The redundancy of the high-spectrum data is mainly caused by the correlation between the bands of the high-spectrum data, the dimension reduction is an important preprocessing method, the characteristics of the high-dimensional data are expressed by using the low-dimensional data, the image information can be effectively reserved, and the information redundancy is reduced. Common hyperspectral image dimension reduction algorithms are divided into linear dimension reduction and nonlinear dimension reduction, although the linear dimension reduction algorithms such as PCA (principal Component analysis) and LDA (linear Discriminant analysis) are simple to implement, the manifold learning algorithm can better mine the nonlinear structure of hyperspectral data and improve the data analysis capability because the hyperspectral images have nonlinear characteristics. Classical manifold learning algorithms include le (laplacian egenmap), lle (local Linear embedding) and ltsa (local satellite Space alignment) algorithms, and can be used for a feature extraction algorithm of a hyperspectral image. However, most classical nonlinear manifold learning algorithms have no generalization capability, cannot directly learn new hyperspectral data, and can obtain a new dimension reduction result only by learning together with all old data.
The generalization problem of the manifold learning algorithm can be described as: known hyperspectral dataset X ═ { X ═ X1,x2,...,xN}∈RD ×NN is the number of sample sets, D is the dimension of the sample set, xjFor the j-th hyperspectral data sample shown, it is also known that a certain manifold learning algorithm reduces the dimension of X to obtain a reduced-dimension data set Y ═ { Y ═ Y1,y2,...,yN}∈Rd×ND is the dimensionality reduction, yjIs xjFor a new hyperspectral data xnew∈RD×1Find xnewCorresponding dimensionality reduction result ynew∈Rd×1。
Many attempts have been made by many researchers to generalize manifold learning. The common ideas are to linearize the nonlinear learning algorithm, and extend the kernel of the nonlinear learning algorithm. However, linearization changes the dimensionality reduction of the original manifold learning algorithm, and kernel development requires the construction of kernel functions for each specific manifold learning algorithm.
Disclosure of Invention
The invention aims to provide a manifold learning generalization method which is applicable to any manifold learning algorithm, can keep the original manifold learning dimensionality reduction result and is based on local linear regression, so that the manifold learning algorithm without generalization capability, such as classical manifold learning algorithms of LTSA, LLE, LE and the like, obtains generalization capability, and the manifold learning algorithm is applicable to the dimensionality reduction processing process of a hyperspectral image.
The purpose of the invention is realized by the following technical scheme:
data distributed over a high-dimensional manifold can be approximated as being distributed over a low-dimensional hyperplane in a small local area, and in this neighborhood, a linear mapping between high-dimensional data and low-dimensional embedding can be assumed. Therefore, for a new sample point, its neighborhood is first found in the original data space, then a linear mapping from high dimension to low dimension is constructed in the neighborhood, and finally generalization of the new sample is realized through the linear mapping.
As shown in fig. 1, the manifold learning generalization algorithm based on local linear regression provided by the present invention specifically comprises the following steps:
step one, searching the field:
for a new data sample xnewFinding X in the hyperspectral dataset XnewThe k nearest sample points form a neighborhood data setD is a sample set dimension, and is obtainedCorresponding dimension reduction neighborhood data set in dimension reduction data set Yd is dimension reduction, k is required to be more than or equal to d, and R represents a real number domain.
Calculating xnewX of the jth hyperspectral data sample in XjDistance b (j):
B(j)=||xj-xnew||2j=1,2,...,N。
sorting the elements in the distance vector B from small to large, and taking the data samples corresponding to the first k smallest elements to form the data samplesAnd in YoldIn which a correspondence is found
Step two, calculating a projection matrix:
1) constructing a matrix C:
wherein HkIn order to centre the operator in the k-dimension,ek=[1,1,...1]T∈Rk×1is a column vector of length k with all "1" elements, IkIs an identity matrix of k × k.
2) Performing characteristic decomposition on the matrix C:
Cv=λv,
where v is the eigenvector of the matrix C, and λ is the eigenvalue corresponding to the eigenvector v.
3) Calculating a local projection matrix V:
wherein v isiIth large eigenvalue λ of CiCorresponding eigenvectors, V (i) is the ith column of the projection matrix V, and d is the dimension reduction dimension.
Step three, solving a linear regression coefficient matrix:
1) computing neighborhood data setsTangential space coordinate Z:
wherein, VTIs the transpose of the local projection matrix V.
2) Calculating a linear regression coefficient matrix L:
wherein Z is a group represented byThe mapping relation of (1) is as follows:Eiis a linear regression error (·)+Representing the Moore-Penrose generalized inverse operator.
Step four, calculating a new sample dimension reduction result:
wherein,Mkan averaging operator is calculated for the k-dimension,ynewis xnewAnd (4) reducing the dimension.
Compared with the prior art, the invention has the following advantages:
the method can realize generalization of a new sample on the basis of keeping the original manifold learning dimensionality reduction result, constructs a linear mapping from a high dimension to a low dimension, and can enable any manifold learning algorithm without generalization capability, such as LE, LLE, LTSA and the like, to have generalization capability, so that the time-consuming manifold learning algorithms are suitable for the dimensionality reduction processing process of the hyperspectral image.
Drawings
FIG. 1 is a flow chart of the steps of the present invention;
FIG. 2 is a feature scatter plot of hyperspectral image data in the present invention;
FIG. 3 is a generalized error curve of hyperspectral data in the present invention.
Detailed Description
The technical solution of the present invention is further described below with reference to the accompanying drawings, but the present invention is not limited thereto, and modifications or equivalent substitutions may be made to the technical solution of the present invention without departing from the spirit and scope of the technical solution of the present invention.
The hyperspectral image data was obtained by the kennedy space center in the united states by taking a picture of a farmland in indiana, which had a total of 16 different crops, and the spatial resolution of the image was 20 × 20m2Each pixel point has 224 wave bands, the coverage spectrum range is a spectrum range of 0.2-2.4 mu m, and the spectrum resolution is 10 nm.
Randomly extracting 1000 samples from all data of the IND PINE as training samples, and taking the other 1000 samples as testing samples as an original data set X to be subjected to dimensionality reductionallThe training sample set is XtrainThe test sample set is XtestIn this embodiment, the LLE algorithm is used to perform manifold learning on the training samples and the test samples to obtain the reduced-dimension dataset Y of all the samplesallTest sample and training sample correspondenceThe dimensionality reduction result of is YtrainAnd Ytest. The algorithm proposed by the invention uses YtrainAnd XallFor input, a generalization result Y of the test sample is obtainednewThen compare YtestAnd YnewError of (2):
performing manifold learning on all samples through LLE manifold learning algorithm to obtain dimension reduction result Y of all samplesallThe dimension reduction result of the training sample and the test sample is YtrainAnd Ytest. For each data sample in a given sample set, performing the following steps:
step one, finding a neighborhood:
x for the test sample settestOf one data sample xnewIn training sample set XtrainFind x innewThe 10 nearest sample points form a neighborhood data setAnd obtainCorresponding dimension reduction neighborhood data set in dimension reduction data set Y
Calculating xnewAnd XtrainX of the jth hyperspectral data samplejDistance b (j):
B(j)=||xj-xnew||2j=1,2,...,1000。
sequencing the elements in the distance vector B to obtainAnd in YtrainIn which a correspondence is found。
Step two, calculating a projection matrix:
1) constructing a matrix C:
wherein H10For the purpose of the 10-dimensional centering operator,e10=[1,1,...1]T∈R10×1is a column vector of length 10 with all "1" elements, I10Is an identity matrix of 10 × 10.
2) Performing characteristic decomposition on the matrix C:
Cv=λv。
3) calculating a local projection matrix V:
wherein v isiIth large eigenvalue λ of CiThe corresponding eigenvector, V (i), is the V (i) th column of the projection matrix.
Step three, solving a linear regression coefficient matrix:
1) computing neighborhood data setsTangential space coordinate Z:
Z=VTXH10。
2) calculating a linear regression coefficient matrix L:
step four, calculating a new sample dimension reduction result:
wherein,M10an averaging operator is calculated for the 10 dimensions,ynewis xnewAnd (4) reducing the dimension.
The results of the experiment are shown in FIG. 2. The left column in fig. 2 is the dimension reduction results of the hyperspectral image test data corresponding to the LE, LLE and LTSA algorithms, and the right column is the generalization results of the LE-LLR, LLE-LLR and LTSA-LLR algorithms on the hyperspectral test samples. It can be seen that the generalization result of the algorithm provided by the invention is approximately the same as the true value of the algorithm, and the method has a better generalization effect on the hyperspectral data. The generalization errors corresponding to the three algorithms LE, LLE and LTSA in the experiment were 1.9%, 2.0% and 1.3%.
Obtaining a generalization result Y of the test sample according to the previous stepsnewOf the through typeCalculating YtestAnd YnewAnd obtaining a generalization error curve. Because the sampling density of the data also influences the final generalization error, the influence of the sampling density on the generalization error is studied by changing the number of sampling points in the experiment, and the experimental result is shown in fig. 3. As can be seen from the figure, the generalized refinement of the algorithm proposed by the present inventionThe degree is related to the sampling density of the data, the higher the sampling density is, the better the linearity of the local neighborhood is, and the smaller the generalization error of the algorithm is.
By analyzing the comprehensive embodiment, the manifold learning generalization method based on local linear regression improves the existing manifold learning algorithm, can well solve the problem that the existing manifold learning has no generalization capability, and has strong engineering practicability.
Claims (2)
1. A manifold learning generalization method based on local linear regression is characterized in that the manifold learning generalization method comprises the following steps:
step one, finding a neighborhood:
for a new data sample xnewFinding X in the hyperspectral dataset XnewThe k nearest sample points form a neighborhood data setD is a sample set dimension, and is obtainedCorresponding dimension reduction neighborhood data set in dimension reduction data set Yd is dimension reduction dimension, k is required to be more than or equal to d, and R represents a real number domain;
calculating xnewX of the jth hyperspectral data sample in XjDistance b (j):
B(j)=||xj-xnew||2j=1,2,...,N;
sequencing the elements in the distance vector B to obtainAnd find the corresponding in Y
Step two, calculating a projection matrix:
1) constructing a matrix C:
wherein HkIn order to centre the operator in the k-dimension,ek=[1,1,...1]T∈Rk×1is a column vector of length k with all "1" elements, IkAn identity matrix of k × k;
2) performing characteristic decomposition on the matrix C:
Cv=λv,
wherein v is the eigenvector of the matrix C, and λ is the eigenvalue corresponding to the eigenvector v;
3) calculating a local projection matrix V:
wherein v isiIth large eigenvalue λ of CiCorresponding eigenvectors, V (i) is the ith column of the projection matrix V, and d is the dimension reduction dimension;
step three, solving a linear regression coefficient matrix:
1) computing neighborhood data setsTangential space coordinate Z:
wherein, VTIs a transposed matrix of the local projection matrix V;
2) calculating a linear regression coefficient matrix L:
wherein Z is a group represented byThe mapping relation of (1) is as follows:Eiis a linear regression error (·)+The generalized inverse operator of Moore-Penrose is solved;
step four, calculating a new sample dimension reduction result:
wherein,Mkan averaging operator is calculated for the k-dimension,ynewis xnewAnd (4) reducing the dimension.
2. The generalization method of manifold learning based on local linear regression of claim 1, wherein in the first step, the distance vector B is selected fromSorting the data samples in a small-to-large order, and taking the data sample composition corresponding to the first k minimum elementsAnd find the corresponding in Y
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201410288959.XA CN104050482B (en) | 2014-06-24 | 2014-06-24 | A kind of manifold learning generalization algorithm based on local linear smoothing |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201410288959.XA CN104050482B (en) | 2014-06-24 | 2014-06-24 | A kind of manifold learning generalization algorithm based on local linear smoothing |
Publications (2)
Publication Number | Publication Date |
---|---|
CN104050482A CN104050482A (en) | 2014-09-17 |
CN104050482B true CN104050482B (en) | 2017-06-13 |
Family
ID=51503294
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN201410288959.XA Active CN104050482B (en) | 2014-06-24 | 2014-06-24 | A kind of manifold learning generalization algorithm based on local linear smoothing |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN104050482B (en) |
Families Citing this family (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN105069482B (en) * | 2015-08-21 | 2018-06-15 | 中国地质大学(武汉) | Classification of hyperspectral remote sensing image method based on spatial regularization manifold learning arithmetic |
CN105184281A (en) * | 2015-10-12 | 2015-12-23 | 上海电机学院 | Face feature library building method based on high-dimensional manifold learning |
CN106199544B (en) * | 2016-06-24 | 2018-07-17 | 电子科技大学 | Differentiate the Recognition of Radar Target Using Range Profiles method of local tangent space alignment based on core |
CN107480688B (en) * | 2017-06-20 | 2020-06-19 | 广东工业大学 | Fine-grained image identification method based on zero sample learning |
Citations (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN102867191A (en) * | 2012-09-04 | 2013-01-09 | 广东群兴玩具股份有限公司 | Dimension reducing method based on manifold sub-space study |
CN103294647A (en) * | 2013-05-10 | 2013-09-11 | 上海大学 | Head-related transfer function dimensionality reduction method based on orthogonal tensor neighbourhood preserving embedding |
Family Cites Families (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US8538195B2 (en) * | 2007-09-17 | 2013-09-17 | Raytheon Company | Hyperspectral image dimension reduction system and method |
-
2014
- 2014-06-24 CN CN201410288959.XA patent/CN104050482B/en active Active
Patent Citations (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN102867191A (en) * | 2012-09-04 | 2013-01-09 | 广东群兴玩具股份有限公司 | Dimension reducing method based on manifold sub-space study |
CN103294647A (en) * | 2013-05-10 | 2013-09-11 | 上海大学 | Head-related transfer function dimensionality reduction method based on orthogonal tensor neighbourhood preserving embedding |
Also Published As
Publication number | Publication date |
---|---|
CN104050482A (en) | 2014-09-17 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
Liu et al. | Multiview dimension reduction via Hessian multiset canonical correlations | |
CN102208034B (en) | Semi-supervised dimension reduction-based hyper-spectral image classification method | |
CN104050482B (en) | A kind of manifold learning generalization algorithm based on local linear smoothing | |
CN109615008B (en) | Hyperspectral image classification method and system based on stack width learning | |
CN109961096B (en) | Multimode hyperspectral image migration classification method | |
CN104036289A (en) | Hyperspectral image classification method based on spatial and spectral features and sparse representation | |
CN104866871B (en) | Hyperspectral image classification method based on projection structure sparse coding | |
CN105261000A (en) | Hyperspectral image fusion method based on end member extraction and spectrum unmixing | |
CN102541954A (en) | Method and system for searching trademarks | |
CN104794681B (en) | Remote sensing image fusion method based on more redundant dictionaries and sparse reconstruct | |
CN103440512A (en) | Identifying method of brain cognitive states based on tensor locality preserving projection | |
CN107491793B (en) | Polarized SAR image classification method based on sparse scattering complete convolution | |
Jin et al. | Spatial-spectral feature extraction of hyperspectral images for wheat seed identification | |
CN102799858A (en) | Automatic medical ultrasonic image recognition method based on redundant characteristic reduction | |
CN109034213B (en) | Hyperspectral image classification method and system based on correlation entropy principle | |
CN105550712A (en) | Optimized convolution automatic encoding network-based auroral image sorting method | |
Liu et al. | Local linear Laplacian eigenmaps: A direct extension of LLE | |
CN106886793A (en) | Hyperspectral image band selection method based on discriminant information and manifold information | |
Zhao et al. | Spatial-spectral classification of hyperspectral image via group tensor decomposition | |
Wang | Real local-linearity preserving embedding | |
CN112381144A (en) | Heterogeneous deep network method for non-European and European domain space spectrum feature learning | |
CN107273919B (en) | Hyperspectral unsupervised classification method for constructing generic dictionary based on confidence | |
Ning et al. | Multi-scale gradational-order fusion framework for breast lesions classification using ultrasound images | |
Vieira et al. | Characterization of welding defects by fractal analysis of ultrasonic signals | |
Li et al. | Hyperspectral image feature extraction using maclaurin series function curve fitting |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
C06 | Publication | ||
PB01 | Publication | ||
C10 | Entry into substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
GR01 | Patent grant | ||
GR01 | Patent grant |