CN110929801B - Improved Euclid distance KNN classification method and system - Google Patents

Improved Euclid distance KNN classification method and system Download PDF

Info

Publication number
CN110929801B
CN110929801B CN201911215801.9A CN201911215801A CN110929801B CN 110929801 B CN110929801 B CN 110929801B CN 201911215801 A CN201911215801 A CN 201911215801A CN 110929801 B CN110929801 B CN 110929801B
Authority
CN
China
Prior art keywords
sample
training
class
projection
value
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201911215801.9A
Other languages
Chinese (zh)
Other versions
CN110929801A (en
Inventor
徐承俊
朱国宾
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Wuhan University WHU
Original Assignee
Wuhan University WHU
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Wuhan University WHU filed Critical Wuhan University WHU
Priority to CN201911215801.9A priority Critical patent/CN110929801B/en
Publication of CN110929801A publication Critical patent/CN110929801A/en
Application granted granted Critical
Publication of CN110929801B publication Critical patent/CN110929801B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G06F18/2413—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on distances to training or reference patterns
    • G06F18/24147—Distances to closest patterns, e.g. nearest neighbour classification
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/213—Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods
    • G06F18/2132—Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods based on discrimination criteria, e.g. discriminant analysis
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting

Landscapes

  • Engineering & Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Biology (AREA)
  • Evolutionary Computation (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
  • Image Analysis (AREA)

Abstract

The invention discloses a method based onThe improved Euclid distance KNN classification method and system firstly acquire a data set from a database, and divide the data set into a test set and a training set; setting a neighbor parameter K value; calculating a projection vector w according to an LDA (Linear Discriminant analysis) algorithm; constructing a neighbor graph G (V, E) from the training set; for each data sample x in the test settextFinding data sample x from neighbor maptextK neighbors in the training set; return pair data sample xtextIs estimated value of
Figure DDA0002299461640000011
And the determination of the sample class is made. The invention has the following advantages: (1) the noise-resistant KNN has good noise resistance, and can solve the problem that the traditional KNN is sensitive to noise. (2) The method adopts the improved Euclid distance to replace the Euclid distance measurement adopted by the traditional KNN, can distinguish samples more, improves the accuracy of classification, and does not increase the complexity of calculation.

Description

Improved Euclid distance KNN classification method and system
Technical Field
The invention relates to the technical field of data classification, in particular to a classification method and system based on an improved Euclid distance KNN.
Background
In the current big data era, various data are large in scale and wide in range, and data need to be classified and processed so as to facilitate subsequent further analysis and processing. The data are classified by using a KNN algorithm, and the basic idea of the KNN algorithm is as follows: for the nearest K neighbors of any given sample to be classified, the classification of the K neighbors is then voted according to the classification attributes of the K neighbors. The distance measurement method of the KNN algorithm mainly adopts the Euclid distance (Euclidean distance) of a sample to be measured and a training sample. The KNN algorithm assumes that all samples correspond to the n-dimensional space RnThe nearest neighbor of a sample is defined according to the standard Euclid distance. The KNN algorithm is only related to a very small number of adjacent samples when the class is judged, the class is mainly determined by the limited adjacent samples around rather than by a method for judging the class domain, and therefore, the KNN algorithm is more suitable for a sample set to be detected with more overlapping or crossing of the class domain than other classification methods.
The KNN algorithm is an inert learning method, so that the problems of low classification speed, strong sample library capacity dependence, low data processing efficiency and the like are caused easily when the distance calculation standard is sensitive to noise characteristics and the sample data volume is large, particularly under the condition that a sample contains noise.
Disclosure of Invention
The invention provides an improved Euclid distance KNN classification method, which is used for solving the problem that the metric for calculating the distance in the background technology is sensitive to noise characteristics.
In order to achieve the above object, the present invention provides an improved euclidd distance KNN-based classification method, which comprises the following specific steps:
step1, acquiring a data set from the database, and dividing the data set into a test set and a training set;
step2, setting a neighbor parameter K value;
step3, solving a projection vector w of a training set according to a Linear Discriminant Analysis algorithm;
step4, constructing a neighbor graph G (V, E) according to the training set, wherein G represents the neighbor graph, V represents a node, namely each training sample in the training set, and E represents an edge connecting each training sample;
step5, for each data sample x in the test settextFinding data sample x from neighbor maptextK neighbors in the training set;
step6, return data sample xtextIs estimated value of
Figure BDA0002299461620000021
Wherein the content of the first and second substances,
Figure BDA0002299461620000022
f(xi) Problem function, x, representing a classificationiRepresents the ith training sample, V represents the class corresponding to the training sample, and V ═ V1,v2,…,vsV denotes a set of data categories,
Figure BDA0002299461620000023
is the data sample xtextIn the final category of (a) to (b),
Figure BDA0002299461620000024
further, Step2 sets K to 1,3,5,7,9,11,13, 15.
Further, the projection vector w in Step3 is calculated as follows,
taking the two classifications as an example, the optimal projection vector w is solved by quantitative analysis:
given N training samples characterized by d dimensions
Figure BDA0002299461620000025
First, the mean value, i.e. the center point, of each class of training samples is found, where i is 1,2,
Figure BDA0002299461620000026
specifically, there is N1A training sample belonging to the category w1Having N of2A training sample belongs to the category w2，N＝N1+N2，μiRepresenting the mean of the ith class of training samples;
the projection of the training samples x to w is represented by y ═ wTx is calculated, and the mean value of sample points after x to w projection of the training samples is represented as:
Figure BDA0002299461620000027
therefore, the projected average value is the projection of the center point of the sample;
the straight line that can make the two types of sample central points after projection separate as much as possible is the best straight line, and the quantitative expression is:
Figure BDA0002299461620000028
obtaining a hash value of the projected class, specifically:
Figure BDA0002299461620000029
final pass metric formula
Figure BDA00022994616200000210
Measuring a projection vector w;
according to the above formula, it is sufficient to find w that maximizes J (w), and the solution is as follows:
expanding the hash value formula:
Figure BDA00022994616200000211
wherein order
Figure BDA00022994616200000212
Namely a hash matrix;
then, let Sw＝S1+S2，SwCalled the intra-class dispersion degree matrix, SB＝(μ1-μ2)(μ1-μ2)T，SBCalled inter-class dispersion degree matrix;
j (w) is finally expressed as:
Figure BDA0002299461620000031
performing derivation on the derivative, and performing normalization processing on the denominator before derivation; then let | | wTSWW | | | 1, after adding lagrange multiplier, the derivation:
Figure BDA0002299461620000032
it follows that w is a matrix
Figure BDA0002299461620000033
The feature vector of (2);
in particular, because of SBw＝(μ1-μ2)(μ1-μ2)Tw, where the product of the latter two terms is a constant, denoted λwThen, then
Figure BDA0002299461620000034
Since any expansion or reduction of w by a factor does not affect the result, the unknown constants λ, λ on both sides are reduced for simplicitywTo obtain
Figure BDA0002299461620000036
Therefore, only the mean and equation of the original training sample are required to calculate the optimal w.
Further, in Step4, the size of the edge in the neighbor graph is specifically represented by the formula:
Figure BDA0002299461620000035
determination of where xlThe l-th feature vector, x, representing the training sample xi,xjRespectively representing the ith training sample and the jth training sample, wherein m is the number of the characteristic vectors, t is an arbitrary constant, and w is the projection vector obtained in the step 2.
Further, the value of m is 5, and the value of m respectively comprises a stroke, a contour, a cross point, an end point and a gray level feature vector of the image.
The invention also provides an improved Euclid distance KNN classification system, which comprises the following modules:
the data set acquisition module is used for acquiring a data set from a database and dividing the data set into a test set and a training set;
the parameter setting module is used for setting a neighbor parameter K value;
the projection vector w solving module is used for solving a training set projection vector w according to a Linear discriminatant Analysis algorithm;
the neighbor graph constructing module is used for constructing a neighbor graph G (V, E) according to the training set, wherein G represents the neighbor graph, V represents a node, namely each training sample in the training set, and E represents an edge connecting each training sample;
k neighbor search modules for each data sample x in the test settextFinding K neighbors in the training set;
a sample class determination module for returning the data sample xtextIs estimated value of
Figure BDA0002299461620000041
Wherein the content of the first and second substances,
Figure BDA0002299461620000042
f(xi) Problem function, x, representing a classificationiRepresents the ith training sample, v represents the class corresponding to the training sample,
Figure BDA0002299461620000043
is the data sample xtextIn the final category of (a) to (b),
Figure BDA0002299461620000044
further, the setting K in the parameter setting module is 1,3,5,7,9,11,13, 15.
Furthermore, the projection vector w in the projection vector w solving module is calculated as follows,
taking the two classifications as an example, the optimal projection vector w is solved by quantitative analysis:
given N training samples characterized by d dimensions
Figure BDA0002299461620000045
First, the mean value, i.e. the center point, of each class of training samples is found, where i is 1,2,
Figure BDA0002299461620000046
specifically, there is N1A training sample belonging to the category w1Having N of2A training sample belonging to the category w2，N＝N1+N2，μiRepresenting the mean of the ith class of training samples;
the projection of the training samples x to w is represented by y ═ wTx is calculated, and the mean value of sample points after x to w projection of the training samples is represented as:
Figure BDA0002299461620000047
therefore, the projected average value is the projection of the center point of the sample;
the straight line that can make the two types of sample central points after projection separate as much as possible is the best straight line, and the quantitative expression is:
Figure BDA0002299461620000048
obtaining a hash value of the projected class, specifically:
Figure BDA0002299461620000049
final pass metric formula
Figure BDA00022994616200000410
Measuring a projection vector w;
according to the above formula, it is sufficient to find w that maximizes J (w), and the solution is as follows:
expanding the hash value formula:
Figure BDA00022994616200000411
wherein order
Figure BDA00022994616200000412
Namely a hash matrix;
then, let Sw＝S1+S2，SwCalled the intra-class dispersion degree matrix, SB＝(μ1-μ2)(μ1-μ2)T，SBCalled inter-class dispersion degree matrix;
j (w) is finally expressed as:
Figure BDA00022994616200000413
performing derivation on the derivative, and performing normalization processing on the denominator before derivation; then let | | wTSWW | | | 1, after adding lagrange multiplier, the derivation:
Figure BDA0002299461620000051
it follows that w is a matrix
Figure BDA0002299461620000052
The feature vector of (2);
in particular, because of SBw＝(μ1-μ2)(μ1-μ2)Tw, where the product of the latter two terms is a constant, denoted λwThen, then
Figure BDA0002299461620000053
Since any expansion or reduction of w by a factor does not affect the result, the unknown constants λ, λ on both sides are reduced for simplicitywTo obtain
Figure BDA0002299461620000054
Therefore, only the original training sample is requiredThe mean and equation of the present method find the optimal w.
Further, in the neighbor graph constructing module, the size of the edge in the neighbor graph is specifically represented by the formula:
Figure BDA0002299461620000055
determination of where xlThe l-th feature vector, x, representing a training sample xi,xjRespectively representing the ith training sample and the jth training sample, wherein m is the number of the characteristic vectors, t is an arbitrary constant, and w is the projection vector obtained in the step 2.
Further, the value of m is 5, and the value of m respectively comprises a stroke, a contour, a cross point, an end point and a gray level feature vector of the image.
Compared with the prior art, the invention has the following beneficial effects: the invention provides an improved Euclidid distance KNN classification method, which is characterized in that a neighbor parameter K is preset, a projection vector w is calculated according to an LDA (Linear cognitive analysis) algorithm, a training data set is constructed into a neighbor graph G (V, E), wherein G represents the neighbor graph, V represents a node which is each data sample, E represents an edge connecting each data sample, and the size of the edge is specifically represented by a formula:
Figure BDA0002299461620000056
wherein x islValues representing the l-th feature of the sample x, xi,xjRespectively representing the ith and jth samples, t representing an arbitrary constant, w representing the projection vector, x for each data sample in the test settextFinding K neighbors in the training set, the return value of the KNN algorithm
Figure BDA0002299461620000057
Is to the data sample xtextEstimation of class, i.e. distance sample xtextAnd performing class judgment on the most common f value in the latest K training samples. Since the traditional KNN algorithm adopts Euclid measurement, the measurement standard for calculating the distance is sensitive to noise characteristicsThe KNN algorithm is improved by replacing the traditional Euclid distance with the Euclid distance improved by the method. The method has good distinguishability, noise immunity and robustness of the projection vector of the LDA algorithm, can distinguish multidimensional data and carry out good classification, can keep high resolution and good calculation performance, and can be used as a reference for similar KNN research.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the present invention is further described below with reference to the accompanying drawings and the embodiments.
FIG. 1 is a simplified flow chart of the classification method based on the improved Euclid distance KNN according to the present invention;
FIG. 2 is a schematic view of a training sample of the present invention projected onto a straight line;
FIG. 3 is a schematic view of a sample center projection of the present invention;
FIG. 4 is a diagram illustrating the present invention using LDA to solve the optimal projection vector w;
FIG. 5 is a graphical representation of the classification performance of the USPS data set in accordance with the present invention;
fig. 6 is a diagram illustrating the classification performance of the MNIST data set according to the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. The detailed description of the embodiments of the present invention generally described and illustrated in the figures herein is not intended to limit the scope of the invention, which is claimed, but is merely representative of selected embodiments of the invention.
It should be noted that: like reference symbols in the following drawings indicate like items, and thus, once an item is defined in one drawing, it need not be further defined and explained in subsequent drawings.
Referring to fig. 1, fig. 1 is a simplified flow chart of the classification method based on the improved euclidd distance KNN according to the present invention. The embodiment is particularly applicable to classification of data, and the embodiment of the invention is executed in a development environment of lie group machine learning.
Step1, in this embodiment, the USPS data set is downloaded over the network, and the data set includes 10 categories, each of which is a 0-9 category number, and includes 20000 ten thousand pictures, and each picture is an image of 32 × 32 (unit: pixel) size. An MNIST data set is downloaded from a network, and the data set comprises 10 categories of numbers of 0-9 categories respectively and 70000 pictures in total, wherein each picture is an image with the size of 28 x 28 (unit: pixel). Furthermore, the classification test is carried out on the two data sets, and the two data sets are respectively divided into a training data set and a test data set by programming through matlab language.
It should be noted that the picture data in this embodiment has the following advantages: (1) the data volume is large, the categories are many, and the method is necessary for the plum blossom machine learning. (2) The diversity of the sample images, which is a standard data set in this embodiment, covers various handwritten data volumes, the sample images have diversity, and the images in the data set are strictly screened for different angles, illumination and definition, so that the observation angles and the like of each category of images have larger differences.
Step2, setting a neighbor parameter K value, wherein K in the method is 1,3,5,7,9,11,13 and 15;
step3, calculating a projection vector w of a training set according to an LDA (Linear Discriminant analysis) algorithm;
given N training samples characterized by d dimensions
Figure BDA0002299461620000071
In which is N1A training sample belonging to the category w1Having N of2A training sample belonging to the category w2，N＝N1+N2；
Reducing the dimension of the d-dimensional feature and ensuring that data feature information is not lost after the dimension reduction, namely, determining the class of each sample after the dimension reduction, and referring the optimal vector as w (d dimension), wherein the projection of the training sample x (d dimension) onto w can be represented by y ═ wTAnd x is calculated.
In order to be simple and easy to understand,in the present invention, we first consider the case where the training sample x is two-dimensional, and intuitively see, as shown in fig. 2, the circular and triangular tables represent two different types of training samples, the training sample x is two-dimensional and includes two eigenvalues, x1 represents one eigenvalue, x2 represents another eigenvalue, the straight line obtained is a straight line capable of separating the two types of training samples, and the straight line y ═ w in fig. 2Tx may well separate training samples of different classes. This is actually the idea of LDA: maximizing inter-class variance and minimizing intra-class variance, i.e., reducing the differences between the intra-class, broaden the differences between different classes.
The specific process of the quantitative analysis to find the optimal w is described below.
First, find the mean (center point) of each class of training samples, where i is only two (i ═ 1, 2):
Figure BDA0002299461620000072
specifically, there is N1A training sample belonging to the category w1Having N of2A training sample belonging to the category w2，N＝N1+N2，μiRepresenting the mean of the ith class of training samples;
the mean of the sample points after x to w projection is given by:
Figure BDA0002299461620000073
the meaning of each symbol is consistent with the above description, and therefore, the projected mean value is the projection of the center point of the training sample.
The straight line that can make the two types of sample central points after projection separate as much as possible is the best straight line, and the quantitative expression is:
Figure BDA0002299461620000074
the larger the J (w), the better.
In practical applications, J (w) is not as large as possible, and as shown in FIG. 3, the sample points are uniformly distributed in an ellipse projected to the horizontal axis x1A larger center point separation j (w) can be achieved in the above, but sample points cannot be separated on the x-axis due to the overlap. Projected onto the longitudinal axis x2In the above-mentioned manner,although j (w) is small, sample points can be separated. Therefore, we must also consider the variance between sample points, the larger the variance, the more difficult it is to separate the sample points.
The projected class is hashed using another metric, called hash (scatter), specifically:
Figure BDA0002299461620000081
the geometric meaning of a hash value is the density of sample points, with larger values being more dispersed and vice versa more concentrated.
In the present invention, it is necessary to separate different sample points better, and the more similar samples are gathered better, i.e. the larger the mean difference is, the better the hash value is. Measured using j (w) and S, the measurement formula:
Figure BDA0002299461620000082
according to the above formula, it is necessary to find w that maximizes J (w).
Expanding the hash value formula:
Figure BDA0002299461620000083
wherein order
Figure BDA0002299461620000084
I.e. a hash matrix.
Then, let Sw＝S1+S2，SwCalled the Within-class dispersion degree matrix (Within-class scatter matrix). SB＝(μ1-μ2)(μ1-μ2)T，SBReferred to as the inter-class dispersion matrix (Between-class scanner matrix).
J (w) is finally expressed as:
Figure BDA0002299461620000085
and (4) carrying out derivation on the derivative, carrying out normalization processing on the denominator before derivation, and if the normalization processing is not carried out, w is expanded by any multiple, and the formula is established, the w cannot be determined. Therefore, in the present invention, let | | wTSWW | | | 1, after adding lagrange multiplier, the derivation:
Figure BDA0002299461620000086
it follows that w is a matrix
Figure BDA0002299461620000087
The feature vector of (2).
In particular, because of SBw＝(μ1-μ2)(μ1-μ2)Tw, where the product of the latter two terms is a constant, denoted λwThen, then
Figure BDA0002299461620000088
Since any expansion or reduction of w by a factor does not affect the result, the unknown constants λ, λ on both sides can be reduced for simplicitywTo obtain
Figure BDA0002299461620000089
We need only find the mean and equation of the original samples to find the best w, as shown in fig. 4.
The above conclusions, although coming from 2 dimensions, are true for multi-dimensions as well. The feature vector segmentation performance corresponding to the large feature value is the best.
Step4, constructing a neighbor graph G (V, E) according to the training set;
constructing a neighbor graph G (V, E) according to the training set, wherein G represents the neighbor graph, V represents a node, namely each data sample, E represents an edge connecting each data sample, and the size of the edge is specifically represented by a formula:
Figure BDA0002299461620000091
wherein x islThe method comprises the steps that the ith characteristic vector of a training sample x is represented, m refers to the number of the characteristic vectors, the value of m is related to the selection of a data set, the characteristic vectors mainly take 5 strokes, outlines, cross points, end points and gray levels of images, and the solution of the characteristic vectors is the prior art and is not written in the invention; x is the number ofi,xjRespectively, the ith sample and the jth sample, t represents an arbitrary constant, and w represents the projection vector.
Step5, for each data sample x in the test settextFinding data sample x from neighbor maptextK neighbors in the training set;
step6, return data sample xtextIs estimated value of
Figure BDA0002299461620000092
And the determination of the sample class is made.
The present invention discusses the case where the objective function is a discrete value (classification problem), i.e. the classification problem function can be described as: f is Rn→ V, where V ═ V1,v2,…,vsV represents a set of data categories, corresponding to s classes. Estimation of KNN algorithm
Figure BDA0002299461620000093
Is to the data sample xtextEstimation of class, i.e. distance sample xtextThe most common f-number of the most recent K training samples:
Figure BDA0002299461620000094
wherein the content of the first and second substances,
Figure BDA0002299461620000095
is the data sample xtextFinal class of f (x)i) Problem function, x, representing a classificationiRepresents the ith training sample, v represents the class corresponding to the training sample,
Figure BDA0002299461620000096
table 1 shows the comparison of classification performance of the inventive method with the conventional KNN classification method on USPS datasets. As can be seen from the table, the classification accuracy of the method is obviously higher than that of the traditional KNN classification method.
TABLE 1 comparison of classification Performance of the inventive method with other methods on USPS datasets
Figure BDA0002299461620000101
Table 2 shows the comparison of the classification performance of the inventive method with the conventional KNN classification method on the MNIST dataset. As can be seen from the table, the classification accuracy of the method is obviously higher than that of the traditional KNN classification method.
TABLE 2 comparison of classification performance of the method of the present invention on MNIST datasets with other methods
Figure BDA0002299461620000102
With reference to fig. 5 to 6, fig. 5 is a classification performance diagram of the USPS data set according to the embodiment of the present invention, and fig. 6 is a classification performance diagram of the MNIST data set according to the embodiment of the present invention. FIG. 5 is applied to the USPS data set, and the average classification accuracy is 96%, while the average classification accuracy of the conventional KNN is 72%, which is 24% higher than that of the method provided by the present invention; fig. 6 is applied to MNIST datasets with an average classification accuracy of 95%, whereas the conventional KNN average classification accuracy is 88%, which is 7% higher than the method proposed by the present invention. The statistical result shows that the method of the invention is obviously superior to the traditional KNN method and has strong practicability.
The invention also provides an improved Euclid distance KNN classification system, which comprises the following modules:
the data set acquisition module is used for acquiring a data set from a database and dividing the data set into a test set and a training set;
the parameter setting module is used for setting a neighbor parameter K value;
the projection vector w solving module is used for solving a training set projection vector w according to a Linear discriminatant Analysis algorithm;
the neighbor graph constructing module is used for constructing a neighbor graph G (V, E) according to the training set, wherein G represents the neighbor graph, V represents a node, namely each training sample in the training set, and E represents an edge connecting each training sample;
k neighbor search modesBlock, for each data sample x in the test settextFinding data sample x from neighbor maptextK neighbors in the training set;
a sample class determination module for returning the data sample xtextIs estimated value of
Figure BDA0002299461620000111
Wherein the content of the first and second substances,
Figure BDA0002299461620000112
f(xi) Problem function, x, representing a classificationiRepresents the ith training sample, V represents the class corresponding to the training sample, and V ═ V1,v2,…,vsV denotes a set of data categories,
Figure BDA0002299461620000113
is the data sample xtextIn the final category of (a) to (b),
Figure BDA0002299461620000114
and setting K in the parameter setting module to be 1,3,5,7,9,11,13 and 15.
Wherein, the calculation mode of the projection vector w in the projection vector w solving module is as follows,
taking the two classifications as an example, the optimal projection vector w is solved by quantitative analysis:
given N training samples characterized by d dimensions
Figure BDA0002299461620000115
First, the mean value, i.e. the center point, of each class of training samples is found, where i is 1,2,
Figure BDA0002299461620000116
specifically, there is N1A training sample belonging to the category w1Having N of2A training sample belonging to the category w2，N＝N1+N2，μiRepresenting the mean of the ith class of training samples;
training sampleY ═ w for projection on x to wTx is calculated, and the mean value of sample points after x to w projection of the training samples is represented as:
Figure BDA0002299461620000117
therefore, the projected average value is the projection of the center point of the sample;
the straight line that can make the two types of sample central points after projection separate as much as possible is the best straight line, and the quantitative expression is:
Figure BDA0002299461620000121
obtaining a hash value of the projected class, specifically:
Figure BDA0002299461620000122
final pass metric formula
Figure BDA0002299461620000123
Measuring a projection vector w;
according to the above formula, it is sufficient to find w that maximizes J (w), and the solution is as follows:
expanding the hash value formula:
Figure BDA0002299461620000124
wherein order
Figure BDA0002299461620000125
Namely a hash matrix;
then, let Sw＝S1+S2，SwCalled the intra-class dispersion degree matrix, SB＝(μ1-μ2)(μ1-μ2)T，SBCalled inter-class dispersion degree matrix;
j (w) is finally expressed as:
Figure BDA0002299461620000126
performing derivation on the derivative, and performing normalization processing on the denominator before derivation; then let | | wTSWW | | | 1, after adding lagrange multiplier, the derivation:
Figure BDA0002299461620000127
it follows that w is a matrix
Figure BDA0002299461620000128
The feature vector of (2);
in particular, because of SBw＝(μ1-μ2)(μ1-μ2)Tw, where the product of the latter two terms is a constant, denoted λwThen, then
Figure BDA0002299461620000129
Since any expansion or reduction of w by a factor does not affect the result, the unknown constants λ, λ on both sides are reduced for simplicitywTo obtain
Figure BDA00022994616200001210
Therefore, only the mean and equation of the original training sample are required to calculate the optimal w.
In the neighbor graph constructing module, the size of the edge in the neighbor graph is specifically represented by a formula:
Figure BDA00022994616200001211
determination of where xlThe l-th feature vector, x, representing a training sample xi,xjRespectively representing the ith training sample and the jth training sample, wherein m is the number of the feature vectors, t is an arbitrary constant, and w is the projection vector obtained in the step 2.
The specific implementation of each module corresponds to each step, and the invention is not described.
The above description is only a part of the embodiments of the present invention, and is not intended to limit the present invention, and it will be apparent to those skilled in the art that various modifications can be made in the present invention. Any changes, equivalent substitutions or improvements made within the spirit and principle of the present invention should be included within the scope of the present invention. Note that like reference numerals and letters denote like items in the following drawings. Thus, once an item is defined in one drawing, it need not be further defined and explained in subsequent drawings.

Claims (6)

1. A classification method based on an improved Euclid distance KNN is characterized by comprising the following steps:
step1, acquiring a data set from the database, and dividing the data set into a test set and a training set;
step2, setting a neighbor parameter K value;
step3, solving a projection vector w of a training set according to a Linear Discriminant Analysis algorithm;
step4, constructing a neighbor graph G (V, E) according to the training set, wherein G represents the neighbor graph, V represents a node, namely each training sample in the training set, and E represents an edge connecting each training sample;
in Step4, the size of the edge in the neighbor graph is specifically represented by the formula:
Figure FDA0003583947560000011
determination of where xlThe l-th feature vector, x, representing a training sample xi,xjRespectively representing the ith training sample and the jth training sample, wherein m is the number of the characteristic vectors, t represents an arbitrary constant, and w represents a projection vector obtained by Step 3;
the value of m is 5, and the value of m respectively comprises a stroke, a contour, a cross point, an end point and a gray level feature vector of the image;
step5, for each data sample x in the test settextFinding data samples x from neighbor mapstextK neighbors in the training set;
step6, return data sample xtextIs estimated value of
Figure FDA0003583947560000012
Wherein the content of the first and second substances,
Figure FDA0003583947560000013
f(xi) Problem function, x, representing a classificationiRepresents the ith training sample, V represents the class corresponding to the training sample, V' represents the set of data classes,
Figure FDA0003583947560000014
is the data sample xtextIn the final category of (a) to (b),
Figure FDA0003583947560000015
2. the improved Euclid distance KNN classification method according to claim 1, characterized in that: and setting the value range of K to be {1,3,5,7,9,11,13,15} in the Step 2.
3. The improved Euclid distance KNN classification method according to claim 1, characterized in that: the projection vector w in Step3 is calculated as follows,
taking the two classifications as an example, the optimal projection vector w is solved by quantitative analysis:
given N training samples characterized by d dimensions
Figure FDA0003583947560000016
First, the mean value, i.e. the center point, of each type of training sample is found, and then
Figure FDA0003583947560000017
Specifically, there is N1A training sample belonging to class v1Having N of2A training sample belonging to class v2，N＝N1+N2，μzRepresenting the mean of the class z training samples;
the projection of the training samples x to w is represented by y-wTx is calculated, and the mean value of sample points after x to w projection of the training samples is represented as:
Figure FDA0003583947560000021
therefore, the projected average value is the projection of the center point of the sample;
the straight line that can make the two types of sample central points after projection separate as much as possible is the best straight line, and the quantitative expression is:
Figure FDA0003583947560000022
obtaining a hash value of the projected class, specifically:
Figure FDA0003583947560000023
final pass metric formula
Figure FDA0003583947560000024
Measuring a projection vector w;
according to the above formula, it is sufficient to find w that maximizes J (w), and the solution is as follows:
expanding the hash value formula:
Figure FDA0003583947560000025
wherein order
Figure FDA0003583947560000026
Namely a hash matrix;
then, let Sw＝S1+S2，SwCalled the intra-class dispersion degree matrix, SB＝(μ1-μ2)(μ1-μ2)T，SBCalled inter-class dispersion degree matrix;
j (w) is finally expressed as:
Figure FDA0003583947560000027
performing derivation on the derivative, and performing normalization processing on the denominator before derivation; then let | | wTSWW | | | 1, after adding lagrange multiplier, the derivation:
Figure FDA0003583947560000028
it follows that w is a matrix
Figure FDA0003583947560000029
The feature vector of (2);
in particular, because of SBw＝(μ1-μ2)(μ1-μ2)Tw, where the product of the latter two terms is a constant, denoted λwThen, then
Figure FDA00035839475600000210
Since any expansion or reduction of w by a factor does not affect the result, the unknown constants λ, λ on both sides are reduced for simplicitywTo obtain
Figure FDA00035839475600000211
Therefore, only the mean and equation of the original training sample are required to calculate the optimal w.
4. An improved Euclid distance KNN classification system is characterized by comprising the following modules:
the data set acquisition module is used for acquiring a data set from a database and dividing the data set into a test set and a training set;
the parameter setting module is used for setting a neighbor parameter K value;
the projection vector w solving module is used for solving a training set projection vector w according to a Linear discriminatant Analysis algorithm;
the neighbor graph constructing module is used for constructing a neighbor graph G (V, E) according to the training set, wherein G represents the neighbor graph, V represents a node, namely each training sample in the training set, and E represents an edge connecting each training sample;
in the neighbor graph constructing module, the size of the edge in the neighbor graph is specifically represented by a formula:
Figure FDA0003583947560000031
determination of where xlThe l-th feature vector, x, representing a training sample xi,xjRespectively representing the ith training sample and the jth training sample, m is the number of the characteristic vectors, and t represents the random numberAn intention constant, wherein w represents a projection vector obtained by the projection vector w solving module;
the value of m is 5, and the value of m respectively comprises a stroke, a contour, a cross point, an end point and a gray level feature vector of the image;
k neighbor search modules for each data sample x in the test settextFinding data sample x from neighbor maptextK neighbors in the training set;
a sample class determination module for returning the data sample xtextIs estimated value of
Figure FDA0003583947560000032
Wherein the content of the first and second substances,
Figure FDA0003583947560000033
f(xi) Problem function, x, representing a classificationiRepresents the ith training sample, V represents the class corresponding to the training sample, V' represents the set of data classes,
Figure FDA0003583947560000034
is the data sample xtextIn the final category of (a) to (b),
Figure FDA0003583947560000035
5. the improved Euclid distance KNN classification system according to claim 4, characterized in that: the value range of K is set to be {1,3,5,7,9,11,13,15} in the parameter setting module.
6. The improved Euclid distance KNN classification system according to claim 4, characterized in that: the projection vector w in the projection vector w solving module is calculated as follows,
taking the two classifications as an example, the optimal projection vector w is solved by quantitative analysis:
given N training samples characterized by d dimensions
Figure FDA0003583947560000036
First, the mean value, i.e., the center point, of each type of training sample is found, where z is 1,2,
Figure FDA0003583947560000037
specifically, there is N1A training sample belonging to class v1Having N of2A training sample belonging to class v2，N＝N1+N2，μzRepresenting the mean of the class z training samples;
the projection of the training samples x to w is represented by y ═ wTx is calculated, and the mean value of sample points after x to w projection of the training samples is represented as:
Figure FDA0003583947560000041
therefore, the projected average value is the projection of the center point of the sample;
the straight line that can make the two types of sample central points after projection separate as much as possible is the best straight line, and the quantitative expression is:
Figure FDA0003583947560000042
obtaining a hash value of the projected class, specifically:
Figure FDA0003583947560000043
final pass metric formula
Figure FDA0003583947560000044
Measuring a projection vector w;
according to the above formula, it is sufficient to find w that maximizes J (w), and the solution is as follows:
expanding the hash value formula:
Figure FDA0003583947560000045
wherein order
Figure FDA0003583947560000046
Namely a hash matrix;
then, let Sw＝S1+S2，SwReferred to as the internal dispersion degree matrix, SB＝(μ1-μ2)(μ1-μ2)T，SBCalled inter-class dispersion degree matrix;
j (w) is finally expressed as:
Figure FDA0003583947560000047
performing derivation on the derivative, and performing normalization processing on the denominator before derivation; then let | | wTSWW | | | 1, after adding lagrange multiplier, the derivation:
Figure FDA0003583947560000048
it follows that w is a matrix
Figure FDA0003583947560000049
The feature vector of (2);
in particular, because of SBw＝(μ1-μ2)(μ1-μ2)Tw, where the product of the latter two terms is a constant, denoted λwThen, then
Figure FDA00035839475600000410
Since any expansion or reduction of w by a factor does not affect the result, the unknown constants λ, λ on both sides are reduced for simplicitywTo obtain
Figure FDA00035839475600000411
Therefore, only the mean and equation of the original training sample are required to calculate the optimal w.
CN201911215801.9A 2019-12-02 2019-12-02 Improved Euclid distance KNN classification method and system Active CN110929801B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201911215801.9A CN110929801B (en) 2019-12-02 2019-12-02 Improved Euclid distance KNN classification method and system

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201911215801.9A CN110929801B (en) 2019-12-02 2019-12-02 Improved Euclid distance KNN classification method and system

Publications (2)

Publication Number Publication Date
CN110929801A CN110929801A (en) 2020-03-27
CN110929801B true CN110929801B (en) 2022-05-13

Family

ID=69848393

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201911215801.9A Active CN110929801B (en) 2019-12-02 2019-12-02 Improved Euclid distance KNN classification method and system

Country Status (1)

Country Link
CN (1) CN110929801B (en)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112613184A (en) * 2020-12-29 2021-04-06 煤炭科学研究总院 Artificial intelligence method for judging distance of side slope collapse rockfall during earthquake occurrence
CN113162926B (en) * 2021-04-19 2022-08-26 西安石油大学 KNN-based network attack detection attribute weight analysis method

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102208020A (en) * 2011-07-16 2011-10-05 西安电子科技大学 Human face recognition method based on optimal dimension scale cutting criterion

Family Cites Families (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20030051554A (en) * 2003-06-07 2003-06-25 전명근 Face Recognition using fuzzy membership value
CN101673348B (en) * 2009-10-20 2012-05-09 哈尔滨工程大学 Human face recognition method based on supervision isometric projection
CN102073799A (en) * 2011-01-28 2011-05-25 重庆大学 Tumor gene identification method based on gene expression profile
CN103679207A (en) * 2014-01-02 2014-03-26 苏州大学 Handwriting number identification method and system
CN103854645B (en) * 2014-03-05 2016-08-24 东南大学 A kind of based on speaker's punishment independent of speaker's speech-emotion recognition method
CN107045621A (en) * 2016-10-28 2017-08-15 北京联合大学 Facial expression recognizing method based on LBP and LDA
CN107463920A (en) * 2017-08-21 2017-12-12 吉林大学 A kind of face identification method for eliminating partial occlusion thing and influenceing

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102208020A (en) * 2011-07-16 2011-10-05 西安电子科技大学 Human face recognition method based on optimal dimension scale cutting criterion

Also Published As

Publication number Publication date
CN110929801A (en) 2020-03-27

Similar Documents

Publication Publication Date Title
CN107368807B (en) Monitoring video vehicle type classification method based on visual word bag model
US8718380B2 (en) Representing object shapes using radial basis function support vector machine classification
CN110443281B (en) Text classification self-adaptive oversampling method based on HDBSCAN (high-density binary-coded decimal) clustering
CN108122008B (en) SAR image recognition method based on sparse representation and multi-feature decision-level fusion
Niu et al. Meta-metric for saliency detection evaluation metrics based on application preference
Rahtu et al. A new convexity measure based on a probabilistic interpretation of images
CN110942091B (en) Semi-supervised few-sample image classification method for searching reliable abnormal data center
JP4376145B2 (en) Image classification learning processing system and image identification processing system
EP2948877A1 (en) Content based image retrieval
Ling et al. How many clusters? A robust PSO-based local density model
CN107633065B (en) Identification method based on hand-drawn sketch
US9165184B2 (en) Identifying matching images
CN110008844B (en) KCF long-term gesture tracking method fused with SLIC algorithm
CN110929801B (en) Improved Euclid distance KNN classification method and system
Duin et al. Mode seeking clustering by KNN and mean shift evaluated
WO2015146113A1 (en) Identification dictionary learning system, identification dictionary learning method, and recording medium
CN111027609B (en) Image data weighted classification method and system
Chebbout et al. Comparative study of clustering based colour image segmentation techniques
CN111738319B (en) Clustering result evaluation method and device based on large-scale samples
CN103345621A (en) Face classification method based on sparse concentration index
Wang et al. SpecVAT: Enhanced visual cluster analysis
CN106980878B (en) Method and device for determining geometric style of three-dimensional model
CN110910497B (en) Method and system for realizing augmented reality map
CN113378620B (en) Cross-camera pedestrian re-identification method in surveillance video noise environment
CN107607723A (en) A kind of protein-protein interaction assay method based on accidental projection Ensemble classifier

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant