CN106778875A - A kind of multi-class classification method based on projection with SVMs - Google Patents
A kind of multi-class classification method based on projection with SVMs Download PDFInfo
- Publication number
- CN106778875A CN106778875A CN201611189741.4A CN201611189741A CN106778875A CN 106778875 A CN106778875 A CN 106778875A CN 201611189741 A CN201611189741 A CN 201611189741A CN 106778875 A CN106778875 A CN 106778875A
- Authority
- CN
- China
- Prior art keywords
- classification
- sample
- divided
- highest
- class
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
- G06F18/2411—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on the proximity to a decision surface, e.g. support vector machines
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/24765—Rule-based classification
Landscapes
- Engineering & Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a kind of multi-class classification method based on projection with SVMs, support vector machine classifier is primarily based on, man-to-man grader is treated a point sample and voted, when highest gained vote classification number is unique, sample class to be divided determines；When highest that and if only if wins the vote classification number equal to participation ballot classification number, sample to be divided and supporting vector concentration vector in the heart to projection vectorial between sample to be divided and class center are calculated, superseded to project a maximum highest number of votes obtained classification, diminution is according to knowledge region.Then recall between remaining highest number of votes obtained classification using man-to-man class device ballot screening devices, recognize sample class to be divided, until decision-making blind area disappears；When highest gained vote classification number is less than participation ballot classification number, continue to recall between highest number of votes obtained classification using man-to-man class device ballot screening devices.The method of the present invention, improves the classification accuracy of sample, shortens the decision-making time of grader, and decision-making blind area can be completely eliminated.
Description
Technical field
It is specifically a kind of based on projection the present invention relates to a kind of SVMs (SVM) man-to-man multi-class classification method
With the multi-class classification method of SVMs.
Background technology
SVMs be initially be solve two classification problems design, for practical application in multiple pattern classifications,
Mainly use one-to-many, one-to-one two kinds of multi-class classification methods.One-to-many sorting technique, is on the m data set of classification
M sub-classifier of training, each sub-classifier builds SVM classifier with the i-th class and remaining classification respectively, with sample point to be divided
Value size on each grader, determines the belonging kinds of sample；Man-to-man sorting technique, be m classification two-by-two it
Between set up m (m-1)/2 sub-classifier, according to each grader ballot how much, it is determined that the belonging kinds of sample to be divided.
Using one-to-many, one-to-one two kinds of support vector machine classifiers, diagnosis identification is carried out to some fault types, sometimes
Can occur can not identification region decision-making blind zone problem.Such as, to improve accuracy and the flight of certain type unmanned plane tasks carrying
Security, on the hardware platform of TMS320F28335, based on the machine algorithm of SVMs (SVM), constructs gyroscope
Fault on-line diagnostic device.Biasing failure, stuck failure, drifting fault, multiplying property failure are the common top of four kinds of common four kind
Spiral shell instrument fault type, gyroscope fault on-line diagnostic device is respectively adopted both the above method and four kinds of fault types is diagnosed
Identification, will occur sometimes can not identification region decision-making blind zone problem.Similar Problems are there is also in other application occasion.
Chinese invention patent CN103839071A discloses a kind of many sorting techniques based on fuzzy support vector machine (referred to as
Fuzzy Decision Method), the decision-making blind area produced to many sorting techniques calculates the degree of membership of sample based on fuzzy technology, and foundation is subordinate to
Size is spent to differentiate the belonging kinds of sample.Fuzzy technology classifying quality when the classification problem of noise or isolated point sample is processed
Substantially.But decision-making blind area is typically a region for very little, and sample is all close apart from all kinds of classifying faces in this region, each sample
The experience cut off value pole of degree of membership is difficult determination, is also easy to produce erroneous judgement.
Chinese periodical paper《Improved one-to-one SVMs multi-classification algorithm》(abbreviation KNN tight ness ratings method) (《Calculate
Machine engineering and design》, vol33 (5), 1837-1841,2012) and a kind of sample that is based on is described to Euclidean distance between class center, and
Sample builds discriminant function and determines returning for classification to KNN (k nearest neighbor) ways and means of Euclidean distance between sample
Category method.Its method for solving decision-making blind area is the method based on similarity between Euclidean distance sign sample between class, belongs to weak typing
The decision scheme of type, have impact on diagnostic accuracy；The intensive of Euclidean distance between sample to be divided and class center, sample, and
It is difficult to meet the real time problems of Fault Identification in engineer applied.
The content of the invention
The technical problem to be solved in the present invention is to provide a kind of multi-class classification method based on projection with SVMs, should
Sorting technique while sample classification accuracy rate is improved, can as much as possible shorten the decision-making time of grader, and completely
Eliminate decision-making blind area.
For achieving the above object, the present invention uses following technical scheme：It is primarily based on SVMs strong classifier
Mechanism, man-to-man multiple two graders are treated a point sample and are voted, and count gained vote of the sample to be divided in grader of all categories
Number, when highest gained vote classification number is unique, sample class to be divided determines；Highest that and if only if gained vote classification number is equal to and participates in voting
During classification number, calculate sample to be divided and supporting vector concentrates vector in the heart to the throwing of vector between sample to be divided and class center
Shadow, eliminates a maximum highest number of votes obtained classification of projection, reduces according to knowledge region.Then proceed in remaining highest gained vote several classes of
Other backtracking recognizes sample class to be divided using man-to-man two grader ballot screening devices, until decision-making blind area disappears；When
Highest gained vote classification number is less than when participating in ballot classification number, continues to recall using man-to-man two points between highest number of votes obtained classification
Class device ballot screening devices.
Specifically, a kind of multi-class classification method based on projection with SVMs, comprises the following steps：
1) one-to-one support vector machine classifier is built between classification two-by-two, sample to be divided is input to grader, uses a pair
One support vector machine classifier method is treated a point sample and is classified, and adds up number of votes obtained of all categories.
2) number of votes obtained of all categories is counted, the classification number of highest number of votes obtained and highest gained vote is asked for, compares highest gained vote
Size between classification number and the classification number for participating in ballot.
3) if the classification number of highest number of votes obtained is identical with the classification number for participating in ballot, following steps are carried out：
311) sample to be divided and corresponding classification supporting vector are calculated and concentrates vector in the heart to sample to be divided and class center
Between vector projection；
312) relatively more each projection value size, eliminates the maximum classification of projection value；
313) each parameter is reinitialized：Number of votes obtained assignment 0 of all categories, the classification number for participating in ballot subtracts 1；
314) by sample to be divided be input to remaining highest gained vote classification between the one-to-one SVMs point that constitutes two-by-two
Additional ballot is carried out in class device, then is transferred to step 2).
If the classification number of highest gained vote is not unique and less than the classification number for participating in ballot, following steps are carried out：
321) each parameter is reinitialized：Number of votes obtained assignment 0 of all categories, the classification number for participating in ballot is entered as number of votes obtained
Highest classification number；
322) partner a support vector machine classifier two-by-two between the classification of highest gained vote, and continuation will sample input be divided
One-to-one support vector machine classifier to firm composition carries out additional ballot.Then it is transferred to step 2).
If highest votes are unique, terminate differentiating, number of votes obtained highest class is the belonging kinds of sample to be divided.
Further, the step 1) following steps can be taken to realize：
11) initialization number of votes obtained of all categories and number of votes obtained highest classification number are 0；The classification number for participating in ballot is entered as
Classification number；
12) by the most optimal sorting of one-to-one support vector machine classifier between the acquisition of SVMs learning method two-by-two classification
Class decision function；
13) for each grader, the supporting vector collection of two classifications of all one-to-one support vector machine classifiers is calculated
Center and class center；
14) sample to be divided is input in a pair of support vector machine classifier；If sample to be divided belongs to certain class, should
The number of votes obtained of classification adds 1；Add up number of votes obtained of the sample to be divided in of all categories.
Another kind is improved, the step 311) calculate sample divide and corresponding classification supporting vector concentration vector in the heart and arrive
The process of the projection of vector is between sample to be divided and class center：
Assuming that O is the class center of certain classification, R is the supporting vector collection center of the category, and A is the sample to be divided near R,
Definition vectorTo vectorBe projected as：
When data set linearly inseparable, using nonlinear transformationBy sample from luv space
After being mapped to high-dimensional feature space, projective representation is：
Wherein, xo, xa, xrRepresent that the class center O of certain classification, supporting vector collection center R, sample A to be divided are corresponding respectively
Variable,It is the kernel function in SVM.
Compared to existing technology, the present invention includes advantages below and beneficial effect：
(1) it is proposed by the present invention to be eliminated according to the Weak Classifier for knowing similarity minimum classification in area and being based on projecting method, and most
Whole decision-making is based on the two-stage classification mechanism that the strong classifier of support vector machine method determines, has broken existing single based on Euclidean
The Weak Classifier method of sample belonging kinds in rejection area is determined apart from size, decision accuracy is improve；
(2) the superseded method based on projection proposed by the present invention, because supporting vector sample size is few, its amount of calculation is less than base
The amount of calculation of Euclidean distance between sample to be divided and learning sample collection, accelerates the decision-making speed of rejection area sample belonging kinds to be divided
Degree；
(3) sorting technique of the invention compared to current rejection area recognize categorised decision process end mechanism,
The mechanism that projecting method is enabled certainly in the middle of categorised decision process, with the nothing with many sorting technique decision processes of SVMs
Seam docking advantage, further increases automaticity of the multi-class classification method in cases of engineering.
Brief description of the drawings
Fig. 1 is class center of the invention, supporting vector collection center, sample triadic relation schematic diagram to be divided；
Fig. 2 is the multi-class classification method workflow diagram based on projection with SVMs of the invention；
Fig. 3 is the m classification two based on projection with multi-class classification method one embodiment of SVMs of the invention
Two build two grader schematic diagrames.
Specific embodiment
Below in conjunction with the accompanying drawings, to it is proposed by the present invention it is a kind of based on projection carried out with the multi-class classification method of SVMs
Describe in detail.
Fig. 1 is angle between class center of the invention, supporting vector collection center, sample relation schematic diagram to be divided, two vectors
Triangle cosine embody the position relationship of three, projected size then embodies the far and near journey of sample convergence class center to be divided
Degree.
As shown in Fig. 2 a kind of multi-class classification method based on projection with SVMs of the invention, including following step
Suddenly：
1) one-to-one support vector machine classifier is built between classification two-by-two, sample to be divided is input to grader, uses a pair
One support vector machine classifier method is treated a point sample and is classified, and adds up number of votes obtained of all categories.
2) number of votes obtained of all categories is counted, the classification number of highest number of votes obtained and highest gained vote is asked for, compares highest gained vote
Size between classification number and the classification number for participating in ballot.
3) if the classification number of highest number of votes obtained is identical with the classification number for participating in ballot, following steps are carried out：
311) sample to be divided and corresponding classification supporting vector are calculated and concentrates vector in the heart to sample to be divided and class center
Between vector projection；
312) relatively more each projection value size, eliminates the maximum classification of projection value；
313) each parameter is reinitialized：Number of votes obtained assignment 0 of all categories, the classification number for participating in ballot subtracts 1；
314) by sample to be divided be input to remaining highest gained vote classification between the one-to-one SVMs point that constitutes two-by-two
Additional ballot is carried out in class device, then is transferred to step 2).
If the classification number of highest gained vote is not unique and less than the classification number for participating in ballot, following steps are carried out：
321) each parameter is reinitialized：Number of votes obtained assignment 0 of all categories, the classification number for participating in ballot is entered as number of votes obtained
Highest classification number；
322) partner a support vector machine classifier two-by-two between the classification of highest gained vote, and continuation will sample input be divided
One-to-one support vector machine classifier to firm composition carries out additional ballot.Then it is transferred to step 2).
If highest votes are unique, terminate differentiating, number of votes obtained highest class is the belonging kinds of sample to be divided.
Further, the step 1) following steps can be taken to realize：
11) initialization number of votes obtained of all categories and number of votes obtained highest classification number are 0；The classification number for participating in ballot is entered as
Classification number；
12) by the most optimal sorting of one-to-one support vector machine classifier between the acquisition of SVMs learning method two-by-two classification
Class decision function；
13) for each grader, the supporting vector collection of two classifications of all one-to-one support vector machine classifiers is calculated
Center and class center；
14) sample to be divided is input in a pair of support vector machine classifier；If sample to be divided belongs to certain class, should
The number of votes obtained of classification adds 1；Add up number of votes obtained of the sample to be divided in of all categories.
Another kind is improved, the step 311) calculate sample divide and corresponding classification supporting vector concentration vector in the heart and arrive
The process of the projection of vector is between sample to be divided and class center：
Assuming that O is the class center of certain classification, R is the supporting vector collection center of the category, and A is the sample to be divided near R,
Definition vectorTo vectorBe projected as：
When data set linearly inseparable, using nonlinear transformationBy sample from luv space
After being mapped to high-dimensional feature space, projective representation is：
Wherein,It is the kernel function in SVM.
Below by example, it is further elaborated with reference to accompanying drawing.Using from the speed in the type flight test of unmanned aerial vehicle
Rate output from Gyroscope, as normal signal；Again from fault case storehouse, 4 kinds of fault types are chosen (biasing failure, stuck
Failure, drifting fault, multiplying property failure) data are tested.
1) support vector machine classifier is built between classification two-by-two.Sample to be divided is input to grader, uses one-against-one
Device method is treated a point sample and is classified, and adds up number of votes obtained of all categories.
Step 1) it is subdivided into following 4 steps：
11) m (m=5 in the specific embodiment) individual classification number, then have m (m-1)/2 grader ballot, construction method
See Fig. 3.The m number of votes obtained V of classification of initializationk(k=1,2 ..., m) with number of votes obtained highest classification number nh0 is, is participated in a pair
The classification number n of one ballotaIt is m；
12) the optimal classification decision function of grader is obtained by SVMs learning method between classification two-by-two, optimal
Categorised decision function is defined as grader Cij(i=1,2 ..., m-1；J ≠ i, j=2,3 ..., m)；
13) class center of all categories and supporting vector collection center are calculated；
14) sample x to be divided is input to the grader Cij(i=1,2 ..., m-1；J ≠ i, j=2,3 ..., m) in, if
X belongs to class i, then the number of votes obtained V of class iiPlus 1；If x belongs to class j, the number of votes obtained V of class jjPlus 1.Add up sample to be divided of all categories
In number of votes obtained.
2) number of votes obtained of all categories is counted, highest number of votes obtained argMax (V are asked fork), and the classification number n that highest is won the voteh.Than
Compared with the classification number n that highest is won the votehWith the classification number n for participating in ballotaSize.
3) n is worked ash=na, then claim sample to be divided that decision-making blind area is generated in these graders：
311) sample to be divided and corresponding classification supporting vector are calculated and concentrates vector in the heart to sample to be divided and class center
Between vector projection；
312) compared projections value size, eliminates the maximum classification of projection value；
313) number of votes obtained V is reinitializedk(k=1,2 ..., m)=0, and the classification number n for taking part in a votea=nh-1；
314) by sample to be divided be input to remaining highest gained vote classification between carry out additional ballot in the grader that constitutes.Again
It is transferred to step 2.
As ((nh≠1)&(nh<na)) when：
321) following parameter assignment is reinitialized：Vk(k=1,2 ..., m)=0, number of votes obtained highest classification number is assigned to
na；
322) two graders are constituted two-by-two between the classification of highest gained vote, continue that sample to be divided is input to two points of firm composition
Class device carries out additional ballot.Then it is transferred to step 2.
Work as nh=1, i.e. highest votes are unique, terminate differentiating, number of votes obtained highest class is the ownership class of sample x to be divided
Not.
Based on description of the preferred embodiment of the present invention, it should be apparent that the sheet being defined by the appended claims
Invention is not limited only to the specific detail that is illustrated in specification above, without departing from present inventive concept or scope to this hair
Bright many obviously change equally possible reaches the purpose of the present invention.
Claims (3)
1. it is a kind of based on the multi-class classification method projected with SVMs, it is characterised in that to comprise the following steps：
1) one-to-one support vector machine classifier is built between classification two-by-two, sample to be divided is input to grader, uses one-to-one branch
Hold vector machine classifier method and treat a point sample and classified, add up number of votes obtained of all categories；
2) number of votes obtained of all categories is counted, the classification number of highest number of votes obtained and highest gained vote is asked for, compares the classification of highest gained vote
Size between number and the classification number for participating in ballot；
3) if the classification number of highest number of votes obtained is identical with the classification number for participating in ballot, following steps are carried out：
311) calculate sample to be divided and corresponding classification supporting vector concentrate vector in the heart between sample to be divided and class center to
The projection of amount；
312) relatively more each projection value size, eliminates the maximum classification of projection value；
313) each parameter is reinitialized：Number of votes obtained assignment 0 of all categories, the classification number for participating in ballot subtracts 1；
314) by sample to be divided be input to remaining highest gained vote classification between the one-to-one support vector machine classifier that constitutes two-by-two
In carry out additional ballot, then be transferred to step 2)；
If the classification number of highest gained vote is not unique and less than the classification number for participating in ballot, following steps are carried out：
321) each parameter is reinitialized：Number of votes obtained assignment 0 of all categories, the classification number for participating in ballot is entered as number of votes obtained highest
Classification number；
322) partner a support vector machine classifier two-by-two between the classification of highest gained vote, and be input to sample to be divided just by continuation
The one-to-one support vector machine classifier of composition carries out additional ballot；Then it is transferred to step 2)；
If highest votes are unique, terminate differentiating, number of votes obtained highest class is the belonging kinds of sample to be divided.
2. it is according to claim 1 based on the multi-class classification method projected with SVMs, it is characterised in that the step
It is rapid 1) following steps to be taken to realize：
11) initialization number of votes obtained of all categories and number of votes obtained highest classification number are 0；The classification number for participating in ballot is classification number；
12) determined by the optimal classification of one-to-one support vector machine classifier between the acquisition of SVMs learning method two-by-two classification
Plan function；
13) for each grader, the supporting vector collection center of two classifications of all one-to-one support vector machine classifiers is calculated
With class center；
14) sample to be divided is input in a pair of support vector machine classifier；If sample to be divided belongs to certain class, the category
Number of votes obtained add 1；Add up number of votes obtained of the sample to be divided in of all categories.
3. it is according to claim 1 based on the multi-class classification method projected with SVMs, it is characterised in that the step
Rapid sample to be divided and the corresponding classification supporting vector of 311) calculating concentrates vector in the heart to arrive vectorial between sample to be divided and class center
The process of projection be：
Assuming that O is the class center of certain classification, R is the supporting vector collection center of the category, and A is the sample to be divided near R,
Definition vectorTo vectorBe projected as：
When data set linearly inseparable, using nonlinear transformationSample is mapped to from luv space
After high-dimensional feature space, projective representation is：
In, xo, xa, xrClass center O, supporting vector collection center R, the corresponding variables of sample A to be divided of certain classification are represented respectively,It is the kernel function in SVM.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201611189741.4A CN106778875A (en) | 2016-12-21 | 2016-12-21 | A kind of multi-class classification method based on projection with SVMs |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201611189741.4A CN106778875A (en) | 2016-12-21 | 2016-12-21 | A kind of multi-class classification method based on projection with SVMs |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN106778875A true CN106778875A (en) | 2017-05-31 |
Family
ID=58896676
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201611189741.4A Pending CN106778875A (en) | 2016-12-21 | 2016-12-21 | A kind of multi-class classification method based on projection with SVMs |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106778875A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111351516A (en) * | 2018-12-21 | 2020-06-30 | 波音公司 | Sensor fault detection and identification using residual fault pattern recognition |
-
2016
- 2016-12-21 CN CN201611189741.4A patent/CN106778875A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111351516A (en) * | 2018-12-21 | 2020-06-30 | 波音公司 | Sensor fault detection and identification using residual fault pattern recognition |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Singla et al. | Understanding failures of deep networks via robust feature extraction | |
| Fridman et al. | Arguing machines: Human supervision of black box AI systems that make life-critical decisions | |
| CN100458831C (en) | Human face model training module and method, human face real-time certification system and method | |
| Verma et al. | Cluster-oriented ensemble classifier: Impact of multicluster characterization on ensemble classifier learning | |
| Verma et al. | Age group predictive models for the real time prediction of the university students using machine learning: Preliminary results | |
| Wahono et al. | Metaheuristic optimization based feature selection for software defect prediction. | |
| CN107830996B (en) | Fault diagnosis method for aircraft control surface system | |
| CN104239858A (en) | Method and device for verifying facial features | |
| CN102156871A (en) | Image classification method based on category correlated codebook and classifier voting strategy | |
| Morales et al. | LAMDA-HAD, an Extension to the LAMDA Classifier in the Context of Supervised Learning | |
| CN103745213A (en) | Optical character recognition method based on LVQ neural network | |
| CN110135327A (en) | A kind of driving behavior recognition methods based on multi-region feature learning model | |
| Rokhana et al. | Multi-class image classification based on mobilenetv2 for detecting the proper use of face mask | |
| Deepak et al. | Identification of fake notes and denomination recognition | |
| CN106778875A (en) | A kind of multi-class classification method based on projection with SVMs | |
| Ghahnavieh et al. | Enhancing the license plates character recognition methods by means of SVM | |
| Çetinkaya et al. | Driver activity recognition using deep learning and human pose estimation | |
| CN106682604A (en) | Method for detecting blurred image based on deep learning | |
| Wanas et al. | Feature-based decision aggregation in modular neural network classifiers | |
| Agarwal et al. | Towards deeper neural networks for Fast Radio Burst detection | |
| CN110503205A (en) | A kind of multi-modal data processing method and system | |
| Nunes et al. | Dynamic feature selection for classifier ensembles | |
| Fonseca et al. | Optical character recognition using automatically generated Fuzzy classifiers | |
| Thamizhkkanal et al. | A Neural Based Approach to Evaluate an Answer Script | |
| Poh et al. | Making better biometric decisions with quality and cohort information: A case study in fingerprint verification |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20170531 |