US20160162757A1 - Multi-class object classifying method and system - Google Patents
Multi-class object classifying method and system Download PDFInfo
- Publication number
- US20160162757A1 US20160162757A1 US14/562,787 US201414562787A US2016162757A1 US 20160162757 A1 US20160162757 A1 US 20160162757A1 US 201414562787 A US201414562787 A US 201414562787A US 2016162757 A1 US2016162757 A1 US 2016162757A1
- Authority
- US
- United States
- Prior art keywords
- images
- characteristic
- parameter ranges
- classifier
- classes
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 55
- 238000013507 mapping Methods 0.000 claims abstract description 31
- 238000004422 calculation algorithm Methods 0.000 claims description 13
- 238000012706 support-vector machine Methods 0.000 claims description 13
- 239000000284 extract Substances 0.000 claims description 10
- 230000006870 function Effects 0.000 description 9
- 238000010586 diagram Methods 0.000 description 7
- 238000004364 calculation method Methods 0.000 description 4
- 230000008569 process Effects 0.000 description 4
- 238000013473 artificial intelligence Methods 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000004590 computer program Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000008707 rearrangement Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
-
- G06K9/6256—
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/243—Classification techniques relating to the number of classes
- G06F18/2431—Multiple classes
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/245—Classification techniques relating to the decision surface
- G06F18/2451—Classification techniques relating to the decision surface linear, e.g. hyperplane
-
- G06K9/4609—
-
- G06K9/4642—
-
- G06K9/6228—
-
- G06K9/6277—
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/20—Linear translation of whole images or parts thereof, e.g. panning
-
- G06T5/003—
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/40—Image enhancement or restoration using histogram techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10004—Still image; Photographic image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20172—Image enhancement details
- G06T2207/20201—Motion blur correction
Definitions
- the disclosure relates to a classifying technology. More particularly, the disclosure relates to a multi -classifying object classifying method and system.
- the multi-class object classification often directly uses multi-class classification, or binary classification to solve a lot of binary classification problems divided from the multi-class classification.
- Directly performing multi-class classification for example: boosting algorithm, uses multiple stage voting to gradually classify the target object by analyzing all the representative points of the input images; because the involved calculation is more complicated, more time-consuming and less accurate, the multi-class classification is not a good solution; however, the binary classification often encounters problems that there are too many pending characteristic points of images and the time of calculation is excessively long when different objects are in a near distance, so there is still room for improvements.
- conventionally binary classification can merely distinguish a single class through a process of classification in optimism condition; if the number of the classes is huge, the number of the classification process has to increase to distinguish every class and results in a lower efficiency of the classification.
- an aspect of the present disclosure provides a multi-class object classifying method, which including the following steps: (a) a plurality of classes, a plurality of first training images and a plurality of second training images are received and stored, and a plurality of first characteristic images are extracted from the first training images; the first training images is used to generate a plurality of classifiers corresponding to the classes through a linear mapping classifying method, wherein the first training images and the second training images respectively correspond to the classes; (b) a plurality of second characteristic images are extracted from the second training images and a classifier is selected from the classifiers; (c) the classifier and the second characteristic images are used to determine a plurality of parameter ranges corresponding to the classes and a threshold corresponding to the classifier; (d) when two of the parameter ranges overlap, the remaining parameter ranges are recorded except for the two parameter ranges; after another classifier is selected from the classifiers except for the classifier that has
- step (a) further includes: the first characteristic images are divided into a plurality of characteristic blocks, and a plurality of variances of the characteristic blocks is calculated; a plurality of characteristic variances in decrement order beginning with a maximum of the variances and the characteristic blocks corresponding to the characteristic variances are selected; the characteristic blocks are used to generate a plurality of classifiers corresponding to the classes through the linear mapping classifying method.
- the linear mapping classifying method is support vector machine (SVM) algorithm
- the parameter ranges and the threshold are all between ⁇ 1 and 1.
- the steps (a), (b), and (f) include: the first characteristic images are extracted from the first training images through histogram of oriented gradients (HOG); the second characteristic images are extracted from the second training images through histogram of oriented gradients: the pending characteristic image is extracted from the pending image through histogram of oriented gradients.
- HOG histogram of oriented gradients
- step (d) includes: when the two parameter ranges overlap, the remaining parameter ranges are recorded except for the two parameter ranges; after one of the two classifiers corresponding to the two parameter ranges is selected as said another classifier, the step (c) is repeated until the parameter ranges don't overlap with each other.
- the first training images include a plurality of positive sample images and a plurality of negative sample images; there is an object respectively corresponding to the classes in the positive sample images, and there is a lack of the object respectively corresponding to the classes in the negative sample images.
- a multi-class object classifying system which includes an input device, a database and a processor, wherein the database is coupled to the input device and the processor is coupled to the database.
- the input device receives a plurality of classes, a plurality of first training images, a plurality of second training images and a pending image, wherein the first training images and the second training images respectively correspond to the classes.
- the database stores the classes, the first training images, the second training images and the pending image.
- the processor executes following steps: (a) a plurality of first characteristic images are extracted from the first training images; the first training images are used to generate a plurality of classifiers corresponding to the classes through a linear mapping classifying method, wherein the first training images and the second training images respectively correspond to the classes; (b) a plurality of second characteristic images are extracted from the second training images and a classifier is selected from the classifiers; (c) the classifier and the second characteristic images are used to determine a plurality of parameter ranges corresponding to the classes and a threshold corresponding to the classifier; (d) when two of the parameter ranges overlap, the remaining parameter ranges are recorded except for the two parameter ranges; after another classifier is selected from the classifiers except for the classifier that has been selected, the step (c) is repeated until the parameter ranges don't overlap with each other; (e) when the parameter ranges don't overlap with each other, the parameter ranges are recorded; (f) a pending image is received and stored and a pending characteristic image is
- the processor divides the first characteristic images into a plurality of characteristic blocks, calculates a plurality of variances of the characteristic blocks, selects a plurality of characteristic variances in decrement order beginning with a maximum of the variances and the characteristic blocks corresponding to the characteristic variances, and uses the characteristic blocks to generate a plurality of classifiers corresponding to the classes through the linear mapping classifying method.
- the linear mapping classifying method is support vector machine algorithm
- the parameter ranges and the threshold are all between ⁇ 1 and 1.
- the processor extracts the first characteristic images from the first training images through histogram of oriented gradients, extracts the second characteristic images from the second training images through histogram of oriented gradients, and extracts the pending characteristic image from the pending image through histogram of oriented gradients.
- the processor records the remaining parameter ranges except for the two parameter ranges; after selecting one of the two classifiers corresponding to the two parameter ranges as said another classifier, the processor repeats the step (c) until the parameter ranges don't overlap with each other.
- the first training images include a plurality of positive sample images and a plurality of negative sample images; there is an object respectively corresponding to the classes in the positive sample images, and there is a lack of the object respectively corresponding to the classes in the negative sample images.
- Another aspect of the present disclosure provides a non-transitory computer-readable storage medium storing a program that is loaded and executed by a computer, performs a multi-class object classifying method, which includes: (a) a plurality of classes a plurality of first training images and a plurality of second training images are received and stored, and a plurality of first characteristic images are extracted from the first training images; the first training images is used to generate a plurality of classifiers corresponding to the classes through a linear mapping classifying method, wherein the first training images and the second training images respectively correspond to the classes; (b) a plurality of second characteristic images are extracted from the second training images and a classifier is selected from the classifiers; (c) the classifier and the second characteristic images are used to determine a plurality of parameter ranges corresponding to the classes and a threshold corresponding to the classifier; (d) when two of the parameter ranges overlap, the remaining parameter ranges are recorded except for the two parameter ranges; after another classifier is selected from the classifiers except for
- step (a) further includes: the first characteristic images are divided into a plurality of characteristic blocks, and a plurality of variances of the characteristic blocks is calculated; a plurality of characteristic variances in decrement order beginning with a maximum of the variances and the characteristic blocks corresponding to the characteristic variances are selected; the characteristic blocks are used to generate a plurality of classifiers corresponding to the classes through the linear mapping classifying method.
- the linear mapping classifying method is support vector machine algorithm
- the parameter ranges and the threshold are all between ⁇ 1 and 1.
- the steps (a), (b), and (f) include: the first characteristic images are extracted from the first training images through histogram of oriented gradients; the second characteristic images are extracted from the second training images through histogram of oriented gradients; the pending characteristic image is extracted from the pending image through histogram of oriented gradients.
- step (d) includes: when the two parameter ranges overlap, the remaining parameter ranges are recorded except for the two parameter ranges; after one of the two classifiers corresponding to the two parameter ranges is selected as said another classifier, the step (c) is repeated until the parameter ranges don't overlap with each other.
- the first training images include a plurality of positive sample images and a plurality of negative sample images; there is an object respectively corresponding to the classes in the positive sample images, and there is a lack of the object respectively corresponding to the classes in the negative sample images.
- the present disclosure originates from efficiency improvements of the binary classifying method with regard to multi-class object classification and achieves multi-class object classification with less classifying steps by recording the parameter ranges that are generated from training images of different classes and one of the classifiers; moreover, the present disclosure reduces the sample number, saves the operation time by dividing the characteristic images into image blocks and selecting the image blocks with higher variances as representative to generate classifiers through the linear mapping classifying method, and apparently improves the efficiency of the multi-class object classification.
- FIG. 1A-1B are flow charts of a multi-class object classifying method according to the first embodiment of the present disclosure
- FIG. 2 is a schematic diagram of parameter ranges determined by the second training images respectively corresponding to the classes A-D and the parameters of classifier A;
- FIG. 3 is a schematic diagram of parameter ranges determined by the second training images respectively corresponding to the classes A-D and the parameters of classifier B;
- FIG. 4 is a schematic diagram of a multi-class object classifying system according to the second embodiment of the present disclosure.
- FIG. 1A-1B show flow charts of a multi-class object classifying method 100 according to the first embodiment of the present disclosure, and point A is a connection point of steps in FIG. 1A and FIG. 1B .
- the multi-class object classifying method 100 can be applied to a multi-class object classifying system 400 as shown in FIG. 4 .
- the multi-class object classifying method 100 can be implemented as a computer program stored in a computer-readable medium, which is loaded by a computer to make the computer execute the multi-class object classifying method.
- the non-transitory computer-readable medium can be read only memory (ROM), flash memory, soft disk, hard disk, optical disk, pen drive, magnetic tape, network accessible database, or other computer-readable medium with the same function that are obvious for those skilled in the art.
- ROM read only memory
- flash memory soft disk
- hard disk hard disk
- optical disk pen drive
- magnetic tape magnetic tape
- network accessible database or other computer-readable medium with the same function that are obvious for those skilled in the art.
- step S 102 classes, first training images and second training images are received and stored; wherein the first training images and the second training images respectively correspond to the classes.
- users can define classes A, B, C and D, and every class respectively has first training images and second training images.
- First characteristic images are extracted from the first training images to execute the first training stage and generate classifiers; wherein a linear mapping classifying method is used in the first training stage and the number of the classifiers equals to the number of the classes.
- the generated classifiers are classifiers A-D corresponding to classes A-D, and the classifiers have functions of distinguishing corresponding classes from non-corresponding classes.
- classifier A can be used to distinguish class A from non-class A, and so on.
- the linear mapping classifying method can be support vector machine algorithm, Fisher's liner discriminant, or naive Bayes classifier.
- step S 104 second characteristic images are extracted from the second training images and a classifier is selected from the classifiers to execute the second training stage.
- step S 106 the classifier and the second characteristic images are used to determine parameter ranges corresponding to the classes and a threshold corresponding to the classifier; for example, the parameter ranges are all between ⁇ 1 and 1.
- the threshold can be defined as a standard of being classified into the class according to the demands; for example, the threshold of classifier A is defined as 0.8; class A is determined when a parameter is larger than 0.8; non-class A is determined when a parameter is smaller than 0.8.
- the above-mentioned parameter ranges are determined through parameters calculated from characteristic values of second characteristic images in a class and parameters of the classifier.
- the equation is as follows:
- S i is the characteristic value of the second characteristic images
- V i , ⁇ i , ⁇ are all the parameters of the selected classifier
- Sum is the calculated parameter. Therefore, second characteristic images generate parameters through calculation and forms a parameter range.
- Parameter ranges corresponding to the classes are generated from the characteristic values of the second characteristic images corresponding to the classes and parameters of a classifier through the above-mentioned equation. As shown in FIG. 2 , there are corresponding relation between the parameter ranges and the classes, wherein the parameter range corresponding to the same class as the classifier is nearest to 1.
- step S 108 the parameter ranges are checked whether the parameter ranges overlap; when two of the parameter ranges overlap, the remaining parameter ranges are recorded except for the two parameter ranges; after another classifier is selected from the classifiers except for the classifier that has been selected, the step S 106 is repeated until the parameter ranges don't overlap with each other (step S 110 ); when the parameter ranges don't overlap with each other, the parameter ranges are recorded (step S 112 ).
- FIG. 2 is a schematic diagram of parameter ranges determined by the second training images respectively corresponding to the classes A-D and the parameters of classifier A.
- a parameter range 202 corresponding to class A is between the threshold and 1, parameter ranges 204 - 208 corresponding to classes B-D are between ⁇ 1 and the threshold.
- the parameter range 204 corresponding to class B overlaps with the parameter range 206 corresponding to class C, which indicates that classifier A can't distinguish class B from class C appropriately; therefore, the distinguishable parameter range 202 corresponding to class A and parameter range 208 corresponding to class D recorded; after one of the classifiers B-D is selected except for classifier A, the step S 106 is repeated until all the parameter ranges don't overlap with each other and the non-overlapping parameter ranges are recorded.
- FIG. 3 is a schematic diagram of parameter ranges determined by the second training images respectively corresponding to the classes A-D and the parameters of classifier B, and all the parameter ranges 302 - 308 don't overlap; therefore the classifier B has a function of distinguishing classes A-D after recording the parameter ranges.
- any of the classifiers A-D has a potential of distinguishing classes A-D after recording the parameter ranges and improved capability of classification compared to the classifiers generated in the first training stage.
- Detecting stage starts from step S 114 , a pending image is received and stored, and a pending characteristic image is extracted from the pending image; a classifier and the pending image are used to determine a pending parameter; in step S 116 the pending parameter is checked whether it is larger than the threshold corresponding to the classifier; when the pending parameter is larger than the threshold corresponding to the classifier, the pending image is classified into the class corresponding to the classifier (step S 118 ); when the pending parameter is smaller than the threshold corresponding to the classifier, in step S 120 , the pending parameter s checked whether it lies in the parameter ranges corresponding to the classes; when the pending parameter lies in the parameter ranges, the pending image is classified into the classes corresponding to the parameter ranges.
- parameters of classifier A and the pending characteristic image are used to determine the pending parameter through the above-mentioned equation.
- the pending parameter is larger than the threshold of classifier A
- the pending image is classified into the class A: when the pending parameter is smaller than the threshold of classifier A and lies in the parameter range of class B, then the pending image is classified into class B; when the pending parameter is smaller than the threshold of classifier A and lies in the parameter range of class C, then the pending image is classified into class C.
- the classifiers A-D have functions of distinguishing not only the classes corresponding to the classifiers but also other classes, and therefore a multi-class object classification with less steps and higher efficiency can be achieved.
- the first characteristic images are divided into characteristic blocks, and variances of the characteristic blocks is calculated.
- Characteristic variances in decrement order beginning with a maximum of the variances and the characteristic blocks corresponding to the characteristic variances are selected.
- the characteristic blocks with large variances are selected to represent for the image, and then the characteristic blocks is used to generate classifiers corresponding to the classes through the linear mapping classifying method.
- the number of selected characteristic blocks is adjustable according to the demands so as to reduce the number of samples needed for generating the classifiers, and then the efficiency of operation is improved.
- the linear mapping classifying method is support vector machine algorithm.
- the first training images corresponding to classes are inputted into the support vector machine algorithm to determine hyperplanes that can distinguish corresponding classes from non-corresponding classes and then generate the classifiers corresponding to the classes.
- characteristic images are extracted through histogram of oriented gradients; namely, the first characteristic images, the second characteristic images and the pending characteristic image are respectively extracted from the first training images, the second training images and the pending image through histogram of oriented gradients.
- the step S 106 is repeated until the parameter ranges don't overlap with each other. For example, as shown in FIG.
- the parameter range 202 corresponding to class A is between the threshold and 1
- the parameter ranges corresponding to B-D are between ⁇ 1 and the threshold
- the parameter range 204 corresponding to class B overlaps with the parameter range 206 corresponding to class C, which indicates that classifier A can't distinguish class B from class C appropriately; therefore, the distinguishable parameter range 202 corresponding to class A and parameter range 208 corresponding to class D are recorded, one of the classifiers B and C is selected, the step S 106 is repeated until all the parameter ranges don't overlap with each other and the non-overlapping parameter ranges are recorded. Directly selecting from the classifiers B and C with overlapped parameter ranges is a faster way to find out the classifiers that can distinguish class B from class C.
- the first training images include positive sample images and negative sample images; there is an object respectively corresponding to the classes in the positive sample images, and there is a lack of the object respectively corresponding to the classes in the negative sample images.
- images with objects of class A is taken as the positive sample images, and images without objects of class A is taken as the negative sample images; therefore, images merely with objects of classes B-D can also be taken as the negative sample images to generate classifier A.
- the positive sample images and the negative sample images are inputted into the linear mapping classifying method to improve the recognizing functions of the generated classifiers.
- FIG. 4 shows a multi-class object classifying system 400 according to the second embodiment of the present disclosure.
- the multi-class object classifying system 400 includes an input device 410 , a database 420 and a processor 430 .
- the input device 410 can be transmission port (for example: USB, GPIB, or other interfaces).
- the database 420 can be a hard disk, flash memory, or other storing media; the processor 430 can be CPUs, microprocessors or other circuits.
- the input device 410 receives and stores user-defined classes 442 , first training images 444 , second training images 446 in the database 420 ; wherein the first training images 444 and the second training images 446 respectively correspond to the classes 442 .
- users can define classes A, B, C and D, and every class respectively has first training images 444 and second training images 446 .
- First characteristic images is extracted from the first training images 444 to execute the first training stage and generate classifiers; wherein a linear mapping classifying method is used in the first training stage and the number of the classifiers equals to the number of the classes 442 .
- the generated classifiers are classifiers A-D corresponding to classes A-D, and the classifiers have functions of distinguishing corresponding classes from non-corresponding classes.
- classifier A can be used to distinguish class A from non-class A, and so on.
- the linear mapping classifying method can be support vector machine algorithm, Fishers liner discriminant, or naive Bayes classifier.
- the processor 430 extracts second characteristic images from the second training images 446 and selects a classifier from the classifiers to execute the second training stage.
- the processor 430 uses the classifier and the second characteristic images to determine parameter ranges corresponding to the classes 442 and a threshold corresponding to the classifier; for example, the parameter ranges are all between ⁇ 1 and 1.
- the threshold can be defined as a standard of being classified into the class according to the demands; for example, the threshold of classifier A is defined as 0.8; class A is determined when a parameter is larger than 0.8; non-class A is determined when a parameter is smaller than 0.8.
- the above-mentioned parameter ranges are determined by the processor 430 through parameters calculated from characteristic values of second characteristic images in a class and parameters of the classifier.
- the equation is as follows:
- S i is the characteristic value of of the second characteristic images
- V i , ⁇ i , ⁇ are all the parameters of the selected classifier
- Sum is the calculated parameter. Therefore, second characteristic images generates parameters through calculation and forms a parameter range.
- Parameter ranges corresponding to the classes is generated from the characteristic values of the second characteristic images corresponding to the classes and parameters of a classifier through the above-mentioned equation. As shown in FIG. 2 , there are corresponding relation between the parameter ranges and the classes, wherein the parameter range corresponding to the same class as the classifier is nearest to 1.
- the processor 430 checks the parameter ranges whether the parameter ranges overlap; when two of the parameter ranges overlap, the processor 430 records the remaining parameter ranges except for the two parameter ranges; after selecting another classifier from the classifiers except for the classifier that has been selected, the processor 430 repeats execution until the parameter ranges don't overlap with each other (step S 110 ) when the parameter ranges don't overlap with each other, the processor 430 records the parameter ranges (step S 112 ).
- FIG. 2 is a schematic diagram of parameter ranges determined by the second training images respectively corresponding to the classes A-D and the parameters of classifier A.
- a parameter range 202 corresponding to class A is between the threshold and 1, parameter ranges 204 - 208 corresponding to classes B-D are between ⁇ 1 and the threshold.
- the parameter range 204 corresponding to class B overlaps with the parameter range 206 corresponding to class C, which indicates that classifier A can't distinguish class B from class C appropriately; therefore, the processor 430 records the distinguishable parameter range 202 corresponding to class A and parameter range 208 corresponding to class D; after selecting one of the classifiers B-D except for classifier A, the processor 430 repeats execution until all the parameter ranges don't overlap with each other and records the non-overlapping parameter ranges.
- FIG. 3 is a schematic diagram of parameter ranges determined by the second training images respectively corresponding to the classes A-D and the parameters of classifier B, and all the parameter ranges 302 - 308 don't overlap; therefore, the classifier B has a function of distinguishing classes A-D after the processor 430 records the parameter ranges.
- any of the classifiers A-D has a potential of distinguishing classes A-D after recording the parameter ranges and improved capability of classification compared to the classifiers generated in the first training stage.
- the processor 430 receives and stores a pending image 448 in the database 420 , extracts a pending characteristic image from the pending image 448 , and uses a classifier and the pending image to determine a pending parameter.
- the processor 430 checks the pending parameter whether t is larger than the threshold corresponding to the classifier; when the pending parameter is larger than the threshold corresponding to the classifier, the processor 430 classifies the pending image 448 into the class corresponding to the classifier; when the pending parameter is smaller than the threshold corresponding to the classifier, the processor 430 checks the pending parameter whether it lies in the parameter ranges corresponding to the classes 442 ; when the pending parameter lies in the parameter ranges, the processor 430 classifies the pending image 448 into the classes 442 corresponding to the parameter ranges.
- the processor 430 uses parameters of classifier A and the pending characteristic image to determine the pending parameter through the above-mentioned equation.
- the processor 430 classifies the pending image 448 into the class A; when the pending parameter is smaller than the threshold of classifier A and lies in the parameter range of class B, then the processor 430 classifies the pending image 448 into class B; when the pending parameter is smaller than the threshold of classifier A and lies in the parameter range of class C, then the processor 430 classifies the pending image 448 into class C.
- the classifiers A-D have functions of distinguishing not only the classes corresponding to the classifiers but also other classes, and therefore a multi-class object classification with less steps and higher efficiency can be achieved.
- the processor 430 divides the first characteristic images into characteristic blocks, and calculates variances of the characteristic blocks.
- the processor 430 selects characteristic variances in decrement order beginning with a maximum of the variances and the characteristic blocks corresponding to the characteristic variances.
- the processor 430 selects the characteristic blocks with large variances to represent for the image, and then uses the characteristic blocks to generate classifiers corresponding to the classes through the linear mapping classifying method.
- the number of selected characteristic blocks is adjustable according to the demands so as to reduce the number of samples needed for generating the classifiers, and then the efficiency of operation is improved.
- the linear mapping classifying method is support vector machine algorithm.
- the first training images corresponding to classes are inputted into the support vector machine algorithm to determine hyperplanes that can distinguish corresponding classes from non-corresponding classes and then the processor 430 generates the classifiers corresponding to the classes.
- the processor 430 extracts characteristic images through histogram of oriented gradients; namely, the processor 430 respectively extracts the first characteristic images, the second characteristic images and the pending characteristic image from the first training images 444 , the second training images 446 and the pending image 448 through histogram of oriented gradients.
- the processor 430 when the two parameter ranges overlap, the processor 430 records the remaining parameter ranges except for the two parameter ranges; after selecting one of the two classifiers corresponding to the two parameter ranges as said another classifier, the processor 430 repeats execution until the parameter ranges don't overlap with each other. For example, as shown in FIG. 2 , the parameter range 202 corresponding to class A is between the threshold and 1, and the parameter ranges corresponding to B-D are between ⁇ 1 and the threshold.
- the parameter range 204 corresponding to class B overlaps with the parameter range 206 corresponding to class C, which indicates that classifier A can't distinguish class B from class C appropriately; therefore, the processor 430 records the distinguishable parameter range 202 corresponding to class A and parameter range 208 corresponding to class D, selects one of the classifiers B and C, repeats execution until all the parameter ranges don't overlap with each other and records the non-overlapping parameter ranges. Directly selecting from the classifiers B and C with overlapped parameter ranges is a faster way to find out the classifiers that can distinguish class B from class C.
- the first training images include positive sample images and negative sample images; there is an object respectively corresponding to the classes in the positive sample images, and there is a lack of the object respectively corresponding to the classes in the negative sample images.
- the processor 430 takes images with objects of class A as the positive sample images and takes images without objects of class A as the negative sample images; therefore, images merely with objects of classes B-D can also be taken as the negative sample images to generate classifier A.
- the positive sample images and the negative sample images are inputted into the linear mapping classifying method to improve the recognizing functions of the classifiers generated by the processor 430 .
- the present disclosure can record the parameter ranges that are generated from training images of different classes and one of the classifiers through the above-mentioned embodiments so as to achieve multi-class object classification with less classifying steps; moreover, the present disclosure reduces the sample number, saves the operation time by dividing the characteristic images into image blocks and selecting the image blocks with higher variances as representative to generate classifiers through the linear mapping classifying method, and apparently improves the efficiency of the multi-class object classification.
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- General Physics & Mathematics (AREA)
- Physics & Mathematics (AREA)
- Artificial Intelligence (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Computation (AREA)
- Life Sciences & Earth Sciences (AREA)
- Bioinformatics & Cheminformatics (AREA)
- General Engineering & Computer Science (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Computational Biology (AREA)
- Databases & Information Systems (AREA)
- Computing Systems (AREA)
- Health & Medical Sciences (AREA)
- Multimedia (AREA)
- Software Systems (AREA)
- General Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- Image Analysis (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
Description
- This application claims priority to China Application Serial Number 201410724928.4, filed Dec. 3, 2014, which is herein incorporated by reference.
- 1. Field of Invention
- The disclosure relates to a classifying technology. More particularly, the disclosure relates to a multi -classifying object classifying method and system.
- 2. Description of Related Art
- In the well-known pattern recognizing application of artificial intelligence or computer vision, the multi-class object classification often directly uses multi-class classification, or binary classification to solve a lot of binary classification problems divided from the multi-class classification. Directly performing multi-class classification, for example: boosting algorithm, uses multiple stage voting to gradually classify the target object by analyzing all the representative points of the input images; because the involved calculation is more complicated, more time-consuming and less accurate, the multi-class classification is not a good solution; however, the binary classification often encounters problems that there are too many pending characteristic points of images and the time of calculation is excessively long when different objects are in a near distance, so there is still room for improvements. In addition, conventionally binary classification can merely distinguish a single class through a process of classification in optimism condition; if the number of the classes is huge, the number of the classification process has to increase to distinguish every class and results in a lower efficiency of the classification.
- In order to improve the efficiency of the binary classifying method with regard to multi-class object classification, an aspect of the present disclosure provides a multi-class object classifying method, which including the following steps: (a) a plurality of classes, a plurality of first training images and a plurality of second training images are received and stored, and a plurality of first characteristic images are extracted from the first training images; the first training images is used to generate a plurality of classifiers corresponding to the classes through a linear mapping classifying method, wherein the first training images and the second training images respectively correspond to the classes; (b) a plurality of second characteristic images are extracted from the second training images and a classifier is selected from the classifiers; (c) the classifier and the second characteristic images are used to determine a plurality of parameter ranges corresponding to the classes and a threshold corresponding to the classifier; (d) when two of the parameter ranges overlap, the remaining parameter ranges are recorded except for the two parameter ranges; after another classifier is selected from the classifiers except for the classifier that has been selected, the step (c) is repeated until the parameter ranges don't overlap with each other; (e) when the parameter ranges don't overlap with each other, the parameter ranges are recorded; (f) a pending image is received and stored and a pending characteristic image is extracted from the pending image; a classifier and the pending image are used to determine a pending parameter; (g) when the pending parameter is larger than the threshold corresponding to the classifier, the pending image is classified into the class corresponding to the classifier; (h) when the pending parameter lies in the parameter ranges, the pending image is classified into the classes corresponding to the parameter ranges.
- In an embodiment of the present disclosure, wherein the step (a) further includes: the first characteristic images are divided into a plurality of characteristic blocks, and a plurality of variances of the characteristic blocks is calculated; a plurality of characteristic variances in decrement order beginning with a maximum of the variances and the characteristic blocks corresponding to the characteristic variances are selected; the characteristic blocks are used to generate a plurality of classifiers corresponding to the classes through the linear mapping classifying method.
- In an embodiment of the present disclosure, wherein the linear mapping classifying method is support vector machine (SVM) algorithm; the parameter ranges and the threshold are all between −1 and 1.
- In an embodiment of the present disclosure, wherein the steps (a), (b), and (f) include: the first characteristic images are extracted from the first training images through histogram of oriented gradients (HOG); the second characteristic images are extracted from the second training images through histogram of oriented gradients: the pending characteristic image is extracted from the pending image through histogram of oriented gradients.
- In an embodiment of the present disclosure, wherein the step (d) includes: when the two parameter ranges overlap, the remaining parameter ranges are recorded except for the two parameter ranges; after one of the two classifiers corresponding to the two parameter ranges is selected as said another classifier, the step (c) is repeated until the parameter ranges don't overlap with each other.
- In an embodiment of the present disclosure, wherein the first training images include a plurality of positive sample images and a plurality of negative sample images; there is an object respectively corresponding to the classes in the positive sample images, and there is a lack of the object respectively corresponding to the classes in the negative sample images.
- Another aspect of the present disclosure provides a multi-class object classifying system, which includes an input device, a database and a processor, wherein the database is coupled to the input device and the processor is coupled to the database. The input device receives a plurality of classes, a plurality of first training images, a plurality of second training images and a pending image, wherein the first training images and the second training images respectively correspond to the classes. The database stores the classes, the first training images, the second training images and the pending image. The processor executes following steps: (a) a plurality of first characteristic images are extracted from the first training images; the first training images are used to generate a plurality of classifiers corresponding to the classes through a linear mapping classifying method, wherein the first training images and the second training images respectively correspond to the classes; (b) a plurality of second characteristic images are extracted from the second training images and a classifier is selected from the classifiers; (c) the classifier and the second characteristic images are used to determine a plurality of parameter ranges corresponding to the classes and a threshold corresponding to the classifier; (d) when two of the parameter ranges overlap, the remaining parameter ranges are recorded except for the two parameter ranges; after another classifier is selected from the classifiers except for the classifier that has been selected, the step (c) is repeated until the parameter ranges don't overlap with each other; (e) when the parameter ranges don't overlap with each other, the parameter ranges are recorded; (f) a pending image is received and stored and a pending characteristic image is extracted from the pending image; a classifier and the pending image are used to determine a pending parameter; (g) when the pending parameter is larger than the threshold corresponding to the classifier, the pending image is classified into the class corresponding to the classifier; (h) when the pending parameter lies in the parameter ranges, the pending image is classified into the classes corresponding to the parameter ranges.
- In an embodiment of the present disclosure, wherein the processor divides the first characteristic images into a plurality of characteristic blocks, calculates a plurality of variances of the characteristic blocks, selects a plurality of characteristic variances in decrement order beginning with a maximum of the variances and the characteristic blocks corresponding to the characteristic variances, and uses the characteristic blocks to generate a plurality of classifiers corresponding to the classes through the linear mapping classifying method.
- In an embodiment of the present disclosure, wherein the linear mapping classifying method is support vector machine algorithm; the parameter ranges and the threshold are all between −1 and 1.
- In an embodiment of the present disclosure, wherein the processor extracts the first characteristic images from the first training images through histogram of oriented gradients, extracts the second characteristic images from the second training images through histogram of oriented gradients, and extracts the pending characteristic image from the pending image through histogram of oriented gradients.
- In an embodiment of the present disclosure, wherein when the two parameter ranges overlap, the processor records the remaining parameter ranges except for the two parameter ranges; after selecting one of the two classifiers corresponding to the two parameter ranges as said another classifier, the processor repeats the step (c) until the parameter ranges don't overlap with each other.
- In an embodiment of the present disclosure, wherein the first training images include a plurality of positive sample images and a plurality of negative sample images; there is an object respectively corresponding to the classes in the positive sample images, and there is a lack of the object respectively corresponding to the classes in the negative sample images.
- Another aspect of the present disclosure provides a non-transitory computer-readable storage medium storing a program that is loaded and executed by a computer, performs a multi-class object classifying method, which includes: (a) a plurality of classes a plurality of first training images and a plurality of second training images are received and stored, and a plurality of first characteristic images are extracted from the first training images; the first training images is used to generate a plurality of classifiers corresponding to the classes through a linear mapping classifying method, wherein the first training images and the second training images respectively correspond to the classes; (b) a plurality of second characteristic images are extracted from the second training images and a classifier is selected from the classifiers; (c) the classifier and the second characteristic images are used to determine a plurality of parameter ranges corresponding to the classes and a threshold corresponding to the classifier; (d) when two of the parameter ranges overlap, the remaining parameter ranges are recorded except for the two parameter ranges; after another classifier is selected from the classifiers except for the classifier that has been selected, the step (c) is repeated until the parameter ranges don't overlap with each other; (e) when the parameter ranges don't overlap with each other, the parameter ranges are recorded; (f) a pending image is received and stored and a pending characteristic image is extracted from the pending image; a classifier and the pending image are used to determine a pending parameter; (g) when the pending parameter is larger than the threshold corresponding to the classifier, the pending image is classified into the class corresponding to the classifier; (h) when the pending parameter lies in the parameter ranges,the pending image is classified into the classes corresponding to the parameter ranges.
- In an embodiment of the present disclosure, wherein the step (a) further includes: the first characteristic images are divided into a plurality of characteristic blocks, and a plurality of variances of the characteristic blocks is calculated; a plurality of characteristic variances in decrement order beginning with a maximum of the variances and the characteristic blocks corresponding to the characteristic variances are selected; the characteristic blocks are used to generate a plurality of classifiers corresponding to the classes through the linear mapping classifying method.
- In an embodiment of the present disclosure, wherein the linear mapping classifying method is support vector machine algorithm; the parameter ranges and the threshold are all between −1 and 1.
- In an embodiment of the present disclosure, wherein the steps (a), (b), and (f) include: the first characteristic images are extracted from the first training images through histogram of oriented gradients; the second characteristic images are extracted from the second training images through histogram of oriented gradients; the pending characteristic image is extracted from the pending image through histogram of oriented gradients.
- In an embodiment of the present disclosure, wherein the step (d) includes: when the two parameter ranges overlap, the remaining parameter ranges are recorded except for the two parameter ranges; after one of the two classifiers corresponding to the two parameter ranges is selected as said another classifier, the step (c) is repeated until the parameter ranges don't overlap with each other.
- In an embodiment of the present disclosure, wherein the first training images include a plurality of positive sample images and a plurality of negative sample images; there is an object respectively corresponding to the classes in the positive sample images, and there is a lack of the object respectively corresponding to the classes in the negative sample images.
- In conclusion, the present disclosure originates from efficiency improvements of the binary classifying method with regard to multi-class object classification and achieves multi-class object classification with less classifying steps by recording the parameter ranges that are generated from training images of different classes and one of the classifiers; moreover, the present disclosure reduces the sample number, saves the operation time by dividing the characteristic images into image blocks and selecting the image blocks with higher variances as representative to generate classifiers through the linear mapping classifying method, and apparently improves the efficiency of the multi-class object classification.
- The following is detailed description of the above-mentioned contents through embodiments, and provides further explanation of the technical aspects of the present disclosure.
- In order to make the above-mentioned contents, and other purposes, features, advantages, and embodiments more clear and understandable, with description made to the accompanying drawings as follows:
-
FIG. 1A-1B are flow charts of a multi-class object classifying method according to the first embodiment of the present disclosure; -
FIG. 2 is a schematic diagram of parameter ranges determined by the second training images respectively corresponding to the classes A-D and the parameters of classifier A; -
FIG. 3 is a schematic diagram of parameter ranges determined by the second training images respectively corresponding to the classes A-D and the parameters of classifier B; and -
FIG. 4 is a schematic diagram of a multi-class object classifying system according to the second embodiment of the present disclosure. - In order to make the description of the disclosure more detailed and comprehensive, reference will now be made in detail to the accompanying drawings and the following embodiments. However, the provided embodiments are not used to limit the ranges covered by the present disclosure; orders of step description are not used to limit the execution sequence either. Any devices with equivalent effect through rearrangement are also covered by the present disclosure.
-
FIG. 1A-1B show flow charts of a multi-classobject classifying method 100 according to the first embodiment of the present disclosure, and point A is a connection point of steps inFIG. 1A andFIG. 1B . The multi-classobject classifying method 100 can be applied to a multi-classobject classifying system 400 as shown inFIG. 4 . The multi-classobject classifying method 100 can be implemented as a computer program stored in a computer-readable medium, which is loaded by a computer to make the computer execute the multi-class object classifying method. The non-transitory computer-readable medium can be read only memory (ROM), flash memory, soft disk, hard disk, optical disk, pen drive, magnetic tape, network accessible database, or other computer-readable medium with the same function that are obvious for those skilled in the art. However, those skilled in the art should understand that the mentioned steps in the present embodiment are in an adjustable execution sequence according to the actual demands except for the steps in a specially described sequence, and even the steps or parts of the steps can be executed simultaneously. - First, in step S102, classes, first training images and second training images are received and stored; wherein the first training images and the second training images respectively correspond to the classes. For example, users can define classes A, B, C and D, and every class respectively has first training images and second training images. First characteristic images are extracted from the first training images to execute the first training stage and generate classifiers; wherein a linear mapping classifying method is used in the first training stage and the number of the classifiers equals to the number of the classes. In the present embodiment, the generated classifiers are classifiers A-D corresponding to classes A-D, and the classifiers have functions of distinguishing corresponding classes from non-corresponding classes. For example, classifier A can be used to distinguish class A from non-class A, and so on. Specifically, the linear mapping classifying method can be support vector machine algorithm, Fisher's liner discriminant, or naive Bayes classifier.
- Then, in step S104, second characteristic images are extracted from the second training images and a classifier is selected from the classifiers to execute the second training stage. In step S106, the classifier and the second characteristic images are used to determine parameter ranges corresponding to the classes and a threshold corresponding to the classifier; for example, the parameter ranges are all between −1 and 1. The threshold can be defined as a standard of being classified into the class according to the demands; for example, the threshold of classifier A is defined as 0.8; class A is determined when a parameter is larger than 0.8; non-class A is determined when a parameter is smaller than 0.8.
- The above-mentioned parameter ranges are determined through parameters calculated from characteristic values of second characteristic images in a class and parameters of the classifier. The equation is as follows:
-
- In the above-mentioned equation: Si is the characteristic value of the second characteristic images, Vi, αi, ρ are all the parameters of the selected classifier, Sum is the calculated parameter. Therefore, second characteristic images generate parameters through calculation and forms a parameter range. Parameter ranges corresponding to the classes are generated from the characteristic values of the second characteristic images corresponding to the classes and parameters of a classifier through the above-mentioned equation. As shown in
FIG. 2 , there are corresponding relation between the parameter ranges and the classes, wherein the parameter range corresponding to the same class as the classifier is nearest to 1. - In step S108, the parameter ranges are checked whether the parameter ranges overlap; when two of the parameter ranges overlap, the remaining parameter ranges are recorded except for the two parameter ranges; after another classifier is selected from the classifiers except for the classifier that has been selected, the step S106 is repeated until the parameter ranges don't overlap with each other (step S110); when the parameter ranges don't overlap with each other, the parameter ranges are recorded (step S112). For example,
FIG. 2 is a schematic diagram of parameter ranges determined by the second training images respectively corresponding to the classes A-D and the parameters of classifier A. Aparameter range 202 corresponding to class A is between the threshold and 1, parameter ranges 204-208 corresponding to classes B-D are between −1 and the threshold. Theparameter range 204 corresponding to class B overlaps with theparameter range 206 corresponding to class C, which indicates that classifier A can't distinguish class B from class C appropriately; therefore, thedistinguishable parameter range 202 corresponding to class A andparameter range 208 corresponding to class D recorded; after one of the classifiers B-D is selected except for classifier A, the step S106 is repeated until all the parameter ranges don't overlap with each other and the non-overlapping parameter ranges are recorded. For example,FIG. 3 is a schematic diagram of parameter ranges determined by the second training images respectively corresponding to the classes A-D and the parameters of classifier B, and all the parameter ranges 302-308 don't overlap; therefore the classifier B has a function of distinguishing classes A-D after recording the parameter ranges. Thus, any of the classifiers A-D has a potential of distinguishing classes A-D after recording the parameter ranges and improved capability of classification compared to the classifiers generated in the first training stage. - Detecting stage starts from step S114, a pending image is received and stored, and a pending characteristic image is extracted from the pending image; a classifier and the pending image are used to determine a pending parameter; in step S116 the pending parameter is checked whether it is larger than the threshold corresponding to the classifier; when the pending parameter is larger than the threshold corresponding to the classifier, the pending image is classified into the class corresponding to the classifier (step S118); when the pending parameter is smaller than the threshold corresponding to the classifier, in step S120, the pending parameter s checked whether it lies in the parameter ranges corresponding to the classes; when the pending parameter lies in the parameter ranges, the pending image is classified into the classes corresponding to the parameter ranges. For example, parameters of classifier A and the pending characteristic image are used to determine the pending parameter through the above-mentioned equation. When the pending parameter is larger than the threshold of classifier A, the pending image is classified into the class A: when the pending parameter is smaller than the threshold of classifier A and lies in the parameter range of class B, then the pending image is classified into class B; when the pending parameter is smaller than the threshold of classifier A and lies in the parameter range of class C, then the pending image is classified into class C. In conclusion, the classifiers A-D have functions of distinguishing not only the classes corresponding to the classifiers but also other classes, and therefore a multi-class object classification with less steps and higher efficiency can be achieved.
- In order to reduce the number of image samples used to generate classifiers, in an embodiment, the first characteristic images are divided into characteristic blocks, and variances of the characteristic blocks is calculated. Characteristic variances in decrement order beginning with a maximum of the variances and the characteristic blocks corresponding to the characteristic variances are selected. The characteristic blocks with large variances are selected to represent for the image, and then the characteristic blocks is used to generate classifiers corresponding to the classes through the linear mapping classifying method. The number of selected characteristic blocks is adjustable according to the demands so as to reduce the number of samples needed for generating the classifiers, and then the efficiency of operation is improved.
- In an embodiment, the linear mapping classifying method is support vector machine algorithm. The first training images corresponding to classes are inputted into the support vector machine algorithm to determine hyperplanes that can distinguish corresponding classes from non-corresponding classes and then generate the classifiers corresponding to the classes.
- In an alternative embodiment, characteristic images are extracted through histogram of oriented gradients; namely, the first characteristic images, the second characteristic images and the pending characteristic image are respectively extracted from the first training images, the second training images and the pending image through histogram of oriented gradients.
- In an embodiment, when the two parameter ranges overlap, the remaining parameter ranges are recorded except for the two parameter ranges; after one of the two classifiers corresponding to the two parameter ranges is selected as said another classifier, the step S106 is repeated until the parameter ranges don't overlap with each other. For example, as shown in
FIG. 2 , theparameter range 202 corresponding to class A is between the threshold and 1, and the parameter ranges corresponding to B-D are between −1 and the threshold Theparameter range 204 corresponding to class B overlaps with theparameter range 206 corresponding to class C, which indicates that classifier A can't distinguish class B from class C appropriately; therefore, thedistinguishable parameter range 202 corresponding to class A andparameter range 208 corresponding to class D are recorded, one of the classifiers B and C is selected, the step S106 is repeated until all the parameter ranges don't overlap with each other and the non-overlapping parameter ranges are recorded. Directly selecting from the classifiers B and C with overlapped parameter ranges is a faster way to find out the classifiers that can distinguish class B from class C. - In an embodiment, the first training images include positive sample images and negative sample images; there is an object respectively corresponding to the classes in the positive sample images, and there is a lack of the object respectively corresponding to the classes in the negative sample images. For example, in the process of generating classifier A, images with objects of class A is taken as the positive sample images, and images without objects of class A is taken as the negative sample images; therefore, images merely with objects of classes B-D can also be taken as the negative sample images to generate classifier A. The positive sample images and the negative sample images are inputted into the linear mapping classifying method to improve the recognizing functions of the generated classifiers.
-
FIG. 4 shows a multi-classobject classifying system 400 according to the second embodiment of the present disclosure. The multi-classobject classifying system 400 includes aninput device 410, adatabase 420 and aprocessor 430. In practice, theinput device 410 can be transmission port (for example: USB, GPIB, or other interfaces). Thedatabase 420 can be a hard disk, flash memory, or other storing media; theprocessor 430 can be CPUs, microprocessors or other circuits. - First, the
input device 410 receives and stores user-definedclasses 442,first training images 444,second training images 446 in thedatabase 420; wherein thefirst training images 444 and thesecond training images 446 respectively correspond to theclasses 442. For example, users can define classes A, B, C and D, and every class respectively hasfirst training images 444 andsecond training images 446. First characteristic images is extracted from thefirst training images 444 to execute the first training stage and generate classifiers; wherein a linear mapping classifying method is used in the first training stage and the number of the classifiers equals to the number of theclasses 442. In the present embodiment, the generated classifiers are classifiers A-D corresponding to classes A-D, and the classifiers have functions of distinguishing corresponding classes from non-corresponding classes. For example, classifier A can be used to distinguish class A from non-class A, and so on. Specifically, the linear mapping classifying method can be support vector machine algorithm, Fishers liner discriminant, or naive Bayes classifier. - Next, the
processor 430 extracts second characteristic images from thesecond training images 446 and selects a classifier from the classifiers to execute the second training stage. Theprocessor 430 uses the classifier and the second characteristic images to determine parameter ranges corresponding to theclasses 442 and a threshold corresponding to the classifier; for example, the parameter ranges are all between −1 and 1. The threshold can be defined as a standard of being classified into the class according to the demands; for example, the threshold of classifier A is defined as 0.8; class A is determined when a parameter is larger than 0.8; non-class A is determined when a parameter is smaller than 0.8. - The above-mentioned parameter ranges are determined by the
processor 430 through parameters calculated from characteristic values of second characteristic images in a class and parameters of the classifier. The equation is as follows: -
- In the above-mentioned equation: Si is the characteristic value of of the second characteristic images, Vi, αi, ρ are all the parameters of the selected classifier, Sum is the calculated parameter. Therefore, second characteristic images generates parameters through calculation and forms a parameter range. Parameter ranges corresponding to the classes is generated from the characteristic values of the second characteristic images corresponding to the classes and parameters of a classifier through the above-mentioned equation. As shown in
FIG. 2 , there are corresponding relation between the parameter ranges and the classes, wherein the parameter range corresponding to the same class as the classifier is nearest to 1. - Next, the
processor 430 checks the parameter ranges whether the parameter ranges overlap; when two of the parameter ranges overlap, theprocessor 430 records the remaining parameter ranges except for the two parameter ranges; after selecting another classifier from the classifiers except for the classifier that has been selected, theprocessor 430 repeats execution until the parameter ranges don't overlap with each other (step S110) when the parameter ranges don't overlap with each other, theprocessor 430 records the parameter ranges (step S112). For example,FIG. 2 is a schematic diagram of parameter ranges determined by the second training images respectively corresponding to the classes A-D and the parameters of classifier A. Aparameter range 202 corresponding to class A is between the threshold and 1, parameter ranges 204-208 corresponding to classes B-D are between −1 and the threshold. Theparameter range 204 corresponding to class B overlaps with theparameter range 206 corresponding to class C, which indicates that classifier A can't distinguish class B from class C appropriately; therefore, theprocessor 430 records thedistinguishable parameter range 202 corresponding to class A andparameter range 208 corresponding to class D; after selecting one of the classifiers B-D except for classifier A, theprocessor 430 repeats execution until all the parameter ranges don't overlap with each other and records the non-overlapping parameter ranges. For example,FIG. 3 is a schematic diagram of parameter ranges determined by the second training images respectively corresponding to the classes A-D and the parameters of classifier B, and all the parameter ranges 302-308 don't overlap; therefore, the classifier B has a function of distinguishing classes A-D after theprocessor 430 records the parameter ranges. Thus, any of the classifiers A-D has a potential of distinguishing classes A-D after recording the parameter ranges and improved capability of classification compared to the classifiers generated in the first training stage. - The
processor 430 receives and stores apending image 448 in thedatabase 420, extracts a pending characteristic image from the pendingimage 448, and uses a classifier and the pending image to determine a pending parameter. Theprocessor 430 checks the pending parameter whether t is larger than the threshold corresponding to the classifier; when the pending parameter is larger than the threshold corresponding to the classifier, theprocessor 430 classifies thepending image 448 into the class corresponding to the classifier; when the pending parameter is smaller than the threshold corresponding to the classifier, theprocessor 430 checks the pending parameter whether it lies in the parameter ranges corresponding to theclasses 442; when the pending parameter lies in the parameter ranges, theprocessor 430 classifies thepending image 448 into theclasses 442 corresponding to the parameter ranges. For example, theprocessor 430 uses parameters of classifier A and the pending characteristic image to determine the pending parameter through the above-mentioned equation. When the pending parameter is larger than the threshold of classifier A, theprocessor 430 classifies thepending image 448 into the class A; when the pending parameter is smaller than the threshold of classifier A and lies in the parameter range of class B, then theprocessor 430 classifies thepending image 448 into class B; when the pending parameter is smaller than the threshold of classifier A and lies in the parameter range of class C, then theprocessor 430 classifies thepending image 448 into class C. In conclusion, the classifiers A-D have functions of distinguishing not only the classes corresponding to the classifiers but also other classes, and therefore a multi-class object classification with less steps and higher efficiency can be achieved. - In order to reduce the number of image samples used to generate classifiers, in an embodiment, the
processor 430 divides the first characteristic images into characteristic blocks, and calculates variances of the characteristic blocks. Theprocessor 430 selects characteristic variances in decrement order beginning with a maximum of the variances and the characteristic blocks corresponding to the characteristic variances. Theprocessor 430 selects the characteristic blocks with large variances to represent for the image, and then uses the characteristic blocks to generate classifiers corresponding to the classes through the linear mapping classifying method. The number of selected characteristic blocks is adjustable according to the demands so as to reduce the number of samples needed for generating the classifiers, and then the efficiency of operation is improved. - In an embodiment, the linear mapping classifying method is support vector machine algorithm. The first training images corresponding to classes are inputted into the support vector machine algorithm to determine hyperplanes that can distinguish corresponding classes from non-corresponding classes and then the
processor 430 generates the classifiers corresponding to the classes. - In an alternative embodiment, the
processor 430 extracts characteristic images through histogram of oriented gradients; namely, theprocessor 430 respectively extracts the first characteristic images, the second characteristic images and the pending characteristic image from thefirst training images 444, thesecond training images 446 and thepending image 448 through histogram of oriented gradients. - In an embodiment, when the two parameter ranges overlap, the
processor 430 records the remaining parameter ranges except for the two parameter ranges; after selecting one of the two classifiers corresponding to the two parameter ranges as said another classifier, theprocessor 430 repeats execution until the parameter ranges don't overlap with each other. For example, as shown inFIG. 2 , theparameter range 202 corresponding to class A is between the threshold and 1, and the parameter ranges corresponding to B-D are between −1 and the threshold. Theparameter range 204 corresponding to class B overlaps with theparameter range 206 corresponding to class C, which indicates that classifier A can't distinguish class B from class C appropriately; therefore, theprocessor 430 records thedistinguishable parameter range 202 corresponding to class A andparameter range 208 corresponding to class D, selects one of the classifiers B and C, repeats execution until all the parameter ranges don't overlap with each other and records the non-overlapping parameter ranges. Directly selecting from the classifiers B and C with overlapped parameter ranges is a faster way to find out the classifiers that can distinguish class B from class C. - In an embodiment, the first training images include positive sample images and negative sample images; there is an object respectively corresponding to the classes in the positive sample images, and there is a lack of the object respectively corresponding to the classes in the negative sample images. For example, in the process of generating classifier A, the
processor 430 takes images with objects of class A as the positive sample images and takes images without objects of class A as the negative sample images; therefore, images merely with objects of classes B-D can also be taken as the negative sample images to generate classifier A. The positive sample images and the negative sample images are inputted into the linear mapping classifying method to improve the recognizing functions of the classifiers generated by theprocessor 430. - In conclusion, the present disclosure can record the parameter ranges that are generated from training images of different classes and one of the classifiers through the above-mentioned embodiments so as to achieve multi-class object classification with less classifying steps; moreover, the present disclosure reduces the sample number, saves the operation time by dividing the characteristic images into image blocks and selecting the image blocks with higher variances as representative to generate classifiers through the linear mapping classifying method, and apparently improves the efficiency of the multi-class object classification.
- Even though the present disclosure is disclosed as above, the disclosure is not used to limit the present disclosure. It will be apparent to those skilled in the art that various modifications and variations can be made to the present disclosure without departing from the spirit or scope of the invention; thus, it is intended that the range protected by the present disclosure should refer to the scope of the following claims.
Claims (18)
Applications Claiming Priority (3)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201410724928.4A CN105718937B (en) | 2014-12-03 | 2014-12-03 | Multi-class object classification method and system |
CN201410724928.4 | 2014-12-03 | ||
CN201410724928 | 2014-12-03 |
Publications (2)
Publication Number | Publication Date |
---|---|
US9361544B1 US9361544B1 (en) | 2016-06-07 |
US20160162757A1 true US20160162757A1 (en) | 2016-06-09 |
Family
ID=56083156
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US14/562,787 Active 2035-02-20 US9361544B1 (en) | 2014-12-03 | 2014-12-08 | Multi-class object classifying method and system |
Country Status (3)
Country | Link |
---|---|
US (1) | US9361544B1 (en) |
CN (1) | CN105718937B (en) |
TW (1) | TWI567660B (en) |
Cited By (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20160335521A1 (en) * | 2015-05-14 | 2016-11-17 | Canon Kabushiki Kaisha | Method and apparatus for generating, updating classifier, detecting objects and image processing device |
WO2019218334A1 (en) * | 2018-05-18 | 2019-11-21 | Beijing Didi Infinity Technology And Development Co., Ltd. | Systems and methods for user analysis |
WO2020215672A1 (en) * | 2019-08-05 | 2020-10-29 | 平安科技(深圳)有限公司 | Method, apparatus, and device for detecting and locating lesion in medical image, and storage medium |
WO2021248932A1 (en) * | 2020-06-11 | 2021-12-16 | 广东浪潮智慧计算技术有限公司 | Image data processing method and apparatus, device and readable storage medium |
Families Citing this family (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN106778801A (en) * | 2016-11-16 | 2017-05-31 | 卡尔蔡司(上海)管理有限公司 | Method and apparatus for analyzing the classification of object of observation |
TWI649659B (en) * | 2017-10-27 | 2019-02-01 | 財團法人工業技術研究院 | Automatic optical detection image classification method, system and computer readable medium containing the same |
US11315231B2 (en) | 2018-06-08 | 2022-04-26 | Industrial Technology Research Institute | Industrial image inspection method and system and computer readable recording medium |
CN110751617A (en) * | 2018-07-06 | 2020-02-04 | 台达电子工业股份有限公司 | Oral cavity image analysis system and method |
CN110348484A (en) * | 2019-06-10 | 2019-10-18 | 天津大学 | A method of classify for polarity electronic device |
Family Cites Families (19)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US7769228B2 (en) * | 2004-05-10 | 2010-08-03 | Siemens Corporation | Method for combining boosted classifiers for efficient multi-class object detection |
US7899253B2 (en) * | 2006-09-08 | 2011-03-01 | Mitsubishi Electric Research Laboratories, Inc. | Detecting moving objects in video by classifying on riemannian manifolds |
US8244044B2 (en) * | 2008-04-25 | 2012-08-14 | Microsoft Corporation | Feature selection and extraction |
CN101853389A (en) * | 2009-04-01 | 2010-10-06 | 索尼株式会社 | Detection device and method for multi-class targets |
US8725660B2 (en) * | 2009-07-30 | 2014-05-13 | Hewlett-Packard Development Company, L.P. | Applying non-linear transformation of feature values for training a classifier |
TWI415032B (en) * | 2009-10-30 | 2013-11-11 | Univ Nat Chiao Tung | Object tracking method |
US8509526B2 (en) * | 2010-04-13 | 2013-08-13 | International Business Machines Corporation | Detection of objects in digital images |
CN101901354B (en) * | 2010-07-09 | 2014-08-20 | 浙江大学 | Method for detecting and tracking multi targets at real time in monitoring videotape based on characteristic point classification |
US8687851B2 (en) * | 2010-09-10 | 2014-04-01 | Panasonic Corporation | Generic object-based image recognition apparatus with exclusive classifier, and method for the same |
KR20120064581A (en) * | 2010-12-09 | 2012-06-19 | 한국전자통신연구원 | Mehtod of classfying image and apparatus for the same |
US8781218B2 (en) * | 2011-11-02 | 2014-07-15 | Nec Laboratories America, Inc. | Receptive field learning for pooled image features |
US8761498B1 (en) * | 2012-01-26 | 2014-06-24 | Google Inc. | Face and license plate detection in street level images with 3-D road width features estimated from laser data |
CN102831447B (en) * | 2012-08-30 | 2015-01-21 | 北京理工大学 | Method for identifying multi-class facial expressions at high precision |
US9213892B2 (en) * | 2012-12-21 | 2015-12-15 | Honda Motor Co., Ltd. | Real-time bicyclist detection with synthetic training data |
TWI497449B (en) * | 2012-12-26 | 2015-08-21 | Ind Tech Res Inst | Unsupervised adaptation method and image automatic classification method applying the same |
US9112984B2 (en) * | 2013-03-12 | 2015-08-18 | Nuance Communications, Inc. | Methods and apparatus for detecting a voice command |
TW201441849A (en) * | 2013-04-30 | 2014-11-01 | Ibm | Method, system and computer program product of automatic determination of image sample group for analysis |
CN103996017B (en) * | 2014-02-24 | 2018-08-31 | 航天恒星科技有限公司 | One kind being based on HuThe not Ship Detection of bending moment and support vector machines |
CN103942564B (en) * | 2014-04-08 | 2017-02-15 | 武汉大学 | High-resolution remote sensing image scene classifying method based on unsupervised feature learning |
-
2014
- 2014-12-03 CN CN201410724928.4A patent/CN105718937B/en active Active
- 2014-12-08 US US14/562,787 patent/US9361544B1/en active Active
-
2015
- 2015-01-19 TW TW104101625A patent/TWI567660B/en active
Cited By (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20160335521A1 (en) * | 2015-05-14 | 2016-11-17 | Canon Kabushiki Kaisha | Method and apparatus for generating, updating classifier, detecting objects and image processing device |
US10242295B2 (en) * | 2015-05-14 | 2019-03-26 | Canon Kabushiki Kaisha | Method and apparatus for generating, updating classifier, detecting objects and image processing device |
WO2019218334A1 (en) * | 2018-05-18 | 2019-11-21 | Beijing Didi Infinity Technology And Development Co., Ltd. | Systems and methods for user analysis |
US10990500B2 (en) | 2018-05-18 | 2021-04-27 | Beijing Didi Infinity Technology And Development Co., Ltd. | Systems and methods for user analysis |
WO2020215672A1 (en) * | 2019-08-05 | 2020-10-29 | 平安科技(深圳)有限公司 | Method, apparatus, and device for detecting and locating lesion in medical image, and storage medium |
US11961227B2 (en) | 2019-08-05 | 2024-04-16 | Ping An Technology (Shenzhen) Co., Ltd. | Method and device for detecting and locating lesion in medical image, equipment and storage medium |
WO2021248932A1 (en) * | 2020-06-11 | 2021-12-16 | 广东浪潮智慧计算技术有限公司 | Image data processing method and apparatus, device and readable storage medium |
Also Published As
Publication number | Publication date |
---|---|
TWI567660B (en) | 2017-01-21 |
TW201621754A (en) | 2016-06-16 |
CN105718937A (en) | 2016-06-29 |
US9361544B1 (en) | 2016-06-07 |
CN105718937B (en) | 2019-04-05 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US9361544B1 (en) | Multi-class object classifying method and system | |
JP6941123B2 (en) | Cell annotation method and annotation system using adaptive additional learning | |
US9552536B2 (en) | Image processing device, information storage device, and image processing method | |
US10395136B2 (en) | Image processing apparatus, image processing method, and recording medium | |
US11416710B2 (en) | Feature representation device, feature representation method, and program | |
AU2014278408B2 (en) | Method for detecting a plurality of instances of an object | |
WO2019127451A1 (en) | Image recognition method and cloud system | |
WO2016107103A1 (en) | Method and device for recognizing main region of image | |
CN106897746B (en) | Data classification model training method and device | |
US20180189610A1 (en) | Active machine learning for training an event classification | |
EP3226176B1 (en) | Method for learning rejector by forming classification tree in use of training images and detecting object in test images, and rejector using the same | |
CN110852983A (en) | Method for detecting defects in semiconductor device | |
US20110158535A1 (en) | Image processing apparatus and image processing method | |
US10147015B2 (en) | Image processing device, image processing method, and computer-readable recording medium | |
EP3620980A1 (en) | Learning method, learning device for detecting lane by using cnn and testing method, testing device using the same | |
JP2018512567A (en) | Barcode tag detection in side view sample tube images for laboratory automation | |
US20120163706A1 (en) | Shape description and modeling for image subscene recognition | |
WO2020024584A1 (en) | Method, device and apparatus for training object detection model | |
US20160155068A1 (en) | Information processing apparatus, information processing method, and recording medium for classifying input data | |
Liang et al. | A supervised figure-ground segmentation method using genetic programming | |
KR20210028966A (en) | Method and apparatus for disease classification of plant leafs | |
CN107924493A (en) | Learning device and study identifying system | |
KR102050422B1 (en) | Apparatus and method for recognizing character | |
US20210042933A1 (en) | Image processing apparatus and image processing method | |
KR20180082680A (en) | Method for learning classifier and prediction classification apparatus using the same |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: INSTITUTE FOR INFORMATION INDUSTRY, TAIWAN Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:CHEN, YEN-LIN;CHIANG, CHUAN-YEN;YU, CHAO-WEI;AND OTHERS;REEL/FRAME:034511/0652 Effective date: 20141203 |
|
STCF | Information on status: patent grant |
Free format text: PATENTED CASE |
|
MAFP | Maintenance fee payment |
Free format text: PAYMENT OF MAINTENANCE FEE, 4TH YEAR, LARGE ENTITY (ORIGINAL EVENT CODE: M1551); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY Year of fee payment: 4 |
|
MAFP | Maintenance fee payment |
Free format text: PAYMENT OF MAINTENANCE FEE, 8TH YEAR, LARGE ENTITY (ORIGINAL EVENT CODE: M1552); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY Year of fee payment: 8 |