CN105205486B - A kind of automobile logo identification method and device - Google Patents

A kind of automobile logo identification method and device Download PDF

Info

Publication number
CN105205486B
CN105205486B CN201510586228.8A CN201510586228A CN105205486B CN 105205486 B CN105205486 B CN 105205486B CN 201510586228 A CN201510586228 A CN 201510586228A CN 105205486 B CN105205486 B CN 105205486B
Authority
CN
China
Prior art keywords
logo
region
fusion
candidate region
confidence level
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201510586228.8A
Other languages
Chinese (zh)
Other versions
CN105205486A (en
Inventor
陈羽飞
陈鑫嘉
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Zhejiang Uniview Technologies Co Ltd
Original Assignee
Zhejiang Uniview Technologies Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Zhejiang Uniview Technologies Co Ltd filed Critical Zhejiang Uniview Technologies Co Ltd
Priority to CN201510586228.8A priority Critical patent/CN105205486B/en
Publication of CN105205486A publication Critical patent/CN105205486A/en
Application granted granted Critical
Publication of CN105205486B publication Critical patent/CN105205486B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/20Image preprocessing
    • G06V10/24Aligning, centring, orientation detection or correction of the image
    • G06V10/245Aligning, centring, orientation detection or correction of the image by locating a pattern; Special marks for positioning
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00Pattern recognition
    • G06F18/20Analysing
    • G06F18/25Fusion techniques

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • General Physics & Mathematics (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Evolutionary Biology (AREA)
  • Evolutionary Computation (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • General Engineering & Computer Science (AREA)
  • Artificial Intelligence (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Multimedia (AREA)
  • Image Analysis (AREA)
  • Image Processing (AREA)

Abstract

The application provides a kind of automobile logo identification method and device, this method comprises: training logo classifier;Obtain license plate position favored area at the beginning of to determine logo;Favored area at the beginning of logo is detected using logo classifier, obtains the first logo candidate region;Calculate the first logo confidence level of the first logo candidate region;Logo position confidence level is calculated according to the position of the first logo candidate region;It is filtered out apart from central axes closer first logo candidate region according to logo position confidence level as the second logo candidate region;Second logo candidate region is identified, the second logo confidence level is obtained;Fusion is carried out to the second logo candidate region and generates fusion candidate region;According to the fusion confidence level of the corresponding fusion candidate region of the first logo confidence level, logo position confidence level and the second logo confidence calculations;Select the fusion highest fusion candidate region of confidence level as the logo identified.The vehicle-logo recognition rate under complex scene can be improved in the application.

Description

A kind of automobile logo identification method and device
Technical field
This application involves technical field of video monitoring more particularly to a kind of automobile logo identification method and devices.
Background technique
Logo is the important information of vehicle, is the significant image of vehicle, and vehicle-logo recognition can propose vehicle monitoring and tracking For strong Informational support.But since logo is compared with small, similitude is big, and be illuminated by the light, the reasons such as background, shape influence, difficult To accurately identify.
Current vehicle-logo recognition technology depends on the precise positioning of logo, and vehicle-logo location mostly use image procossing with And mode identification technology, the vehicle-logo location mode is to extreme sensitivities such as surrounding's texture of logo, angle rotation, inclinations, therefore, It is difficult to accomplish to be accurately positioned under complex scene, and then causes the discrimination of logo low.
Summary of the invention
In view of this, the application provides a kind of automobile logo identification method and device.
Specifically, the application is achieved by the following technical solution:
The application provides a kind of automobile logo identification method, this method comprises:
Using logo detection algorithm training logo classifier;
Obtain the license plate location information in image to be detected;
Favored area at the beginning of determining logo according to the license plate location information;
The logo classifier obtained using training carries out logo detection to favored area at the beginning of the logo, obtains several first vehicles Mark candidate region;
Calculate the first logo confidence level of each the first logo candidate region;
Corresponding logo position confidence level is calculated according to the position of the first logo candidate region;
It is filtered out at the beginning of the logo according to logo position confidence level from several first logos candidate region The central axes of favored area closer first logo candidate region is as the second logo candidate region;
Second logo candidate region is identified using machine learning algorithm, obtains the second logo candidate region Second logo confidence level;
Region fusion is carried out to multiple second logos candidate region and generates multiple fusion candidate regions;
According to the first logo confidence level, the logo position confidence level of the second logo candidate region for generating fusion candidate region And second the corresponding fusion candidate region of logo confidence calculations fusion confidence level;
Select the fusion highest fusion candidate region of confidence level as the logo identified.
The application also provides a kind of vehicle-logo recognition device, which includes:
Training unit, for using logo detection algorithm training logo classifier;
Acquiring unit, for obtaining the license plate location information in image to be detected;
Determination unit, favored area at the beginning of for determining logo according to the license plate location information;
Detection unit, the logo classifier for being obtained using training carry out logo detection to favored area at the beginning of the logo, Obtain several first logos candidate region;
First computing unit, for calculating the first logo confidence level of each the first logo candidate region;
Second computing unit, for calculating corresponding logo position confidence level according to the position of the first logo candidate region;
Screening unit, for according to logo position confidence level from several first logos candidate region filter out away from Central axes closer first logo candidate region from favored area at the beginning of the logo is as the second logo candidate region;
Recognition unit obtains second for identifying using machine learning algorithm to second logo candidate region Second logo confidence level of logo candidate region;
Integrated unit generates multiple fusion candidate regions for carrying out region fusion to multiple second logos candidate region;
Third computing unit, for the first logo confidence according to the second logo candidate region for generating fusion candidate region The fusion confidence level of the corresponding fusion candidate region of degree, logo position confidence level and the second logo confidence calculations;
Selecting unit, for selecting the fusion highest fusion candidate region of confidence level as the logo identified.
Precise positioning of the application independent of logo it can be seen from above description, but it is based on deep learning algorithm, Vehicle-logo recognition is carried out by the way of a variety of confidence level Weighted Fusions, improves the vehicle-logo recognition rate under complex scene.
Detailed description of the invention
Fig. 1 is a kind of automobile logo identification method flow chart shown in one exemplary embodiment of the application;
Fig. 2 is the positive sample and negative sample example shown in one exemplary embodiment of the application;
Fig. 3 is the logo primary election area schematic shown in one exemplary embodiment of the application;
Fig. 4 is a kind of underlying hardware structure of vehicle-logo recognition device place equipment shown in one exemplary embodiment of the application Schematic diagram;
Fig. 5 is a kind of structural schematic diagram of vehicle-logo recognition device shown in one exemplary embodiment of the application.
Specific embodiment
Example embodiments are described in detail here, and the example is illustrated in the accompanying drawings.Following description is related to When attached drawing, unless otherwise indicated, the same numbers in different drawings indicate the same or similar elements.Following exemplary embodiment Described in embodiment do not represent all embodiments consistent with the application.On the contrary, they be only with it is such as appended The example of the consistent device and method of some aspects be described in detail in claims, the application.
It is only to be not intended to be limiting the application merely for for the purpose of describing particular embodiments in term used in this application. It is also intended in the application and the "an" of singular used in the attached claims, " described " and "the" including majority Form, unless the context clearly indicates other meaning.It is also understood that term "and/or" used herein refers to and wraps It may be combined containing one or more associated any or all of project listed.
It will be appreciated that though various information, but this may be described using term first, second, third, etc. in the application A little information should not necessarily be limited by these terms.These terms are only used to for same type of information being distinguished from each other out.For example, not departing from In the case where the application range, the first information can also be referred to as the second information, and similarly, the second information can also be referred to as One information.Depending on context, word as used in this " if " can be construed to " ... when " or " when ... When " or " in response to determination ".
Current vehicle-logo recognition technology mainly uses image procossing and mode identification technology, in the base of license plate recognition technology On plinth, using the relative positional relationship of license plate and logo, just positioning is carried out to logo;It recycles based on Haar feature Adaboost algorithm carries out logo detection, obtains several doubtful car mark regions;Then using based on HOG (Histogram of Oriented Gradient, histograms of oriented gradients) feature SVM (Support Vector Machine, support vector machines) Algorithm screens doubtful car mark region, chooses the maximum region of confidence level as vehicle-logo location region;Finally to determining Vehicle-logo location region carries out vehicle-logo recognition.
During above-mentioned vehicle-logo recognition, it usually needs train multiple logos according to the difference of logo length-width ratio and classify Device, and due to length-width ratio difference, necessarily cause to be detected using different scale, different step-lengths, different sliding windows, therefore, Need to consume a large amount of system resource.Secondly as the inherent shortcoming of HOG algorithm, is difficult the problems such as processing blocks, is incomplete, and The algorithm is quite sensitive to image noise.In addition, choosing discriminant approach of the maximum region of confidence level as vehicle-logo location region Too simple violence be easy to cause erroneous judgement.
In view of the above-mentioned problems, the embodiment of the present application proposes a kind of automobile logo identification method, essence of this method independent of logo Certainly position, but it is based on deep learning algorithm, vehicle-logo recognition is carried out with the mode of a variety of confidence level Weighted Fusions.
It is one embodiment flow chart of the application automobile logo identification method, the embodiment is to vehicle-logo recognition process referring to Fig. 1 It is described.
Step 101, using logo detection algorithm training logo classifier.
Therefore the embodiment of the present application carries out vehicle-logo recognition using logo classifier during vehicle-logo recognition need to instruct in advance Practice mark classifier.Training (the step 101) of the logo classifier need to only be completed before logo detects (step 104). The inspection such as Adaboost algorithm, HOG algorithm, DPM (Deformable Parts Model, deformable member model) algorithm specifically can be used Method of determining and calculating carries out logo classifier training.
In a preferred embodiment, the training of logo classifier is carried out using Adaboost algorithm, specifically:
The positive sample and negative sample of logo pattern are obtained, positive sample is the sample comprising logo pattern, and negative sample is not wrap The sample of the pattern containing logo.In the embodiment of the present application, all positive samples are demarcated according to the ratio of width to height of 1:1, such as Fig. 2 (a) It is shown, for being unsatisfactory for the logo of 1:1 the ratio of width to height, part of the car mark region with identification can be intercepted as positive sample, for example, Audi's logo can choose the ring in Fourth Ring as positive sample, as shown in Fig. 2 (b).
In addition, the negative sample in the present embodiment is divided into two parts, a part is only comprising logo just favored area (logo primary election Region be license plate area just above, can be discussed in detail in subsequent descriptions) negative sample, hereinafter referred to as the first negative sample, such as Shown in Fig. 2 (c), which mainly considers that the embodiment of the present application is initial based on logo during subsequent vehicle-logo recognition Region carries out vehicle-logo recognition, therefore, selects the first favored area of logo that can effectively exclude as negative sample non-in the first favored area of logo Car mark region improves the detection efficiency of logo classifier.Another part negative sample is the negative sample comprising whole front face, is such as schemed Shown in 2 (d), hereinafter referred to as the second negative sample.Although the first negative sample has very strong elimination ability, as at the beginning of logo The range of favored area is smaller, and the textural characteristics in the region are limited, when logo type increases, is located at the logo just constituency The otherness very little of car mark region and non-car mark region within the scope of domain, the simple logo classifier generated using the first negative sample The ineffective of vehicle-logo recognition is carried out, the embodiment of the present application increases the second negative sample, which includes whole Chinese herbaceous peony Face, textural characteristics are abundant, it is possible to increase the difference of car mark region and non-car mark region is conducive to vehicle-logo recognition, therefore, passes through increase Second negative sample can promote the convergence of logo classifier.Certainly, either the first negative sample or the second negative sample are both needed to scratch Take down logo pattern.
After getting training required positive sample and negative sample, the embodiment of the present application is negative according to positive sample and first first Sample training obtains preceding N grades of strong classifiers, rear M grades of strong classifier is obtained further according to positive sample and the training of the second negative sample, by preceding N Grade strong classifier and the cascade of rear M grades of strong classifier generate logo classifier.As previously mentioned, before being obtained according to the training of the first negative sample N grades of strong classifiers can effectively exclude the non-car mark region in the first favored area of logo;It is obtained according to the training of the second negative sample latter M grades strong Classifier can promote the convergence of logo classifier, and details are not described herein.
Step 102, the license plate location information in image to be detected is obtained.
Existing more mature license plate recognition technology identification license plate can be used, obtain the position letter of license plate in image to be detected Breath, details are not described herein.
Step 103, favored area at the beginning of determining logo according to the license plate location information.
It can be found by largely observing, logo is usually located at license plate area just above, but its locating height has differences, For example, some pony car logos are usually located at adjacent domain above license plate, and some oversize vehicle logos are usually located at license plate Top upper zone (such as truck, medium truck).By the statistics discovery to a large amount of logo positions, logo height is usually not More than twice license plate width.Therefore, the embodiment of the present application determines the logo being located above license plate according to above-mentioned statistical result First favored area, the lower edge of favored area at the beginning of the logo connects with license plate upper edge, the height on side edge is license plate width 2 times, As shown in Figure 3.The logo just favored area can cover the overwhelming majority logo position, and as at the beginning of the logo favored area only with vehicle The width of board is high related with position, and therefore, under varying environment and different angle, the first favored area of the logo is relative to entire front face Variation is smaller, provides the relatively stable and lesser detection zone of range for subsequent vehicle-logo recognition.
Step 104, the logo classifier obtained using training carries out logo detection to favored area at the beginning of the logo, if obtaining Dry first logo candidate region.
The training of logo classifier is completed by step 101, using the logo classifier to the vehicle in image to be detected It marks just favored area and carries out logo detection, exclude non-car mark region, get multiple doubtful car mark regions, hereinafter referred to as first Logo candidate region.The the first logo candidate region obtained by above-mentioned logo detection of classifier is in true logo or similar logo Surrounding has apparent aggregation, lays the foundation for subsequent sections fusion.
Step 105, the first logo confidence level of each the first logo candidate region is calculated.
After getting the first logo candidate region, the confidence level of each the first logo candidate region is calculated, it is simple below Referred to as the first logo confidence level, specific calculating process are as follows:
FFstWeight=K/P formula (1)
Wherein, K is the number that the Weak Classifier of the first logo candidate region is capable of detecting when in logo classifier;P is vehicle Mark the total number of Weak Classifier in classifier;FFstWeightFor the first logo confidence level of the first logo candidate region.
In foregoing description it is found that logo classifier is made of multiple cascade strong classifiers, and strong classifier is by training The multiple Weak Classifiers composition generated in the process.Above-mentioned formula (1) exactly using the Weak Classifier generated in training process as Calculate the basis of the first logo candidate region.During logo classifier training, each Weak Classifier has a correspondence Threshold value, each Weak Classifier can have a corresponding output valve to the area to be tested of input, by the output valve with it is right The threshold value answered is compared, so that it is determined that whether area to be tested can pass through the detection of weak typing.If area to be tested can Add one with the Weak Classifier number by the detection of current Weak Classifier, then counted, when area to be tested passes through all weak point Final statistical result, i.e. K value can be obtained after class device (P Weak Classifier) detection.In confirmation, the area to be tested passes through logo After the detection of classifier, which can be used as the first logo candidate region and calculates corresponding first vehicle according to formula (1) Mark confidence level.A possibility that first logo confidence level is bigger, then corresponding first logo candidate region is logo is bigger.
Step 106, corresponding logo position confidence level is calculated according to the position of the first logo candidate region.
Since the embodiment of the present application does not carry out precise positioning to car mark region, by algorithm of target detection pre- If logo when just favored area carries out logo detection, may will detect that structural texture characteristic area similar with logo, and This feature cluster around regions a large amount of first logo candidate region.
In order to exclude above structure texture characteristic area similar with logo, the embodiment of the present application be based on logo position make into One step investigation.Although the horizontal position of logo is usually located at vehicle it is well known that the size of logo, highly having differences On central axes, therefore, the embodiment of the present application calculates the logo of each the first logo candidate region using the position characteristic of logo Position confidence level.
The calculating process of logo position confidence level is as follows:
Formula (2)
Wherein, D is distance of the first logo candidate region central point to the first favored area central axes of logo;W is the first logo The width of candidate region;FLocWeightFor the logo position confidence level of the first logo candidate region.
It needs to add explanation a bit, the vehicle for logo not on central axes, still vehicle can be calculated using formula (2) Cursor position confidence level, this is needed in logo classifier training, selects to be located in the first favored area of logo some bright on central axes Aobvious characteristic area replaces true logo as positive sample, can equally exclude the bigger similar car mark region of position deviation.
Step 107, it is filtered out from several first logos candidate region apart from institute according to logo position confidence level The central axes closer first logo candidate region of the first favored area of logo is stated as the second logo candidate region.
Specifically, the logo position confidence level that the embodiment of the present application is calculated according to formula (2) judges that current first logo is waited Whether favored area can be used as the second logo candidate region and carries out subsequent vehicle-logo recognition.
In a preferred embodiment, the logo position confidence level F being calculatedLocWeightWhen between 0 to 1, Think that corresponding first logo candidate region is located near central axes, can be used as the second logo candidate region and further identify;And Work as FLocWeightWhen less than 0, illustrate that corresponding first logo candidate region farther out, should reject first logo time apart from central axes Favored area.It is respectively positioned near central axes by the second logo candidate region that this step is screened, and more intensively clustered In different height, subsequent vehicle-logo recognition range is reduced.
Step 108, second logo candidate region is identified using machine learning algorithm, obtains the second logo and waits Second logo confidence level of favored area.
There are many existing machine learning algorithm, for example, CNN (Convolutional Neural Networks, convolution mind Through network) algorithm, SVM algorithm, BOW (Bag of words, bag of words) algorithm etc..The embodiment of the present application is right by taking CNN algorithm as an example Make further identification in the second logo candidate region.
When carrying out CNN classifier training, the training sample of CNN can increase on the basis of preceding aim detection algorithm sample Add incomplete part, rotation, the logo sample translated, to enhance the robustness of CNN classifier identification logo.
In this step, set by the second logo that the identification of CNN classifier can get each the second logo candidate region Reliability FSndWeight, the acquisition methods of the second logo confidence level are the prior art, and details are not described herein.
Step 109, region fusion is carried out to multiple second logos candidate region and generates multiple fusion candidate regions.
Since above-mentioned recognizer has error, and the second logo candidate region identified generally more intensively clusters Around true logo or similar logo, therefore, the embodiment of the present application is close with size to position and machine learning result is consistent The second logo candidate region merged.After carrying out S wheel fusion to the second logo candidate region of acquisition, S are generated Candidate region is merged, specific fusion process is as follows:
Execute new round mixing operation: selection one did not merged with other logo candidate regions and was not chosen as initially melting Original fusion region of the second logo candidate region in region as new round mixing operation is closed, which is current Take turns the first intermediate integration region of mixing operation.
Execute work as front-wheel mixing operation: select one have neither part nor lot in when the second logo candidate region of front-wheel mixing operation as Region to be fused;Position, width, height and the machine recognition result of intermediate integration region and region to be fused are obtained respectively (result after machine learning algorithm identifies);Calculate current fusion threshold value;According to fusion threshold value, intermediate integration region and to Position, width, height and the machine recognition result of integration region judge whether intermediate integration region and region to be fused meet Fusion conditions;When intermediate integration region and region to be fused meet fusion conditions, by intermediate integration region and region to be fused It is merged, as new intermediate integration region.
Judge whether that each second logo candidate region has participated in the mixing operation when front-wheel;It is executed if it is not, returning When front-wheel mixing operation;If so, judging whether the second logo candidate region for being also not chosen as original fusion region;If nothing, Then there is currently intermediate integration region be fusion candidate region;New round mixing operation is executed if so, then returning.
The calculating process of above-mentioned fusion threshold value is as follows:
DDelta=θ × MIN (iRectWdt1, iRectWdt2) formula (3)
Wherein, θ is threshold value adjustment factor, for example, θ=0.3;IRectWdt1 is the width of intermediate integration region; IRectWdt2 is the width in region to be fused;MIN (iRectWdt1, iRectWdt2) be take intermediate integration region width and to The minimum value of integration region width;DDelta is fusion threshold value.
It is above-mentioned judge among integration region and region to be fused whether meet fusion conditions process it is as follows:
IType1=iType2 formula (4)
| iRectX1-iRectX2 |≤dDelta formula (5)
| iRectY1-iRectY2 |≤dDelta formula (6)
| iRectX1+iRectWdt1-iRectX2-iRectWdt2 |≤dDelta formula (7)
| iRectY1+iRectHgt1-iRectY2-iRectHgt2 |≤dDelta formula (8)
Wherein, iType1 is the machine recognition result of intermediate integration region;IType2 is the machine recognition in region to be fused As a result;(iRectX1, iRectY1) is the position of intermediate integration region;(iRectX2, iRectY2) is the position in region to be fused It sets;IRectWdt1 is the width of intermediate integration region;IRectWdt2 is the width in region to be fused;IRectHgt1 is centre The height of integration region;IRectHgt2 is the height in region to be fused;DDelta is fusion threshold value.
As can be seen from the above formula that formula (4) indicates the machine recognition result of intermediate integration region and region to be fused It is identical, for example, being identified as the logo that " runs quickly ";Formula (5) and formula (6) indicate the position of intermediate integration region and region to be fused It sets close;Formula (7) and formula (8) indicate that the size in intermediate integration region and region to be fused is close.
When intermediate integration region and region to be fused meet simultaneously, machine recognition result is identical, position is close, size is close When, confirm that intermediate integration region and region to be fused meet fusion conditions.
Step 110, according to the first logo confidence level of the second logo candidate region for generating fusion candidate region, logo position Set the fusion confidence level of confidence level and the corresponding fusion candidate region of the second logo confidence calculations.
After the fusion by step 109, however it remains multiple fusion candidate regions, it is final to determine to need to continue screening Logo.
Used recognizer all has respective advantage and disadvantage during vehicle-logo recognition, for example, CNN algorithm is to flat Shifting, scaling, inclination or the deformation of his total form have height invariance;Adaboost algorithm is to accurate candidate regions The recognition confidence in domain is higher, and the recognition confidence of the candidate region offset for position is slightly lower, the candidate region of erroneous detection Recognition confidence it is lower.Certainly, being also not excluded for certain candidate regions has pole with the sample in training library in structure and form Big similitude.Therefore, multiple obtained in the embodiment of the present application combination above process in order to improve the accuracy of vehicle-logo recognition The fusion confidence level of each fusion candidate region of confidence calculations.
Merge the calculating process of confidence level are as follows:
Formula (9)
Wherein, t is the number for constituting the second logo candidate region of present fusion candidate region;FSndWeightIt (i) is i-th Second logo confidence level of a second logo candidate region;FFstWeightIt (i) is the first vehicle of i-th of second logo candidate regions Mark confidence level;FLocWeightIt (i) is the logo position confidence level of i-th of second logo candidate regions;α, β are respectively the first logo The weight coefficient of confidence level and logo position confidence level, and alpha+beta=1;FWeightFor the fusion confidence of present fusion candidate region Degree.
The fusion confidence level that candidate region is merged it can be seen from formula (9) is constitute the fusion candidate region second The confidence level of logo candidate region cumulative and.Wherein, when calculating the confidence level of each the second logo candidate region, to table The the first logo confidence level and logo position confidence level for showing the second logo candidate region and true logo similarity degree are weighted Processing takes α=0.35 for example, in a preferred embodiment, β=0.65, and recognition effect is preferable.
Step 111, select the fusion highest fusion candidate region of confidence level as the logo identified.
It is also higher to merge a possibility that higher fusion candidate region of confidence level is logo.
The application is not necessarily to carry out precise positioning to logo it can be seen from foregoing description, but by being obtained to depth recognition The a variety of confidence levels taken are weighted fusion, obtain final recognition result.The application is suitable for the logo under complex scene and knows Not, discrimination is higher.
Corresponding with the embodiment of aforementioned automobile logo identification method, present invention also provides the embodiments of vehicle-logo recognition device.
The embodiment of the application vehicle-logo recognition device can be using on an electronic device.Installation practice can pass through software It realizes, can also be realized by way of hardware or software and hardware combining.Taking software implementation as an example, as on a logical meaning Device, be that corresponding computer program instructions are formed in the processor run memory by equipment where it.From hardware For level, as shown in figure 4, for a kind of hardware structure diagram of the application vehicle-logo recognition device place equipment, in addition to shown in Fig. 4 Except processor, network interface and memory, the practical function of equipment in embodiment where device generally according to the equipment Can, it can also include other hardware, this is repeated no more.
Referring to FIG. 5, for the structural schematic diagram of the vehicle-logo recognition device in the application one embodiment.Vehicle-logo recognition dress It sets including training unit 501, acquiring unit 502, determination unit 503, detection unit 504, the first computing unit 505, second meter Unit 506, screening unit 507, recognition unit 508, integrated unit 509, third computing unit 510 and selecting unit 511 are calculated, Wherein:
Training unit 501, for using logo detection algorithm training logo classifier;
Acquiring unit 502, for obtaining the license plate location information in image to be detected;
Determination unit 503, favored area at the beginning of for determining logo according to the license plate location information;
Detection unit 504, the logo classifier for being obtained using training carry out logo inspection to favored area at the beginning of the logo It surveys, obtains several first logos candidate region;
First computing unit 505, for calculating the first logo confidence level of each the first logo candidate region;
Second computing unit 506, for calculating corresponding logo position confidence according to the position of the first logo candidate region Degree;
Screening unit 507, for being screened according to logo position confidence level from several first logos candidate region Central axes closer first logo candidate region out apart from favored area at the beginning of the logo is as the second logo candidate region;
Recognition unit 508 obtains for being identified to second logo candidate region using machine learning algorithm Second logo confidence level of two logo candidate regions;
Integrated unit 509 generates multiple fusion candidate regions for carrying out region fusion to multiple second logos candidate region Domain;
Third computing unit 510, for the first logo according to the second logo candidate region for generating fusion candidate region The fusion confidence level of the corresponding fusion candidate region of confidence level, logo position confidence level and the second logo confidence calculations;
Selecting unit 511, for selecting the fusion highest fusion candidate region of confidence level as the logo identified.
Further, first computing unit 505, specifically:
FFstWeight=K/P
Wherein,
K is the number that the Weak Classifier of the first logo candidate region is capable of detecting when in the logo classifier;
P is the total number of Weak Classifier in the logo classifier;
FFstWeightFor the first logo confidence level of first logo candidate region.
Further, second computing unit 506, specifically:
Wherein,
D is distance of the first logo candidate region central point to the first favored area central axes of logo;
W is the width of the first logo candidate region;
FLocWeightFor the logo position confidence level of the first logo candidate region.
Further, the integrated unit 509, comprising:
Original fusion region selection module was not merged and was not chosen as with other logo candidate regions for selecting one Original fusion region of the second logo candidate region in original fusion region as new round mixing operation, the original fusion area Domain is the first intermediate integration region when front-wheel mixing operation;
Region selection module to be fused, for selecting one to have neither part nor lot in when the second logo candidate region of front-wheel mixing operation As region to be fused;
Data obtaining module, for obtain respectively the position of the intermediate integration region and the region to be fused, width, Height and machine recognition result;
Threshold calculation module, for calculating current fusion threshold value;
Judgment module is merged, for according to the fusion threshold value, the intermediate integration region and the region to be fused Position, width, height and machine recognition result judge whether the intermediate integration region and the region to be fused meet and melt Conjunction condition;
Region Fusion Module will for when the intermediate integration region and the region to be fused meet fusion conditions The intermediate integration region and the region to be fused are merged, as new intermediate integration region;
As a result judgment module, for judging whether that each second logo candidate region has participated in grasping when the fusion of front-wheel Make;If it is not, then executing region selection module to be fused;If so, judging whether also to be not chosen as the second of original fusion region Logo candidate region;If nothing, there is currently intermediate integration region be fusion candidate region;If so, then executing original fusion Region selection module.
Further, the threshold calculation module, specifically:
DDelta=θ × MIN (iRectWdt1, iRectWdt2)
Wherein,
θ is threshold value adjustment factor;
IRectWdt1 is the width of intermediate integration region;
IRectWdt2 is the width in region to be fused;
MIN (iRectWdt1, iRectWdt2) is the minimum value for taking intermediate integration region width and peak width to be fused;
DDelta is fusion threshold value.
Further, the fusion conditions are as follows:
IType1=iType2
|iRectX1-iRectX2|≤dDelta
|iRectY1-iRectY2|≤dDelta
|iRectX1+iRectWdt1-iRectX2-iRectWdt2|≤dDelta
|iRectY1+iRectHgt1-iRectY2-iRectHgt2|≤dDelta
Wherein,
IType1 is the machine recognition result of intermediate integration region;
IType2 is the machine recognition result in region to be fused;
(iRectX1, iRectY1) is the position of intermediate integration region;
(iRectX2, iRectY2) is the position in region to be fused;
IRectWdt1 is the width of intermediate integration region;
IRectWdt2 is the width in region to be fused;
IRectHgt1 is the height of intermediate integration region;
IRectHgt2 is the height in region to be fused;
DDelta is fusion threshold value.
Further, the third computing unit 510, specifically:
Wherein,
T is the number for constituting the second logo candidate region of present fusion candidate region;
FSndWeightIt (i) is the second logo confidence level of i-th of second logo candidate regions;
FFstWeightIt (i) is the first logo confidence level of i-th of second logo candidate regions;
FLocWeightIt (i) is the logo position confidence level of i-th of second logo candidate regions;
α, β are respectively the weight coefficient of the first logo confidence level and logo position confidence level, and alpha+beta=1;
FWeightFor the fusion confidence level of present fusion candidate region.
The function of each unit and the realization process of effect are specifically detailed in the above method and correspond to step in above-mentioned apparatus Realization process, details are not described herein.
For device embodiment, since it corresponds essentially to embodiment of the method, so related place is referring to method reality Apply the part explanation of example.The apparatus embodiments described above are merely exemplary, wherein described be used as separation unit The unit of explanation may or may not be physically separated, and component shown as a unit can be or can also be with It is not physical unit, it can it is in one place, or may be distributed over multiple network units.It can be according to actual The purpose for needing to select some or all of the modules therein to realize application scheme.Those of ordinary skill in the art are not paying Out in the case where creative work, it can understand and implement.
The foregoing is merely the preferred embodiments of the application, not to limit the application, all essences in the application Within mind and principle, any modification, equivalent substitution, improvement and etc. done be should be included within the scope of the application protection.

Claims (14)

1. a kind of automobile logo identification method, which is characterized in that this method comprises:
Using logo detection algorithm training logo classifier;
Obtain the license plate location information in image to be detected;
Favored area at the beginning of determining logo according to the license plate location information;
The logo classifier obtained using training carries out logo detection to favored area at the beginning of the logo, obtains several first logos and waits Favored area;
Calculate the first logo confidence level of each the first logo candidate region;
Corresponding logo position confidence level is calculated according to the position of the first logo candidate region;
It is filtered out from several first logos candidate region apart from constituency at the beginning of the logo according to logo position confidence level The central axes in domain closer first logo candidate region is as the second logo candidate region;
Second logo candidate region is identified using machine learning algorithm, obtains the second of the second logo candidate region Logo confidence level;
According to position, width, height and the machine recognition of the second logo candidate region as a result, to meeting the second of fusion conditions Logo candidate region is merged, and fusion candidate region is generated;
According to generate the first logo confidence level of the second logo candidate region of fusion candidate region, logo position confidence level and The fusion confidence level of the corresponding fusion candidate region of second logo confidence calculations;
Select the fusion highest fusion candidate region of confidence level as the logo identified.
2. the method as described in claim 1, which is characterized in that first vehicle for calculating each the first logo candidate region Mark confidence level, comprising:
The calculation method of first logo confidence level of each first logo candidate region is identical, specifically:
FFstWeight=K/P
Wherein,
K is the number that the Weak Classifier of the first logo candidate region is capable of detecting when in the logo classifier;
P is the total number of Weak Classifier in the logo classifier;
FFstWeightFor the first logo confidence level of first logo candidate region.
3. the method as described in claim 1, which is characterized in that described calculated according to the position of the first logo candidate region corresponds to Logo position confidence level, comprising:
Wherein,
D is distance of the first logo candidate region central point to the first favored area central axes of logo;
W is the width of the first logo candidate region;
FLocWeightFor the logo position confidence level of the first logo candidate region.
4. the method as described in claim 1, which is characterized in that the position according to the second logo candidate region, width, height As a result, merging to the second logo candidate region for meeting fusion conditions, candidate region is merged in generation for degree and machine recognition, Include:
Execute new round mixing operation: selection one did not merged with other logo candidate regions and was not chosen as original fusion area Original fusion region of the second logo candidate region in domain as new round mixing operation, the original fusion region are to work as front-wheel The first intermediate integration region of mixing operation;
It executes and works as front-wheel mixing operation: selecting one to have neither part nor lot in the second logo candidate region when front-wheel mixing operation as wait melt Close region;Position, width, height and the machine recognition knot of the intermediate integration region and the region to be fused are obtained respectively Fruit;Calculate current fusion threshold value;According to the fusion threshold value, the position of the intermediate integration region and the region to be fused It sets, width, height and machine recognition result judge whether the intermediate integration region and the region to be fused meet fusion Condition;When the intermediate integration region and the region to be fused meet fusion conditions, by the intermediate integration region and institute It states region to be fused to be merged, as new intermediate integration region;
Judge whether that each second logo candidate region has participated in the mixing operation when front-wheel;It is executed currently if it is not, returning Take turns mixing operation;If so, judging whether the second logo candidate region for being also not chosen as original fusion region;If nothing, when Preceding existing intermediate integration region is fusion candidate region;New round mixing operation is executed if so, then returning.
5. method as claimed in claim 4, which is characterized in that described to calculate current fusion threshold value, comprising:
DDelta=θ × MIN (iRectWdt1, iRectWdt2)
Wherein,
θ is threshold value adjustment factor;
IRectWdt1 is the width of intermediate integration region;
IRectWdt2 is the width in region to be fused;
MIN (iRectWdt1, iRectWdt2) is the minimum value for taking intermediate integration region width and peak width to be fused;
DDelta is fusion threshold value.
6. method as claimed in claim 4, which is characterized in that the fusion conditions are as follows:
IType1=iType2
|iRectX1-iRectX2|≤dDelta
|iRectY1-iRectY2|≤dDelta
|iRectX1+iRectWdt1-iRectX2-iRectWdt2|≤dDelta
|iRectY1+iRectHgt1-iRectY2-iRectHgt2|≤dDelta
Wherein,
IType1 is the machine recognition result of intermediate integration region;
IType2 is the machine recognition result in region to be fused;
(iRectX1, iRectY1) is the position of intermediate integration region;
(iRectX2, iRectY2) is the position in region to be fused;
IRectWdt1 is the width of intermediate integration region;
IRectWdt2 is the width in region to be fused;
IRectHgt1 is the height of intermediate integration region;
IRectHgt2 is the height in region to be fused;
DDelta is fusion threshold value.
7. the method as described in claim 1, which is characterized in that described candidate according to the second logo for generating fusion candidate region Melt the corresponding fusion candidate region of first logo confidence level, logo position confidence level and the second logo confidence calculations in region Close confidence level, comprising:
Wherein,
T is the number for constituting the second logo candidate region of present fusion candidate region;
FSndWeightIt (i) is the second logo confidence level of i-th of second logo candidate regions;
FFstWeightIt (i) is the first logo confidence level of i-th of second logo candidate regions;
FLocWeightIt (i) is the logo position confidence level of i-th of second logo candidate regions;
α, β are respectively the weight coefficient of the first logo confidence level and logo position confidence level, and alpha+beta=1;
FweightFor the fusion confidence level of present fusion candidate region.
8. a kind of vehicle-logo recognition device, which is characterized in that the device includes:
Training unit, for using logo detection algorithm training logo classifier;
Acquiring unit, for obtaining the license plate location information in image to be detected;
Determination unit, favored area at the beginning of for determining logo according to the license plate location information;
Detection unit, the logo classifier for being obtained using training are carried out logo detection to favored area at the beginning of the logo, obtained Several first logos candidate region;
First computing unit, for calculating the first logo confidence level of each the first logo candidate region;
Second computing unit, for calculating corresponding logo position confidence level according to the position of the first logo candidate region;
Screening unit, for being filtered out from several first logos candidate region apart from institute according to logo position confidence level The central axes closer first logo candidate region of the first favored area of logo is stated as the second logo candidate region;
Recognition unit obtains the second logo for identifying using machine learning algorithm to second logo candidate region Second logo confidence level of candidate region;
Integrated unit, for according to position, width, height and the machine recognition of the second logo candidate region as a result, to satisfaction Second logo candidate region of fusion conditions is merged, and fusion candidate region is generated;
Third computing unit, the first logo confidence level of the second logo candidate region for merging candidate region according to generation, Logo position confidence level and the fusion confidence level of the corresponding fusion candidate region of the second logo confidence calculations;
Selecting unit, for selecting the fusion highest fusion candidate region of confidence level as the logo identified.
9. device as claimed in claim 8, which is characterized in that first computing unit, specifically:
FFstWeight=K/P
Wherein,
K is the number that the Weak Classifier of the first logo candidate region is capable of detecting when in the logo classifier;
P is the total number of Weak Classifier in the logo classifier;
FFstWeightFor the first logo confidence level of first logo candidate region.
10. device as claimed in claim 8, which is characterized in that second computing unit, specifically:
Wherein,
D is distance of the first logo candidate region central point to the first favored area central axes of logo;
W is the width of the first logo candidate region;
FLocWeightFor the logo position confidence level of the first logo candidate region.
11. device as claimed in claim 8, which is characterized in that the integrated unit, comprising:
Original fusion region selection module, for selecting one not merge and be not chosen as with other logo candidate regions initially Original fusion region of the second logo candidate region of integration region as new round mixing operation, the original fusion region is When the first intermediate integration region of front-wheel mixing operation;
Region selection module to be fused, for select one have neither part nor lot in when the second logo candidate region of front-wheel mixing operation as Region to be fused;
Data obtaining module, for obtaining position, the width, height of the intermediate integration region and the region to be fused respectively And machine recognition result;
Threshold calculation module, for calculating current fusion threshold value;
Merge judgment module, for according to the position of the fusion threshold value, the intermediate integration region and the region to be fused, Width, height and machine recognition result judge whether the intermediate integration region and the region to be fused meet fusion item Part;
Region Fusion Module will be described for when the intermediate integration region and the region to be fused meet fusion conditions Intermediate integration region and the region to be fused are merged, as new intermediate integration region;
As a result judgment module, for judging whether that each second logo candidate region has participated in the mixing operation when front-wheel; If it is not, then executing region selection module to be fused;If so, judging whether the second logo for being also not chosen as original fusion region Candidate region;If nothing, there is currently intermediate integration region be fusion candidate region;If so, then executing original fusion region Selecting module.
12. device as claimed in claim 11, which is characterized in that the threshold calculation module, specifically:
DDelta=θ × MIN (iRectWdt1, iRectWdt2)
Wherein,
θ is threshold value adjustment factor;
IRectWdt1 is the width of intermediate integration region;
IRectWdt2 is the width in region to be fused;
MIN (iRectWdt1, iRectWdt2) is the minimum value for taking intermediate integration region width and peak width to be fused;
DDelta is fusion threshold value.
13. device as claimed in claim 11, which is characterized in that the fusion conditions are as follows:
IType1=iType2
|iRectX1-iRectX2|≤dDelta
|iRectY1-iRectY2|≤dDelta
|iRectX1+iRectWdt1-iRectX2-iRectWdt2|≤dDelta
|iRectY1+iRectHgt1-iRectY2-iRectHgt2|≤dDelta
Wherein,
IType1 is the machine recognition result of intermediate integration region;
IType2 is the machine recognition result in region to be fused;
(iRectX1, iRectY1) is the position of intermediate integration region;
(iRectX2, iRectY2) is the position in region to be fused;
IRectWdt1 is the width of intermediate integration region;
IRectWdt2 is the width in region to be fused;
IRectHgt1 is the height of intermediate integration region;
IRectHgt2 is the height in region to be fused;
DDelta is fusion threshold value.
14. device as claimed in claim 8, which is characterized in that the third computing unit, specifically:
Wherein,
T is the number for constituting the second logo candidate region of present fusion candidate region;
FSndWeightIt (i) is the second logo confidence level of i-th of second logo candidate regions;
FFstWeightIt (i) is the first logo confidence level of i-th of second logo candidate regions;
FLocWeightIt (i) is the logo position confidence level of i-th of second logo candidate regions;
α, β are respectively the weight coefficient of the first logo confidence level and logo position confidence level, and alpha+beta=1;
FWeightFor the fusion confidence level of present fusion candidate region.
CN201510586228.8A 2015-09-15 2015-09-15 A kind of automobile logo identification method and device Active CN105205486B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201510586228.8A CN105205486B (en) 2015-09-15 2015-09-15 A kind of automobile logo identification method and device

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201510586228.8A CN105205486B (en) 2015-09-15 2015-09-15 A kind of automobile logo identification method and device

Publications (2)

Publication Number Publication Date
CN105205486A CN105205486A (en) 2015-12-30
CN105205486B true CN105205486B (en) 2018-12-07

Family

ID=54953158

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201510586228.8A Active CN105205486B (en) 2015-09-15 2015-09-15 A kind of automobile logo identification method and device

Country Status (1)

Country Link
CN (1) CN105205486B (en)

Families Citing this family (16)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105608441B (en) * 2016-01-13 2020-04-10 浙江宇视科技有限公司 Vehicle type recognition method and system
CN105957071B (en) * 2016-04-26 2019-04-12 浙江宇视科技有限公司 A kind of lamp group localization method and device
CN106339445B (en) * 2016-08-23 2019-06-18 东方网力科技股份有限公司 Vehicle retrieval method and device based on big data
CN106529424B (en) * 2016-10-20 2019-01-04 中山大学 A kind of logo detection recognition method and system based on selective search algorithm
CN106503710A (en) * 2016-10-26 2017-03-15 北京邮电大学 A kind of automobile logo identification method and device
CN106529460A (en) * 2016-11-03 2017-03-22 贺江涛 Object classification identification system and identification method based on robot side
CN108256404B (en) * 2016-12-29 2021-12-10 北京旷视科技有限公司 Pedestrian detection method and device
CN107590492B (en) * 2017-08-28 2019-11-19 浙江工业大学 A kind of vehicle-logo location and recognition methods based on convolutional neural networks
CN108229308A (en) * 2017-11-23 2018-06-29 北京市商汤科技开发有限公司 Recongnition of objects method, apparatus, storage medium and electronic equipment
CN108171274B (en) * 2018-01-17 2019-08-09 百度在线网络技术(北京)有限公司 The method and apparatus of animal for identification
CN109409159A (en) * 2018-10-11 2019-03-01 上海亿保健康管理有限公司 A kind of fuzzy two-dimensional code detection method and device
CN109919154B (en) * 2019-02-28 2020-10-13 北京科技大学 Intelligent character recognition method and device
CN109697719B (en) * 2019-03-05 2021-12-24 北京康夫子健康技术有限公司 Image quality evaluation method and device and computer readable storage medium
CN112069862A (en) * 2019-06-10 2020-12-11 华为技术有限公司 Target detection method and device
CN110852252B (en) * 2019-11-07 2022-12-02 厦门市美亚柏科信息股份有限公司 Vehicle weight-removing method and device based on minimum distance and maximum length-width ratio
CN113470347B (en) * 2021-05-20 2022-07-26 上海天壤智能科技有限公司 Congestion identification method and system combining bayonet vehicle passing record and floating vehicle GPS data

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101630361A (en) * 2008-12-30 2010-01-20 北京邮电大学 Plate number, body color and mark identification-based equipment and plate number, body color and mark identification-based method for identifying fake plate vehicles
CN102968646A (en) * 2012-10-25 2013-03-13 华中科技大学 Plate number detecting method based on machine learning
CN103077384A (en) * 2013-01-10 2013-05-01 北京万集科技股份有限公司 Method and system for positioning and recognizing vehicle logo
CN103310231A (en) * 2013-06-24 2013-09-18 武汉烽火众智数字技术有限责任公司 Auto logo locating and identifying method
CN104268596A (en) * 2014-09-25 2015-01-07 深圳市捷顺科技实业股份有限公司 License plate recognizer and license plate detection method and system thereof
CN104281851A (en) * 2014-10-28 2015-01-14 浙江宇视科技有限公司 Extraction method and device of car logo information
CN104331691A (en) * 2014-11-28 2015-02-04 深圳市捷顺科技实业股份有限公司 Vehicle logo classifier training method, vehicle logo recognition method and device

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6735337B2 (en) * 2001-02-02 2004-05-11 Shih-Jong J. Lee Robust method for automatic reading of skewed, rotated or partially obscured characters

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101630361A (en) * 2008-12-30 2010-01-20 北京邮电大学 Plate number, body color and mark identification-based equipment and plate number, body color and mark identification-based method for identifying fake plate vehicles
CN102968646A (en) * 2012-10-25 2013-03-13 华中科技大学 Plate number detecting method based on machine learning
CN103077384A (en) * 2013-01-10 2013-05-01 北京万集科技股份有限公司 Method and system for positioning and recognizing vehicle logo
CN103310231A (en) * 2013-06-24 2013-09-18 武汉烽火众智数字技术有限责任公司 Auto logo locating and identifying method
CN104268596A (en) * 2014-09-25 2015-01-07 深圳市捷顺科技实业股份有限公司 License plate recognizer and license plate detection method and system thereof
CN104281851A (en) * 2014-10-28 2015-01-14 浙江宇视科技有限公司 Extraction method and device of car logo information
CN104331691A (en) * 2014-11-28 2015-02-04 深圳市捷顺科技实业股份有限公司 Vehicle logo classifier training method, vehicle logo recognition method and device

Also Published As

Publication number Publication date
CN105205486A (en) 2015-12-30

Similar Documents

Publication Publication Date Title
CN105205486B (en) A kind of automobile logo identification method and device
CN108319964B (en) Fire image recognition method based on mixed features and manifold learning
CN106407981B (en) A kind of licence plate recognition method, apparatus and system
Zhou et al. Principal visual word discovery for automatic license plate detection
CN103679168B (en) Detection method and detection device for character region
CN101799867B (en) Improved detection of people in real world videos and images
CN106682586A (en) Method for real-time lane line detection based on vision under complex lighting conditions
CN107103317A (en) Fuzzy license plate image recognition algorithm based on image co-registration and blind deconvolution
CN105740886B (en) A kind of automobile logo identification method based on machine learning
Lee et al. Shape discovery from unlabeled image collections
CN109255375A (en) Panoramic picture method for checking object based on deep learning
CN106355138A (en) Face recognition method based on deep learning and key features extraction
CN103366181A (en) Method and device for identifying scene integrated by multi-feature vision codebook
CN106778687A (en) Method for viewing points detecting based on local evaluation and global optimization
CN106682641A (en) Pedestrian identification method based on image with FHOG- LBPH feature
CN110263712A (en) A kind of coarse-fine pedestrian detection method based on region candidate
CN105224945B (en) A kind of automobile logo identification method based on joint-detection and identification algorithm
CN106250824A (en) Vehicle window localization method and system
CN105279492A (en) Iris identification method and device
CN106503710A (en) A kind of automobile logo identification method and device
CN101984453A (en) Human eye recognition system and method
Ruta et al. Towards real-time traffic sign recognition by class-specific discriminative features
CN106056132A (en) Garment style identification method based on Fourier descriptor and support vector machine
CN107886066A (en) A kind of pedestrian detection method based on improvement HOG SSLBP
CN104156690A (en) Gesture recognition method based on image space pyramid bag of features

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant