CN107092876A - The low-light (level) model recognizing method combined based on Retinex with S SIFT features - Google Patents

The low-light (level) model recognizing method combined based on Retinex with S SIFT features Download PDF

Info

Publication number
CN107092876A
CN107092876A CN201710237135.3A CN201710237135A CN107092876A CN 107092876 A CN107092876 A CN 107092876A CN 201710237135 A CN201710237135 A CN 201710237135A CN 107092876 A CN107092876 A CN 107092876A
Authority
CN
China
Prior art keywords
image
vehicle
retinex
wavelet
algorithm
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201710237135.3A
Other languages
Chinese (zh)
Inventor
张斯尧
马昊辰
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Hunan Source Letter Photoelectric Polytron Technologies Inc
Original Assignee
Hunan Source Letter Photoelectric Polytron Technologies Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Hunan Source Letter Photoelectric Polytron Technologies Inc filed Critical Hunan Source Letter Photoelectric Polytron Technologies Inc
Priority to CN201710237135.3A priority Critical patent/CN107092876A/en
Publication of CN107092876A publication Critical patent/CN107092876A/en
Pending legal-status Critical Current

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00Scenes; Scene-specific elements
    • G06V20/50Context or environment of the image
    • G06V20/56Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
    • G06V20/58Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
    • G06V20/584Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads of vehicle lights or traffic lights
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00Pattern recognition
    • G06F18/20Analysing
    • G06F18/21Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00Pattern recognition
    • G06F18/20Analysing
    • G06F18/24Classification techniques
    • G06F18/241Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G06F18/2411Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on the proximity to a decision surface, e.g. support vector machines

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Artificial Intelligence (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Evolutionary Biology (AREA)
  • Evolutionary Computation (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • General Engineering & Computer Science (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Multimedia (AREA)
  • Image Analysis (AREA)

Abstract

The present invention relates to computer vision field, the model recognizing method under a kind of low light conditions combined based on Retinex with S SIFT features is refered in particular to.This method includes four steps:1) the low-light (level) image enhaucament that is converted based on Retinex and wavelet adaptive threshold, 2) License Plate, 3) the car face region interception comprising vehicle information and 4) set up the S SIFT features model of vehicle area image and combine SVM training aids and carry out vehicle cab recognition.The method proposed in the present invention can actually be embedded in FPGA realizations, apply in camera or video camera with the vehicle cab recognition function of real-time output image function under low-light (level) environment, effectively improve the accuracy and reliability of system, it is met real-time demand.

Description

The low-light (level) model recognizing method combined based on Retinex with S-SIFT features
Technical field
The present invention relates to computer vision field, one kind is refered in particular to based on Retinex and S-SIFT (sparse scale Invariant feature transform) feature combine low light conditions under model recognizing method.
Background technology
With developing rapidly for the industries such as modern transportation, security protection, vehicle automatic identification technology is increasingly by people's Pay attention to, be in recent years computer vision and mode identification technology in one of important subject of intelligent transportation field.Vehicle Automatic recognition system can be used for the vehicle management in the places such as toll station, parking lot, crossroad, it can also be used to modernize small The vehicles while passing management of area or industrial park, for public safety, community security protection, road traffic and parking lot vehicle management are all With important facilitation.
Vehicle cab recognition generally comprises the research of three aspects, and domestic and international experts and scholars have also carried out substantial amounts of work, mainly Including:The classification of the positioning and identification of car plate, the detection and identification of logo, and vehicle size.Wherein, according to Chinese herbaceous peony face image To recognize that the research method of specific vehicle is the hot research direction of recent years.
In reality, the usual background complexity of picture that gathers in practice, uneven illumination, resolution ratio is low, vehicle is old, vehicle is dirty Deng especially when night or dim weather, because light illumination is low, causing automobile video frequency image blurring unclear, regard simultaneously Frequency monitoring system is as a rule different from for the angle that vehicle image is shot, and these all bring very big to vehicle cab recognition It is difficult.
The content of the invention
The technical problem to be solved in the present invention is:In the case of for existing night low-light (level), model recognition system is deposited Difficulties, propose a kind of based on Retinex and S-SIFT (sparse scale invariant feature Transform) model recognizing method under the low light conditions that feature is combined, is effectively improved the vehicle cab recognition of monitoring system Order of accuarcy, and it is met real-time demand.
Technical solution of the present invention specifically includes following steps:
S1) the low-light (level) image enhaucament based on Retinex algorithm and wavelet transformation;
S2) License Plate;
S3 car face area image) is intercepted;
S4 the S-SIFT characteristic models of car face area image) are set up and SVM training aids is combined and carry out vehicle cab recognition.
It is used as the further improvement of technical solution of the present invention, the step S1) specifically include:
S1.1) recover the shape constancy of the colourity of license plate image by using single scale Retinex algorithm and strengthen image Definition;
S1.2) Wavelet Denoising Method for realizing vehicle image by improved wavelet threshold Denoising Algorithm is handled, and is specifically included:
A) N layers of wavelet decomposition are carried out to the recovery image after Retinex algorithm is handled;
B) a wavelet threshold T is selectedj, compare wavelet coefficient and wavelet threshold TjDifference so as to judging correlation, correlation Big is main information, it is necessary to strengthen.Correlation is small, then is noise section, by the noise section wavelet coefficient zero setting;
C) all yardsticks are traveled through, all carrying out a threshold correlation in adjacent yardstick every bit judges to put with wavelet coefficient Zero or enhancing processing;
D) choosing an enhancing function strengthens image after step c) processing, then carries out wavelet reconstruction.
It is used as the further improvement of technical solution of the present invention, the step S2) specifically include S2.1) training car plate sample spy Levy extraction and feature organization and S2.2) the detection positioning of car plate;
The S2.1) include:
S2.1.1 any normal national standard car plate) is taken out manually;
S2.1.2 channel characteristics extraction) is integrated to the license plate image taken out:Set up LUV passages, ladder respectively first Amplitude passage and histogram of gradients passage are spent, and correspondence is obtained according to LUV passages, gradient magnitude passage and histogram of gradients passage The license plate image feature of passage;
S2.1.3 detector) is trained based on Adaboost algorithm:Training stage, utilize product of the Adaboost algorithm to extraction Subchannel features training goes out strong classifier;Differentiating the stage, calculating the integrating channel feature for detecting positioning licence plate window, using Strong classifier differentiates the Confidence of car plate position, finally stores self-confident that frame of highest or a few two field pictures in one section of video;
The S2.2) include:
S2.2.1) target image by strengthening processing is scanned using sliding window method, by the figure of each scanning interception As being integrated channel characteristics calculating, it is compared with the strong detector that Adaboost algorithm is trained, chooses similarity highest Image-region be used as first positioning licence plate image;
S2.2.2) the first positioning image for exporting detector carries out the first positioning result progress after non-maxima suppression processing Slant correction based on Hough transformation, and it is integrated the strong inspection of input after channel characteristics are extracted again to the image after slant correction Survey device and carry out secondary positioning, obtain the license plate image after secondary positioning.
It is used as the further improvement of technical solution of the present invention, the step S3) specifically include:Orienting accurate car plate Behind position, generally according to the length and width of car plate, choose certain ratio and enter the interception of face image of driving a vehicle.
It is used as the further improvement of technical solution of the present invention, the step S4) specifically include:
S4.1 the S-SIFT characteristic models of car face area image) are set up, it is included using S-SIFT algorithm process Che Lian areas Area image SIFT feature describes son and obtains sparse coding, and uses pond method statistic sparse coding result, obtains car face region The summary statistics feature of image;
S4.2) car face area image vehicle tagsort and identification, it is included in after extraction S-SIFT features, utilized SVM training aids is trained classification;After training classification, the car face area image comprising vehicle characteristic information of interception is inputted In SVM training aids, and export the vehicle information of identification.
Compared with prior art, the invention has the advantages that:
1st, the inventive method carries out color recovery processing first with Retinex algorithm and uses a kind of improved small echo threshold It is worth Denoising Algorithm, while for the limitation of Traditional Wavelet Denoising Algorithm treatment effect under dark condition, adding Retinex Algorithm adjusts the color of license plate image, effectively keeps the color constancy of original image, can effectively strengthen the color pair of image Degree of ratio.
2nd, the inventive method combines the vehicle characteristic image for passing through enhancing processing, extracts the low layer SIFT of target image Characteristic vector, then trained acquisition encoder dictionary and sparse SIFT feature, obtain deeper level characteristics of image, are regarded with adapting to difference Angle, illumination variation, shade, the complex scene such as block, further improve discrimination;Finally realized with linear SVM sparse SIFT feature is classified, and reduces time complexity, it is ensured that real-time.
3rd, the inventive method reliability is high, debates that resolution is good, can be carried out for the vehicle image of dim complex condition Car license recognition processing, while this algorithm robustness is good, step calculates simple, can keep high efficiency, real-time can also meet demand.
Brief description of the drawings
Fig. 1 is holistic approach flow chart described in the present embodiment;
Fig. 2 is based on the contrast of Retinex image enhancement effects described in the present embodiment;
Fig. 3 is based on Wavelet Denoising Method and Retinex combination algorithm image procossing comparison diagrams described in the present embodiment;
Fig. 4 is three kinds of integration feature channel images of license plate image described in the present embodiment;
Fig. 5 is four direction gradient operator schematic diagram described in the present embodiment;
Fig. 6 is pixel direction schematic diagram described in the present embodiment;
Fig. 7 is the license plate image of Hough transformation slant correction described in the present embodiment;
Fig. 8 is each examples of parameters of car face image scope intercepted described in the present embodiment.
Embodiment
The present invention is described in further details in conjunction with accompanying drawing, the present embodiment proposes a kind of based on Retinex and S- Vehicle under the low light conditions that SIFT (sparse scale invariant feature transform) feature is combined is known Other method, the model recognizing method comprises the following steps that shown:
S1. the low-light (level) image enhaucament based on Retinex and wavelet transformation:
S1.1 recovers the shape constancy of the colourity of license plate image by using single scale Retinex algorithm and strengthens image Definition;
The theory of Retinex algorithm is a kind of algorithm for image enhancement set up on the basis of human-eye visual characteristic, Its general principle is:Target object determines the color of gathered target image to the albedo of the light of different wave length, The light and shade change of illumination will not be impacted to color in kind, i.e. the color of object has shape constancy.Retinex algorithm can be with The effective color constancy using image, is carried out to image at dynamic range compression, edge enhancing and color of image shape constancy Reason, therefore self-adaptive processing can be carried out to low-light (level) image.
Based on above content, the theory of Retinex algorithm has good processing to the similar dark scape image by illumination effect Effect, because build-in attribute-color of object and the intensity size of light source do not have dependence, and the color of image is by pixel Value determines that the correction of pixel can be drawn by the relative relationship between light and dark between pixel.
Shown in the general principle formula specific as follows of Retinex algorithm:If the image that S (x, y), which is computer, to be received, R (x, Y) it is reflection subject image, L (x, y) is incident light images, and r (x, y) is the image after Retinex algorithm processing, and Retinex is calculated Method solves luminance picture L (x, y) using realistic objective S (x, y).
S (x, y)=R (x, y) L (x, y)
Solution procedure is usually required by Gaussian convolution function as center ring around function, and resolution principle is specific such as following public affairs Shown in formula:
∫ ∫ G (x, y) dxdy=1
L (x, y)=S (x, y) * G (x, y)
Wherein, if G (x, y) be center ring around function, compared to inverse square function and exponential type central function, Gaussian Function operating distance is near, and details enhancing effect and dynamic compression effect are all fine.
Formula (1) c is filter radius, is scale parameters of the G (x, y) around scope, the selection of filter radius is Retinex The key of algorithm process.C values are big compared with small compression of images scope, and details protrudes obvious, but can cause cross-color.The larger face of c values Color is close to actual value, but compression of images scope is small.So select suitable scale parameter critically important, usual c values choose 15,80 or 20, it is corresponded to respectively, and yardstick is small, it is big to neutralize.
For low-light (level) environment, the embodiment of the present invention only carries out enhanced algorithm by wavelet theory to tradition and changed Enter, i.e., the processing of single scale Retinex algorithm is added before adaptively strengthen, the processing can effectively strengthen image Detail section, moreover it is possible to keep the contrast of color.
According to the actual requirements, a suitable scale parameter c value (usual c values are selected between 15,80,20) is chosen, then Retinex algorithm processing is carried out to image using gaussian kernel function, specific steps include:
A) calculate pending image R, G, B component size respectively first, and carry out data type normalized, and by picture Plain value is converted to floating number, to be calculated.
B) scale parameter is set and gaussian kernel function is created:Calculated including definition template matrix size, and by template size Go out template center i.e. filter radius c.
C) pending image R, G, B component are subjected to process of convolution with gaussian kernel function respectively, and result taken pair Number.
D) logarithm of the logarithm for obtaining step c) and input picture matrix is made the difference, and R, G, B component are carried out respectively Contrast stretching processing.
E) step d) is obtained into result and carries out fetching number processing, the R now obtained, G, B component constitute recovery image.
Retinex algorithm has preferable recovery effects, and it can strengthen image key message and increase image definition.This Inventive embodiments Contrast on effect result is as shown in Figure 2.
But although Retinex algorithm has recovered image color contrast after handling, image is also dry containing much noise Disturb, and single scale Retinex algorithm has limitation in terms of detail recovery.Therefore combination wavelet threshold of the embodiment of the present invention Enhancing algorithm is handled the noise and detailed information of the image after Retinex algorithm processing.
S1.2 realizes that the Wavelet Denoising Method of vehicle image is handled by improved wavelet threshold Denoising Algorithm:
Multiscale Wavelet Decomposition process is:One-dimensional wavelet decomposition first is carried out to image, low pass, high-pass filtering are carried out respectively, Then binary sampling is carried out, is averaged, details two parts coefficient.By constantly decomposing obtained low frequency coefficient to upper level Wavelet decomposition is carried out, a series of recursive procedures are to constitute small echo multilayer to decompose.
Wavelet function is:
In formula, R+=R- { 0 } be not equal to zero all real numbers, functionIt isFourier transformation, It is mother wavelet function.
For real number to (a, b), parameter a is non-zero real, function:
That is wavelet function, it is by wavelet mother functionThe continuous wavelet function obtained by flexible translation, wherein (a, B) contraction-expansion factor and shift factor are represented respectively.
F (x) continuous wavelet transform is then defined as:
It is inversely transformed into:
In the environment of low-light (level), the characteristics of IMAQ is influenceed greatly by environmental factor change, the embodiment of the present invention is adopted With the Threshold Filter Algorithms of most suitable small wave self-adaption, i.e. adaptive-filtering enhancing algorithm, its general principle is:
First with Multiscale Wavelet Decomposition, the details and noise coefficient of noisy image are obtained, then strengthens detail section Wavelet coefficient, by noise coefficient zero setting.The key wherein recovered is the suitable threshold function table of selection, and the threshold function table can be adaptive Wavelet coefficient under multiple dimensioned is handled with answering.The adaptive-filtering enhancing algorithm improves traditional algorithm and handled merely High frequency or the noise scale-up problem for by image wavelet coefficient bring during disposed of in its entirety.But the adaptive-filtering strengthens Algorithm can effectively strengthen image and suppress noise, and the image weaker to complex condition detail signal has enhancing effect well Really.
In order to further enhance treatment effect, image color pair has been recovered by Retinex algorithm processing in original image After degree, the embodiment of the present invention takes improved Adaptive Wavelet Thrinkage algorithm, and the Adaptive Wavelet Thrinkage algorithm can be effective Self-adaptive processing is carried out according to different Wavelet Components.The step of Adaptive Wavelet Thrinkage algorithm, specifically includes:
A) N layers of wavelet decomposition are carried out to the recovery image after Retinex algorithm is handled.
B) a wavelet threshold T is selectedj, compare wavelet coefficient and wavelet threshold TjDifference so as to judging correlation, correlation Big is main information, it is necessary to strengthen.Correlation is small, then is noise section, by the noise section wavelet coefficient zero setting.
C) all yardsticks are traveled through, all carrying out a threshold correlation in adjacent yardstick every bit judges to put with wavelet coefficient Zero or enhancing processing.
D) choosing an enhancing function strengthens image after step c) processing, then carries out wavelet reconstruction.
Wherein, wavelet threshold TjBasis for selecting be the information such as Noise Variance Estimation value, sub-band coefficients Energy distribution.If When threshold value chooses too small, much noise can be caused to remain, image effect is not good.If threshold value chooses excessive, can be by detailed information Noise signal removal is mistaken for, causes image information to lose.Wavelet decomposition scales increase, and the wavelet conversion coefficient of noise section subtracts It is small, so wavelet threshold T under different decomposition leveljSelected value is different, therefore the embodiment of the present invention uses effective algorithm for estimating Determine every layer of threshold value.
For wavelet threshold TjThe selection of (λ), using generic threshold value estimation functionWherein σ and N points Noise variance and signal length are not represented.The embodiment of the present invention chooses the white Gaussian noise of σ=0.01.And in real image processing During, because environment Complex Noise information and the variance of noise are uncertain, so first having to estimate noise level.This The estimating algorithm that invention is used is estimated using the intermediate value of the absolute value of the wavelet coefficient of details under each decomposition scale, is estimated Calculate formula as follows:
σ=median (| Wj,k|)/0.6745
The design sketch of the improved wavelet threshold Denoising Algorithm processing by result as shown in figure 3, understand the application present invention The image clearly that the improved wavelet threshold Denoising Algorithm of embodiment is recovered, simultaneously effective retains the useful letter of image Breath.
S2 License Plates:
S2.1 training car plate sample characteristics are extracted and feature organization;
S2.1.1 takes out any normal national standard car plate manually;
S2.1.2 is integrated channel characteristics extraction to the license plate image taken out;
Integrating channel feature was proposed by Doll á r P et al. in 2009, is generally used for pedestrian detection earliest, is to comment at present Estimate the preferable detective operators of effect.The basic thought of integrating channel feature is by carrying out various linear and non-thread to tablet pattern Property conversion, many common features of image, such as local summation, histogram, Haar and their mutation can be by products Component is fast and efficiently calculated.An input picture matrix I is given, its corresponding passage refers to original input picture Certain output response.For gray-scale map, its corresponding access matrix C=I, i.e. artwork are in itself;
For coloured picture, each of which Color Channel all corresponds to a passage.Other similar passages can be by various linear Calculate and obtain with non-linear method.Certain path computation function of Ω representative images is made, then corresponding channel C=Ω (I).
In the calculation, different conversion, which can be formed, chooses 3 kinds of different passages works in different channel types, the present invention For integrating channel feature, to ensure its accuracy.Wherein LUV Color Channels can describe car plate brightness well and colourity becomes Change, gradient magnitude passage reflects the profile of car plate well, and histogram of gradients passage is then comprehensive pair from different gradient directions The change of car plate position and attitude is described.3 kinds of passage transform effects are as shown in Figure 4.
The foundation of S2.1.2.1 LUV passages
In image procossing, LUV color spaces (full name CIE1976 (L*, U*, V*)) are better than rgb color space.LUV colors The purpose of color space is to set up the color space unified with the vision of people, is possessed between uniformity and uniformity and each color component It is uncorrelated.In LUV color spaces, L represents brightness, and U, V represent colourity.General pattern color is all RGB color, is led to Following formula is crossed to may switch in LUV color spaces.
Finally calculate L, U, V passage obtained in LUV color spaces.
S2.1.2.2 gradient magnitude passages:
Gradient magnitude is a kind of description method for Image Edge-Detection.Each pixel has eight neighbours in piece image Domain and four edge direction detections.In order to detect edge in pixel X-direction, Y-direction, Z-direction, the present invention is implemented Example determines pixel using X-direction Y-direction, the first-order partial derivative finite difference average of Z-direction is calculated respectively in the window The method of gradient magnitude.The gradient operator of four direction is respectively shown in Fig. 5.Wherein I [i, j] is that coordinate is 3 × 3 window centers The gradient magnitude of pixel centered on the gray value of pixel, M [i, j], its calculation formula is as follows, on correspondence four direction Calculation formula be:
M [i, j]=(| Px[i,j]|+|P45°[i,j]|+|Py[i,j]|+|P135°[i,j]|)
The gradient magnitude figure of entire image is finally obtained by above-mentioned formula.
S2.1.2.3 histogram of gradients passages:
Histogram of gradients thought source in gradient orientation histogram (Histograms of Oriented Gradients, HOG) to be Dalal in 2005 et al. be used for pedestrian by it recognizes and gains the name.HOG as a kind of local feature description son, to direction, Yardstick, illumination-insensitive.HOG is successfully applied to recognition of face by later Deniz et al., has obtained relatively good effect.
Histogram of gradients characteristic extraction procedure is as follows:
Step 1 takes 3 × 3 neighborhood of pixels as sampling window centered on image I [i, j].
Step 2 calculates the gradient direction θ [i, j] and gradient magnitude M [i, j] of the pixel [i, j].
θ [i, j]=arctan (I [i, j+1]-I [i, j-1])/I [i+1, j]-I [i-1, j]
As shown in fig. 6, arrow represents the direction of the pixel [i, j].
Gradient direction is divided into 6 directions by step 3, i.e., be divided into 6 parts, 30 ° of equispaced by 180 °.According to ellipse circle Gauss weighting scope all gradient direction angle identical pixel gradient magnitudes in the neighborhood of pixels are added.
The gradient magnitude that step 4 is finally counted on 6 directions adds up and obtains the gradient width on 6 directions of entire image Value figure.
The image for including 10 passages such as LUV passages, gradient magnitude passage, histogram of gradients passage finally obtained is as schemed Shown in 4.
S2.1.3 is based on Adaboost algorithm and trains detector:
Training stage, strong classifier is gone out to the integrating channel features training of extraction using Adaboost algorithm, differentiating rank Section, calculates and detects the integrating channel feature of positioning licence plate window, given a mark with strong classifier (differentiate car plate position oneself Reliability), finally store one section of that frame of video mid-score highest or a few two field pictures.
AdaBoost algorithms proposed that its essence is the classification of Weak Classifier by Schapire, Freund et al. in 1996 Learning process, is one kind of ensemble machine learning method, with computational efficiency is high, regulation parameter is few, structure for Weak Classifier Make compatible by force and to sample priori and the low advantage of data format requirement, therefore, be widely popularized. Each feature corresponds to a Weak Classifier in AdaBoost algorithms, but is not that each feature can describe prospect well The characteristics of target.How optimal characteristics are picked out from big measure feature and be fabricated to Weak Classifier, then it is integrated by Weak Classifier, High-precision strong classifier is finally obtained, is AdaBoost Algorithm for Training processes key issue to be solved.
The definition of Weak Classifier is:
Wherein, fjRepresent a feature, pjRepresent inequality direction, θjRepresent threshold value.
The specific training algorithms of S2.1.3.1
(1) n sample image, x are giveniIt is input sample image, yiIt is class formative, wherein yi=0 is expressed as negative sample This, yi=1 is expressed as positive sample.
(2) weight is initialized:
Wherein m and l are respectively the quantity of incorrect car plate sample and correct car plate sample, n=m+l.
(3) For t=1,2,3 ..., T
1st, normalized weight:Wherein ωtFor statistical distribution.
2nd, random selection integrating channel feature j:
Randomly choose passage index bink(k=1,2 ..., 10);
Randomly choose rectangular area RectjAnd calculate pixel value sum
3rd, to each feature j, a Weak Classifier h is trainedj, calculate corresponding ωtError rate:
εj=∑iωi|hj(xi)-yi|
4th, selection minimal error rate εtWeak Classifier ht
5th, weight is updated:Wherein, x is worked asiWhen correctly being classified, ei=0, conversely, ei=1;
(4) final strong classifier is h (x):
Wherein,
S2.2. the detection positioning of car plate;
S2.2.1 is scanned with sliding window method to the target image by strengthening processing, obtains just positioning licence plate image;
The embodiment of the present invention sets the sliding window of a fixed size according to the fixed proportion of domestic car plate, from acquisition video Image apex is proceeded by be scanned one by one, in order to improve scanning accuracy, and it is 4 pixels generally to set sliding window step-length, will be each Scanning truncated picture is integrated channel characteristics calculating, is compared, obtains with the strong detector that AdaBoost Algorithm for Training goes out (i.e. similarity highest) image-region, i.e. preliminary judgement to highest scoring is car plate position, intercepts the figure of the highest scoring As region is just positioning image and output detector.
The first positioning image that S2.2.2 exports detector carries out the first positioning result after non-maxima suppression processing and carried out Slant correction based on Hough transformation obtains the license plate image after secondary positioning;
Non-maxima suppression in object detection using quite varied, its main purpose be in order to eliminate unnecessary interference because Element, finds the position of optimal object detection.Non-maxima suppression is the last handling process of detection, is one of key link.
Heuristic window blending algorithm is fine to non-coincidence target detection effect, but for vehicle license plate detection and discomfort Close.Heuristic window blending algorithm, is divided into several misaligned subsets by initial detecting window, then calculates each subset Center, last each subset only retains a detection window, it is clear that the heuristic window blending algorithm easily causes a large amount of leakages Inspection.
Dalal etc. proposes average drifting non-maxima suppression method, and it not only calculates complexity, it is necessary to which detection window is existed 3-dimensional space (abscissa, ordinate, yardstick) represents that detection fraction is changed, the uncertain matrix of calculating, iteration optimization, but also Need to adjust parameter much associated with the step-length of detector etc., it is therefore, less at present to use.
Currently, most target detection generally uses the non-maxima suppression algorithm based on Greedy strategy, because it is simple Single efficient, key step is as follows:
(1) initial detecting window is sorted from high to low according to detection fraction;
(2) it regard the 1st initial detecting window as current suppression window;
(3) non-maxima suppression.All detection score ratios are currently suppressed into the low home window of window and are used as suppressed window Mouthful.Calculate the current overlapping area ratio for suppressing window and suppressed window:The union of common factor/area of area.Reject and overlap Area ratio is higher than the window of given threshold;
(4) if only remain last initial detecting window if terminate, otherwise according to the order sequenced, take it is next not by The window of suppression goes to step (3) as window is suppressed.
The embodiment of the present invention equally uses the simple efficient non-maxima suppression algorithm based on Greedy strategy, and will License plate image after non-maxima suppression is handled carries out the slant correction based on Hough transformation.
Hough transformation is a kind of strong feature extracting method, it using topography's information effectively accumulate it is all can Can model instance foundation, this causes it easily to obtain extra information from external data, again can observantly from Only effective information is showed in the example of some.Hough transformation is commonly utilized in shape in computer vision, position, geometry In the judgement of transformation parameter.Since being proposed from Hough transformation, it is widely used.In recent years, experts and scholars were to suddenly The theory property of husband's conversion has carried out further discussion again with application process.Hough transformation is used as a kind of effective identification straight line Algorithm, with good anti-interference and robustness.
Mapping of the Hough transformation method comprising one from the feature in image space to the set at parameter space midpoint.It is each Point in individual parameter space characterizes an example of model in image space, and characteristics of image is mapped to ginseng using a function In the middle of number space, this function is produced being capable of the compatible characteristics of image observed and all parameter groups for the model assumed Close.Each characteristics of image will produce a different plane in the parameter space of multidimensional, but be produced by all characteristics of image One section of the raw example for belonging to same model, which can all intersect, is describing the point of common example, Hough transformation it is basic It is these planes of generation and recognizes intersecting therewith parameter point.
License plate image after the slant correction based on Hough transformation is the image after secondary system positioning.Hough transformation The license plate image example of slant correction is as shown in Figure 7.
License plate image after the secondary positioning of output is inputted strong detector by S2.2.3, draws final positioning licence plate result.
After license plate image by non-maxima suppression processing and after the slant correction based on Hough transformation is exported again It is integrated after channel characteristics are extracted and inputs the secondary positioning of strong detector progress, including the strong inspection gone out with AdaBoost Algorithm for Training Survey device to be compared, obtain (i.e. similarity highest) image-region of highest scoring, that is, be determined as car plate position, interception is somebody's turn to do It is secondary positioning image and output detector to divide highest image-region, obtains final positioning result.
The interception of S3 car face area images:
After accurate car plate position is oriented, generally according to the length and width of car plate, choose certain ratio and enter to drive a vehicle face figure The interception of picture, so that the positive face of bayonet socket camera shoots vehicle region as an example, generally, 1.3 times of cars is respectively intercepted with the right and left of car plate Board length is the length in car face region, and vehicle picture height is for 0.3 times of car under 0.8 times of car plate length on car plate, car plate Board length, parameter can be adjusted in real time as needed.Each examples of parameters of car face image scope of interception is as shown in Figure 8.
S4, which sets up the S-SIFT characteristic models of license plate area image and combines SVM training aids, carries out vehicle cab recognition:
For the car face area image including vehicle information that has intercepted, it is necessary to which final car could be exported by being identified Type result, it is theoretical that the embodiment of the present invention is based on deep learning, it is proposed that a kind of model recognizing method based on sparse SIFT feature, The model recognizing method extracts the low layer SIFT feature vector of target license plate area image, then trained acquisition encoder dictionary first With sparse SIFT feature, deeper level characteristics of image is obtained, to adapt to different visual angles, illumination variation, shade, the complicated field such as block Scape, further improves discrimination;Sparse SIFT feature classification is finally realized with linear SVM, time complexity is reduced, Ensure real-time.Model recognizing method is comprised the following steps that:
S4.1 sets up S-SIFT characteristic models
S4.1.1 S-SIFT characteristics algorithms:
S-SIFT characteristics algorithms are on the basis of image SIFT feature, super complete dictionary base further to be trained, in L1 models The sparse SIFT of the lower coding of number constraint, it is possible to achieve higher level vehicle image is abstract.
Matrix X is defined comprising image in M S-SIFT Feature Descriptor of D dimensional feature spaces, X=(x1,…,xM)T, then X It can be expressed as:
X=WC
In formula:W is the coefficient of sparse coding, C=(c1,…,cK)TIt is K base vector.The sparse coding for solving X can be with table Levy and optimization problem is solved to W and C for following formula:
In formula:| | | | and | | L2 norms and L1 norms are represented respectively.From L1 norm constraint properties, penalty term | Wm| it ensure that the openness of coding result, sparse coefficient β controls | Wm| weight, i.e., it is openness.Base vector was complete (K> D), therefore C is usedgL2 constraint avoid trivial solution.
Although the formula of solutionWhen W and C change simultaneously, object function be not it is convex optimization ask Topic, but when fixing W and C respectively, object function deteriorates to the convex function on C and W respectively.During fixed W, object function is degenerated For the least square problem on C:
Lagrange duality algorithm rapid solving can be used.Fixed C, object function is deteriorated to individually to each WmAsk most The linear regression problem of excellent solution:
It can be solved with characteristic symbol searching algorithm.
D=128 is chosen in experiment of the embodiment of the present invention, and β=0.15, K is from 8,32,128,512,1024 totally 5 kinds of codings Dimension.M depends on image size.By taking the image of 256 × 256 pixels as an example, SIFT tile sizes are defined as 16 × 16 Pixel, step-length is 6, then laterally makees (256-16)/6=40 matching, longitudinal direction is madeSecondary matching, M=40 × 40, i.e., 1600, with 512 dimension S-SIFT algorithm process SIFT features, the sparse coding of final output for 1600 512 dimensions to Amount.
S4.1.2 ponds
Pond is the process for counting sparse coding result, and it simulates the physiological mechanism of human eye vision cortex, it is possible to reduce defeated Incoming vector dimension, advantageously reduces the time complexity of training grader.With 256 × 256 image described in the embodiment of the present invention Exemplified by, its sparse SIFT coding dimension is 1600 × 512=819200, trains a classification of the input vector dimension more than 800,000 Device difficulty is very big, and over-fitting easily occurs.Therefore the present embodiment uses pond method, and the summary statistics for obtaining piece image are special Levy, not only reduce the difficulty of training grader, and avoid over-fitting.
Pond method common at present has average pondization and maximum pond etc., and computational methods are:
Average pond:
Maximum pond:pj=max | w1j|,…,|wMj|}
In formula:WmIt is sparse coding vector;P is pond result;wijRepresent j-th of element of i-th of sparse coding vector. Feature behind pond is with simple Linear SVM grader with regard to that can reach preferable classifying quality, and time complexity is only O (n).
S4.2 car face area image vehicle tagsorts:
The parameter training of S4.2.1 SVM classifiers
Vehicle tagsort is primarily referred to as the car face area image including vehicle information to be identified with passing through study Training vehicle feature is contrasted to be identified by a certain algorithm.Conventional grader mainly includes minimum distance classification Device, K- nearest neighbor classifiers, Bayes classifier, decision tree, Adaboost cascade classifiers, artificial neural network and support to Amount machine (SVM).The characteristics of training vehicle picture characteristics and the different classifications device of classification as needed, the present invention is main using support Vector machine is classified.The core concept of SVMs is using an Optimal Separating Hyperplane when the curved surface made decision, and comes most Change the Edge Distance of both positive class and negative class greatly.
In the present invention, Q={ (x are definedi,yi), i=1 ..., n, wherein Q are n input data point sets;xiRepresent input Variable;yiRepresent desired value, the y in two class problemsi∈{1,-1}.Classification function is defined as:
In formula,Represent the mapping from the input space to high-dimensional feature space.According to sequential minimal optimization algorithm (sequential minimal optimization, SMO) can be as follows in the hope of decision function:
In formula:aiRepresent Lagrange multiplier;k<xi,x>Kernel function is represented, is mapped to for quickly calculating after higher dimensional space Two vectorial inner products.
The linear core of common kernel function, Gaussian kernel, polynomial kernel.Non-linear Kernel SVM classifier is used, its training time answers Miscellaneous degree is O (n2~n3), classification time complexity is O (n);And training time complexity can be then reduced to by O using linear kernel (n), classification time complexity is still O (n).In actual applications, generally use linear kernel function to improve training effectiveness, it is ensured that System real time.
In summary, after feature is extracted, classification is trained using SVM.After training classification, interception is included into car In the car face area image input SVM training aids of type characteristic information, and export the vehicle information of identification.
The method proposed in the present invention can actually be embedded in FPGA realizations, apply to the car with real-time output image function In the camera of type identification function or camera supervised system.
Those skilled in the art will be clear that the scope of the present invention is not restricted to example discussed above, it is possible to which it is carried out Some changes and modification, the scope of the present invention limited without departing from appended claims.Although oneself is through in accompanying drawing and explanation The present invention is illustrated and described in book in detail, but such explanation and description are only explanations or schematical, and it is nonrestrictive. The present invention is not limited to the disclosed embodiments.

Claims (5)

1. the vehicle targets under a kind of low light conditions combined based on Retinex with S-SIFT features, it is characterised in that Comprise the following steps:
S1) the low-light (level) image enhaucament based on Retinex algorithm and wavelet transformation;
S2) License Plate;
S3 car face area image) is intercepted;
S4 the S-SIFT characteristic models of car face area image) are set up and SVM training aids is combined and carry out vehicle cab recognition.
2. the vehicle under a kind of low light conditions combined based on Retinex with S-SIFT features according to claim 1 Recognizer, it is characterised in that the step S1) specifically include:
S1.1) recover the shape constancy of the colourity of license plate image by using single scale Retinex algorithm and strengthen the clear of image Degree;
S1.2) Wavelet Denoising Method for realizing vehicle image by improved wavelet threshold Denoising Algorithm is handled, and is specifically included:
A) N layers of wavelet decomposition are carried out to the recovery image after Retinex algorithm is handled;
B) a wavelet threshold T is selectedj, compare wavelet coefficient and wavelet threshold TjDifference so as to judge correlation, correlation is big It is main information, it is necessary to strengthen.Correlation is small, then is noise section, by the noise section wavelet coefficient zero setting;
C) all yardsticks are traveled through, adjacent yardstick every bit all carry out a threshold correlation judge and wavelet coefficient zero setting or Enhancing is handled;
D) choosing an enhancing function strengthens image after step c) processing, then carries out wavelet reconstruction.
3. the vehicle under a kind of low light conditions combined based on Retinex with S-SIFT features according to claim 1 Recognizer, it is characterised in that the step S2) specifically include S2.1) training car plate sample characteristics extract and feature organization and S2.2) the detection positioning of car plate;
The S2.1) include:
S2.1.1 any normal national standard car plate) is taken out manually;
S2.1.2 channel characteristics extraction) is integrated to the license plate image taken out:Set up LUV passages, gradient width respectively first It is worth passage and histogram of gradients passage, and respective channel is obtained according to LUV passages, gradient magnitude passage and histogram of gradients passage License plate image feature;
S2.1.3 detector) is trained based on Adaboost algorithm:Training stage, the integration of extraction is led to using Adaboost algorithm Road features training goes out strong classifier;Differentiating the stage, calculating the integrating channel feature for detecting positioning licence plate window, with strong point Class device differentiates the Confidence of car plate position, finally stores self-confident that frame of highest or a few two field pictures in one section of video;
The S2.2) include:
S2.2.1) target image by strengthening processing is scanned using sliding window method, each scanning truncated picture is entered Row integrating channel feature calculation, is compared with the strong detector that AdaBoost Algorithm for Training goes out, and chooses similarity highest figure As region is used as first positioning licence plate image;
S2.2.2) the first positioning result that the first positioning image for exporting detector is carried out after non-maxima suppression processing is based on The slant correction of Hough transformation, and the image after slant correction is integrated again channel characteristics extract after input strong detector Secondary positioning is carried out, the license plate image after secondary positioning is obtained.
4. under a kind of low light conditions combined based on Retinex with S-SIFT features according to claim 1 or 2 or 3 Vehicle targets, it is characterised in that the step S3) specifically include:After accurate car plate position is oriented, usual root According to the length and width of car plate, choose certain ratio and enter the interception of face image of driving a vehicle.
5. the vehicle under a kind of low light conditions combined based on Retinex with S-SIFT features according to claim 4 Recognizer, it is characterised in that the step S4) specifically include:
S4.1 the S-SIFT characteristic models of car face area image) are set up, it is included using S-SIFT algorithm process car face administrative division maps Sparse coding is obtained as SIFT feature describes son, and uses pond method statistic sparse coding result, car face area image is obtained Summary statistics feature;
S4.2) car face area image vehicle tagsort and identification, it is included in after extraction S-SIFT features, instructed using SVM Practice device and be trained classification;After training classification, by the input SVM training of the car face area image comprising vehicle characteristic information of interception In device, and export the vehicle information of identification.
CN201710237135.3A 2017-04-12 2017-04-12 The low-light (level) model recognizing method combined based on Retinex with S SIFT features Pending CN107092876A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201710237135.3A CN107092876A (en) 2017-04-12 2017-04-12 The low-light (level) model recognizing method combined based on Retinex with S SIFT features

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201710237135.3A CN107092876A (en) 2017-04-12 2017-04-12 The low-light (level) model recognizing method combined based on Retinex with S SIFT features

Publications (1)

Publication Number Publication Date
CN107092876A true CN107092876A (en) 2017-08-25

Family

ID=59637903

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201710237135.3A Pending CN107092876A (en) 2017-04-12 2017-04-12 The low-light (level) model recognizing method combined based on Retinex with S SIFT features

Country Status (1)

Country Link
CN (1) CN107092876A (en)

Cited By (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108717529A (en) * 2018-05-17 2018-10-30 山东深图智能科技有限公司 A kind of extremely low quality license plate character recognition method and system based on vehicle
CN109087268A (en) * 2018-08-17 2018-12-25 凌云光技术集团有限责任公司 Image enchancing method under a kind of low light environment
CN109840809A (en) * 2019-03-12 2019-06-04 中国联合网络通信集团有限公司 Advertisement placement method and device
CN109903545A (en) * 2019-03-27 2019-06-18 湖北医药学院 A kind of method and system of car networking data transmission
CN110210608A (en) * 2019-06-05 2019-09-06 国家广播电视总局广播电视科学研究院 The enhancement method of low-illumination image merged based on attention mechanism and multi-level features
CN110378236A (en) * 2019-06-20 2019-10-25 西安电子科技大学 Testing vehicle register identification model construction, recognition methods and system based on deep learning
CN110580469A (en) * 2019-09-10 2019-12-17 广州麦仑信息科技有限公司 palm vein recognition system and method based on embedded equipment
CN110796697A (en) * 2019-11-04 2020-02-14 刘勇 General surgery ward object size identification platform and method
CN111081383A (en) * 2020-01-20 2020-04-28 罗晖 Epidemic disease monitoring system based on cloud computing
CN112070812A (en) * 2020-08-21 2020-12-11 广东电网有限责任公司广州供电局 Feature descriptor generation method for electrical equipment

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101520841A (en) * 2009-03-10 2009-09-02 北京航空航天大学 Real-time and anti-interference method for positioning license plate in high-definition TV video
CN103544489A (en) * 2013-11-12 2014-01-29 公安部第三研究所 Device and method for locating automobile logo
CN103903018A (en) * 2014-04-02 2014-07-02 浙江师范大学 Method and system for positioning license plate in complex scene
CN105279512A (en) * 2015-10-22 2016-01-27 东方网力科技股份有限公司 Tilt vehicle license plate recognition method and device
CN105335743A (en) * 2015-10-28 2016-02-17 重庆邮电大学 Vehicle license plate recognition method
CN106056101A (en) * 2016-06-29 2016-10-26 哈尔滨理工大学 Non-maximum suppression method for face detection
CN106204470A (en) * 2016-07-01 2016-12-07 湖南源信光电科技有限公司 Low-light-level imaging method based on fuzzy theory
CN106503748A (en) * 2016-11-07 2017-03-15 湖南源信光电科技有限公司 A kind of based on S SIFT features and the vehicle targets of SVM training aids

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101520841A (en) * 2009-03-10 2009-09-02 北京航空航天大学 Real-time and anti-interference method for positioning license plate in high-definition TV video
CN103544489A (en) * 2013-11-12 2014-01-29 公安部第三研究所 Device and method for locating automobile logo
CN103903018A (en) * 2014-04-02 2014-07-02 浙江师范大学 Method and system for positioning license plate in complex scene
CN105279512A (en) * 2015-10-22 2016-01-27 东方网力科技股份有限公司 Tilt vehicle license plate recognition method and device
CN105335743A (en) * 2015-10-28 2016-02-17 重庆邮电大学 Vehicle license plate recognition method
CN106056101A (en) * 2016-06-29 2016-10-26 哈尔滨理工大学 Non-maximum suppression method for face detection
CN106204470A (en) * 2016-07-01 2016-12-07 湖南源信光电科技有限公司 Low-light-level imaging method based on fuzzy theory
CN106503748A (en) * 2016-11-07 2017-03-15 湖南源信光电科技有限公司 A kind of based on S SIFT features and the vehicle targets of SVM training aids

Non-Patent Citations (4)

* Cited by examiner, † Cited by third party
Title
张鹏等: ""采用稀疏SIFT特征的车型识别方法"", 《西安交通大学学报》 *
焦振等: ""基于小波分析和Retinex算法的图像增强算法"", 《吉林大学学报(理学版)》 *
赵阳等: ""基于车脸特征的车型识别技术及其在公安领域的应用"", 《警察技术》 *
陈金辉等: ""行人检测中非极大值抑制算法的改进"", 《华东理工大学学报(自然科学版)》 *

Cited By (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108717529A (en) * 2018-05-17 2018-10-30 山东深图智能科技有限公司 A kind of extremely low quality license plate character recognition method and system based on vehicle
CN109087268A (en) * 2018-08-17 2018-12-25 凌云光技术集团有限责任公司 Image enchancing method under a kind of low light environment
CN109840809A (en) * 2019-03-12 2019-06-04 中国联合网络通信集团有限公司 Advertisement placement method and device
CN109903545A (en) * 2019-03-27 2019-06-18 湖北医药学院 A kind of method and system of car networking data transmission
CN110210608A (en) * 2019-06-05 2019-09-06 国家广播电视总局广播电视科学研究院 The enhancement method of low-illumination image merged based on attention mechanism and multi-level features
CN110378236A (en) * 2019-06-20 2019-10-25 西安电子科技大学 Testing vehicle register identification model construction, recognition methods and system based on deep learning
CN110580469A (en) * 2019-09-10 2019-12-17 广州麦仑信息科技有限公司 palm vein recognition system and method based on embedded equipment
CN110580469B (en) * 2019-09-10 2022-08-09 广州麦仑信息科技有限公司 Palm vein recognition system and method based on embedded equipment
CN110796697A (en) * 2019-11-04 2020-02-14 刘勇 General surgery ward object size identification platform and method
CN111081383A (en) * 2020-01-20 2020-04-28 罗晖 Epidemic disease monitoring system based on cloud computing
CN112070812A (en) * 2020-08-21 2020-12-11 广东电网有限责任公司广州供电局 Feature descriptor generation method for electrical equipment

Similar Documents

Publication Publication Date Title
CN107092876A (en) The low-light (level) model recognizing method combined based on Retinex with S SIFT features
CN107103317A (en) Fuzzy license plate image recognition algorithm based on image co-registration and blind deconvolution
CN108520216B (en) Gait image-based identity recognition method
CN105354572B (en) A kind of automatic license plate identification system based on simplified convolutional neural networks
CN104809443B (en) Detection method of license plate and system based on convolutional neural networks
CN102509091B (en) Airplane tail number recognition method
CN104050471B (en) Natural scene character detection method and system
CN103218831B (en) A kind of video frequency motion target classifying identification method based on profile constraint
CN105718889B (en) Based on GB (2D)2The face personal identification method of PCANet depth convolution model
CN106650786A (en) Image recognition method based on multi-column convolutional neural network fuzzy evaluation
CN107122776A (en) A kind of road traffic sign detection and recognition methods based on convolutional neural networks
CN107066933A (en) A kind of road sign recognition methods and system
CN105447503B (en) Pedestrian detection method based on rarefaction representation LBP and HOG fusion
CN106529532A (en) License plate identification system based on integral feature channels and gray projection
CN104408469A (en) Firework identification method and firework identification system based on deep learning of image
CN102509112A (en) Number plate identification method and identification system thereof
CN105740886B (en) A kind of automobile logo identification method based on machine learning
CN106503748A (en) A kind of based on S SIFT features and the vehicle targets of SVM training aids
CN109635784A (en) Traffic sign recognition method based on improved convolutional neural networks
CN104504395A (en) Method and system for achieving classification of pedestrians and vehicles based on neural network
CN106778742B (en) Car logo detection method based on Gabor filter background texture suppression
CN109886086B (en) Pedestrian detection method based on HOG (histogram of oriented gradient) features and linear SVM (support vector machine) cascade classifier
CN106529461A (en) Vehicle model identifying algorithm based on integral characteristic channel and SVM training device
CN112464731B (en) Traffic sign detection and identification method based on image processing
CN102147867A (en) Method for identifying traditional Chinese painting images and calligraphy images based on subject

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication
RJ01 Rejection of invention patent application after publication

Application publication date: 20170825