CN110443245A - Localization method, device and the equipment of a kind of license plate area under unrestricted scene - Google Patents
Localization method, device and the equipment of a kind of license plate area under unrestricted scene Download PDFInfo
- Publication number
- CN110443245A CN110443245A CN201910747526.9A CN201910747526A CN110443245A CN 110443245 A CN110443245 A CN 110443245A CN 201910747526 A CN201910747526 A CN 201910747526A CN 110443245 A CN110443245 A CN 110443245A
- Authority
- CN
- China
- Prior art keywords
- vehicle
- image
- license plate
- size
- vehicle image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 23
- 230000004807 localization Effects 0.000 title claims abstract description 9
- 238000003062 neural network model Methods 0.000 claims abstract description 27
- PXFBZOLANLWPMH-UHFFFAOYSA-N 16-Epiaffinine Natural products C1C(C2=CC=CC=C2N2)=C2C(=O)CC2C(=CC)CN(C)C1C2CO PXFBZOLANLWPMH-UHFFFAOYSA-N 0.000 claims abstract description 19
- 238000001514 detection method Methods 0.000 claims abstract description 17
- 230000009466 transformation Effects 0.000 claims abstract description 17
- 238000013528 artificial neural network Methods 0.000 claims description 22
- 238000013527 convolutional neural network Methods 0.000 claims description 11
- 238000007781 pre-processing Methods 0.000 claims description 7
- 230000005284 excitation Effects 0.000 claims description 6
- 238000012549 training Methods 0.000 claims description 5
- 238000012545 processing Methods 0.000 claims description 3
- 238000013519 translation Methods 0.000 claims description 3
- 230000006870 function Effects 0.000 description 13
- 238000010586 diagram Methods 0.000 description 8
- 238000005516 engineering process Methods 0.000 description 4
- 230000008569 process Effects 0.000 description 3
- 241000208340 Araliaceae Species 0.000 description 2
- 235000005035 Panax pseudoginseng ssp. pseudoginseng Nutrition 0.000 description 2
- 235000003140 Panax quinquefolius Nutrition 0.000 description 2
- 235000008434 ginseng Nutrition 0.000 description 2
- 238000007689 inspection Methods 0.000 description 2
- 230000000644 propagated effect Effects 0.000 description 2
- 238000000844 transformation Methods 0.000 description 2
- 208000004350 Strabismus Diseases 0.000 description 1
- 230000003321 amplification Effects 0.000 description 1
- 230000008901 benefit Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000012937 correction Methods 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 238000003199 nucleic acid amplification method Methods 0.000 description 1
- 230000008439 repair process Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
- G06F18/2413—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on distances to training or reference patterns
- G06F18/24133—Distances to prototypes
- G06F18/24137—Distances to cluster centroïds
- G06F18/2414—Smoothing the distance, e.g. radial basis function networks [RBFN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/02—Affine transformations
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/80—Geometric correction
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/32—Normalisation of the pattern dimensions
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/60—Type of objects
- G06V20/62—Text, e.g. of license plates, overlay texts or captions on TV images
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/60—Type of objects
- G06V20/62—Text, e.g. of license plates, overlay texts or captions on TV images
- G06V20/625—License plates
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Engineering & Computer Science (AREA)
- Computing Systems (AREA)
- Software Systems (AREA)
- Molecular Biology (AREA)
- Computational Linguistics (AREA)
- Biophysics (AREA)
- Biomedical Technology (AREA)
- Mathematical Physics (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- Multimedia (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Biology (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses localization method, device and the equipment of the license plate area under a kind of unrestricted scene, wherein, acquire the image of vehicle, the image is inputted into default neural network model, obtain the vehicle image comprising the vehicle image, wherein, which has the image of the vehicle in the image for identification；According to the ratio between the size and vehicle bounding box of license plate, the scaling of picture size is carried out to the vehicle image, obtains the vehicle image of uniform sizes；The vehicle image is inputted into full convolutional network, wherein, the full convolutional network identifies and determines license plate in the region of the vehicle image, wherein, the full convolutional network is used to calculate the probability for being covered with the license plate with target point corresponding unit in the vehicle image, and the full convolutional network solves the problems, such as unrestricted scene license plate area detection inaccuracy, improves the precision in License Plate region for exporting affine transformation parameter to correct the size of the license plate area.
Description
Technical field
The present invention relates to field of image recognition, in particular to a kind of positioning of the license plate area under unrestricted scene
Method, device and equipment.
Background technique
Common business software is to be identified in the case where scene is limited to license plate, such as parking lot, receive mostly
Take the monitoring at station, these scenes are all very fixed, and more demanding to the angle and distance of vehicle shooting.Current many Car license recognitions
System, it is good to front shooting license plate photo recognition effect, but unrestricted scene, such as freeway surveillance and control, mobile phone are clapped
Image etc. is taken the photograph, the license plate recognition technology limited under scene is unable to reach practical standard.And car plate detection skill under unrestricted scene
Art is the prerequisite of Car license recognition.License plate area is only correctly detected in the picture, could carry out Car license recognition process.
For in the related technology, unrestricted scene license plate area detects inaccurate problem, not yet proposes effective solution at present
Certainly scheme.
Summary of the invention
Inaccurate problem is detected for unrestricted scene license plate area in the related technology, the present invention provides a kind of unrestricted
Localization method, device and the equipment of license plate area under scene.
According to an aspect of the invention, there is provided a kind of localization method of the license plate area under unrestricted scene, special
Sign is, which comprises
Described image is inputted default neural network model, obtained comprising the vehicle image by the image for acquiring vehicle
Vehicle image, wherein the default neural network model has the image of the vehicle in described image for identification；
According to the ratio between the size and vehicle bounding box of license plate, the contracting of picture size is carried out to the vehicle image
It puts, obtains the vehicle image of uniform sizes；
The vehicle image is inputted into full convolutional network, wherein the full convolutional network identifies and determines license plate in the vehicle
The region of image, wherein the full convolutional network is covered with for calculating with target point corresponding unit in the vehicle image
The probability of the license plate and the full convolutional network are for exporting affine transformation parameter to correct the big of the license plate area
It is small.
It further, include YOLO network and shallow-layer neural network in the default neural network, wherein the shallow-layer mind
It include the shallow-layer convolutional neural networks of 2 convolutional layers and 1 full articulamentum through network.
Further, the ratio between the size and vehicle bounding box according to license plate, carries out the vehicle image
The scaling of picture size, the vehicle image for obtaining uniform sizes include:
In the case that ratio between the size of license plate and vehicle bounding box is lower than preset first threshold value, amplify the vehicle
Image；
In the case that ratio between the size of license plate and vehicle bounding box is greater than default second threshold, the vehicle is reduced
Image；
According to the preset first threshold value and the default second threshold, the size of the unified vehicle image.
Further, the full convolutional network includes the first convolution module and the second convolution module, wherein first volume product module
Block has used Softmax function for calculating the probability that target point corresponding unit in the vehicle image is covered with the license plate
As excitation function, second convolution module is not use excitation function for obtaining the affine transformation parameter.
Further, the method for the training dataset of the described image of the extension input default neural network model includes
At least one of:
Adjusting license plate center described in described image becomes picture centre；
License plate described in described image is zoomed in and out, its width is made to match 40px to the value between 208px；
The rotation of 3D angle is carried out to described image at random；
50% probability carries out mirror image processing to described image；
Random translation, the license plate is mobile from described image center；
Described image is cut centered on the license plate；
It modifies to the tone saturation degree lightness hsv color space of described image.
According to another aspect of the present invention, a kind of positioning device of license plate area is additionally provided, unrestricted scene is used for
Under license plate area positioning, described device includes:
Described image is inputted default neural network model, obtaining includes institute for acquiring the image of vehicle by input module
State the vehicle image of vehicle image, wherein the default neural network model has the vehicle in described image for identification
Image；
Preprocessing module, for according to license plate size and vehicle bounding box between ratio, to the vehicle image into
The scaling of row picture size obtains the vehicle image of uniform sizes；
Recognition detection module, for the vehicle image to be inputted full convolutional network, wherein the full convolutional network identification is simultaneously
Determine license plate in the region of the vehicle image, wherein the full convolutional network is for calculating and target in the vehicle image
The probability and the full convolutional network that point corresponding unit is covered with the license plate are for exporting affine transformation parameter to correct
State the size of license plate area.
It further, include YOLO network and shallow-layer neural network in the default neural network, wherein the shallow-layer mind
It include the shallow-layer convolutional neural networks of 2 convolutional layers and 1 full articulamentum through network.
Further, the preprocessing module is also used to the ratio between the size and vehicle bounding box of license plate lower than pre-
If in the case where first threshold, amplifying the vehicle image, the ratio between the size and vehicle bounding box of license plate is greater than pre-
If in the case where second threshold, reducing the vehicle image, according to the preset first threshold value and the default second threshold, system
The size of one vehicle image.
According to another aspect of the present invention, additionally provide a kind of car license recognition equipment, the equipment include camera and
Processor,
The camera is used to acquire the image of vehicle, and described image is inputted default neural network mould by the processor
Type obtains the vehicle image comprising the vehicle image, wherein the default neural network model is for identification in described image
There is the image of the vehicle；
The processor, the ratio being also used between the size and vehicle bounding box according to license plate, to the vehicle image
The scaling for carrying out picture size, obtains the vehicle image of uniform sizes；
The processor is also used to the vehicle image inputting full convolutional network, wherein the full convolutional network identification is simultaneously
Determine license plate in the region of the vehicle image, wherein the full convolutional network is for calculating and target in the vehicle image
The probability and the full convolutional network that point corresponding unit is covered with the license plate are for exporting affine transformation parameter to correct
State the size of license plate area.
It further, include YOLO network and shallow-layer neural network in the default neural network, wherein the shallow-layer mind
It include the shallow-layer convolutional neural networks of 2 convolutional layers and 1 full articulamentum through network.
Through the invention, which is inputted default neural network model by the image for acquiring vehicle, and obtaining includes the vehicle
The vehicle image of image, wherein the default neural network model has the image of the vehicle in the image for identification；According to license plate
Size and vehicle bounding box between ratio, to the vehicle image carry out picture size scaling, obtain the vehicle of uniform sizes
Image；The vehicle image is inputted into full convolutional network, wherein the full convolutional network identifies and determines license plate in the vehicle image
Region, wherein the full convolutional network is covered with the general of the license plate with target point corresponding unit in the vehicle image for calculating
Rate and the full convolutional network solve unrestricted field for exporting affine transformation parameter to correct the size of the license plate area
Scape license plate area detects inaccurate problem, improves the precision in License Plate region.
Detailed description of the invention
The drawings described herein are used to provide a further understanding of the present invention, constitutes part of this application, this hair
Bright illustrative embodiments and their description are used to explain the present invention, and are not constituted improper limitations of the present invention.In the accompanying drawings:
Fig. 1 is the flow chart of the localization method of the license plate area under a kind of unrestricted scene according to an embodiment of the present invention；
Fig. 2 is the schematic diagram of vehicle detection neural networks principles according to an embodiment of the present invention；
Fig. 3 is the schematic diagram of full convolutional network LPnet framework according to an embodiment of the present invention；
Fig. 4 is a kind of structural block diagram of license plate area positioning device according to an embodiment of the present invention；
Fig. 5 is a kind of structural block diagram of car license recognition equipment according to an embodiment of the present invention.
Specific embodiment
Hereinafter, the present invention will be described in detail with reference to the accompanying drawings and in combination with Examples.It should be noted that not conflicting
In the case of, the features in the embodiments and the embodiments of the present application can be combined with each other.
In the present embodiment, a kind of localization method of the license plate area under unrestricted scene is provided, Fig. 1 is according to this hair
The flow chart of the localization method of license plate area under a kind of unrestricted scene of bright embodiment, as shown in Figure 1, this method includes such as
Lower step:
Step S102 acquires the image of vehicle, which is inputted default neural network model, obtaining includes the vehicle figure
The vehicle image of picture, wherein the default neural network model has the image of the vehicle in the image for identification；
Step S104 carries out image ruler to the vehicle image according to the ratio between the size and vehicle bounding box of license plate
Very little scaling obtains the vehicle image of uniform sizes；
The vehicle image is inputted full convolutional network, wherein the full convolutional network identifies and determines that license plate exists by step S106
The region of the vehicle image, wherein the full convolutional network is covered with for calculating with target point corresponding unit in the vehicle image
The probability of the license plate and the full convolutional network are for exporting affine transformation parameter to correct the size of the license plate area.
Through the above steps, which is inputted default neural network model by the image for acquiring vehicle, and obtaining includes the vehicle
The vehicle image of image, wherein the default neural network model has the image of the vehicle in the image for identification；According to vehicle
Ratio between the size and vehicle bounding box of board carries out the scaling of picture size to the vehicle image, obtains uniform sizes
Vehicle image；The vehicle image is inputted into full convolutional network, wherein the full convolutional network identifies and determines license plate in the vehicle figure
The region of picture solves the problems, such as unrestricted scene license plate area detection inaccuracy, improves the precision in License Plate region.
In the present embodiment, vehicle image is detected, it is desirable that higher discrimination, because any missing inspection vehicle will be direct
The missing inspection for leading to entire license plate requires in addition that the propagated forward time of vehicle detection network is shorter, so just reality with higher
Shi Xing.YOLO network can be used in the default neural network model, it has the advantage for executing that speed is fast and accuracy of identification is high, needle
To YOLO network, in the present embodiment, the relevant output of vehicle (i.e. automobile and bus) is incorporated, and ignore other classes
Not.
In order to improve vehicle detection network to the accuracy of identification under a certain scene, the present embodiment increases after YOLO network
One shallow-layer neural network.Fig. 2 is the schematic diagram of vehicle detection neural networks principles according to an embodiment of the present invention, wherein should
Shallow-layer neural network includes the shallow-layer convolutional neural networks of 2 convolutional layers and 1 full articulamentum, by the full articulamentum output of YOLO
Feature Conversion, as input picture, the training for having supervision is carried out to shallow-layer convolutional neural networks to two-dimensional space.It is shallow by training
Layer convolutional neural networks, can learn the complex relationship between higher level of abstraction feature.The structure and ginseng of shallow-layer convolutional neural networks
It is convolutional layer that number, which is layers 1 and 2, and convolution kernel size is 5 × 5, and level 1 volume product nuclear volume is 96, level 2 volume product nucleus number
Amount is 72.Then 1 full articulamentum, size 1000 are connect.
In an embodiment of the present invention, according to the ratio between the size and vehicle bounding box of license plate, to the vehicle image
The scaling of picture size is carried out, the vehicle image for obtaining uniform sizes may include:
In the case that ratio between the size of license plate and vehicle bounding box is lower than preset first threshold value, amplify the vehicle
Image；
In the case that ratio between the size of license plate and vehicle bounding box is greater than default second threshold, the vehicle is reduced
Image；
According to the preset first threshold value and the default second threshold, unify the size of the vehicle image.
In an embodiment of the present invention, for detected comprising vehicle image, vehicle image size is located in advance
After reason, it is sent into the subsequent recognition detection for carrying out license plate.Before and after the vehicle in front view, license plate size and vehicle bounding box it
Between ratio it is higher.For strabismus/side view, vehicle bounding box is often bigger, therefore the ratio is much smaller.In order to guarantee
The identifiability of license plate area, when vehicle photo is angled, it should which amplification picture is to amplify license plate area；In vehicle photo sheet
When coming just very big, then corresponding diminution is done.Here it is pre-process to vehicle image size.The zoom factor meter of vehicle image
The following formula 1 of calculation method:
Wherein, WvAnd HvIt is the width and height of vehicle bounding box, D respectivelyminFor the minimum dimension picture of vehicle image size
Element value, DmaxFor the full-size pixel value of vehicle image size, wherein Dmin≤fsc*min(Wv, Hv)≤Dmax.In experiment
On the basis of, in order to keep good balance between precision and runing time, select Dmin=240px and Dmax=580px.It is logical
It crosses after this scaling calculating, includes that the picture of vehicle image can be unified into the picture that size is 240*580 size.
In the present embodiment, Fig. 3 is the schematic diagram of full convolutional network LPnet framework according to an embodiment of the present invention, such as Fig. 3
Shown, which generates the probability of a characteristic pattern and coded object and the ginseng of affine transformation
Number.If the destination probability of the unit centered on (m, n) is greater than the threshold value of detection, illustrate there is vehicle in the region of unit covering
Board reuses the affine transformation parameter of output to correct license plate.
In order to guarantee the speed of car plate detection, the embodiment of the present invention selects MobileNet V2 algorithm model, the network
It is to be wanted using the network of design so that can also reach ideal speed on a processor for mobile terminal and embedded end deep learning
It asks.In context of detection, embodiment devises tool, and there are two parallel convolution module (convolution kernel are CONV 3 × 3,2): the first volume
Volume module is: for calculating the probability of license plate, having used softmax function as excitation function.Second convolution module is: using
In obtaining affine parameter, excitation function is not used.
For LPnet frame above-mentioned in this implementation, a kind of Loss function is devised, which has detection license plate and output
The function of affine parameter.If pi=[xi, yi]T, i=1...4 indicates four angles of license plate.The size of domestic license plate is 440mm
Under this ratio the apex coordinate of standard license plate is arranged are as follows: q in × 140mm1=[- 0.5,0.3]T, q2=[0.5,0.3]T, q3
=[0.5, -0.3]Tq4=[- 0.5, -0.3]T。
For the input picture of height H and width W, network output characteristic pattern is made of the convolution of M × N × 8, wherein M=H/
Ns, N=W/Ns, NsFor the ratio of image down after propagated forward.For the corresponding unit of each point (m, n) in characteristic pattern,
There are eight values to be estimated: the first two value (v1And v2) it is target and non-targeted probability, rear six values (v3To v8) for constructing
Local affine transformations Tmn, local affine transformations TmnCalculate following formula 2:
For matching network output resolution ratio, point piCoordinate needs re-scaling, and according in characteristic pattern each point (m,
N) again placed in the middle.It is realized by normalized function.
Wherein α is a proportionality constant, is used for Unitary coordinate into 0-1 numberical range.
Assuming that there is a license plate target at (m, n) corresponding unit, then the first part of loss function is license plate motion correction
The statement of error between standard license plate coordinate such as formula 4 afterwards:
The second part of loss function is to judge in (m, n) corresponding unit with and without the probability of license plate such as formula 5
Statement.
fprobs(m, n)=logloss (IIobj, v1)+logloss(1-IIobj, v2) formula 5
Wherein Ι ΙobjIt is object indicator function, if having license plate in (m, n) corresponding unit, returns to 1, otherwise
Logloss (y, p)=- ylog (p).
Final loss function combines the statement such as formula 6 as defined in formula.
In an embodiment of the present invention, the side of the training dataset of the image of the extension input default neural network model
Method includes at least one of:
Adjusting the license plate center in the image becomes picture centre；
The license plate in the image is zoomed in and out, its width is made to match 40px to the value between 208px；
The rotation of 3D angle is carried out to the image at random；
50% probability carries out mirror image processing to the image；
Random translation, the license plate is mobile from the picture centre；
The image is cut centered on the license plate；
It modifies to the tone saturation degree lightness hsv color space of the image.
Fig. 4 is a kind of structural block diagram of license plate area positioning device according to an embodiment of the present invention, as shown in figure 4, the dress
The positioning for license plate area is set, which includes: input module 42, preprocessing module 44 and recognition detection module 46.
The image is inputted default neural network model, obtained comprising being somebody's turn to do by input module 42 for acquiring the image of vehicle
The vehicle image of vehicle image, wherein the default neural network model has the image of the vehicle in the image for identification；
Preprocessing module 44, for according to license plate size and vehicle bounding box between ratio, to the vehicle image into
The scaling of row picture size obtains the vehicle image of uniform sizes；
Recognition detection module 46, for the vehicle image to be inputted full convolutional network, wherein the full convolutional network identification is simultaneously
Determine license plate in the region of the vehicle image, wherein the full convolutional network is corresponding with target point in the vehicle image for calculating
The probability and the full convolutional network that unit is covered with the license plate are for exporting affine transformation parameter to correct the license plate area
Size.
By above-mentioned apparatus, input module 42 acquires the image of vehicle, which is inputted default neural network model, is obtained
Take the vehicle image comprising the vehicle image, preprocessing module 44 according to the ratio between the size and vehicle bounding box of license plate,
The scaling that picture size is carried out to the vehicle image, obtains the vehicle image of uniform sizes, the vehicle figure of recognition detection module 46
As inputting full convolutional network, wherein the full convolutional network identifies and determines that license plate in the region of the vehicle image, solves non-limit
Scene license plate area processed detects inaccurate problem, improves the precision in License Plate region.
In the implementation process of the license plate area positioning device of embodiment, such as the statement in above-described embodiment, the default mind
It include YOLO network and shallow-layer neural network through network, wherein the shallow-layer neural network includes 2 convolutional layers and 1 full connection
The shallow-layer convolutional neural networks of layer.
Fig. 5 is a kind of structural block diagram of car license recognition equipment according to an embodiment of the present invention, as shown in figure 5, the license plate is known
Other equipment 500 includes camera 52 and processor 54,
The camera 52 is used to acquire the image of vehicle, which inputs default neural network model for the image,
Obtain include the vehicle image vehicle image, wherein the default neural network model has the vehicle in the image for identification
Image；
The processor 54, the ratio being also used between the size and vehicle bounding box according to license plate, to the vehicle image into
The scaling of row picture size obtains the vehicle image of uniform sizes；
The processor 55 is also used to the vehicle image inputting full convolutional network, wherein the full convolutional network identification is simultaneously true
License plate is determined in the region of the vehicle image, wherein the full convolutional network is corresponding with target point in the vehicle image single for calculating
The probability and the full convolutional network that member is covered with the license plate are for exporting affine transformation parameter to correct the big of the license plate area
It is small.
By above equipment, which identifies and determines that license plate in the region of the vehicle image, solves non-limit
Scene license plate area processed detects inaccurate problem, improves the precision in License Plate region.
In another embodiment, a kind of software is additionally provided, the software is for executing above-described embodiment and preferred reality
Apply technical solution described in example.
In another embodiment, a kind of storage medium is additionally provided, above-mentioned software is stored in the storage medium, it should
Storage medium includes but is not limited to CD, floppy disk, hard disk, scratch pad memory etc..
Obviously, those skilled in the art should be understood that each module of the above invention or each step can be with general
Computing device realize that they can be concentrated on a single computing device, or be distributed in multiple computing devices and formed
Network on, optionally, they can be realized with the program code that computing device can perform, it is thus possible to which they are stored
It is performed by computing device in the storage device, and in some cases, it can be to be different from shown in sequence execution herein
Out or description the step of, perhaps they are fabricated to each integrated circuit modules or by them multiple modules or
Step is fabricated to single integrated circuit module to realize.In this way, the present invention is not limited to any specific hardware and softwares to combine.
Above this is merely a preferred embodiment of the present invention, and is not intended to restrict the invention, for the technology of this field
For personnel, the invention may be variously modified and varied.All within the spirits and principles of the present invention, made any to repair
Change, equivalent replacement, improvement etc., should all be included in the protection scope of the present invention.
Claims (10)
1. a kind of localization method of the license plate area under unrestricted scene, which is characterized in that the described method includes:
Described image is inputted default neural network model, obtains the vehicle comprising the vehicle image by the image for acquiring vehicle
Image, wherein the default neural network model has the image of the vehicle in described image for identification；
According to the ratio between the size and vehicle bounding box of license plate, the scaling of picture size is carried out to the vehicle image, is obtained
Take the vehicle image of uniform sizes；
The vehicle image is inputted into full convolutional network, wherein the full convolutional network identifies and determines license plate in the vehicle figure
The region of picture, wherein the full convolutional network is covered with for calculating with target point corresponding unit in the vehicle image described
The probability of license plate and the full convolutional network are for exporting affine transformation parameter to correct the size of the license plate area.
2. method according to claim 1, which is characterized in that
It include YOLO network and shallow-layer neural network in the default neural network, wherein the shallow-layer neural network includes 2
The shallow-layer convolutional neural networks of convolutional layer and 1 full articulamentum.
3. method according to claim 1, which is characterized in that the ratio between the size and vehicle bounding box according to license plate
Rate, the scaling of picture size is carried out to the vehicle image, and the vehicle image for obtaining uniform sizes includes:
In the case that ratio between the size of license plate and vehicle bounding box is lower than preset first threshold value, amplify the vehicle figure
Picture；
In the case that ratio between the size of license plate and vehicle bounding box is greater than default second threshold, the vehicle figure is reduced
Picture；
According to the preset first threshold value and the default second threshold, the size of the unified vehicle image.
4. method according to any one of the claim 1 to 3, which is characterized in that
The full convolutional network includes the first convolution module and the second convolution module, wherein the first convolution module is for calculating institute
The probability that target point corresponding unit in vehicle image is covered with the license plate is stated, has used Softmax function as excitation function,
Second convolution module is not use excitation function for obtaining the affine transformation parameter.
5. method according to any one of the claim 1 to 3, which is characterized in that extension inputs the default neural network model
The method of training dataset of described image include at least one of:
Adjusting license plate center described in described image becomes picture centre；
License plate described in described image is zoomed in and out, its width is made to match 40px to the value between 208px；
The rotation of 3D angle is carried out to described image at random；
50% probability carries out mirror image processing to described image；
Random translation, the license plate is mobile from described image center；
Described image is cut centered on the license plate；
It modifies to the tone saturation degree lightness hsv color space of described image.
6. a kind of positioning device of license plate area, which is characterized in that described for the positioning of the license plate area under unrestricted scene
Device includes:
Described image is inputted default neural network model, obtaining includes the vehicle for acquiring the image of vehicle by input module
The vehicle image of image, wherein the default neural network model has the image of the vehicle in described image for identification；
Preprocessing module, for carrying out figure to the vehicle image according to the ratio between the size and vehicle bounding box of license plate
As the scaling of size, the vehicle image of uniform sizes is obtained；
Recognition detection module, for the vehicle image to be inputted full convolutional network, wherein the full convolutional network is identified and determined
License plate is in the region of the vehicle image, wherein the full convolutional network is for calculating and target point pair in the vehicle image
The probability and the full convolutional network for answering unit to be covered with the license plate are for exporting affine transformation parameter to correct the vehicle
The size in board region.
7. device according to claim 6, which is characterized in that
It include YOLO network and shallow-layer neural network in the default neural network, wherein the shallow-layer neural network includes 2
The shallow-layer convolutional neural networks of convolutional layer and 1 full articulamentum.
8. device according to claim 6, which is characterized in that the preprocessing module is also used to size and vehicle in license plate
Ratio between bounding box amplifies the vehicle image, in the size and vehicle of license plate lower than in the case where preset first threshold value
In the case that ratio between bounding box is greater than default second threshold, the vehicle image is reduced, according to default first threshold
Value and the default second threshold, the size of the unified vehicle image.
9. a kind of car license recognition equipment, which is characterized in that the equipment includes camera and processor,
The camera is used to acquire the image of vehicle, and described image is inputted default neural network model, obtained by the processor
Take the vehicle image comprising the vehicle image, wherein the default neural network model for identification in described image
State the image of vehicle；
The processor, the ratio being also used between the size and vehicle bounding box according to license plate, carries out the vehicle image
The scaling of picture size obtains the vehicle image of uniform sizes；
The processor is also used to the vehicle image inputting full convolutional network, wherein the full convolutional network is identified and determined
License plate is in the region of the vehicle image, wherein the full convolutional network is for calculating and target point pair in the vehicle image
The probability and the full convolutional network for answering unit to be covered with the license plate are for exporting affine transformation parameter to correct the vehicle
The size in board region.
10. equipment according to claim 9, which is characterized in that
It include YOLO network and shallow-layer neural network in the default neural network, wherein the shallow-layer neural network includes 2
The shallow-layer convolutional neural networks of convolutional layer and 1 full articulamentum.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910747526.9A CN110443245B (en) | 2019-08-14 | 2019-08-14 | License plate region positioning method, device and equipment in non-limited scene |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910747526.9A CN110443245B (en) | 2019-08-14 | 2019-08-14 | License plate region positioning method, device and equipment in non-limited scene |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110443245A true CN110443245A (en) | 2019-11-12 |
| CN110443245B CN110443245B (en) | 2022-02-15 |
Family
ID=68435360
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910747526.9A Active CN110443245B (en) | 2019-08-14 | 2019-08-14 | License plate region positioning method, device and equipment in non-limited scene |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110443245B (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111079744A (en) * | 2019-12-06 | 2020-04-28 | 鲁东大学 | Intelligent vehicle license plate identification method and device suitable for complex illumination environment |
| CN111797713A (en) * | 2020-06-16 | 2020-10-20 | 浙江大华技术股份有限公司 | License plate recognition method and photographing device |
| CN112560608A (en) * | 2020-12-05 | 2021-03-26 | 江苏爱科赛尔云数据科技有限公司 | Vehicle license plate recognition method |
| CN113628206A (en) * | 2021-08-25 | 2021-11-09 | 深圳市捷顺科技实业股份有限公司 | License plate detection method, device and medium |
| CN113780278A (en) * | 2021-09-10 | 2021-12-10 | 北京精英路通科技有限公司 | Method and device for identifying license plate content, electronic equipment and storage medium |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103065135A (en) * | 2013-01-25 | 2013-04-24 | 上海理工大学 | License number matching algorithm based on digital image processing |
| CN104408430A (en) * | 2014-12-01 | 2015-03-11 | 广东中星电子有限公司 | License plate positioning method and device |
| CN106778745A (en) * | 2016-12-23 | 2017-05-31 | 深圳先进技术研究院 | A kind of licence plate recognition method and device, user equipment |
| CN106874907A (en) * | 2017-01-19 | 2017-06-20 | 博康智能信息技术有限公司北京海淀分公司 | A kind of method and device for setting up Car license recognition model |
| CN108548820A (en) * | 2018-03-28 | 2018-09-18 | 浙江理工大学 | Cosmetics paper labels defect inspection method |
| CN110008360A (en) * | 2019-04-09 | 2019-07-12 | 河北工业大学 | Vehicle target image data base method for building up comprising specific background image |
| CN110059683A (en) * | 2019-04-15 | 2019-07-26 | 广州广电银通金融电子科技有限公司 | A kind of license plate sloped antidote of wide-angle based on end-to-end neural network |
-
2019
- 2019-08-14 CN CN201910747526.9A patent/CN110443245B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103065135A (en) * | 2013-01-25 | 2013-04-24 | 上海理工大学 | License number matching algorithm based on digital image processing |
| CN104408430A (en) * | 2014-12-01 | 2015-03-11 | 广东中星电子有限公司 | License plate positioning method and device |
| CN106778745A (en) * | 2016-12-23 | 2017-05-31 | 深圳先进技术研究院 | A kind of licence plate recognition method and device, user equipment |
| CN106874907A (en) * | 2017-01-19 | 2017-06-20 | 博康智能信息技术有限公司北京海淀分公司 | A kind of method and device for setting up Car license recognition model |
| CN108548820A (en) * | 2018-03-28 | 2018-09-18 | 浙江理工大学 | Cosmetics paper labels defect inspection method |
| CN110008360A (en) * | 2019-04-09 | 2019-07-12 | 河北工业大学 | Vehicle target image data base method for building up comprising specific background image |
| CN110059683A (en) * | 2019-04-15 | 2019-07-26 | 广州广电银通金融电子科技有限公司 | A kind of license plate sloped antidote of wide-angle based on end-to-end neural network |
Non-Patent Citations (3)
| Title |
|---|
| JOSEPH REDMON ET AL: ""YOLO9000:Better, Faster, Stronger"", 《ARXIV》 * |
| JOSEPH REDMON ET AL: ""You Only Look Once:Unified, Real-Time Object Detection"", 《ARXIV》 * |
| S´ERGIO MONTAZZOLLI SILVA ET AL: ""License Plate Detection and Recognition in Unconstrained Scenarios"", 《ECCV》 * |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111079744A (en) * | 2019-12-06 | 2020-04-28 | 鲁东大学 | Intelligent vehicle license plate identification method and device suitable for complex illumination environment |
| CN111797713A (en) * | 2020-06-16 | 2020-10-20 | 浙江大华技术股份有限公司 | License plate recognition method and photographing device |
| CN112560608A (en) * | 2020-12-05 | 2021-03-26 | 江苏爱科赛尔云数据科技有限公司 | Vehicle license plate recognition method |
| CN112560608B (en) * | 2020-12-05 | 2024-05-24 | 江苏爱科赛尔云数据科技有限公司 | Vehicle license plate recognition method |
| CN113628206A (en) * | 2021-08-25 | 2021-11-09 | 深圳市捷顺科技实业股份有限公司 | License plate detection method, device and medium |
| CN113780278A (en) * | 2021-09-10 | 2021-12-10 | 北京精英路通科技有限公司 | Method and device for identifying license plate content, electronic equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110443245B (en) | 2022-02-15 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110443245A (en) | Localization method, device and the equipment of a kind of license plate area under unrestricted scene | |
| CN106971185B (en) | License plate positioning method and device based on full convolution network | |
| CN109583483B (en) | Target detection method and system based on convolutional neural network | |
| CN111640157B (en) | Checkerboard corner detection method based on neural network and application thereof | |
| CN110378837B (en) | Target detection method and device based on fish-eye camera and storage medium | |
| CN108154149B (en) | License plate recognition method based on deep learning network sharing | |
| CN112201078B (en) | Automatic parking space detection method based on graph neural network | |
| CN105303514A (en) | Image processing method and apparatus | |
| CN106934806B (en) | It is a kind of based on text structure without with reference to figure fuzzy region dividing method out of focus | |
| CN109741241B (en) | Fisheye image processing method, device, equipment and storage medium | |
| CN110766002B (en) | Ship name character region detection method based on deep learning | |
| CN113221897B (en) | Image correction method, image text recognition method, identity verification method and device | |
| CN111383264A (en) | Positioning method, positioning device, terminal and computer storage medium | |
| CN112183517A (en) | Certificate card edge detection method, equipment and storage medium | |
| CN115810133A (en) | Welding control method based on image processing and point cloud processing and related equipment | |
| CN112950528A (en) | Certificate posture determining method, model training method, device, server and medium | |
| CN113808033A (en) | Image document correction method, system, terminal and medium | |
| CN111709377A (en) | Feature extraction method, target re-identification method and device and electronic equipment | |
| CN115147389A (en) | Image processing method, apparatus, and computer-readable storage medium | |
| CN113850100A (en) | Method and device for correcting two-dimensional code | |
| CN111028290B (en) | Graphic processing method and device for drawing book reading robot | |
| CN109376653B (en) | Method, apparatus, device and medium for locating vehicle | |
| CN109859263B (en) | Wide-view angle positioning method based on fisheye lens | |
| CN112700428A (en) | Method and device for identifying backboard element of switch | |
| CN112560606A (en) | Trailer angle identification method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information | ||
| CB02 | Change of applicant information |
Address after: Room 5288, building e, 555 Dongchuan Road, Minhang District, Shanghai 200241 Applicant after: Shanghai Shimao Internet of Things Technology Co.,Ltd. Address before: World Trade Building, 55 Weifang West Road, Pudong New Area, Shanghai, 200120 Applicant before: Shanghai Shimao Internet of Things Technology Co.,Ltd. |
|
| GR01 | Patent grant | ||
| GR01 | Patent grant |