CN107657275B - License plate pre-detection method based on improved BING algorithm - Google Patents

License plate pre-detection method based on improved BING algorithm Download PDF

Info

Publication number
CN107657275B
CN107657275B CN201710854432.2A CN201710854432A CN107657275B CN 107657275 B CN107657275 B CN 107657275B CN 201710854432 A CN201710854432 A CN 201710854432A CN 107657275 B CN107657275 B CN 107657275B
Authority
CN
China
Prior art keywords
image
license plate
gradient
classifier
sliding window
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201710854432.2A
Other languages
Chinese (zh)
Other versions
CN107657275A (en
Inventor
马争
解梅
李佩伦
秦方
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Houpu Clean Energy Group Co ltd
Original Assignee
University of Electronic Science and Technology of China
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by University of Electronic Science and Technology of China filed Critical University of Electronic Science and Technology of China
Priority to CN201710854432.2A priority Critical patent/CN107657275B/en
Publication of CN107657275A publication Critical patent/CN107657275A/en
Application granted granted Critical
Publication of CN107657275B publication Critical patent/CN107657275B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/245—Classification techniques relating to the decision surface
    • G06F18/2451—Classification techniques relating to the decision surface linear, e.g. hyperplane
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/42—Global feature extraction by analysis of the whole pattern, e.g. using frequency domain transformations or autocorrelation
    • G06V10/422—Global feature extraction by analysis of the whole pattern, e.g. using frequency domain transformations or autocorrelation for representing the structure of the pattern or shape of an object therefor
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/60—Type of objects
    • G06V20/62—Text, e.g. of license plates, overlay texts or captions on TV images
    • G06V20/625—License plates

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • General Physics & Mathematics (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Evolutionary Biology (AREA)
  • Evolutionary Computation (AREA)
  • Artificial Intelligence (AREA)
  • General Engineering & Computer Science (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Multimedia (AREA)
  • Image Analysis (AREA)

Abstract

The invention discloses a license plate pre-detection method based on an improved BING algorithm, and belongs to the field of license plate recognition. According to the invention, the BING detection window is improved according to the specific shape of the license plate, the BING detection characteristics are enriched, and the recommendation frame given by the BING can completely contain the whole license plate.

Description

License plate pre-detection method based on improved BING algorithm
Technical Field
The invention belongs to the field of computer vision, and particularly relates to the technical field of license plate recognition.
Background
With the rapid development of economy, more and more Chinese people own private cars. Therefore, an intelligent transportation system is needed to help people manage traffic order. The license plate is the identity card of the motor vehicle, and the only corresponding motor vehicle can be found when the license plate is found, so the license plate recognition system is an important component of the intelligent traffic system. The license plate detection is the first step of license plate recognition and is also the most important step, and a good license plate detection algorithm requires high detection speed and high recall ratio. The traditional license plate detection mainly comprises the following methods:
1) the method based on the edge detection mainly depends on the abundant texture characteristics of a license plate area, and if the background environment is complex or the license plate part is too close to an automobile exhaust outlet area (also has abundant textures), the license plate is difficult to detect.
2) According to the color-based license plate positioning method, most license plates are blue or yellow, so that the color area can be used for detecting the license plates, the effect of the license plates is good in many times by using color information, but the blue license plates or yellow license plates of blue cars cannot be completely detected, and meanwhile, the algorithm is sensitive to illumination and cannot detect the (white) license plates of police cars.
3) The BING algorithm detects objects under different scales by using 8-by-8 sliding windows, and the result of 8-by-8 is 64, so that the characteristics of a 64-bit computer can be fully utilized to improve the detection speed. Because the existing BING algorithm adopts 8 × 8 sliding windows, the detected recommendation frame (the position identifier of the detection target, usually a rectangular frame covering the detection target, whose coordinates are located by the coordinates of the upper left corner and the lower right corner of the rectangle) does not just include a complete object, but usually only includes most of the region of the target, and the detection rate accuracy is extremely low when the number of recommendation frames of a single picture reaches a certain number, for example, when the number of recommendation frames of a single picture reaches 5000, the recall rate (recall rate) of the target object can reach 99.5%.
Disclosure of Invention
The invention aims to: aiming at the existing problems, the method can quickly detect all regions which may be the license plate, reduce the subsequent detection range and further accelerate the speed of license plate recognition.
The invention discloses a license plate pre-detection method based on an improved BING algorithm, which comprises the following steps:
step 1: training a classifier:
101: inputting a training sample set, wherein the training sample set comprises an image containing a license plate and license plate position marking information in the image;
102: extracting a positive sample: extracting a license plate section image from the training sample image, and carrying out size normalization processing on the license plate section image to scale the size to 24 x 8; calculating a gradient image of the license plate section image after size normalization to obtain a gradient license plate section image, and carrying out image column vectorization to obtain a positive sample;
103: extracting a negative sample: randomly extracting slice images of non-license plates from training sample images, scaling the slice images to be 24 x 8, then calculating gradient images of the slice images, and vectorizing the gradient image columns to obtain negative samples;
104: training a classifier: training a classifier based on the positive and negative samples, and after removing a bias term of the classifier, arranging the classifier into 24 × 8 template pictures;
105: dividing the template picture trained in the step 104 into 3 sub-templates with the size of 8 x 8, and calculating the binary gradient amplitude of the pixel in each sub-template
Figure BDA0001413342170000021
Wherein b iskRepresenting the kth bit plane, NgAn estimated hyper-parameter representing a gradient image;
bit surface: the jth bit of each pixel in 8 bit/pixel of a gray level image is extracted to obtain a binary image called a bit plane.
Step 2: detection treatment:
step 201: inputting a picture to be pre-detected;
step 202: constructing an 8-layer image pyramid of a picture to be pre-detected, wherein the scaling coefficient of the image pyramid is 0.8;
step 203: calculating the binarization gradient amplitude of each pixel point of the 8-layer image pyramid, wherein the calculation formula of the binarization gradient amplitude of each layer is
Figure BDA0001413342170000022
Where l denotes the l-th layer image, bk,lK-th bit plane, N, representing the l-th layer imagegEstimated hyper-parameters, N, representing a gradient imagegGenerally, 4 is taken to represent the bit plane of the upper 4 bits.
Adopting 24 × 8 sliding windows to perform sliding window detection on a picture to be detected, extracting the binary gradient amplitude of each pixel point in the window every time of sliding window, sequentially dividing the sliding window result into 3 8 × 8 sub-sliding window results according to the direction of the sliding window, and inputting the sub-sliding window results into a classifier (namely, inputting the sub-sliding window results into the classifier)The sub sliding window results are respectively matched with each sub template of the classifier), and the final judgment score s of each sliding window is obtainedl；
Wherein
Figure BDA0001413342170000023
Symbol<·>Representing an inner product operation;
step 204: final decision score s for each sliding windowlAnd performing non-maximum suppression processing, and outputting a recommended frame of the picture to be pre-detected, namely the position of the sliding window left after the non-maximum suppression processing.
In order to further increase the detection speed, an 8-bit char type is used for storing the gradient amplitude value g of a single pixel pointlAnd when the binary gradient amplitude of each pixel point in the window is extracted every time, only the first four digits (high four digits) are extracted.
In summary, due to the adoption of the technical scheme, the invention has the beneficial effects that: the innovation points of the invention are as follows:
the invention has the innovation that according to the specific shape of the license plate, the BING detection window is improved (three BING detections are superposed), and the BING detection characteristics are enriched, so that the given recommendation frame can almost completely contain the whole license plate. Because the BING detection form is changed, the detection is not carried out under images with different scales like the BING original method, and a good detection effect can be obtained only by adopting the traditional pyramid method; the detection mode of combining a plurality of BING classifiers is suitable for detecting rigid objects with any aspect ratio (by changing the combination form of the BING), expands the size of the sliding window, fully utilizes the operation performance of a modern 64-bit computer and improves the detection speed.
Drawings
FIG. 1 is a training flow diagram of an embodiment of the present invention;
FIG. 2 is a detection process according to an embodiment of the invention;
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention will be described in further detail with reference to the following embodiments and accompanying drawings.
The invention relates to a license plate pre-detection method based on an improved BING algorithm, which comprises the following specific implementation steps of:
step 1: training the classifier, see fig. 1, which comprises the following steps, respectively:
step 101: preparing a training sample set: in the embodiment, the training sample set comprises 1900 pictures containing the license plate and 1900 markup files, and the real position of the license plate in each picture is recorded in the markup files.
Step 102: extracting a positive sample: and respectively intercepting license plate region parts in 1900 pictures, wherein in the specific embodiment, only one license plate in each picture is limited, and then obtaining a slice image of the 1900 license plates.
And (3) carrying out size normalization processing on the slice images: each slice is scaled to a size of 24 x 8. The gradient of the slice after size normalization is calculated to obtain a gradient slice, and 24 × 8 pixels are aligned in a row to form a positive sample.
Step 103: extracting a negative sample: pictures are randomly cut out from 1900 images (the cut-out part cannot be overlapped with the positive sample), 10 slices are cut out from each picture, and 19000 negative sample slices are obtained. Similarly, these slices are scaled to a 24 × 8 picture size, the gradient is calculated, and 24 × 8 pixels of the gradient slice are aligned in a row to form a negative sample.
Step 104: training a classifier: after steps 102 and 103, the total training samples are 1900 (positive) +19000 (negative) ═ 20900. It is put into librinear (open source training program, applicable to training linear svm) for training, resulting in a 193-dimensional classifier, and the final dimension (bias term) is removed to become 192-dimensional. The 192-dimensional classifiers are then arranged into 24-by-8 template pictures.
Step 105: cutting by a classifier: the classifier adopted by the invention is 24 × 8 in size, so the classifier is divided into 3 sub-classifiers 8 × 8 in size, the dimension of each sub-classifier is 64, and each sub-classifier is represented by w. I.e. dividing the template picture into 3 8 x 8 sizesAnd calculating the binary gradient amplitude of the pixel in each sub-template
Figure BDA0001413342170000041
Wherein b iskRepresenting the kth bit plane, NgRepresenting the estimated hyper-parameters of the gradient image.
Step 2: detection processing, referring to fig. 2, the detection processing specifically comprises the steps of:
step 201: inputting a picture to be pre-detected;
step 202: constructing an image pyramid: since the specific sizes of the license plates in different pictures are different, in order to search in images with different scales, an image pyramid with a scaling coefficient of 0.8 is used in the invention. For example, if the original picture size is 640 x 320, the first level of pyramids is 640 x 320, the second level of pyramids is 512 x 256, the third level of pyramids is 410 x 405, and so on. The pyramid has 8 layers in total, and the scaling coefficient can balance the detection speed and the detection effect.
Step 203: since the linear model w can be represented as the sum of a series of binary basis vectors
Figure BDA0001413342170000042
Wherein N iswThe estimated hyperparameter representing the linear classifier, generally taken as 4, represents the approximate estimate of the linear classifier's value with the high 4-bit plane, βj(real number) denotes a calibration coefficient, aj∈{-1,1}64，ajCan be further expressed as a binary vector and its complement:
Figure BDA0001413342170000043
the test model (object to be pre-detected) can be expressed as:
Figure BDA0001413342170000044
where b is the gradient magnitude after binarization.
Calculating the binarization gradient amplitude of each pixel point of the image pyramid obtained in the step 202
Figure BDA0001413342170000045
Where l denotes the l-th layer image, bk,lK-th bit plane, N, representing the l-th layer imagegGenerally, 4 is taken to represent the bit plane of the upper 4 bits.
And an 8bits char type is used for storing the gradient amplitude g of a single pixel pointlFor an 8 × 8 sliding window (the sub-classifier of the present invention), it can be regarded as an 8 × 8 storage cube, and the operation of fetching bit faces can be performed, so as to obtain 8 × 8bit faces. In order to accelerate the operation, only the first four layers (high four bits) are put into the classifier trained in the step 1 for sliding window method detection, namely three sub-classifiers are simultaneously subjected to sliding window detection, and the sliding window scores are added, so that the final judgment score s is obtainedl：
Figure BDA0001413342170000051
Wherein c isj,kCan be obtained quickly by using bit operation and SSE (sequencing SIMD extensions) instruction operation.
Step 204: carrying out non-maximum suppression processing on the sliding window detection result:
after step 203 is executed, a score map can be obtained, a higher score can be obtained at the correct position of the license plate, and the score near the correct position is not too low, so that a non-maximum value is required to inhibit and exclude a sliding window with the peripheral score not being the highest, the number of the finally output recommended frames is obviously reduced, and the accuracy is improved.
Step 205: and outputting the detection result, namely a recommendation box.
The pre-detection method can effectively detect whether the blue license plate, the yellow license plate or the white license plate is detected, has extremely high detection speed and low omission factor, and can obtain correct detection results only by carrying out one-time screening (non-maximum inhibition treatment) on the detection results in the subsequent process.

Claims (2)

1. The license plate pre-detection method based on the improved BING algorithm is characterized by comprising the following steps of:
step 1: training a classifier:
101: inputting a training sample set, wherein the training sample set comprises an image containing a license plate and license plate position marking information in the image;
102: extracting a positive sample: extracting a license plate section image from the training sample image, and carrying out size normalization processing on the license plate section image to scale the size to 24 x 8; calculating a gradient image of the license plate section image after size normalization to obtain a gradient license plate section image, and carrying out image column vectorization to obtain a positive sample;
103: extracting a negative sample: randomly extracting slice images of non-license plates from training sample images, scaling the slice images to be 24 x 8, then calculating gradient images of the slice images, and vectorizing the gradient image columns to obtain negative samples;
104: training a classifier: training a classifier based on the positive and negative samples, and after removing a bias term of the classifier, arranging the classifier into 24 × 8 template pictures;
105: dividing the template picture trained in the step 104 into 3 sub-templates with the size of 8 x 8, and calculating the binary gradient amplitude of the pixel in each sub-template
Figure FDA0002403757370000011
Wherein b iskRepresenting the kth bit plane, NgAn estimated hyper-parameter representing a gradient image;
step 2: detection treatment:
step 201: inputting a picture to be pre-detected;
step 202: constructing an 8-layer image pyramid of a picture to be pre-detected, wherein the scaling coefficient of the image pyramid is 0.8;
step 203: calculating the binarization gradient amplitude of each pixel point of the 8-layer image pyramid, wherein the calculation formula of the binarization gradient amplitude of each layer is
Figure FDA0002403757370000012
Where l denotes the l-th layer image, bk,lRepresenting the kth ratio of the l layer imageSpecial face, NgAn estimated hyper-parameter representing a gradient image;
adopting 24 × 8 sliding windows to perform sliding window detection on a picture to be detected in advance, extracting the binary gradient amplitude of each pixel point in the window every time of sliding window, sequentially dividing the sliding window result into 3 8 × 8 sub-sliding window results according to the sliding window direction, inputting the sub-sliding window results into a classifier, and obtaining the final judgment score s of each sliding windowl：
Wherein
Figure FDA0002403757370000013
NwRepresenting estimated hyper-parameters of the classifier, βjRepresenting calibration coefficients, signs<·>Representing an inner product operation;
step 204: final decision score s for each sliding windowlAnd carrying out non-maximum suppression processing and outputting a recommendation frame of the picture to be pre-detected.
2. The method of claim 1, wherein the estimated hyper-parameter N of the gradient imageg＝4。
CN201710854432.2A 2017-09-20 2017-09-20 License plate pre-detection method based on improved BING algorithm Active CN107657275B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201710854432.2A CN107657275B (en) 2017-09-20 2017-09-20 License plate pre-detection method based on improved BING algorithm

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201710854432.2A CN107657275B (en) 2017-09-20 2017-09-20 License plate pre-detection method based on improved BING algorithm

Publications (2)

Publication Number Publication Date
CN107657275A CN107657275A (en) 2018-02-02
CN107657275B true CN107657275B (en) 2020-07-21

Family

ID=61130523

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201710854432.2A Active CN107657275B (en) 2017-09-20 2017-09-20 License plate pre-detection method based on improved BING algorithm

Country Status (1)

Country Link
CN (1) CN107657275B (en)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108010082B (en) * 2017-12-28 2021-07-23 上海觉感视觉科技有限公司 Geometric matching method
CN109635844B (en) * 2018-11-14 2021-08-27 网易传媒科技（北京）有限公司 Method and device for training classifier and method and device for detecting watermark

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107145905A (en) * 2017-05-02 2017-09-08 重庆大学 The image recognizing and detecting method that elevator fastening nut loosens

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107145905A (en) * 2017-05-02 2017-09-08 重庆大学 The image recognizing and detecting method that elevator fastening nut loosens

Non-Patent Citations (4)

* Cited by examiner, † Cited by third party
Title
BING: Binarized Normed Gradients for Objectness Estimation at 300fps;Ming-Ming Cheng et al.;《2014 IEEE Conference on Computer Vision and Pattern Recognition》;20140628;第3286-3293页 *
BING的硬件并行化设计;杨国超;《中国优秀硕士学位论文全文数据库 信息科技辑》;20160415(第04期);第I138-1017页 *
Object detection based on BING in optical remote sensing images;Jiangbin Zheng et al;《2016 9th International Congress on Image and Signal Processing, BioMedical Engineering and Informatics (CISP-BMEI)》;20161017;第504-509页 *
基于通用对象估计的目标检测与模糊车牌识别算法研究;赵丹凤;《中国优秀硕士学位论文全文数据库 信息科技辑》;20170215(第02期);第I138-3702页 *

Also Published As

Publication number Publication date
CN107657275A (en) 2018-02-02

Similar Documents

Publication Publication Date Title
CN110348319B (en) Face anti-counterfeiting method based on face depth information and edge image fusion
Lee et al. Adaboost for text detection in natural scene
CN108734138B (en) Melanoma skin disease image classification method based on ensemble learning
CN113361495B (en) Method, device, equipment and storage medium for calculating similarity of face images
EP3101594A1 (en) Saliency information acquisition device and saliency information acquisition method
CN105574527B (en) A kind of quick object detecting method based on local feature learning
CN108921057B (en) Convolutional neural network-based prawn form measuring method, medium, terminal equipment and device
CN113723573B (en) Tumor tissue pathological classification system and method based on adaptive proportion learning
CN107330397A (en) A kind of pedestrian&#39;s recognition methods again based on large-spacing relative distance metric learning
CN110647802A (en) Remote sensing image ship target detection method based on deep learning
Kadkhodaei et al. Automatic segmentation of multimodal brain tumor images based on classification of super-voxels
CN111460894B (en) Intelligent car logo detection method based on convolutional neural network
CN114067444A (en) Face spoofing detection method and system based on meta-pseudo label and illumination invariant feature
CN105005798B (en) One kind is based on the similar matched target identification method of structures statistics in part
CN107886539B (en) High-precision gear visual detection method in industrial scene
Yang et al. Real-time traffic sign detection via color probability model and integral channel features
CN107862680B (en) Target tracking optimization method based on correlation filter
CN114783003A (en) Pedestrian re-identification method and device based on local feature attention
CN103778430B (en) Rapid face detection method based on combination between skin color segmentation and AdaBoost
US9715729B2 (en) Method and apparatus for processing block to be processed of urine sediment image
WO2008039480A1 (en) Method and system for learning spatio-spectral features in an image
CN104732534B (en) Well-marked target takes method and system in a kind of image
CN110852327A (en) Image processing method, image processing device, electronic equipment and storage medium
CN114266794A (en) Pathological section image cancer region segmentation system based on full convolution neural network
CN107657275B (en) License plate pre-detection method based on improved BING algorithm

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
CB03 Change of inventor or designer information
CB03 Change of inventor or designer information

Inventor after: Ma Zheng

Inventor after: Xie Mei

Inventor after: Li Peilun

Inventor after: Qin Fang

Inventor before: Ma Zheng

Inventor before: Li Peilun

Inventor before: Qin Fang

SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant
TR01 Transfer of patent right
TR01 Transfer of patent right

Effective date of registration: 20210513

Address after: No.3, 11th floor, building 6, no.599, shijicheng South Road, Chengdu hi tech Zone, China (Sichuan) pilot Free Trade Zone, Chengdu, Sichuan 610041

Patentee after: Houpu clean energy Co.,Ltd.

Address before: 611731, No. 2006, West Avenue, Chengdu hi tech Zone (West District, Sichuan)

Patentee before: University of Electronic Science and Technology of China

CP01 Change in the name or title of a patent holder
CP01 Change in the name or title of a patent holder

Address after: No.3, 11th floor, building 6, no.599, shijicheng South Road, Chengdu hi tech Zone, China (Sichuan) pilot Free Trade Zone, Chengdu, Sichuan 610041

Patentee after: Houpu clean energy (Group) Co.,Ltd.

Address before: No.3, 11th floor, building 6, no.599, shijicheng South Road, Chengdu hi tech Zone, China (Sichuan) pilot Free Trade Zone, Chengdu, Sichuan 610041

Patentee before: Houpu clean energy Co.,Ltd.