CN109815956B - License plate character recognition method based on self-adaptive position segmentation - Google Patents
License plate character recognition method based on self-adaptive position segmentation Download PDFInfo
- Publication number
- CN109815956B CN109815956B CN201811624974.1A CN201811624974A CN109815956B CN 109815956 B CN109815956 B CN 109815956B CN 201811624974 A CN201811624974 A CN 201811624974A CN 109815956 B CN109815956 B CN 109815956B
- Authority
- CN
- China
- Prior art keywords
- license plate
- image
- layer
- network
- character
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Landscapes
- Character Discrimination (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a license plate character recognition method based on self-adaptive position segmentation, which belongs to the technical field of license plate recognition and comprises the steps of constructing a deep neural network model, wherein the deep neural network model comprises a basic network, a license plate position correction network, a license plate character segmentation network and a license plate character recognition network; collecting license plate sample images, and training the constructed deep neural network model to obtain a self-adaptive character segmentation recognition model; and (3) carrying out license plate character recognition on the license plate image to be recognized by using the self-adaptive character segmentation recognition module. The invention does not strictly distinguish the steps of license plate position correction, license plate character segmentation, license plate character recognition and the like, directly completes the license plate position correction, the license plate character segmentation and the character recognition by means of a deep neural network structure model, and gives consideration to the license plate recognition accuracy and the recognition speed.
Description
Technical Field
The invention relates to the technical field of license plate recognition, in particular to a license plate character recognition method based on self-adaptive position segmentation.
Background
License plate recognition is the core technology of intelligent transportation, and comprises two major parts: and detecting the position of the license plate and recognizing characters of the license plate. The license plate character recognition is the most important part of the whole technology, and the quality of a license plate character recognition engine directly determines the overall performance of the license plate recognition technology.
The license plate character recognition means that all Chinese characters, characters and numbers on a license plate are accurately and uninterruptedly recognized in an image with a known license plate position, and specifically comprises the following technical steps: the method comprises the steps of license plate position correction, license plate character segmentation, license plate character recognition and the like.
The license plate position correction refers to space transformation of an initially detected license plate with an unsatisfactory position to form the license plate with the ideal position, so that accurate character segmentation is convenient to perform subsequently, wherein the space transformation comprises translation transformation, rotation transformation, scaling transformation, shear transformation, perspective transformation and the like, and the following common correction methods are adopted:
(1) The method based on the line detection represents a correction method based on hough line detection and a correction method based on radon line detection, and the principle of the method is to directly detect a line on a license plate and correct the position of the license plate according to the inclination angle of the line.
(2) The method based on traversal search represents a "correction method based on rotation projection". The principle is that firstly, the license plate is rotated to each allowed angle position, then, projection is carried out to obtain corresponding characteristic values, the optimal characteristic values are obtained through comparison, and the corresponding angle is the optimal license plate inclination angle.
(3) The typical methods based on the characteristic analysis include a correction method based on principal component analysis and a correction method based on a least square method, and the methods directly perform integral analysis on the gray level image or the binary image to obtain the optimal correction parameters in the integral sense.
The license plate character segmentation means that each single character is accurately segmented from an image with a known license plate position, and the method mainly comprises the following methods:
(1) The method is based on a vertical projection method, and the edge position of each character is obtained according to the positions of wave crests and wave troughs of a vertical projection curve of license plate characters.
(2) The method based on the connected region analysis comprises the steps of firstly carrying out license plate image binarization, carrying out analysis by utilizing the characteristics that all single characters are in a single connected region, and finally obtaining the positions of the characters.
(3) A method based on machine learning, such as a license plate character segmentation method based on a support vector machine, comprises the steps of obtaining the layout rule characteristics of a license plate, training and learning by means of a classifier, and finally completing the segmentation of license plate characters.
The license plate character recognition means that for a single character which is accurately segmented, the real letter meaning of the character is recognized, and the following methods are commonly used:
(1) Global features, which use global transformation to obtain the overall features of the characters, and use ordered overall features or subset features to form feature vectors, wherein the common features include GABOR transformation features, moment features, projection features, stroke density features, HARR features, HOG features, and the like. The advantages of the characteristics are insensitivity to local change and strong anti-interference capability; the disadvantage is that some important local features are easy to ignore, and similar characters cannot be distinguished.
(2) And local features, wherein the corresponding features are calculated in a plurality of local regions of the character, the serial ordered local features are used for forming a final feature vector, and the main features comprise local gray level histogram features, LBP features, threading features, SIFT features and the like. The characteristic has the advantages of strong capability of distinguishing characters; the disadvantage is that the local features of the character are over-focused and often misdistinguish characters with noise interference.
The technology can achieve good effects on clear license plate images, however, license plate images collected in an actual environment often have the characteristics of low resolution, shallow or missing part of characters, fuzzy edges, inclined characters and the like, so that accurate license plate position correction, license plate character segmentation and license plate character recognition are difficult to perform, even failure occurs, and the overall performance of license plate recognition is seriously influenced. Therefore, how to accurately and robustly identify the license plate characters is still a difficulty of the license plate identification system in China.
In recent years, deep learning technology relies on the fact that a human brain neural network can be simulated, accurate nonlinear prediction can be conducted, and wide attention and application are paid to various fields.
Disclosure of Invention
The invention aims to provide a license plate character recognition method based on self-adaptive position segmentation so as to take account of both the license plate character recognition accuracy and the recognition speed.
In order to achieve the above object, in a first aspect, the present invention provides a license plate character recognition method based on adaptive location segmentation, including the following steps:
s1, constructing a deep neural network model, wherein the deep neural network model comprises a basic network, a license plate position correction network, a license plate character segmentation network and a license plate character recognition network;
s2, collecting license plate sample images, and training the constructed deep neural network model to obtain a self-adaptive character segmentation recognition model;
and S3, utilizing the self-adaptive character segmentation and recognition module to recognize the license plate characters of the license plate image to be recognized.
Preferably, the base network comprises: convolution layer conv0, maximum pooling layer pool0 and 5 residual error network basic mechanisms resblock0, resblock1, resblock2, resblock3 and resblock4; the core size of convolutional layer conv0 is 3x3 and the span is 2, the core size of maximum pooling layer pool0 is 2x2 and the span is 2;
the input image of the basic network is any 3-channel RGB image with the width of 192 pixels and the height of 64 pixels, and the output of the image is a high-level feature layer feature map of the input image.
Preferably, the residual network infrastructure comprises a branch 1, a branch 2 and a merge layer eltsum, wherein the branch 2 comprises a branch 2_0 and a branch 2_1;
in the three residual error network basic mechanisms of resblock0, resblock2 and resblock4, a branch 1 is a convolution layer with the kernel size of 1x1 and the span of 1; each of span 2_0 and span 2_1 is a convolutional layer with a core size of 3x3 and a span of 1;
in the resblock1 residual network infrastructure, branch 1 is a convolutional layer with a core size of 1x1 and a span of 2, branch 2_0 is a convolutional layer with a core size of 3x3 and a span of 2, and branch 2_1 is a convolutional layer with a core size of 3x3 and a span of 1;
in the resblock3 residual network infrastructure, branch 1 is a convolutional layer with a core size of 1x1, a height direction span of 2 and a width direction span of 1, branch 2_0 is a convolutional layer with a core size of 3x3, a height direction span of 2 and a width direction span of 1, and branch 2_1 is a convolutional layer with a core size of 3x3 and a span of 1.
Preferably, each convolutional layer in the base network is followed by a batch normalized BN layer and a non-linear activated relu layer.
Preferably, the input feature image of the license plate position correction network is the high-level feature layer feature map, the output feature image is the corrected high-level feature layer correct feature map, and the process of the license plate position correction network processing the high-level feature layer feature map includes:
sequentially passing the high-level feature layer feature map through 1 convolution layer para _ conv _0 with the core size of 3x3 and the span of 1, passing through 2 convolution layers para _ conv _1 and para _ conv _2 with the core size of 3x3 and the span of 2 and passing through 1 fully-connected layer para _ fc, and outputting 9 feature parameters;
sequentially combining the 9 characteristic parameters into a 3x3 matrix, and determining an inverse matrix of the matrix as a space transformation matrix for correcting the position of the license plate;
acquiring a pixel position corresponding relation between an output characteristic image and an input characteristic image based on the space transformation matrix for correcting the license plate position;
calculating the gray value of the corresponding pixel position on the input characteristic image by adopting an improved bilinear interpolation algorithm;
and obtaining a gray value of each pixel position on the output image based on the pixel position corresponding relation between the output characteristic image and the input characteristic image, and obtaining a corrected high-level characteristic layer correct feature map.
Preferably, the license plate character segmentation network comprises a convolution layer conv _ rect0 with the core size of 3x3 and the span of 2, and the output of the convolution layer conv _ rect0 is respectively connected with the input of the q character position segmentation network;
passing the output result of the license plate position correction network through a convolution layer conv _ rect0;
and respectively passing the output results of the convolution layer conv _ rect0 through a q character position segmentation network to obtain q character local feature images.
Preferably, the license plate character recognition network comprises 1 convolutional layer char _ conv0 with the kernel size of 3x3 and the span of 2, and the output of the convolutional layer char _ conv0 is connected with the input of 1 fully-connected layer char _ fc 0;
passing the partial feature images of the q characters through a convolution layer char _ conv0;
and (4) outputting a license plate character recognition result after the output result of the convolution layer char _ conv0 passes through the full-connection layer char _ fc0.
Preferably, the convolutional layer char _ conv0 is followed by a batch normalized BN layer and a non-linearly enabled relu layer, and the fully-connected layer char _ fc0 is followed by a non-linearly enabled relu layer.
Preferably, the step S2 includes:
collecting a license plate image as a license plate sample image, and performing pre-arrangement on the license plate sample image to obtain a local license plate image with a license plate character position rectangle marked;
adopting an image enhancement method to expand the pre-arranged license plate sample images, and taking the expanded license plate sample images as a license plate sample image set;
carrying out size normalization operation on the images in the license plate sample image set to obtain normalized images;
and training the deep neural network model by using the normalized image to obtain the self-adaptive character segmentation recognition model.
Preferably, the step S3 includes:
pre-sorting the license plate image to be recognized to obtain a local license plate image of the license plate image to be recognized;
normalizing the local license plate image of the license plate image to be recognized to obtain a normalized local license plate of the license plate image to be recognized;
and sending the normalized local license plate of the license plate image to be recognized into the self-adaptive character segmentation recognition module to obtain a character recognition result of the license plate image to be recognized.
In a second aspect, a license plate character recognition system based on adaptive position segmentation is provided, and comprises a deep neural network model building module, a training module and a character recognition module;
the deep neural network model building module is used for building a deep neural network model, and the deep neural network model comprises a basic network, a license plate position correction network, a license plate character segmentation network and a license plate character recognition network;
the training module is used for collecting license plate sample images, training the constructed deep neural network model and obtaining a self-adaptive character segmentation recognition model;
and the character recognition module is used for recognizing license plate characters of the license plate image to be recognized by utilizing the self-adaptive character segmentation recognition module.
Compared with the prior art, the invention has the following technical effects: because the license plate characters have definite position arrangement rules and have certain relevance among the license plate characters, the deep neural network model is constructed, the deep neural network model does not strictly distinguish steps of license plate position correction, license plate character segmentation, license plate character recognition and the like, but directly completes the license plate position correction, the license plate character segmentation and the character recognition, and a plurality of license plate character recognition steps in the traditional sense can be completed only by one deep neural network model, so that the accuracy of the license plate character recognition result is ensured, and the speed of license plate character recognition is improved.
Drawings
The following detailed description of embodiments of the invention refers to the accompanying drawings in which:
FIG. 1 is a schematic flow chart of a license plate character recognition method based on adaptive position segmentation;
FIG. 2 is a block diagram of a deep neural network model population;
FIG. 3 is a diagram of an infrastructure network architecture;
FIG. 4 is a diagram of a residual network infrastructure architecture;
FIG. 5 is a diagram of a license plate location correction network architecture;
FIG. 6 is a diagram of a license plate character segmentation network architecture;
FIG. 7 is a diagram of a license plate character recognition network architecture;
FIG. 8 is a schematic diagram of a license plate character recognition system based on adaptive location segmentation.
In the drawings, the identification on the left side of each neural network structure layer graph represents the output characteristic size of the network structure: feature layer width x feature layer height x feature layer channel number.
Detailed Description
To further illustrate the features of the present invention, refer to the following detailed description of the invention and the accompanying drawings. The drawings are for reference and illustration purposes only and are not intended to limit the scope of the present disclosure.
As shown in fig. 1, the embodiment discloses a license plate character recognition method based on adaptive position segmentation, which includes the following steps S1 to S3:
s1, constructing a deep neural network model, wherein the deep neural network model comprises a basic network, a license plate position correction network, a license plate character segmentation network and a license plate character recognition network;
s2, collecting license plate sample images, and training the constructed deep neural network model to obtain a self-adaptive character segmentation recognition model;
and S3, utilizing the self-adaptive character segmentation recognition model to recognize the license plate characters of the license plate image to be recognized.
According to the embodiment, a deep neural network structure model is constructed to directly complete the work of license plate position correction, license plate character segmentation, license plate character recognition and the like, and the overall optimal character recognition result is output. The steps of recognizing the characters of the license plate can be completed only by one deep neural network model, the recognition speed is higher, the error sources are fewer, and the recognition result is more accurate.
It should be noted that the deep neural network adopted in this embodiment is a Convolutional Neural Network (CNN), and is a deep neural network most commonly applied in the field of image processing by virtue of local sparse connection and weight sharing.
Preferably, as shown in fig. 2, the building of the deep neural network model in step S1 includes four parts: the license plate character recognition system comprises a base network, a license plate position correction network, a license plate character segmentation network and a license plate character recognition network, wherein the license plate character segmentation network and the license plate character recognition network share the base network and the license plate position correction network. The specific design steps of the deep neural network model are as follows S11 to S14:
s11, constructing a basic network:
in the embodiment, the basic network is mainly used for acquiring the high-level features with high abstraction and rich expression capability of the input image, and the quality of the high-level feature extraction directly influences the performance of subsequent character recognition. In addition, the processed object is the recognition of characters of a license plate, which is a very special image processing object: firstly, the license plate on the image has a specific length-width ratio, secondly, the characters of the license plate have a definite position arrangement rule, and different characters have certain correlation.
Therefore, the particularity of the license plate character recognition task and the computing capacity of the convolutional neural network are comprehensively considered, an improved ResNet classical network is adopted as a basic network of a deep neural network model, as shown in FIG. 3, an input image of the basic network is a 3-channel RGB image with the width of 192 pixels and the height of 64 pixels, and the output of the basic network is a high-level feature layer feature map. The base network includes: convolutional layer conv0, maximum pooling layer pool0, and 5 residual network infrastructure bodies resblock0, resblock1, resblock2, resblock3, and resblock4.conv 0 is a convolutional layer with a core size of 3x3 and a span of 2; pool0 is the maximum pooling layer with a core size of 2x2 and a span of 2; resblock0, resblock1, resblock2, resblock3, and resblock4 are 5 residual network infrastructure elements.
The specific network structure of the residual network infrastructure is shown in fig. 4, where bottom is the bottom layer feature (input feature) and top is the top layer feature (output feature). In three residual error network basic mechanisms of resblock0, resblock2 and resblock4, branch 1 is a convolution layer with the core size of 1x1 and the span of 1; branch 2_0 and branch 2_1 in branch 2 are convolutional layers with core size of 3x3 and span of 1.
In the resblock1 residual error network infrastructure, branch 1 is a convolutional layer with a core size of 1x1 and a span of 2, branch 2_0 in branch 2 is a convolutional layer with a core size of 3x3 and a span of 2, and branch 2_1 in branch 2 is a convolutional layer with a core size of 3x3 and a span of 1.
In the reblock 3 residual network infrastructure, branch 1 is a convolutional layer with a kernel size of 1x1, a height direction span of 2 and a width direction span of 1, branch 2_0 of branch 2 is a convolutional layer with a kernel size of 3x3, a height direction span of 2 and a width direction span of 1, and branch 2_1 of branch 2 is a convolutional layer with a kernel size of 3x3 and a span of 1; eltsum is the merging layer where the corresponding elements of the different branches are added.
In addition, each convolution layer in the base network is followed by a batch normalized BN layer and a nonlinear activation relu layer. Meanwhile, in this embodiment, unless otherwise specified, the span means that the span includes both the height direction span and the width direction span.
S12, constructing a license plate position correction network:
it should be noted that the position correction mainly converts a license plate with an undesirable position into a license plate with an ideal position through a space conversion operation, so as to facilitate accurate character segmentation in the subsequent process. The spatial transformation includes a translation transformation, a rotation transformation, a scaling transformation, a shearing transformation, a perspective transformation, etc., each of which may be represented as a 2x3 transformation matrix or a 3x3 transformation matrix. In actual environment, the correction of the license plate position generally needs the combination of multiple spatial transformations to achieve an ideal correction effect, so that multiple spatial transformation moments for correcting the license plate position are combined into a composite 3x3 transformation matrix.
The input feature image of the license plate position correction network is the high-level feature layer feature map acquired in step S11, and the output feature image of the license plate position correction network is the corrected high-level feature layer correct feature map. In this embodiment, the license plate position correction network is mainly based on a classical Spatial transform network STN (Spatial transform Networks) structure, and the network structure has an advantage of allowing a loss function to perform back propagation (back propagation) in the network, and can adaptively learn an undesirable position parameter of a license plate position for different license plate position images.
As shown in fig. 5, the license plate position correction network is specifically designed as follows in steps S121 to S123:
s121, obtaining a space transformation matrix for license plate position correction:
on the basis of the high feature layer feature map obtained in step S11, first, 9 feature parameters, that is, non-ideal position parameters of the current license plate position, are output through 1 convolution layer para _ conv _0 with a core size of 3x3 and a span of 1, then through 2 convolution layers para _ conv _1 and para _ conv _2 with a core size of 3x3 and a span of 2, and finally through 1 full-connected layer para _ fc. The 9 characteristic parameters are combined into a 3x3 matrix in sequence, and the inverse matrix of the matrix is the space transformation matrix for correcting the position of the license plate.
Preferably, each convolution layer in the license plate position correction network is followed by a batch normalization BN layer and a nonlinear activation relu layer, and each full-connection layer is followed by a nonlinear activation relu layer.
S122, acquiring a grid generator for correcting the position of the license plate:
the step is mainly based on the spatial transformation matrix for correcting the license plate position obtained in step S121, obtaining a pixel position corresponding relationship between the corrected output characteristic image and the original input characteristic image, and calculating the position of each pixel on the input characteristic image on the output characteristic image according to the following formula, where the corresponding position relationship is unique:
wherein:
in the formula: x is a radical of a fluorine atom i in 、y i in Representing (x) on the input feature image i in ,y i in ) Pixel position, x i out 、y i out Representing (x) on the output feature image i out ,y i out ) Pixel position, M denotes a spatial transformation matrix, M 11 、m 12 、m 13 、m 21 、m 22 、m 23 、m 31 、m 32 、m 33 Each represents an element in the spatial transformation matrix M, which is generally obtained by network structure learning in step S121.
S123, acquiring an output characteristic image generator for license plate position correction:
the step is mainly based on the grid generator model for correcting the license plate position obtained in step S122, calculating the gray value of each pixel position on the output characteristic image, where the gray value of each pixel position on the output characteristic image is equal to the gray value of the corresponding pixel position on the input characteristic image, and because the corresponding pixel position on the input characteristic image may be located at a non-integer position, obtaining the gray value of the pixel by using an improved bilinear interpolation algorithm, where the formula is as follows:
wherein, O i c Representing the gray value of the ith pixel on the c channel of the output feature image, the height and width of the output feature image being the same as those of the input feature image, I nm c Representing the gray value of the pixel (n, m) on the c channel of the input feature image, H and W respectively representing the height and width of the input feature image, max { } representing the function for obtaining the maximum value of the two elements, x i in 、y i in Indicating O on the output characteristic image i c Corresponding toThe pixel position on the input feature image is (x) i in ,y i in )。
S13, constructing a license plate character segmentation network:
the character segmentation network in the embodiment is mainly used for detecting the position rectangle of each character on the basis of the high-level feature layer extracted by the license plate position correction network, extracting a feature layer area corresponding to the position rectangle of each character on the high-level feature layer, forming a local feature image of each character, and further assisting and improving subsequent character recognition performance.
The particularity of the license plate character positions and the computing capacity of the convolutional neural network are comprehensively considered, based on the STN classical network structure idea in the step S12, an improved deep neural network structure of the multitask regression analysis is adopted, as shown in FIG. 6, the license plate character segmentation network comprises a convolutional layer conv _ rect0 with the kernel size of 3x3 and the span of 2, and the output of the convolutional layer conv _ rect0 is the input of the q character position segmentation network. Wherein q is a constant, the value of q is the same as the number of characters of the license plate number to be recognized, and a license plate with commonly used 7 characters is taken as an example for explanation:
on the basis of the high-level feature layer correct map obtained in step S12, a convolution layer conv _ rect0 with a kernel size of 3x3 and a span of 2 is passed first, and then 7 character position division networks loc0, loc1, loc2, loc3, loc4, loc5 and loc6 are passed respectively, and local feature images map0, map1, map2, map3, map4, map5 and map6 of 7 characters are output. The character position segmentation network is similar to the license plate position correction network in the step S12, the spatial transformation matrix M is a 2 × 3 segmentation transformation matrix, and the total number of the character segmentation parameters to be acquired is 4. The height and width of the output feature image are 2w and w, respectively, where w represents a fixed number. The 2 × 3 partition transformation matrix is shown by the following formula:
wherein M is f A character segmentation transformation matrix is represented, in which,s x 、s y 、t x 、t y matrix M for dividing and transforming characters f Where the element to be acquired by learning, in general, s x 、s y Corresponding to zoom operations on the x-coordinate axis, the y-coordinate axis, t x 、t y Corresponding to the translation operation on the x coordinate axis and the y coordinate axis.
S14, constructing a license plate character recognition network:
the license plate character recognition network mainly recognizes the real meaning of characters on the basis of local character images of the characters acquired by the character segmentation network, and then outputs the whole license plate character recognition result.
In this embodiment, a deep neural network structure adopting multi-task classification includes 1 convolution layer char _ conv0 with a kernel size of 3x3 and a span of 2, 1 fully-connected layer char _ fc0, and a connection layer concat connected to an output of the fully-connected layer char _ fc 0; the connection layer concat outputs the recognition result of the character. As shown in fig. 7, on the basis of the character local feature image obtained in step S13, first, 1 convolution layer char _ conv0 with a kernel size of 3x3 and a span of 2 passes through 1 full-connection layer char _ fc0, and then character recognition results reg0, reg1, reg2, reg3, reg4, reg5, reg6 corresponding to each character local feature image are output, and all character recognition results, that is, the final license plate character recognition result, are concatenated through the connection layer concat.
Note that all the character partial feature images share the convolution layer char _ conv0 and the fully-connected layer char _ fc0. In addition, each convolution layer is followed by a batch normalized BN layer and a nonlinear activation relu layer, and each fully-connected layer is followed by a nonlinear activation relu layer.
Preferably, the step S2: and collecting a license plate sample image, and training the constructed deep neural network model to obtain a self-adaptive character segmentation recognition model. The method specifically comprises the following steps S21 to S26:
s21, collecting license plate images:
the method mainly collects license plate images under various scenes, various light rays and various angles.
S22, arranging license plate images:
the position of the license plate is detected in the whole image by using the existing mature method, and the local license plate image is extracted from the whole image through cutting operation and is stored.
S23, marking a license plate character position rectangle:
the method comprises the steps of using the existing license plate character segmentation technology to segment and position license plate character position rectangles, then manually checking, correcting wrong character position rectangles, and recording letter meanings corresponding to each license plate character position rectangle.
It should be noted that, in the above steps S22 to S23, the license plate sample image is pre-sorted to obtain a local license plate image labeled with a license plate character position rectangle.
S24, expanding the license plate image:
the main method is to adopt the current commonly used image enhancement method, which comprises the following steps: and performing various translation transformation, rotation transformation, scaling transformation, shearing transformation, perspective transformation and color transformation, performing transformation operation on the pre-arranged license plate image, and expanding a license plate image sample library.
S25, license plate image normalization:
the size normalization operation is carried out on the license plate image set, all license plate images are scaled to a fixed size, a bilinear interpolation algorithm is mainly adopted, and the formula is as follows:
wherein (x, y) represents the pixel coordinates of the RGB value to be obtained, (x) 1 ,y 1 )、(x 2 ,y 1 )、(x 1 ,y 2 )、(x 2 ,y 2 ) Pixel coordinates representing the four known BGR values closest to pixel (x, y), g (x, y) representing the RGB value of pixel (x, y), and x representing the product.
S26, training a deep neural network model:
and sending the sorted normalized license plate image set into a defined deep neural network model, and learning related model parameters to obtain a self-adaptive character segmentation recognition model. The target loss function of the license plate character recognition network adopts a common multi-class cross entropy loss function.
Preferably, the step S3: and performing license plate character recognition on the license plate image to be recognized by using the self-adaptive character segmentation recognition module. The method specifically comprises the following steps:
the method comprises the following steps of using a deep neural network model, training the deep neural network model, then using the model in an actual environment, firstly detecting the position of a license plate of any given license plate image, then sending a local license plate image with normalized size into the trained deep neural network model, and outputting a license plate character recognition result, wherein the specific steps are as follows S31 to S32:
s31, detecting a local license plate image of the license plate image to be recognized:
the method comprises the steps of pre-sorting license plate images to be recognized, mainly detecting the positions of license plates in the whole images of the license plate images to be recognized, extracting local license plate images from the whole images through cutting operation, and normalizing the image sizes.
S32, recognizing license plate characters:
and (3) sending the normalized local license plate image into a trained deep neural network model, namely an adaptive character segmentation recognition model, and finally outputting a result, namely an optimal license plate character recognition result.
It should be noted that the invention directly completes the work of license plate position correction, license plate character segmentation, license plate character recognition and the like by constructing an integrated deep neural network model, and outputs the overall optimal character recognition result. The steps of license plate position correction, license plate character segmentation, license plate character recognition and the like are not strictly distinguished, and the steps of license plate character recognition can be completed only by one deep neural network model. The method has the advantages of higher recognition speed, fewer error sources, more accurate recognition result and higher robustness for the low-quality license plate images which are stained, adhered, lost in characters and inaccurate in positioning.
As shown in fig. 8, the embodiment discloses a license plate character recognition system based on adaptive location segmentation, which includes a deep neural network model building module 10, a training module 20, and a character recognition module 30;
the deep neural network model building module 10 is used for building a deep neural network model, and the deep neural network model comprises a basic network, a license plate position correction network, a license plate character segmentation network and a license plate character recognition network;
the training module 20 is used for collecting license plate sample images, training the constructed deep neural network model and obtaining a self-adaptive character segmentation recognition model;
the character recognition module 30 is configured to perform license plate character recognition on the license plate image to be recognized by using the adaptive character segmentation recognition module.
According to the embodiment, a deep neural network structure model is constructed to directly complete the work of license plate position correction, license plate character segmentation, license plate character recognition and the like, and the overall optimal character recognition result is output. The steps of recognizing the characters of the license plate can be completed only by one deep neural network model, the recognition speed is higher, the error sources are fewer, and the recognition result is more accurate.
It should be noted that the neural network model building module 10, the training module 20, and the character recognition module 30 in this embodiment are used to perform the steps in the above method embodiments.
The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents, improvements and the like that fall within the spirit and principle of the present invention are intended to be included therein.
Claims (5)
1. A license plate character recognition method based on self-adaptive position segmentation is characterized by comprising the following steps:
s1, constructing a deep neural network model, wherein the deep neural network model comprises a basic network, a license plate position correction network, a license plate character segmentation network and a license plate character recognition network;
s2, collecting license plate sample images, and training the constructed deep neural network model to obtain a self-adaptive character segmentation recognition model;
s3, utilizing the self-adaptive character segmentation and recognition module to perform license plate character recognition on a license plate image to be recognized;
the base network includes: the convolutional layer conv0, the maximum pooling layer pool0 and 5 residual network infrastructure bodies resblock0, resblock1, resblock2, resblock3 and resblock4; the core size of convolutional layer conv0 is 3x3 and the span is 2, the core size of maximum pooling layer pool0 is 2x2 and the span is 2;
the input image of the basic network is any 3-channel RGB image with the width of 192 pixels and the height of 64 pixels, and the output of the input image is a high-level feature layer feature map of the input image;
the number of the residual error network basic mechanisms is at least 3;
the residual error network infrastructure comprises a branch 1, a branch 2 and a merging layer eltsum, wherein the branch 2 comprises a branch 2_0 and a branch 2_1;
the branch circuit 1, the branch circuit 2 and the merging layer eltsum are convolution layers, and the outputs of the branch circuit 1 and the branch circuit 2 are connected to the merging layer eltsum;
the input feature image of the license plate position correction network is the high-level feature layer feature map, the output feature image is the corrected high-level feature layer correct feature map, and the process of the license plate position correction network for processing the high-level feature layer feature map comprises the following steps:
sequentially passing the high-level feature layer feature map through 1 convolutional layer para _ conv _0 with the core size of 3x3 and the span of 1, passing through 2 convolutional layers para _ conv _1 and para _ conv _2 with the core size of 3x3 and the span of 2 and passing through 1 fully-connected layer para _ fc, and outputting 9 feature parameters;
sequentially combining the 9 characteristic parameters into a 3x3 matrix, and determining an inverse matrix of the matrix as a space transformation matrix for correcting the position of the license plate;
acquiring a pixel position corresponding relation between an output characteristic image and an input characteristic image based on the space transformation matrix for correcting the license plate position;
calculating the gray value of the corresponding pixel position on the input characteristic image by adopting an improved bilinear interpolation algorithm;
obtaining a gray value of each pixel position on the output image based on the pixel position corresponding relation between the output characteristic image and the input characteristic image, and obtaining a corrected high-level characteristic layer correct feature map;
the license plate character segmentation network comprises a convolution layer conv _ rect0 with the core size of 3x3 and the span of 2, and the output of the convolution layer conv _ rect0 is respectively connected with the input of the q character position segmentation network;
enabling the output result of the license plate position correction network to pass through a convolutional layer conv _ rect0;
respectively passing the output result of the convolutional layer conv _ rect0 through a q character position segmentation network to obtain a q character local feature image;
the license plate character recognition network comprises 1 convolution layer char _ conv0 with the kernel size of 3x3 and the span of 2, wherein the output of the convolution layer char _ conv0 is connected with the input of 1 full-connection layer char _ fc0, and the output of the full-connection layer char _ fc0 is connected with the concat of the connection layer;
passing the partial feature images of the q characters through a convolutional layer char _ conv0;
and (4) outputting a license plate character recognition result after the output result of the convolution layer char _ conv0 passes through the full connection layer char _ fc0.
2. The method for recognizing characters on the basis of adaptive position segmentation according to claim 1, wherein the step S2 comprises:
collecting a license plate image as a license plate sample image, and performing pre-arrangement on the license plate sample image to obtain a local license plate image with a license plate character position rectangle marked;
adopting an image enhancement method to expand the pre-arranged license plate sample images, and taking the expanded license plate sample images as a license plate sample image set;
carrying out size normalization operation on the images in the license plate sample image set to obtain normalized images;
and training the deep neural network model by using the normalized image to obtain the self-adaptive character segmentation recognition model.
3. The method for recognizing characters on a license plate based on adaptive position segmentation as claimed in claim 1, wherein said step S3 comprises:
pre-sorting the license plate image to be recognized to obtain a local license plate image of the license plate image to be recognized;
normalizing the local license plate image of the license plate image to be recognized to obtain a normalized local license plate of the license plate image to be recognized;
and sending the local license plate of the normalized license plate image to be recognized into the self-adaptive character segmentation recognition module to obtain a character recognition result of the license plate image to be recognized.
4. The method of claim 1, wherein each convolutional layer in the base network is followed by a batch normalized BN layer and a non-linear activated relu layer.
5. The recognition system of the license plate character recognition method based on the adaptive position segmentation as claimed in claim 1, comprising a deep neural network model construction module, a training module and a character recognition module;
the deep neural network model building module is used for building a deep neural network model, and the deep neural network model comprises a basic network, a license plate position correction network, a license plate character segmentation network and a license plate character recognition network;
the training module is used for collecting license plate sample images, training the constructed deep neural network model and obtaining a self-adaptive character segmentation recognition model;
and the character recognition module is used for recognizing license plate characters of the license plate image to be recognized by utilizing the self-adaptive character segmentation recognition module.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811624974.1A CN109815956B (en) | 2018-12-28 | 2018-12-28 | License plate character recognition method based on self-adaptive position segmentation |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811624974.1A CN109815956B (en) | 2018-12-28 | 2018-12-28 | License plate character recognition method based on self-adaptive position segmentation |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109815956A CN109815956A (en) | 2019-05-28 |
| CN109815956B true CN109815956B (en) | 2022-12-09 |
Family
ID=66602721
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811624974.1A Active CN109815956B (en) | 2018-12-28 | 2018-12-28 | License plate character recognition method based on self-adaptive position segmentation |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109815956B (en) |
Families Citing this family (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110378337B (en) * | 2019-07-05 | 2023-03-31 | 上海交通大学 | Visual input method and system for drawing identification information of metal cutting tool |
| CN110427944A (en) * | 2019-09-06 | 2019-11-08 | 重庆紫光华山智安科技有限公司 | Acquisition methods, device, equipment and the storage medium of car plate detection data |
| CN111008632B (en) * | 2019-10-17 | 2023-06-09 | 安徽清新互联信息科技有限公司 | License plate character segmentation method based on deep learning |
| CN111027539B (en) * | 2019-10-17 | 2023-11-07 | 安徽清新互联信息科技有限公司 | License plate character segmentation method based on spatial position information |
| CN111008634B (en) * | 2019-11-22 | 2023-08-22 | 北京精英路通科技有限公司 | Character recognition method and character recognition device based on instance segmentation |
| CN111444911B (en) * | 2019-12-13 | 2021-02-26 | 珠海大横琴科技发展有限公司 | Training method and device of license plate recognition model and license plate recognition method and device |
| CN111191653A (en) * | 2019-12-25 | 2020-05-22 | 北京精英路通科技有限公司 | License plate recognition method and device, computer equipment and storage medium |
| CN111402164B (en) * | 2020-03-18 | 2023-10-24 | 北京市商汤科技开发有限公司 | Training method and device for correction network model, text recognition method and device |
| CN111444917A (en) * | 2020-03-30 | 2020-07-24 | 合肥京东方显示技术有限公司 | License plate character recognition method and device, electronic equipment and storage medium |
| CN111860690B (en) * | 2020-07-31 | 2024-05-28 | 河北交投智能科技股份有限公司 | Dangerous goods vehicle detection and identification method based on deep learning |
| CN112150452B (en) * | 2020-09-30 | 2022-04-15 | 西南交通大学 | Millimeter wave imaging-based railway sleeper defect detection method |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP1085456A2 (en) * | 1999-09-15 | 2001-03-21 | Siemens Corporate Research, Inc. | Character segmentation method for vehicle license plate recognition |
| CN107563385A (en) * | 2017-09-02 | 2018-01-09 | 西安电子科技大学 | License plate character recognition method based on depth convolution production confrontation network |
| CN108108746A (en) * | 2017-09-13 | 2018-06-01 | 湖南理工学院 | License plate character recognition method based on Caffe deep learning frames |
| WO2018112900A1 (en) * | 2016-12-23 | 2018-06-28 | 深圳先进技术研究院 | License plate recognition method and apparatus, and user equipment |
-
2018
- 2018-12-28 CN CN201811624974.1A patent/CN109815956B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP1085456A2 (en) * | 1999-09-15 | 2001-03-21 | Siemens Corporate Research, Inc. | Character segmentation method for vehicle license plate recognition |
| WO2018112900A1 (en) * | 2016-12-23 | 2018-06-28 | 深圳先进技术研究院 | License plate recognition method and apparatus, and user equipment |
| CN107563385A (en) * | 2017-09-02 | 2018-01-09 | 西安电子科技大学 | License plate character recognition method based on depth convolution production confrontation network |
| CN108108746A (en) * | 2017-09-13 | 2018-06-01 | 湖南理工学院 | License plate character recognition method based on Caffe deep learning frames |
Non-Patent Citations (2)
| Title |
|---|
| 基于BP神经网络的车牌字符识别算法研究;欧阳俊等;《光学与光电技术》;20121010(第05期);全文 * |
| 基于深度学习的车牌字符识别的设计与实现;肖秀春等;《电子技术与软件工程》;20180824(第16期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109815956A (en) | 2019-05-28 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109815956B (en) | License plate character recognition method based on self-adaptive position segmentation | |
| CN109753914B (en) | License plate character recognition method based on deep learning | |
| Zhang et al. | Loop closure detection for visual SLAM systems using convolutional neural network | |
| CN109345508B (en) | Bone age evaluation method based on two-stage neural network | |
| CN107563433B (en) | Infrared small target detection method based on convolutional neural network | |
| Yan et al. | Weakly-and semi-supervised object detection with expectation-maximization algorithm | |
| CN110569738B (en) | Natural scene text detection method, equipment and medium based on densely connected network | |
| CN110321967B (en) | Image classification improvement method based on convolutional neural network | |
| CN112633382B (en) | Method and system for classifying few sample images based on mutual neighbor | |
| CN103761531A (en) | Sparse-coding license plate character recognition method based on shape and contour features | |
| US20200134382A1 (en) | Neural network training utilizing specialized loss functions | |
| Lim et al. | Shape classification using local and global features | |
| CN111126401B (en) | License plate character recognition method based on context information | |
| Thanikkal et al. | Advanced plant leaf classification through image enhancement and canny edge detection | |
| CN113177549B (en) | Few-sample target detection method and system based on dynamic prototype feature fusion | |
| CN113343989B (en) | Target detection method and system based on self-adaption of foreground selection domain | |
| Khellal et al. | Pedestrian classification and detection in far infrared images | |
| Hussain et al. | A survey of traffic sign recognition systems based on convolutional neural networks | |
| CN114723010B (en) | Automatic learning enhancement method and system for asynchronous event data | |
| CN114283431B (en) | Text detection method based on differentiable binarization | |
| Lomanov et al. | Cell detection with deep convolutional networks trained with minimal annotations | |
| Singh et al. | Wavelet based histogram of oriented gradients feature descriptors for classification of partially occluded objects | |
| CN116863293A (en) | Marine target detection method under visible light based on improved YOLOv7 algorithm | |
| CN116543250A (en) | Model compression method based on class attention transmission | |
| CN115761667A (en) | Unmanned vehicle carried camera target detection method based on improved FCOS algorithm |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |