CN115424217A - AI vision-based intelligent vehicle identification method and device and electronic equipment - Google Patents
AI vision-based intelligent vehicle identification method and device and electronic equipment Download PDFInfo
- Publication number
- CN115424217A CN115424217A CN202211056249.5A CN202211056249A CN115424217A CN 115424217 A CN115424217 A CN 115424217A CN 202211056249 A CN202211056249 A CN 202211056249A CN 115424217 A CN115424217 A CN 115424217A
- Authority
- CN
- China
- Prior art keywords
- image
- vehicle
- face
- scene
- features
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects
- G06V20/54—Surveillance or monitoring of activities, e.g. for recognising suspicious objects of traffic, e.g. cars on the road, trains or boats
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/42—Global feature extraction by analysis of the whole pattern, e.g. using frequency domain transformations or autocorrelation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
- G06V10/761—Proximity, similarity or dissimilarity measures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V2201/00—Indexing scheme relating to image or video recognition or understanding
- G06V2201/08—Detecting or categorising vehicles
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Computing Systems (AREA)
- Artificial Intelligence (AREA)
- Health & Medical Sciences (AREA)
- Databases & Information Systems (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- Software Systems (AREA)
- Image Analysis (AREA)
- Image Processing (AREA)
- Traffic Control Systems (AREA)
Abstract
The invention provides an AI vision-based intelligent vehicle identification method, an AI vision-based intelligent vehicle identification device and electronic equipment, wherein the method comprises the following steps of m1, acquiring an image of a vehicle without a license plate as an image to be identified, and acquiring a scene of the image to be identified based on a scene classification model; step m2, extracting vehicle attribute features, vehicle face global features and vehicle face local features from the image to be recognized based on the feature extraction model associated with the scene obtained in step m 1; step m3, performing image search in the sample image library associated with the scene obtained in the step m1 based on the vehicle attribute features and the vehicle face global features to obtain a first search result; step m4, selecting one or more recommended areas from nine car face sub-images of the image to be recognized based on the Sudoku and the attention map; m5, performing image search in the first search result based on the local features of the car face in the recommended region to obtain a car identification result; the method can reduce the operation amount of the similarity measurement and improve the accuracy of the similarity measurement.
Description
Technical Field
The invention relates to a vehicle identification method, in particular to an AI vision-based intelligent vehicle identification method, an AI vision-based intelligent vehicle identification device and electronic equipment.
Background
In order to regulate the over-limit and overload transportation of freight vehicles and practically ensure the smooth road traffic safety and the life and property safety of the masses, the national transportation department promulgates a series of regulation regulations aiming at the over-limit and overload phenomena of the freight vehicles, and along with the continuous promotion of various provinces on the over-limit and overload regulation work of the freight vehicles, individual drivers of the freight vehicles adopt illegal behaviors of intentionally shielding license plate numbers to avoid detection punishment, thereby bringing a plurality of difficulties to the work of law enforcement departments.
Artificial intelligence is a branch of computer science with 4 elements: algorithm, calculation power, data, application scenario. With the progress and enrichment of the above four elements, the application field of artificial intelligence is also continuously expanding, such as machine vision, automatic planning, intelligent control, language and image understanding, and so on. Al vision, one of the core technologies in the field of artificial intelligence, is the most direct information source of a machine vision system, and is important for robots, such as human eyes. In Al vision, a vision sensor and a computer are used for replacing human eyes, so that a machine has the functions of segmenting, classifying, identifying, tracking and judging a target similar to those of the human eyes, and the system realizes the capability of simulating a human's thinking map', namely human thinking logic
At present, the vehicle identification technology based on AI vision is more and more widely applied to the aspects of intelligent traffic monitoring, intelligent traffic information systems and the like, and the vehicle identification comprises the contents of moving vehicle detection, vehicle type classification, vehicle tracking, license plate identification and the like. The invention patent with the bulletin number of CN107729818B discloses a multi-feature fusion vehicle re-identification method based on deep learning, and by designing the vehicle re-identification method in which a license plate identification vector, a vehicle expressive feature vector and a vehicle type attribute feature vector are fused, the violation conditions such as fake license plates can be judged, a plurality of attribute information such as vehicle colors, brands and types can be obtained, and meanwhile, the accuracy of vehicle re-identification in a monitoring video can be effectively improved.
However, in the patent, each time the data is required to be in a large-scale database with different scenes, the data volume is large; the existing deep learning detection method is usually applied in the daytime with good light, and has a plurality of challenges in night scenes, namely the vehicle features are easy to hide due to weak light environment, data sets are relatively lacked, the network does not sufficiently learn the significant features of the vehicle, and the detection means is single, so that only part of targets can be reflected. These make it difficult for existing deep learning methods to exhibit detection effects comparable to daytime scenes. Similarly, under different exposure conditions, the feature vectors of the same vehicle can generate deviation due to environmental factors and exposure degrees, for example, when visibility is low in rainy days and foggy days, the quality of license plate pictures can be affected; at night, some black and white blocks appear in the image of the license plate under the action of the headlamp, and the vehicle identification effect is influenced by the condition.
In order to solve the above problems, people are always seeking an ideal technical solution.
Disclosure of Invention
The invention aims to overcome the defects of the prior art, and provides an AI vision-based intelligent vehicle identification method, an AI vision-based intelligent vehicle identification device and electronic equipment.
In order to achieve the purpose, the invention adopts the technical scheme that: an AI vision-based intelligent vehicle identification method comprises the following steps:
step m1, scene classification
Acquiring a license plate-free vehicle image as an image to be recognized, classifying the image to be recognized based on a scene classification model, and acquiring a scene of the image to be recognized;
step m2, feature extraction
Detecting a target vehicle image from the image to be recognized based on a target detection algorithm, detecting and segmenting a vehicle face global image from the target vehicle image, and dividing the vehicle face global image according to a nine-square grid to obtain nine vehicle face sub-images;
sequentially extracting vehicle attribute features, vehicle face global features and vehicle face local features from the target vehicle image, the vehicle face global image and each vehicle face sub-image on the basis of the feature extraction model associated with the scene obtained in the step m 1;
step m3, first image search
Performing image search in a sample image library associated with the scene obtained in the step m1 based on the vehicle attribute features and the vehicle face global features to obtain a first search result;
step m4, regional recommendation
Selecting one or more recommended areas from nine car face sub-images of the image to be recognized based on the nine palace lattices and the attention map;
step m5, secondary image search
And carrying out image search in the first search result based on the local features of the car face in the recommended region, and obtaining a car identification result.
Based on the above, the specific steps of selecting one or more recommended regions from nine car face sub-images of the image to be recognized based on the nine-square grid and the attention map are as follows:
processing the global image of the car face based on the attention map, and extracting data of the attention map layer as an attention feature matrix;
quantizing each value of the attention feature matrix, taking a median value of the attention feature matrix as a threshold, assigning all values larger than the threshold in the attention feature matrix as 1, assigning all values smaller than the threshold in the attention feature matrix as 0, and obtaining a quantized feature image M;
dividing the characteristic image M according to the Sudoku to obtain nine sub-characteristic images, wherein each sub-characteristic image is a sub-characteristic imageLike M i Wherein i =0,1, ·,8;
calculating the sum S of all values in each sub-feature image i ；
To S i Sorting according to the sequence from big to small, and selecting three S in the top i And the corresponding sub-feature image is used as a recommended area in a corresponding car face sub-image in the car face global image.
Based on the above, the specific steps of step m1 are as follows:
m1.1, acquiring a vehicle image in real time, judging whether the vehicle image has a license plate number, and if not, taking the vehicle image as an image to be identified;
m1.2, carrying out day and night classification on the image to be recognized by using a day and night classifier constructed by a Hue Saturation Value (HSV) color model and an openCV, and determining whether the image to be recognized is a day image or a night image;
m1.3, converting the image to be identified into a gray image, calculating the mean value and the variance of the gray image, carrying out exposure classification on the image to be identified based on the mean value and the variance, and determining that the image to be identified is a normal exposure image, an overexposure image or an underexposure image;
m1.4, determining the scene of the image to be identified as a day overexposure scene, a night overexposure scene, a day underexposure scene, a night underexposure scene, a day exposure normal scene or a night exposure normal scene according to the identification results of m1.2 and m 1.3.
Based on the above, after the vehicle identification results are obtained, each vehicle identification result and the image to be identified are synchronously checked in a linkage manner.
The invention also provides an AI vision-based intelligent vehicle identification device, which comprises:
the image acquisition module is used for acquiring the image of the vehicle without the license plate and taking the image of the vehicle without the license plate as an image to be identified;
the scene selection module is used for classifying the image to be recognized based on the scene classification model and acquiring the scene of the image to be recognized;
the characteristic extraction module is internally provided with a plurality of characteristic extraction models under different scenes; detecting a target vehicle image from the image to be recognized based on a target detection algorithm, detecting and segmenting a vehicle face global image from the target vehicle image, and dividing the vehicle face global image according to a nine-square grid to obtain nine vehicle face sub-images; sequentially extracting vehicle attribute features, vehicle face global features and vehicle face local features from a target vehicle image, a vehicle face global image and each vehicle face sub-image through a feature extraction model in a corresponding scene;
the primary image searching module is used for carrying out image searching in a sample image library of a corresponding scene based on vehicle attributes and global characteristics of a vehicle face to obtain a first searching result;
the area recommendation module is used for selecting one or more recommendation areas from nine car face sub-images of the image to be recognized based on the squared figure and the attention map;
and the secondary image searching module is used for carrying out image searching in the first searching result based on the local characteristics of the vehicle face in the recommended region to obtain a vehicle identification result.
The present invention also provides an electronic device, including:
a memory for storing executable instructions;
and the processor is used for realizing the intelligent vehicle identification method when executing the executable instructions stored in the memory.
Compared with the prior art, the invention has outstanding substantive characteristics and remarkable progress, particularly,
(1) The method comprises the steps of carrying out scene analysis on an image to be recognized, selecting a feature extraction model of a corresponding scene according to an analysis result, and extracting the vehicle attribute and the vehicle face feature of the image to be recognized, so that the vehicle attribute and the vehicle face feature of the image to be recognized can be accurately extracted; furthermore, after the vehicle attribute and the vehicle face feature of the image to be identified are extracted, similarity measurement is carried out in the sample image library of the corresponding scene based on the vehicle attribute and the vehicle face feature, and the operation amount of the similarity measurement can be greatly reduced because the similarity measurement is not required to be carried out on the sample image libraries in all scenes; meanwhile, the similarity measurement accuracy can be improved by searching in a similar scene;
(2) When the features are extracted, the vehicle attributes, the vehicle face global features and the vehicle face local features of the vehicle face global images which are required in the intelligent vehicle identification process are extracted in advance at the same time, so that the extraction time for extracting the vehicle attributes, the vehicle face global features and the vehicle face local features one by one is reduced, and the extraction efficiency is improved;
(3) The method adopts a two-step image searching method, wherein in the first step, similarity retrieval is carried out on the basis of vehicle attributes and vehicle face global characteristics, and vehicle images with similar vehicle attributes and vehicle face global characteristics are found out so as to narrow the range of subsequent secondary retrieval; secondly, secondary retrieval is carried out based on the local features of the car face, and the vehicle pictures with unmatched local region features of the car face are filtered, so that the accuracy of the intelligent recognition result of the vehicle features is improved;
(4) The method intelligently recommends one or more sub-regions to perform local region similarity measurement based on the Sudoku, the attention map and the prior library, can improve the accuracy of the local region similarity measurement, and can reduce the operation amount because similarity measurement is not required to be performed on all the regions;
(5) The invention has the advantages that the vehicle attribute, the face region, the face global feature and the nine-square grid local feature of the face region of the vehicle image with the license plate number are extracted, and the vehicle attribute, the face region, the face global feature and the face local feature of the vehicle image with the license plate number are stored in the sample library, so that the effect of continuously learning and expanding the sample library is achieved;
(6) According to the invention, after the vehicle identification results are obtained, each vehicle identification result and the image to be identified are synchronously checked in a linkage manner, when one of the images is amplified, the other image is automatically amplified and positioned to the same visual area, so that the manual operation cost can be reduced, the comparison and confirmation efficiency of the artificial intelligent identification results is improved, and the assistance is used for quickly generating the identification result report.
Drawings
Fig. 1 is a flow chart of the intelligent vehicle identification method of the invention.
FIG. 2 is a flow chart of the area recommendation of the present invention.
FIG. 3 is a diagram illustrating the results of the region recommendation of the present invention.
FIG. 4 is a flow chart illustrating the simultaneous linkage viewing according to the present invention.
Detailed Description
The technical solution of the present invention is further described in detail by the following embodiments.
According to the invention, a large amount of research is carried out on local short-distance transport vehicle groups, and the method has the characteristics that the method can frequently pass through the same road at different time (particularly at night), and can not shield the number plate when the vehicle is not overweight, and can shield the number plate when the vehicle is overweight.
Example 1
As shown in fig. 1, the present embodiment provides an AI vision-based intelligent vehicle identification method, which includes the following steps:
step m1, scene classification
Acquiring a license plate-free vehicle image as an image to be recognized, classifying the image to be recognized based on a scene classification model, and acquiring a scene of the image to be recognized;
step m2, feature extraction
Detecting a target vehicle image from the image to be recognized based on a target detection algorithm, detecting and segmenting a vehicle face global image from the target vehicle image, and dividing the vehicle face global image according to a nine-square grid to obtain nine vehicle face sub-images;
sequentially extracting vehicle attribute features, vehicle face global features and vehicle face local features from the target vehicle image, the vehicle face global image and each vehicle face sub-image on the basis of the feature extraction model associated with the scene obtained in the step m 1;
step m3, first image search
Performing image search in a sample image library associated with the scene obtained in the step m1 based on the vehicle attribute features and the vehicle face global features to obtain a first search result; preferably, the present embodiment adopts an image retrieval method based on image similarity measurement;
step m4, regional recommendation
Selecting one or more recommended areas from nine car face sub-images of the image to be recognized based on the nine palace lattices and the attention map;
step m5, secondary image search
And carrying out image search in the first search result based on the local features of the car face in the recommended region, and obtaining a car identification result.
Aiming at the identification of the vehicle with the license plate shielded, the invention determines the license plate number corresponding to the vehicle with the license plate shielded through the data of the vehicle when the vehicle does not shield the license plate in other time periods; because most vehicles for shielding license plates are overloaded large trucks and the vehicles can change greatly due to uncertain time span, the recognition of the vehicles for shielding the license plates is carried out by adopting the vehicle face area.
In addition, in different scenes, the feature vectors of the same vehicle can deviate due to environmental factors and exposure degrees, for example, in an overexposure scene in the daytime and an underexposure scene at night, the feature vectors of the same vehicle can deviate greatly, so that the same vehicle cannot be accurately retrieved; therefore, in the embodiment, the scene of the image to be analyzed is automatically judged, and then subsequent identification is performed according to the sign extraction model corresponding to the automatically selected scene, for example, if the image to be identified is judged to be in a daytime exposure normal scene, image retrieval is preferably performed from the daytime exposure normal scene, so that similar images can be accurately retrieved; meanwhile, only the daytime scene library is searched, so that the number of comparison samples can be reduced, and the searching efficiency is further improved.
It can be understood that in the vehicle intelligent identification method, a clear and complete scene sample library is established by using the images of the license plates when the vehicle is not overweight in advance, and feature extraction models in different scenes are trained in advance, wherein the feature extraction models comprise a vehicle attribute extraction model, a vehicle face global feature extraction model and a vehicle face local feature extraction model; the vehicle face local feature extraction model needs to divide a vehicle face global image into nine-square grids and then extract local features of each vehicle face sub-image respectively.
The vehicle attribute characteristics comprise vehicle color, vehicle logo, vehicle type, vehicle lamp, vehicle reflector number and the like; when the vehicle attribute extraction model is trained, a vehicle color recognition model, a vehicle logo recognition model, a vehicle type recognition model, a vehicle lamp recognition model and a vehicle reflector recognition model can be respectively trained, and a vehicle attribute extraction model integrating vehicle color recognition, vehicle logo recognition, vehicle lamp recognition, vehicle type recognition and vehicle reflector recognition can also be trained.
Since the vehicle attribute features are significant, in specific implementation, in step m3, coarse-grained image search may be performed in a sample image library of a corresponding scene based on the vehicle attribute features, and then fine-grained image search may be performed on a search result based on the vehicle face global features, so as to obtain a first search result. For example, data with incomplete car faces are firstly excluded from extraction; then, removing the global image of the car face which does not meet the requirement by detecting the number of the car lamps and the car reflectors in the global image of the car face; and then removing the global image of the car face which does not meet the requirement by taking the car logo, the car type and the car color as the obvious attribute features of the car, thereby improving the accuracy of the subsequent similarity measurement based on the global features of the car face.
In another embodiment, step m3 may also perform similarity measurement based on the vehicle attribute features and the vehicle face global features simultaneously.
In specific implementation, the specific steps of step m1 are as follows:
m1.1, acquiring a vehicle image in real time, judging whether the vehicle image has a license plate number, and if not, taking the vehicle image as an image to be identified;
m1.2, carrying out day and night classification on the image to be recognized by using a day and night classifier constructed by a Hue Saturation Value (HSV) color model and an openCV, and determining whether the image to be recognized is a day image or a night image;
hue Saturation Value (HSV) is a surrogate color model for RGB, hue (H) is the three primary and three secondary colors in a color, and saturation (S) is the purity and intensity of a color, the lower it is, the closer the color is to gray; the value (V) refers to the relative lightness or darkness of the color, each of these values having a limit; h from 0 to 360, S and V from 0 to 100; when in use, an optimal threshold value can be found through basic derivation through training images to carry out day and night classification;
m1.3, converting the image to be identified into a gray image, calculating the mean value and the variance of the gray image, carrying out exposure classification on the image to be identified based on the mean value and the variance, and determining that the image to be identified is an exposed normal image, an overexposed image or an underexposed image;
m1.4, determining the scene of the image to be identified as a day overexposure scene, a night overexposure scene, a day underexposure scene, a night underexposure scene, a day exposure normal scene or a night exposure normal scene according to the identification results of m1.2 and m 1.3.
It should be noted that in step m1.1, if a license plate number is determined in the vehicle image, the license plate number in the vehicle image is identified and compared with the license plate number in the vehicle passing record information reported by the data acquisition service, if the license plate numbers are consistent, the steps m1.2, m1.3 and m1.4 are continuously executed to obtain the scene where the current vehicle image is located, the vehicle face global image, the vehicle attribute feature, the vehicle face global feature and the vehicle face local feature of each vehicle face sub-image are obtained based on the feature extraction model associated with the scene where the current vehicle image is located, and the current vehicle image, the vehicle face global image, the vehicle attribute feature, the vehicle face global feature and the vehicle face local feature are stored in the sample image library of the corresponding scene; and if the license plate numbers are not consistent, storing the current vehicle image into the suspected wrong license plate sample library.
The steps can realize continuous expansion of the sample library by utilizing the images of the number plates when the vehicles are not overweight so as to establish a clear and complete vehicle characteristic space-time chain, and effectively improve the identification accuracy and the reliability of the vehicles without the number plates.
Further, it can be understood that, in the process of determining the scene of the image to be identified, it can also be determined whether the similarity between the scene of the image to be identified and the daytime overexposure scene, the night overexposure scene, the daytime underexposure scene, the night underexposure scene, the daytime exposure normal scene or the night exposure normal scene reaches a threshold value, and if so, it is determined that the scene belongs to the daytime overexposure scene, the night overexposure scene, the daytime underexposure scene, the night underexposure scene, the daytime exposure normal scene or the night exposure normal scene; and if not, judging that the current image to be recognized does not belong to any scene.
When the current image to be recognized does not belong to any scene, the whole vehicle texture features and the texture features of the key points of the image to be recognized are directly extracted by using the trained texture feature extraction model, similarity searching is carried out in a pre-trained texture sample library, and an image recommendation result is given.
For example, when the license plate number in the current vehicle image is judged to be consistent with the license plate number in the vehicle passing record information reported by the data acquisition service and the current vehicle image is judged not to belong to any scene, the whole vehicle texture feature and each key point texture feature of the current vehicle image are extracted by directly utilizing a trained texture feature extraction model, and the vehicle image with the license plate, the whole vehicle texture feature and each key point texture feature are added into a texture sample library to realize the continuous expansion of the texture sample library; and when vehicle retrieval is carried out subsequently, similarity searching is carried out based on the texture sample library.
In specific implementation, the texture feature extraction steps are as follows: color influence in the vehicle image of the feature to be extracted is removed through image binarization operation, and texture features of the whole vehicle and key points (vehicle lamps, rearview mirrors, annual inspection marks and the like) are extracted from the binary image by using a data enhancement method.
It should be noted that, due to the similarity between vehicles of the same brand and the same model, it is difficult to accurately retrieve the vehicle images according to the comparison of the global feature vectors of the vehicle faces, so that the comparison of the local feature vectors can be further performed through stickers, annual check marks, hanging decorations, scratches and the like on the vehicle faces; however, if all local features are compared, the data size is large and the time is long.
Therefore, in this embodiment, one or more recommended regions are selected from nine car face sub-images of the image to be recognized based on the squared figure and the attention map to perform joint similarity measurement, as shown in fig. 2, the specific steps are as follows:
processing the global image of the car face based on the attention map, and extracting data of the attention map layer as an attention feature matrix;
quantizing each value of the attention feature matrix, taking a median value of the attention feature matrix as a threshold, assigning all values larger than the threshold in the attention feature matrix as 1, and assigning all values smaller than the threshold in the attention feature matrix as 0 to obtain a quantized feature image M;
dividing the characteristic image M according to the Sudoku to obtain nine sub-characteristic images, and recording each sub-characteristic image as M i Wherein i =0,1, ·,8;
calculating the sum S of all values in each sub-feature image i ；
To S i Sorting according to the sequence from big to small, and selecting three S with the top sorting i And the corresponding sub-feature image is a corresponding sub-image of the car face in the global car face image, and is used as a recommended area, as shown in fig. 3.
It can be seen that the importance degree of the sub-regions recommended based on the attention map and the Sudoku intelligence is higher than that of the non-recommended sub-regions, so that the accuracy of the local region similarity measurement can be improved by performing the local region similarity measurement based on the sub-regions with high importance degree, and meanwhile, the calculation amount can be reduced because the similarity measurement is not required to be performed on all the regions.
Further, after the recommendation areas are obtained, whether the car face local features of each recommendation area are located in the priori significant feature library or not is judged, if yes, the recommendation areas are reserved, and otherwise, the recommendation areas are omitted. The area where the user is interested in the significant features is screened out based on the prior library, so that the accuracy of image joint similarity measurement can be improved.
Example 2
This example differs from example 1 in that: before or at the same time of selecting one or more recommended regions from nine car face sub-images of an image to be recognized based on a Sudoku and an attention map, judging whether the local car face features of each car face sub-image are located in a priori significant feature library, and if so, taking the car face sub-images as priori recommended regions;
after one or more recommended regions are selected from nine car face sub-images of the image to be identified based on the Sudoku and the attention map, whether each recommended region belongs to a priori recommended region or not is judged, if yes, the recommended region is reserved, and if not, the recommended region is omitted.
Example 3
The information of the unoccluded license plate vehicles with the highest similarity can be obtained through an artificial intelligence algorithm and characteristic vector retrieval, but in consideration of the suspected attitude of a user to artificial intelligence, the scheme provides a manual confirmation function through a left-right double-graph comparison mode after the recognition result with the highest similarity is obtained.
In the prior art, a user needs to respectively amplify the original head images of the prototype diagrams and manually compare the same areas of the faces of the head images to be compared. Through investigation, it is found that the picture viewing tool only supports zooming of a single image and the picture amplifier tool only supports amplifying and displaying of a local area of a single picture, and the functional requirements cannot be met; simultaneously in order to promote user experience, reduce the operation degree of difficulty when artifical comparison is confirmed, improve work efficiency, this embodiment is after obtaining vehicle identification result, looks over each vehicle identification result and the image of waiting to discern in the linkage of synchronizing.
As shown in fig. 4, the specific steps of synchronous linkage viewing are as follows:
acquiring an image to be identified and a vehicle face area in a vehicle identification result to be compared;
respectively calculating the width, height and center coordinates of the vehicle face area in the image to be recognized and the vehicle recognition result to be compared based on the pixel coordinates, wherein the width, height and center coordinates of the image to be recognized are respectively as follows: w, h and (cx, cy); the width, height and center coordinates of the vehicle identification results to be compared are respectively as follows: w ', h', and (cx ', cy');
the method comprises the steps of scaling a vehicle face area in an image to be identified and a vehicle identification result to be compared to 600 pixels by 600 pixels, and obtaining vehicle face scaling ratios rw and rh of the image to be identified and vehicle face scaling ratios rw 'and rh' of the vehicle identification result to be compared;
obtaining coordinates (x, y) of a mouse pointer in an image to be identified, and calculating coordinates of a center point of a pre-zooming area of a vehicle identification result to be compared:
x’ = (cx’ + ((cx - x) * rw) / rw’)
y’ = (cy’ + ((cy - y) * rh) / rh’)；
when the zooming action of the mouse pointer in the image to be recognized is detected, based on the zooming ratios rw and rh of the image to be recognized, the coordinate (x ', y') of the central point of the pre-zooming area of the vehicle recognition result to be compared and the zooming ratios rw 'and rh', the vehicle recognition result to be compared is synchronously zoomed.
It can be understood that the method supports automatic amplification of one picture and positioning of the other picture to the same visual area when the other picture is amplified, so that local features of the car face can be compared conveniently and manually, for example, the left picture is amplified to the position of a license plate, and the right picture is amplified to the position of the license plate automatically.
The tool reduces the manual operation cost, improves the comparison and confirmation efficiency of the artificial intelligent recognition result, and assists in quickly generating a recognition result report.
Example 4
The embodiment provides an AI vision-based intelligent vehicle identification device, which comprises:
the image acquisition module is used for acquiring the image of the vehicle without the license plate and taking the image of the vehicle without the license plate as an image to be identified;
the scene selection module is used for classifying the image to be recognized based on the scene classification model and acquiring the scene of the image to be recognized;
the characteristic extraction module is internally provided with a plurality of characteristic extraction models under different scenes; detecting a target vehicle image from the image to be recognized based on a target detection algorithm, detecting and segmenting a vehicle face overall image from the target vehicle image, and dividing the vehicle face overall image according to nine grids to obtain nine vehicle face sub-images; sequentially extracting vehicle attribute features, vehicle face global features and vehicle face local features from a target vehicle image, a vehicle face global image and each vehicle face sub-image through a feature extraction model in a corresponding scene;
the primary image searching module is used for carrying out image searching in a sample image library of a corresponding scene based on vehicle attributes and global characteristics of a vehicle face to obtain a first searching result;
the area recommendation module is used for selecting one or more recommendation areas from nine vehicle face sub-images of the image to be recognized based on the nine-square and the attention map;
and the secondary image searching module is used for carrying out image searching in the first searching result based on the local characteristics of the vehicle face in the recommended region to obtain a vehicle identification result.
The method comprises the steps of carrying out scene analysis on an image to be recognized, selecting a feature extraction model of a corresponding scene according to an analysis result, and extracting vehicle attributes and vehicle face features of the image to be recognized, so that the vehicle attributes and the vehicle face features of the image to be recognized can be accurately extracted; furthermore, after the vehicle attribute and the vehicle face feature of the image to be identified are extracted, similarity measurement is carried out in the sample image library of the corresponding scene based on the vehicle attribute and the vehicle face feature, and the operation amount of the similarity measurement can be greatly reduced because the similarity measurement is not required to be carried out on the sample image libraries in all scenes;
when the features are extracted, the vehicle attributes, the vehicle face global features and the vehicle face local features of the vehicle face global image, which are required in the intelligent vehicle identification process, are extracted simultaneously in advance, so that the extraction time for extracting the vehicle attributes, the vehicle face global features and the vehicle face local features one by one is reduced, and the extraction efficiency is improved;
in the embodiment, a two-step image searching method is adopted, wherein in the first step, similarity retrieval is carried out on the basis of vehicle attributes and vehicle face global characteristics, and vehicle pictures with similar vehicle attributes and vehicle face global characteristics are found out so as to narrow the range of subsequent secondary retrieval; and secondly, carrying out secondary retrieval based on the local features of the car face, and filtering the vehicle pictures with unmatched local region features of the car face, so that the accuracy of the intelligent recognition result of the vehicle features is improved.
Example 5
The present embodiment also provides an electronic device, including:
a memory for storing executable instructions;
and the processor is used for realizing the intelligent vehicle identification method when executing the executable instructions stored in the memory.
The present embodiment also provides an embodiment of a readable storage medium, which has stored thereon instructions that, when executed by a processor, implement the steps of the intelligent vehicle identification method according to embodiments 1-3.
In the above embodiments, the descriptions of the respective embodiments have respective emphasis, and reference may be made to the related descriptions of other embodiments for parts that are not described or illustrated in a certain embodiment.
Those of ordinary skill in the art will appreciate that the various illustrative modules and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.
In the embodiments provided in the present application, it should be understood that the disclosed system and apparatus may be implemented in other ways. For example, the above-described device embodiments are merely illustrative, and for example, the division of the above-described modules is only one logical functional division, and other divisions may be realized in practice, for example, multiple units or components may be combined or integrated into another system, or some features may be omitted, or not executed.
In addition, functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.
The integrated module may be stored in a computer-readable storage medium if it is implemented in the form of a software functional unit and sold or used as a separate product. Based on such understanding, all or part of the flow in the method of the embodiments described above may be implemented by a computer program, which may be stored in a computer-readable storage medium and can implement the steps of the embodiments of the methods described above when the computer program is executed by a processor. The computer program includes computer program code, and the computer program code may be in a source code form, an object code form, an executable file or some intermediate form.
Finally, it should be noted that: the above examples are only intended to illustrate the technical solution of the present invention and not to limit it; although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art will understand that: modifications to the specific embodiments of the invention or equivalent substitutions for parts of the technical features may be made; without departing from the spirit of the present invention, it is intended to cover all aspects of the invention as defined by the appended claims.
Claims (10)
1. An AI vision-based intelligent vehicle identification method is characterized by comprising the following steps:
step m1, scene classification
Acquiring a license plate-free vehicle image as an image to be recognized, classifying the image to be recognized based on a scene classification model, and acquiring a scene of the image to be recognized;
step m2, feature extraction
Detecting a target vehicle image from the image to be recognized based on a target detection algorithm, detecting and segmenting a vehicle face global image from the target vehicle image, and dividing the vehicle face global image according to a nine-square grid to obtain nine vehicle face sub-images;
sequentially extracting vehicle attribute features, vehicle face global features and vehicle face local features from the target vehicle image, the vehicle face global image and each vehicle face sub-image on the basis of the feature extraction model associated with the scene obtained in the step m 1;
step m3, first image search
Performing image search in a sample image library associated with the scene obtained in the step m1 based on the vehicle attribute features and the vehicle face global features to obtain a first search result;
step m4, regional recommendation
Selecting one or more recommended areas from nine car face sub-images of the image to be recognized based on the nine palace lattices and the attention map;
step m5, secondary image search
And carrying out image search in the first search result based on the local features of the car face in the recommended region, and obtaining a car identification result.
2. The AI vision based vehicle intelligent recognition method as in claim 1, wherein in step m4, the specific steps of selecting one or more recommended regions from nine vehicle face sub-images of the image to be recognized based on the Sudoku and the attention map are as follows:
processing the global image of the car face based on the attention map, and extracting data of the attention map layer as an attention feature matrix;
quantizing each value of the attention feature matrix, taking a median value of the attention feature matrix as a threshold, assigning all values larger than the threshold in the attention feature matrix as 1, and assigning all values smaller than the threshold in the attention feature matrix as 0 to obtain a quantized feature image M;
dividing the characteristic image M according to the Sudoku to obtain nine sub-characteristic images, and recording each sub-characteristic image as M i Wherein i =0,1, ·,8;
calculating the sum S of all values in each sub-feature image i ；
To S i Sorting according to the sequence from big to small, and selecting three S in the top i And the corresponding sub-feature image is used as a recommended area in a corresponding car face sub-image in the car face global image.
3. The AI vision-based intelligent vehicle identification method according to claim 2, wherein: and after the recommendation areas are obtained, judging whether the car face local features of each recommendation area are located in a priori significant feature library, if so, reserving the recommendation areas, and otherwise, omitting the recommendation areas.
4. The AI vision-based intelligent vehicle identification method according to claim 2, wherein: before or at the same time of selecting one or more recommended regions from nine car face sub-images of an image to be recognized based on a Sudoku and an attention map, judging whether the local car face features of each car face sub-image are located in a priori significant feature library, and if so, taking the car face sub-images as priori recommended regions;
after one or more recommended regions are selected from nine car face sub-images of the image to be identified based on the Sudoku and the attention map, whether each recommended region belongs to a priori recommended region or not is judged, if yes, the recommended region is reserved, and if not, the recommended region is omitted.
5. The AI-vision-based vehicle intelligent recognition method according to claim 1, wherein the specific steps of step m1 are as follows:
m1.1, acquiring a vehicle image in real time, judging whether the vehicle image has a license plate number, and if not, taking the vehicle image as an image to be identified;
m1.2, carrying out day and night classification on the image to be recognized by using a day and night classifier constructed by a Hue Saturation Value (HSV) color model and an openCV, and determining whether the image to be recognized is a day image or a night image;
m1.3, converting the image to be identified into a gray image, calculating the mean value and the variance of the gray image, carrying out exposure classification on the image to be identified based on the mean value and the variance, and determining that the image to be identified is a normal exposure image, an overexposure image or an underexposure image;
m1.4, determining the scene of the image to be identified as a day overexposure scene, a night overexposure scene, a day underexposure scene, a night underexposure scene, a day exposure normal scene or a night exposure normal scene according to the identification results of m1.2 and m 1.3.
6. The AI vision-based intelligent vehicle identification method according to claim 5, wherein in step m1.1, if it is determined that there is a license plate number in the vehicle image, then identifying the license plate number in the vehicle image, comparing the license plate number with the license plate number in the past vehicle record information reported by the data acquisition service, if the license plate numbers are consistent, continuing to execute steps m1.2, m1.3 and m1.4 to obtain the scene where the current vehicle image is located, obtaining the global vehicle face image, the vehicle attribute features, the global vehicle face features and the local vehicle face features of each vehicle face sub-image based on the feature extraction model associated with the scene where the current vehicle image is located, and storing the current vehicle image, the global vehicle face image, the vehicle attribute features, the global vehicle face features and the local vehicle face features into the sample image library of the corresponding scene; and if the license plate numbers are not consistent, storing the current vehicle image into the suspected wrong license plate sample library.
7. The AI vision-based intelligent vehicle identification method according to claim 1, wherein: and after the vehicle identification results are obtained, synchronously viewing each vehicle identification result and the image to be identified in a linkage manner.
8. The AI vision-based intelligent vehicle identification method according to claim 7, wherein the step of synchronously linkage viewing each vehicle identification result and the image to be identified comprises the following steps:
acquiring an image to be identified and a vehicle face area in a vehicle identification result to be compared;
respectively calculating the width, height and center coordinates of the vehicle face area in the image to be recognized and the vehicle recognition result to be compared based on the pixel coordinates, wherein the width, height and center coordinates of the image to be recognized are respectively as follows: w, h and (cx, cy); the width, height and center coordinates of the vehicle identification results to be compared are respectively as follows: w ', h' and (cx ', cy');
the method comprises the steps of scaling a vehicle face area in an image to be identified and a vehicle identification result to be compared to 600 pixels by 600 pixels, and obtaining vehicle face scaling ratios rw and rh of the image to be identified and vehicle face scaling ratios rw 'and rh' of the vehicle identification result to be compared;
obtaining coordinates (x, y) of a mouse pointer in an image to be identified, and calculating coordinates of a center point of a pre-zooming area of a vehicle identification result to be compared:
x’ = (cx’ + ((cx - x) * rw) / rw’)
y’ = (cy’ + ((cy - y) * rh) / rh’)；
when the zooming action of the mouse pointer in the image to be recognized is detected, based on the zooming ratios rw and rh of the image to be recognized, the coordinate (x ', y') of the central point of the pre-zooming area of the vehicle recognition result to be compared and the zooming ratios rw 'and rh', the vehicle recognition result to be compared is synchronously zoomed.
9. An AI vision-based intelligent recognition apparatus for a vehicle, the apparatus comprising:
the image acquisition module is used for acquiring the image of the vehicle without the license plate and taking the image of the vehicle without the license plate as an image to be identified;
the scene selection module is used for classifying the image to be recognized based on the scene classification model and acquiring the scene of the image to be recognized;
the characteristic extraction module is internally provided with a plurality of characteristic extraction models under different scenes; detecting a target vehicle image from the image to be recognized based on a target detection algorithm, detecting and segmenting a vehicle face global image from the target vehicle image, and dividing the vehicle face global image according to a nine-square grid to obtain nine vehicle face sub-images; sequentially extracting vehicle attribute features, vehicle face global features and vehicle face local features from a target vehicle image, a vehicle face global image and each vehicle face sub-image through a feature extraction model in a corresponding scene;
the primary image searching module is used for carrying out image searching in a sample image library of a corresponding scene based on vehicle attributes and global characteristics of a vehicle face to obtain a first searching result;
the area recommendation module is used for selecting one or more recommendation areas from nine vehicle face sub-images of the image to be recognized based on the nine-square and the attention map;
and the secondary image searching module is used for carrying out image searching in the first searching result based on the local characteristics of the vehicle face in the recommended region to obtain a vehicle identification result.
10. An electronic device, characterized in that the electronic device comprises:
a memory for storing executable instructions;
a processor for implementing the intelligent vehicle identification method of any one of claims 1 to 8 when executing the executable instructions stored in the memory.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211056249.5A CN115424217A (en) | 2022-08-31 | 2022-08-31 | AI vision-based intelligent vehicle identification method and device and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211056249.5A CN115424217A (en) | 2022-08-31 | 2022-08-31 | AI vision-based intelligent vehicle identification method and device and electronic equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115424217A true CN115424217A (en) | 2022-12-02 |
Family
ID=84201214
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211056249.5A Pending CN115424217A (en) | 2022-08-31 | 2022-08-31 | AI vision-based intelligent vehicle identification method and device and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115424217A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115835448A (en) * | 2022-12-28 | 2023-03-21 | 无锡车联天下信息技术有限公司 | Method and device for adjusting light, endoscope equipment and medium |
| CN116740661A (en) * | 2023-08-11 | 2023-09-12 | 科大国创软件股份有限公司 | Method for reversely tracking Mongolian vehicle based on face recognition |
-
2022
- 2022-08-31 CN CN202211056249.5A patent/CN115424217A/en active Pending
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115835448A (en) * | 2022-12-28 | 2023-03-21 | 无锡车联天下信息技术有限公司 | Method and device for adjusting light, endoscope equipment and medium |
| CN115835448B (en) * | 2022-12-28 | 2024-03-19 | 无锡车联天下信息技术有限公司 | Method and device for adjusting light, endoscope equipment and medium |
| CN116740661A (en) * | 2023-08-11 | 2023-09-12 | 科大国创软件股份有限公司 | Method for reversely tracking Mongolian vehicle based on face recognition |
| CN116740661B (en) * | 2023-08-11 | 2023-12-22 | 科大国创软件股份有限公司 | Method for reversely tracking Mongolian vehicle based on face recognition |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107729801B (en) | Vehicle color recognition system based on multitask deep convolution neural network | |
| CN105373794B (en) | A kind of licence plate recognition method | |
| CN109816024B (en) | Real-time vehicle logo detection method based on multi-scale feature fusion and DCNN | |
| Soilán et al. | Traffic sign detection in MLS acquired point clouds for geometric and image-based semantic inventory | |
| CN112233097B (en) | Road scene other vehicle detection system and method based on space-time domain multi-dimensional fusion | |
| De Charette et al. | Real time visual traffic lights recognition based on spot light detection and adaptive traffic lights templates | |
| CN109636795B (en) | Real-time non-tracking monitoring video remnant detection method | |
| US8620026B2 (en) | Video-based detection of multiple object types under varying poses | |
| CN111104903B (en) | Depth perception traffic scene multi-target detection method and system | |
| CN111582216A (en) | Unmanned vehicle-mounted traffic signal lamp identification system and method | |
| CN105608455B (en) | A kind of license plate sloped correcting method and device | |
| Romdhane et al. | An improved traffic signs recognition and tracking method for driver assistance system | |
| Janahiraman et al. | Traffic light detection using tensorflow object detection framework | |
| CN102867417B (en) | Taxi anti-forgery system and taxi anti-forgery method | |
| CN115424217A (en) | AI vision-based intelligent vehicle identification method and device and electronic equipment | |
| CN112488083B (en) | Identification method, device and medium of traffic signal lamp based on key point extraction of hetmap | |
| Naufal et al. | Preprocessed mask RCNN for parking space detection in smart parking systems | |
| Tung et al. | Large-scale object detection of images from network cameras in variable ambient lighting conditions | |
| Kale et al. | A road sign detection and the recognition for driver assistance systems | |
| CN111860509A (en) | Coarse-to-fine two-stage non-constrained license plate region accurate extraction method | |
| Oruklu et al. | Real-time traffic sign detection and recognition for in-car driver assistance systems | |
| Dousai et al. | Detecting humans in search and rescue operations based on ensemble learning | |
| Coronado et al. | Detection and classification of road signs for automatic inventory systems using computer vision | |
| Shi et al. | License plate localization in complex environments based on improved GrabCut algorithm | |
| Lashkov et al. | Edge-computing-facilitated nighttime vehicle detection investigations with CLAHE-enhanced images |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |