CN111639642A

CN111639642A - Image processing method, device and apparatus

Info

Publication number: CN111639642A
Application number: CN202010379274.1A
Authority: CN
Inventors: 方明超; 邵明
Original assignee: Zhejiang Dahua Technology Co Ltd
Current assignee: Zhejiang Dahua Technology Co Ltd
Priority date: 2020-05-07
Filing date: 2020-05-07
Publication date: 2020-09-08
Anticipated expiration: 2040-05-07
Also published as: CN111639642B

Abstract

The invention relates to an image processing method, equipment and a device, which are used for reducing the times of image correction, improving the image processing efficiency and improving the definition of an image after correction. The method comprises the following steps: determining a target area image containing a target object in the acquired image; determining the inclination angle of the target object in the image based on the recognized preset number of straight lines in the target area image; determining a transformation matrix of the image according to the inclination angle of the target object and the target area image; and performing correction processing on the image based on the transformation matrix of the image.

Description

Image processing method, device and apparatus

Technical Field

The present invention relates to the field of image processing, and in particular, to an image processing method, device and apparatus.

Background

With the opening of the mao bridge, hong kong and macau, more and more vehicles will pass between the continents, hong kong and macau. Because the license plate coding rules of the three places are different, two or even three license plates are usually hung on the vehicle, and a plurality of license plates are closely arranged up and down, so that the difficulty in recognizing the license plates is increased. In particular, due to factors such as the shooting angle of the camera and the inclination of the vehicle (such as the vehicle turning), the license plate in the vehicle picture captured by the camera can incline to different degrees, and the difficulty of license plate recognition is further increased. Therefore, it is necessary to correct a tilt image before recognizing the number plate of a vehicle having a plurality of number plates.

The existing image correction method corrects the image of a vehicle with only one plate, and needs to correct for the condition of hanging multiple plates for many times, so that the efficiency is low. In addition, the existing image correction method needs to detect the corner points of the license plates, however, when a plurality of license plates are hung on a vehicle, the corner points of the license plates are fuzzy, and the boundaries are not clear.

Disclosure of Invention

The invention provides an image processing method, equipment and a device, which are used for reducing the times of image correction, improving the image processing efficiency and improving the definition of an image after correction.

The technical scheme of the invention is as follows:

according to a first aspect of embodiments of the present invention, there is provided an image processing method, the method including:

determining a target area image containing a target object in the acquired image;

determining the inclination angle of the target object in the image based on the preset number of straight lines in the identified target area image;

determining a transformation matrix of the image according to the inclination angle of the target object and the target area image;

and performing correction processing on the image based on the transformation matrix of the image.

In the image processing method provided by the embodiment of the invention, the image containing the target object is corrected for one time by the transformation matrix determined based on the inclination angle of the target object in the image, so that the correction processing of the target image is completed, the calculation amount in the image correction process is reduced, the image processing time is shortened, the image processing efficiency is improved, and the image does not need to be subjected to processing processes such as corner detection and the like, so that the definition of the image is ensured, the image is directly corrected by using the transformation matrix, and compared with the prior art, the definition of the corrected image is improved.

In a possible embodiment, the image processing method provided by the present invention includes that the tilt angle of the target object in the image includes a first tilt angle formed by a long side of the target object and a preset direction and a second tilt angle formed by a short side of the target object and the preset direction;

determining the inclination angle of the target object in the image based on the preset number of straight lines in the identified target area image, wherein the method comprises the following steps:

determining the slope of each straight line according to the included angle between each straight line and the preset direction;

determining a first inclination angle based on the slope of a target straight line, wherein the target straight line is all straight lines of which included angles with a preset direction are smaller than a first angle in a preset number of straight lines;

and determining a second inclination angle based on the slopes of the other ones of the identified straight lines other than the target straight line.

In the image processing method provided by the embodiment of the invention, the first inclination angle and the second inclination angle of each edge of the target object and the preset direction are determined through the included angle between the preset number of straight lines in the target area image containing the target object and the preset direction, so that the inclination state of the target object in the image is accurately reflected.

In one possible embodiment, an image processing method according to the present invention is an image processing method for determining a transformation matrix of an image according to a tilt angle of a target object and a target area image, including:

determining a middle area based on the inclination angle of the target object and the vertex of the target area image;

and determining a transformation matrix of the image according to the coordinates of the vertex of the middle region in a preset coordinate system and the side length of the middle region.

According to the image processing method provided by the embodiment of the invention, the image is corrected through the transformation matrix of the image determined by the middle area, so that the definition and the correction effect of the corrected image are improved.

In one possible embodiment, the image processing method according to the present invention, which determines the intermediate region based on the tilt angle of the target object and the vertex of the target region image, includes:

when the angle interval to which the first inclination angle belongs is the same as the angle interval to which the second inclination angle belongs, generating a first target straight line which forms an included angle with the preset direction as the first inclination angle and a second target straight line which forms an included angle with the preset direction as the second inclination angle through each target vertex, wherein the target vertex is any one of two vertexes on the same diagonal line of the target area image;

and determining a parallelogram formed by the first target straight line and the second target straight line as a middle area.

In the image processing method provided by the embodiment of the invention, the angle interval to which the first inclination angle belongs and the angle interval to which the second inclination angle belongs are the same, so that the inclination condition of the target object in the image compared with the target object entity can be reflected, and the correspondingly determined middle area can reflect the inclination condition, so that the correction effect of the transformation matrix determined based on the middle area on the image correction is improved.

In a possible implementation manner, the image processing method provided by the present invention further includes:

and when the angle interval to which the first inclination angle belongs is different from the angle interval to which the second inclination angle belongs, determining a target external parallelogram of the target region image as a middle region, wherein the included angle between the long edge of the target external parallelogram and the preset direction is equal to the included angle between the short edge of the target external parallelogram of the first inclination angle and the preset direction is equal to the second inclination angle.

In the image processing method provided by the embodiment of the invention, the angle interval to which the first inclination angle belongs and the angle interval to which the second inclination angle belongs are different, so that another inclination condition of the target object in the image compared with the target object entity can be reflected, and the strategy for determining the middle area is also different due to the different inclination conditions, so that the correspondingly determined middle area can reflect the inclination condition, and the correction effect of the transformation matrix determined based on the middle area on the image correction is improved.

In one possible embodiment, the present invention provides an image processing method for determining a target area image including a target object in an image, including:

determining a target circumscribed rectangle containing a target object in the image by using a pre-trained object detection model, wherein the target circumscribed rectangle is a circumscribed rectangle containing the target object and having the smallest area;

performing gray processing on the image in the target external rectangle, performing expansion processing on the image in the target external rectangle after the gray processing, and performing corrosion processing on the image in the target external rectangle after the gray processing;

determining the outline containing the target area image based on the image in the target circumscribed rectangle after the expansion processing and the image in the target circumscribed rectangle after the erosion processing;

and generating a target area image by detecting the edge of the image in the outline.

In the image processing method provided by the embodiment of the invention, the process of determining the target area image has less calculation amount, and the processing time overhead is reduced.

In a possible implementation manner, the image processing method provided by the present invention, determining a preset number of straight lines in a target area image, includes:

determining a plurality of straight lines in the target area image by using a preset straight line detection method;

and selecting a preset number of straight lines from the plurality of straight lines according to the length of each straight line.

According to the image processing method provided by the embodiment of the invention, the preset number of straight lines are selected, the calculation amount for determining the inclination angle of the target object is reduced, and the processing speed is increased.

According to a second aspect of embodiments of the present invention, there is provided an image processing apparatus including:

a processor;

a memory for storing processor-executable instructions;

wherein the processor is configured to execute the instructions and perform the following steps:

determining the inclination angle of a target object in the image based on a preset number of straight lines in the identified target area image;

and carrying out correction processing on the image based on the transformation matrix of the image.

In one possible embodiment, the present invention provides an image processing apparatus, wherein the tilt angle of the target object in the image includes a first tilt angle formed by a long side of the target object and a preset direction and a second tilt angle formed by a short side of the target object and the preset direction;

the processor is specifically configured to:

In a possible implementation manner, the present invention provides an image processing apparatus, wherein the processor is specifically configured to:

In one possible implementation, the present invention provides an image processing apparatus, wherein the processor is further configured to:

According to a third aspect of embodiments of the present invention, there is provided an image processing apparatus including:

a target area image determination unit configured to determine a target area image containing a target object in the acquired image;

the inclination angle determining unit is used for determining the inclination angle of the target object in the image based on the preset number of straight lines in the identified target area image;

a transformation matrix determining unit for determining a transformation matrix of the image according to the inclination angle of the target object and the target area image;

and the processing unit is used for carrying out correction processing on the image based on the transformation matrix of the image.

a tilt angle determination unit, specifically configured to:

In a possible implementation manner, the image processing apparatus provided by the present invention is configured to, with the transform matrix determining unit:

In a possible implementation manner, the image processing apparatus provided by the present invention is configured to, where the target area image determining unit is specifically configured to:

In one possible embodiment, the image processing apparatus according to the present invention, the tilt angle determining unit is specifically configured to:

According to a fourth aspect of embodiments of the present invention, a storage medium, when instructions in the storage medium are executed by a processor of an image processing apparatus, enables the image processing apparatus to perform the image processing method of any one of the first aspects.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and together with the description, serve to explain the principles of the invention and are not to be construed as limiting the invention.

FIG. 1 is an image of a vehicle with two license plates hanging in a real application scene, according to an exemplary embodiment;

FIG. 2 is a schematic illustration of a process for determining a vehicle region image according to an exemplary embodiment;

FIG. 3 is a schematic flow chart diagram illustrating a method of image processing in accordance with an exemplary embodiment;

FIG. 4 is a schematic diagram illustrating a tilt angle of a target object according to an exemplary embodiment;

FIG. 5(a) is a schematic diagram illustrating a tilt of a target object in a target area image according to an exemplary embodiment;

FIG. 5(b) is a schematic diagram illustrating another tilt state of a target object in a target area image according to an exemplary embodiment;

FIG. 5(c) is a schematic diagram illustrating yet another tilt of a target object in a target area image according to an exemplary embodiment;

FIG. 5(d is a schematic diagram illustrating yet another tilt state of a target object in a target area image according to an exemplary embodiment;

FIG. 6(a) is a schematic diagram illustrating an intermediate region of a determined target region image, according to an exemplary embodiment;

FIG. 6(b) is a schematic diagram illustrating another intermediate region of the determined target region image, according to an exemplary embodiment;

FIG. 6(c) is a schematic diagram illustrating yet another intermediate region of a determined target region image, according to an exemplary embodiment;

FIG. 6(d) is a schematic diagram illustrating yet another intermediate region of a determined target region image, according to an exemplary embodiment;

FIG. 7 is a schematic diagram illustrating intermediate region vertex positions and corrected intermediate region vertex positions in accordance with an exemplary embodiment;

FIG. 8 illustrates pre-corrected and post-corrected images of an image according to an exemplary embodiment;

FIG. 9 is a schematic flow chart diagram illustrating another method of image processing in accordance with an exemplary embodiment;

FIG. 10 is a schematic flow chart diagram illustrating yet another method of image processing in accordance with an exemplary embodiment;

FIG. 11 is a schematic diagram illustrating a configuration of an image processing apparatus according to an exemplary embodiment;

FIG. 12 is a schematic diagram of a configuration of an image processing apparatus according to an exemplary embodiment;

fig. 13 is a schematic diagram illustrating a configuration of an image processing terminal according to an exemplary embodiment.

Detailed Description

In order to make the objects, technical solutions and advantages of the present invention clearer, the present invention will be described in further detail with reference to the accompanying drawings, and it is apparent that the described embodiments are only a part of the embodiments of the present invention, not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

The application scenario described in the embodiment of the present invention is for more clearly illustrating the technical solution of the embodiment of the present invention, and does not form a limitation on the technical solution provided in the embodiment of the present invention, and it can be known by a person skilled in the art that with the occurrence of a new application scenario, the technical solution provided in the embodiment of the present invention is also applicable to similar technical problems. In the description of the present invention, the term "plurality" means two or more unless otherwise specified.

More and more vehicles pass between regions with different license plate coding rules, for example, vehicles pass through the bridge of hong Zhu and Macau, Guangdong, hong Kong and Macau. Because the license plate coding rules of Guangdong Zhuhai, hong Kong and Macau are different, two or even three license plates need to be hung on the same vehicle, and a plurality of license plates are closely arranged from top to bottom, as shown in FIG. 1, one vehicle hung with two license plates needs to be explained, the image containing the license plates provided by the embodiment of the invention is only used for assisting understanding of the image processing method provided by the invention, and the license plate information in the image containing the license plates is shielded.

In the field of intelligent transportation, due to factors such as the shooting angle of a camera or the motion state (such as the turning state) of a vehicle, the shape of a license plate in a vehicle image collected by the camera is greatly changed, the license plate is inclined to a certain extent in the horizontal direction and the vertical direction, and the shape of the license plate in the image is not rectangular, so that the difficulty of license plate detection and license plate identification is increased. Therefore, the license plate is corrected before license plate detection and license plate recognition, and license plate detection accuracy and license plate recognition efficiency are improved.

In an actual application scene, the acquired image containing the vehicle is input into a vehicle detection neural network model, and a vehicle area image in the image is determined. The vehicle detection neural network model is a neural network model generated by training with a plurality of images and the position of a vehicle frame in each image as input and the position of the vehicle frame in the image as output. As shown in fig. 2, the captured image 201 containing the vehicle is input to the vehicle detection neural network model 202, and then a vehicle area image 203 in the image can be determined.

And inputting the vehicle region image 203 into a license plate detection neural network model, and determining a license plate region image in the vehicle region image. The license plate detection neural network model is a neural network model generated by training by taking a plurality of images and the position of a license plate frame in each image as input and taking the position of the license plate frame in the image as output. After the vehicle area image 203 shown in fig. 2 is input to the license plate detection neural network model 204, a license plate area 205 in the vehicle area image can be determined, and the license plate area 205 is rectangular in an actual application scene.

The license plate region 205 is subjected to grayscale processing. And respectively performing expansion processing and corrosion processing on the image subjected to the gray level processing. And subtracting the image after the erosion processing from the image after the expansion processing to obtain the outline of the image of the target area. Edge detection can be performed in the horizontal and vertical directions by the Kirsch edge detection method, respectively, to generate a target region image.

Fig. 3 illustrates an image processing method according to an exemplary embodiment, as shown in fig. 3, the image processing method including the steps of:

in step S301, a target area image including a target object is determined in the acquired image.

In specific implementation, the target area image including the image 201 of the vehicle may be obtained through the above process, it should be noted that the target object in the embodiment of the present invention is a vehicle license plate, which may be 1 license plate or multiple license plates, and when a vehicle has multiple license plates, the target area image includes all license plates of the vehicle.

Step S302, determining the inclination angle of the target object in the image based on the preset number of straight lines in the identified target area image.

In specific implementation, a method for detecting straight lines through Radon transformation is adopted, and a plurality of straight lines in the target area image are obtained. The line detection is performed in the horizontal direction and the vertical direction, respectively, and as shown in fig. 1, four lines 101 detected in the horizontal direction and 4 lines 102 detected in the vertical direction in the target area image are detected in the case of a vehicle with two license plates attached. In an actual application scene, a vehicle is hung with three license plates at most, and each license plate is provided with two frames in the horizontal and vertical directions. Therefore, when the straight line detection is performed in the horizontal direction and the vertical direction, the preset number may be set to 6. If the number of the straight lines is more than 6, only 6 straight lines with the longest length are reserved. To avoid interference in the background region of the image, the detected lines may be filtered again.

To facilitate description of the process of determining the tilt angle of the target object in the image according to the embodiment of the present invention, it is assumed that the slopes of N straight lines (tangent values of the included angle formed between the slope and the preset direction) detected in the horizontal direction by the method of detecting the straight lines through Radon transform are respectively recorded as k₁,k₂,…,k_NCalculating the average slope of N straight lines

And the standard deviation of the slopes of the N straight lines

When N straight lines are filtered respectively, the j-th straight line is judged

If so, deleting the straight line, keeping the M straight lines, and calculating the average slope of the M straight lines

The first inclination angle of the target object is determined. And detecting a straight line in the vertical direction by using a Radon transformation detection straight line method, and determining a second inclination angle of the target object by the process.

In the pre-constructed coordinate system, the first inclination angle of the target object is denoted as α, and the second inclination angle is denoted as β. As shown in fig. 4, a frame 401 of the target object entity parallel to the horizontal direction x, a frame 402 of the target object entity perpendicular to the horizontal direction x, the frame 401 in the target area image corresponds to a frame 403, and the frame 402 in the target area image corresponds to a frame 404.

And representing a first inclination angle of the target object, wherein an included angle alpha between a frame of the target object entity parallel to the horizontal direction and the horizontal direction in the target area image. And the second inclination angle of the target object represents an included angle beta between a frame of the target object entity vertical to the horizontal direction and the horizontal direction in the target image.

In step S303, the intermediate region is determined based on the tilt angle of the target object and the vertex of the target region image.

In practical application, fig. 5 shows the tilt states of four types of target objects in the target area image. Fig. 5(a) is a plate shape characteristic of 0 ≦ α <90 °, 0 ≦ β ≦ 90 °, fig. 5(b) is a plate shape characteristic of 0 ≦ α <90 °, 90 ° < β <180 °, fig. 5(c) is a plate shape characteristic of 90 ° < α <180 °, 0 ≦ β ≦ 90 °, fig. 5(d) is a plate shape characteristic of 90 ° < α <180 °, 90 ° < β <180 °.

Fig. 5(a) and 5(d) illustrate the shapes of the license plates corresponding to the same angle intervals to which the first and second tilt angles of the target object belong, and fig. 5(b) and 5(c) illustrate the shapes of the license plates corresponding to the different angle intervals to which the first and second tilt angles of the target object belong.

And selecting two vertexes of the target area image on the same diagonal when determining the middle area based on the inclination angle of the target object and the vertexes of the target area image aiming at the condition that the angle sections to which the first inclination angle and the second inclination angle of the target object belong are the same.

As shown in fig. 6(a), an upper left vertex a and a lower right vertex B are selected, two straight lines forming an included angle with the preset direction X as a first inclination angle and a second inclination angle are respectively generated through the two vertices, and a parallelogram formed by the generated four straight lines is a middle area.

As shown in fig. 6(D), the lower left corner vertex C and the upper right corner vertex D are selected, two straight lines forming an included angle with the preset direction X as the first inclination angle and the second inclination angle are respectively generated through the two vertices, and a parallelogram formed by the generated four straight lines is a middle area.

The target vertex can be determined according to the angle intervals to which the first inclination angle and the second inclination angle belong. For example, when the angle section to which the first inclination angle and the second inclination angle belong is [0, 90 °), the target vertices of the target region image are the upper left vertex and the lower right vertex of the target region image. When the angle section to which the first inclination angle and the second inclination angle belong is [90 DEG, 180 DEG ], the target vertexes of the target region image are the lower left vertex and the upper right vertex of the target region image. It should be noted that the intermediate area in the embodiment of the present invention includes the target area image.

And determining a target external parallelogram of the target area image as a middle area when determining the middle area based on the inclination angle of the target object and the vertex of the target area image aiming at the condition that the angle intervals to which the first inclination angle and the second inclination angle of the target object belong are different, wherein the included angle between the long edge of the target external parallelogram and the preset direction is equal to the included angle between the short edge of the target external parallelogram of the first inclination angle and the preset direction is equal to the second inclination angle.

As shown in fig. 6(b), a straight line having a first inclination angle with respect to the preset direction X is generated through the top left corner E and the bottom right corner F, and a straight line having a second inclination angle with respect to the preset direction X is generated through the bottom left corner G and the top right corner H, respectively, and the generated four straight lines may form a target parallelogram, where the target parallelogram is a middle region.

As shown in fig. 6(c), the lower left corner vertex I and the upper right corner vertex J are selected, a straight line having an included angle with the preset direction X as a first inclination angle is generated, a straight line having an included angle with the preset direction X as a second inclination angle is generated through the lower left corner vertex K and the upper right corner vertex K, and the generated four straight lines can be enclosed into a target parallelogram, which is a middle area.

And step S304, determining a transformation matrix of the image according to the coordinates of the vertex of the middle area in a preset coordinate system and the side length of the middle area.

In specific implementation, taking the middle area of fig. 6(a) as an example, fig. 7 shows that coordinates of four vertices X1, X2, X3, and X4 in the middle area can be determined in a preset coordinate system, coordinates of vertex X1 are (X1, y1), coordinates of vertex X2 are (X2, y2), coordinates of vertex X3 are (X3, y3), and coordinates of vertex X4 are (X4, y 4). And each side length of the middle region can be obtained by calculation through the coordinates of the four vertexes of the middle region, for example, the side length between the vertex X1 and the vertex X2

Side length between vertex X2 and vertex X3

And according to the coordinates of the four vertexes of the middle area and the side lengths of the middle area, the position coordinates of the four vertexes in the middle area after corresponding correction can be determined. The coordinates of the vertex X1 after correction are (X1, y1), the coordinates of the corresponding point N2 after correction of the vertex X2 are (X1+ w, y1), the coordinates of the corresponding point N3 after correction of the vertex X3 are (X1+ w, y1+ h), and the coordinates of the corresponding point N4 after correction of the vertex X4 are (X1, y1+ h).

And determining a transformation matrix of the image according to the coordinates of the four vertexes of the middle area and the coordinates after the four vertexes of the middle area are corrected. For example, the four vertex coordinates of the middle region constitute a matrix M, the coordinates of the middle region after the four vertex correction constitute a matrix P, and the affine transformation matrix Q of the image is M^-1P。

In step S305, correction processing is performed on the image based on the transformation matrix of the image.

In specific implementation, when the image is corrected, affine transformation may be performed on each pixel point in the vehicle region image 203 according to the affine transformation matrix, and fig. 8 shows an image before correction and an image after correction performed on the vehicle region image by the image processing method provided by the embodiment of the present invention.

Fig. 9 illustrates another image processing method according to an exemplary embodiment, as illustrated in fig. 9, the image processing method including the steps of:

in step S901, a target area image including a target object is determined in the acquired image.

In practical application scenarios, the target object may be a license plate, and the acquired image may be an image including the target object acquired in real time or a preprocessed image including the target object. The determined target area image also contains a target object, the target area image can be a license plate position prediction area, and the position of the target area image in the acquired image can be predicted through a pre-trained neural network model.

In one possible implementation, a pre-trained object detection model is used to determine a target bounding rectangle containing the target object in the image, where the target bounding rectangle is the bounding rectangle with the smallest area containing the target object.

And performing gray processing on the image in the target external rectangle, performing expansion processing on the image in the target external rectangle after the gray processing, and performing corrosion processing on the image in the target external rectangle after the gray processing.

Determining the outline containing the target area image based on the image in the target circumscribed rectangle after the expansion processing and the image in the target circumscribed rectangle after the erosion processing; and generating a target area image by detecting the edge of the image in the outline.

Step S902, determining a tilt angle of the target object in the image based on the preset number of straight lines in the identified target area image.

In specific implementation, in order to accurately determine the inclination angle of the target object in the image, the inclination angle of the target object in the image is determined by calculation by using the slopes of a preset number of straight lines in the target area image.

In one possible embodiment, the tilt angle of the target object in the image includes a first tilt angle formed by a long side of the target object and the preset direction and a second tilt angle formed by a short side of the target object and the preset direction. Determining the tilt angle of the target object in the image may employ the following steps:

In practical implementation, in an actual scene, when the target object is a license plate, the long side of the target object is a frame parallel to the horizontal direction when the target object is hung on a vehicle, and the short side of the target object is a frame perpendicular to the horizontal direction when the target object is hung on the vehicle.

The horizontal and vertical lines in the target area image may be detected by Radon transform. The angle formed by the recognized straight lines and the first preset direction is smaller than a first angle, the recognized straight lines can be judged as first straight lines detected in the horizontal direction of the target area image, and the straight lines except the first straight lines in the recognized straight lines can be judged as second straight lines detected in the vertical direction of the target area image. As a vehicle can hang 3 license plates at most, the number of the first straight lines and the number of the second straight lines are not more than 6 respectively, and the preset number of the straight lines is set according to an actual application scene.

And recording the average value of all the slopes of the first straight line as an average slope, wherein the value of the average slope is the tangent value of the first inclination angle, so that the first inclination angle of the target object can be determined according to the average slope of the first straight line. Similarly, a second inclination angle of the target object may be determined according to the average slope of the second line.

Step S903, determining a transformation matrix of the image according to the inclination angle of the target object and the target area image.

In specific implementation, the intermediate region is determined based on the inclination angle of the target object and the vertex of the target region image, and the transformation matrix of the image is determined according to the coordinate of the vertex of the intermediate region in the preset coordinate system and the side length of the intermediate region.

Determining the middle area based on the inclination angle of the target object and the vertex of the target area image can be divided into the following two cases:

the method comprises the steps that under the condition that an angle interval to which a first inclination angle belongs is the same as an angle interval to which a second inclination angle belongs, a first target straight line with an included angle with a preset direction being the first inclination angle and a second target straight line with the included angle with the preset direction being the second inclination angle are generated through each target vertex, and the target vertex is any one of two vertexes on the same diagonal line of a target area image;

In specific implementation, two license plate shapes shown in fig. 5(a) and 5(d) correspond to the situation that the angle intervals to which the first inclination angle and the second inclination angle of the target object belong are the same, the first inclination angle is represented as α, the second inclination angle is represented as β, the license plate shape in fig. 5(a) is characterized by 0 ≦ α <90 °, 0 ≦ β ≦ 90 °, and the license plate shape in fig. 5(d) is characterized by 90 ° < α <180 °, 90 ° < β <180 °.

For the shape characteristics of the license plate shown in fig. 5(a), as shown in fig. 6(a), an upper left corner vertex a and a lower right corner vertex B are selected, two straight lines forming an included angle with the preset direction X as a first inclination angle and a second inclination angle are respectively generated through the two vertices, and a parallelogram surrounded by the generated four straight lines is a middle area.

For the shape characteristics of the license plate shown in fig. 5(D), as shown in fig. 6(D), a lower left corner vertex C and an upper right corner vertex D are selected, two straight lines forming an included angle with the preset direction X as a first inclination angle and a second inclination angle are respectively generated through the two vertices, and a parallelogram formed by the four generated straight lines is a middle area.

And determining a target external parallelogram of the target region image as a middle region when the angle range to which the first inclination angle belongs is different from the angle range to which the second inclination angle belongs, wherein the included angle between the long side of the target external parallelogram and the preset direction is equal to the included angle between the short side of the target external parallelogram at the first inclination angle and the preset direction is equal to the second inclination angle.

In specific implementation, fig. 5(b) and 5(c) correspond to the case where the angle intervals to which the first inclination angle and the second inclination angle of the target object belong are different, where α <90 °, 90 ° < β <180 ° in fig. 5(b), and α <180 °, 0< β <90 ° in fig. 5 (c).

For the shape characteristics of the license plate shown in fig. 5(b), as shown in fig. 6(b), a straight line having an included angle with the preset direction X as a first inclination angle is generated through the top left corner vertex E and the bottom right corner vertex F, a straight line having an included angle with the preset direction X as a second inclination angle is generated through the bottom left corner vertex G and the top right corner vertex H, and the generated four straight lines can enclose a target parallelogram, which is a middle region.

For the shape characteristics of the license plate shown in fig. 5(c), as shown in fig. 6(c), a lower left corner vertex I and an upper right corner vertex J are selected, a straight line having an included angle with the preset direction X as a first inclination angle is generated, a straight line having an included angle with the preset direction X as a second inclination angle is generated through the lower left corner vertex K and the upper right corner vertex K, and the generated four straight lines can enclose a target parallelogram, which is a middle area.

Taking the middle area of fig. 6(a) as an example, fig. 7 shows that coordinates of four vertices X1, X2, X3, and X4 of the middle area can be determined in a preset coordinate system, coordinates of vertex X1 are (X1, y1), coordinates of vertex X2 are (X2, y2), coordinates of vertex X3 are (X3, y3), and coordinates of vertex X4 are (X4, y 4). And the side lengths of the middle area can be obtained by calculating the coordinates of the four vertexes of the middle area, for example:

side length between vertex X1 and vertex X2

Side length between vertex X2 and vertex X3

And according to the coordinates of the four vertexes of the middle area and the side lengths of the middle area, the position coordinates of the four vertexes in the middle area after corresponding correction can be determined. The coordinates of the vertex X1 after correction are (X1, y1), the coordinates of the corresponding point N2 after correction of the vertex X2 are (X1+ w, y1), the coordinates of the corresponding point N3 after correction of the vertex X3 are (X1+ w, y1+ h), and the coordinates of the corresponding point N4 after correction of the vertex X4 are (X1, y1+ h). From the coordinates of the four vertices of the middle area and the corrected coordinates of the four vertices of the middle area, it is possible to determineA transformation matrix of the image. For example, the four vertex coordinates of the middle region constitute a matrix M, the coordinates of the middle region after the four vertex correction constitute a matrix P, and the affine transformation matrix Q of the image is M^-1P。

In step S904, correction processing is performed on the image based on the transformation matrix of the image.

In specific implementation, when the image obtained in step S901 is corrected, affine transformation may be performed on each pixel point in the image according to the affine transformation matrix.

Fig. 10 illustrates yet another image processing method according to an exemplary embodiment, as illustrated in fig. 10, the image processing method including the steps of:

in step S1001, an image is acquired.

In particular, the vehicle image can be acquired by the image acquisition device.

Step S1002 detects the number of vehicles in the image, and extracts a vehicle region image of each vehicle.

In specific implementation, a first pre-trained object detection neural network (e.g., a YOLO object detection neural network) is used to locate a vehicle region, and the number of vehicles included in an image can be detected. And the processes of step S1003 to step S1008 are performed for the vehicle area image of each vehicle.

And step S1003, detecting a license plate area in the vehicle area image.

And during specific implementation, the pre-trained second target is used for detecting the neural network model, and the license plate area is positioned. The license plate area is the minimum circumscribed rectangle containing all license plates.

Step S1004, the determined target area image.

In specific implementation, the license plate area is expanded and grayed. And respectively carrying out expansion and corrosion treatment on the license plate region image subjected to the graying treatment. And subtracting the corroded image from the expanded image to obtain the outline of the license plate area. And performing horizontal direction and vertical direction edge detection by using a Kirsch edge detection operator to obtain a target area image.

In step S1005, straight lines in the target area image are recognized, and the inclination angle of each straight line is determined.

In specific implementation, the slope of the straight line in the horizontal direction and the slope of the straight line in the vertical direction can be determined by detecting the straight line through Radon transformation, and the first inclination angle of the license plate can be obtained by calculating the average slope of the straight line in the horizontal direction. And calculating the average slope of the straight line in the vertical direction to obtain a second inclination angle of the license plate.

In step S1006, the middle area is determined.

In specific implementation, a middle area (parallelogram) surrounding the target area image can be obtained based on the first inclination angle and the second inclination angle of the license plate and the vertex of the target area image.

Step S1007, the determined affine transformation matrix.

In specific implementation, fig. 7 shows that coordinates of four vertices X1, X2, X3, and X4 in the middle area can be determined in a preset coordinate system, coordinates of vertex X1 are denoted as (X1, y1), coordinates of vertex X2 are denoted as (X2, y2), coordinates of vertex X3 are denoted as (X3, y3), and coordinates of vertex X4 are denoted as (X4, y 4). And each side length of the middle region can be obtained by calculation through the coordinates of the four vertexes of the middle region, for example, the side length between the vertex X1 and the vertex X2

Side length between vertex X2 and vertex X3

The coordinates of the vertex X1 after correction are (X1, y1), the coordinates of the corresponding point N2 after correction of the vertex X2 are (X1+ w, y1), the coordinates of the corresponding point N3 after correction of the vertex X3 are (X1+ w, y1+ h), and the coordinates of the corresponding point N4 after correction of the vertex X4 are (X1, y1+ h). And determining a transformation matrix of the image according to the coordinates of the four vertexes of the middle area and the coordinates after the four vertexes of the middle area are corrected. For example, the four vertex coordinates of the middle region constitute a matrix M, the coordinates of the middle region after the four vertex correction constitute a matrix P, and the affine transformation matrix Q of the image is M^-1P。

In step S1008, the vehicle area image is corrected.

During specific implementation, affine transformation is carried out on each pixel point in the vehicle area image based on the determined affine transformation matrix of the vehicle area image, and correction processing is carried out.

Based on the same concept of the embodiment of the present disclosure described above, fig. 11 is an image processing apparatus illustrated according to an exemplary embodiment, which includes, as illustrated in fig. 11:

a target area image determination unit 1101 for determining a target area image containing a target object in the acquired image;

an inclination angle determining unit 1102, configured to determine an inclination angle of the target object in the image based on a preset number of straight lines in the identified target area image;

a transformation matrix determination unit 1103 configured to determine a transformation matrix of the image according to the inclination angle of the target object and the target area image;

and a processing unit 1104 for performing correction processing on the image based on the transformation matrix of the image.

the tilt angle determination unit 1102 is specifically configured to:

In a possible implementation manner, the image processing apparatus provided by the present invention, the transformation matrix determining unit 1103 is specifically configured to:

In a possible embodiment, the image processing apparatus of the present invention, the target area image determining unit 1101, is specifically configured to:

In one possible embodiment, the image processing apparatus provided by the present invention, the tilt angle determining unit 1102 is specifically configured to:

Fig. 12 is a schematic structural diagram of an image processing apparatus 1200 according to an exemplary embodiment, and as shown in fig. 12, the image processing apparatus 1200 shown in the embodiment of the present invention includes:

a processor 1210;

a memory 1220 for storing instructions executable by the processor 1210;

wherein processor 1210 is configured to execute instructions and perform the following steps:

the processor is specifically configured to:

In an exemplary embodiment, there is also provided a storage medium comprising instructions, such as a memory 1220 comprising instructions, executable by a processor 1210 of an image processing apparatus to perform the above-described method. Alternatively, the storage medium may be a non-transitory computer readable storage medium, for example, the non-transitory computer readable storage medium may be a ROM, a Random Access Memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like.

In addition, the image processing method and apparatus provided by the embodiments of the present invention described in conjunction with fig. 3, 9, and 10 can be implemented by an image processing terminal. Fig. 13 is a schematic diagram illustrating a structure of an image processing terminal according to an embodiment of the present invention.

The image processing terminal may include a processor 1301 and a memory 1302 storing computer program instructions.

In particular, the processor 1301 may include a Central Processing Unit (CPU), or an Application Specific Integrated Circuit (ASIC), or may be configured as one or more Integrated circuits implementing embodiments of the present invention.

Memory 1302 may include a mass storage memory for storing data or instructions. By way of example, and not limitation, memory 1302 may include a Hard Disk Drive (HDD), a floppy Disk Drive, flash memory, an optical Disk, a magneto-optical Disk, tape, or a Universal Serial Bus (USB) Drive or a combination of two or more of these. Memory 1302 may include removable or non-removable (or fixed) media, where appropriate. Memory 1302 may be internal or external to the data processing apparatus, where appropriate. In a particular embodiment, the memory 1302 is non-volatile solid-state memory. In a particular embodiment, memory 1302 includes Read Only Memory (ROM). Where appropriate, the ROM may be mask-programmed ROM, Programmable ROM (PROM), Erasable PROM (EPROM), Electrically Erasable PROM (EEPROM), electrically rewritable ROM (EAROM), or flash memory or a combination of two or more of these.

The processor 1301 realizes the image processing method in the above-described embodiments by reading and executing computer program instructions stored in the memory 1302.

In one example, the image processing terminal may further include a communication interface 1303 and a bus 1310. As shown in fig. 13, the processor 1301, the memory 1302, and the communication interface 1303 are connected to each other via a bus 1310 to complete communication therebetween.

The communication interface 1303 is mainly used to implement communication between modules, apparatuses, units and/or devices in the embodiment of the present invention.

The bus 1310 includes hardware, software, or both to couple the components of the image processing terminal to each other. By way of example, and not limitation, a bus may include an Accelerated Graphics Port (AGP) or other graphics bus, an Enhanced Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), a Hypertransport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an infiniband interconnect, a Low Pin Count (LPC) bus, a memory bus, a Micro Channel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a video electronics standards association local (VLB) bus, or other suitable bus or a combination of two or more of these. Bus 1310 may include one or more buses, where appropriate. Although specific buses have been described and shown in the embodiments of the invention, any suitable buses or interconnects are contemplated by the invention.

In addition, in combination with the image processing method in the above embodiments, the embodiments of the present invention may be implemented by providing a computer-readable storage medium. The computer readable storage medium having stored thereon computer program instructions; the computer program instructions, when executed by a processor, implement any of the image processing methods of the above embodiments.

As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.

The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to the application. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

While the preferred embodiments of the present application have been described, additional variations and modifications in those embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. Therefore, it is intended that the appended claims be interpreted as including preferred embodiments and all alterations and modifications as fall within the scope of the application.

It will be apparent to those skilled in the art that various changes and modifications may be made in the present application without departing from the spirit and scope of the application. Thus, if such modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is intended to include such modifications and variations as well.

Claims

1. An image processing method, characterized in that the method comprises:

determining the inclination angle of the target object in the image based on the recognized preset number of straight lines in the target area image;

2. The method according to claim 1, wherein the tilt angle of the target object in the image comprises a first tilt angle formed by a long side of the target object and a preset direction and a second tilt angle formed by a short side of the target object and the preset direction;

the determining the inclination angle of the target object in the image based on the identified preset number of straight lines in the target area image comprises:

determining the first inclination angle based on the slope of a target straight line, wherein the target straight line is all straight lines of which included angles with the preset direction are smaller than a first angle in the preset number of straight lines; and

determining the second inclination angle based on the slopes of the other ones of the identified straight lines except the target straight line.

3. The method of claim 2, wherein determining a transformation matrix for the image based on the tilt angle of the target object and the target area image comprises:

and determining a transformation matrix of the image according to the coordinates of the vertex of the middle area in a preset coordinate system and the side length of the middle area.

4. The method of claim 3, wherein determining an intermediate region based on the tilt angle of the target object and the vertex of the target region image comprises:

when the angle interval to which the first inclination angle belongs is the same as the angle interval to which the second inclination angle belongs, generating a first target straight line, the included angle of which with the preset direction is the first inclination angle, and a second target straight line, the included angle of which with the preset direction is the second inclination angle, through each target vertex, wherein the target vertex is any one of two vertexes on the same diagonal line of the target area image;

determining a parallelogram formed by the first target straight line and the second target straight line as the middle area.

5. The method of claim 4, further comprising:

the angle interval that first inclination belongs to with the angle interval that the second inclination belongs to is different, will the external parallelogram of target of the regional image of target confirms to do the middle zone, wherein the long limit of the external parallelogram of target with the contained angle of predetermineeing the direction equals first inclination the short side of the external parallelogram of target with the contained angle of predetermineeing the direction equals the second inclination.

6. The method of claim 1, wherein the determining a preset number of straight lines in the target area image comprises:

and selecting the preset number of straight lines from the straight lines according to the length of each straight line.

7. An image processing apparatus characterized by comprising:

a processor;

a memory for storing the processor-executable instructions;

8. The apparatus according to claim 7, wherein the tilt angle of the target object in the image includes a first tilt angle formed by a long side of the target object and a preset direction and a second tilt angle formed by a short side of the target object and the preset direction;

the processor is specifically configured to:

9. The device of claim 7, wherein the processor is specifically configured to:

10. An image processing apparatus, characterized in that the apparatus comprises:

the inclination angle determining unit is used for determining the inclination angle of the target object in the image based on the recognized preset number of straight lines in the target area image;

a transformation matrix determining unit, configured to determine a transformation matrix of the image according to the inclination angle of the target object and the target area image;