CN114897987A - Method, device, equipment and medium for determining vehicle ground projection - Google Patents
Method, device, equipment and medium for determining vehicle ground projection Download PDFInfo
- Publication number
- CN114897987A CN114897987A CN202210807361.1A CN202210807361A CN114897987A CN 114897987 A CN114897987 A CN 114897987A CN 202210807361 A CN202210807361 A CN 202210807361A CN 114897987 A CN114897987 A CN 114897987A
- Authority
- CN
- China
- Prior art keywords
- key point
- determining
- coordinate information
- ground
- physical space
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
- G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30244—Camera pose
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30248—Vehicle exterior or interior
- G06T2207/30252—Vehicle exterior; Vicinity of vehicle
Landscapes
- Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Length Measuring Devices By Optical Means (AREA)
Abstract
The invention discloses a method, a device, equipment and a medium for determining ground projection of a vehicle. Even if the target vehicle in the image is blocked or cut off, the key points of the vehicle body of the target vehicle can be obtained, and then the ground projection area of the target vehicle is finally obtained based on the key points of the vehicle body. The problem of among the prior art, when the wheel of vehicle is pasted the place and is lacked, can't the accurate projected vehicle ground that determines is solved.
Description
Technical Field
The present invention relates to the field of image processing technologies, and in particular, to a method, an apparatus, a device, and a medium for determining a vehicle ground projection.
Background
In the field of intelligent transportation, obtaining a vehicle ground projection area is a common technical means in tasks such as line pressing detection, parking detection and the like. In the prior art, the following methods are generally used for analyzing the ground projection area of the vehicle from the image or the video:
the first mode is as follows: and detecting the wheel sticking point of the vehicle through a key point detection algorithm, and acquiring the ground projection area of the vehicle by combining the generated pseudo-3D target frame.
The second mode is as follows: wheel sticking point information forming the target 3D frame, such as angular points of the 3D frame or attitude information and size of the target, is directly given through a monocular 3D detection algorithm, and a ground projection area is obtained accordingly.
The problem that the prior art exists is that the ground projection area is identified by detecting the wheel attachment points of the vehicle, when the vehicle in the image is shielded or cut off, the wheel attachment points of the vehicle are lost, and the three-dimensional information is difficult to restore when the wheel attachment points are lost, so that the ground projection of the vehicle cannot be accurately determined by adopting the conventional scheme.
Disclosure of Invention
The embodiment of the invention provides a method, a device, equipment and a medium for determining vehicle ground projection, which are used for solving the problem that the vehicle ground projection cannot be accurately determined when the wheel attachment points of a vehicle are missing in the prior art.
The embodiment of the invention provides a method for determining vehicle ground projection, which comprises the following steps:
obtaining each body key point of a target vehicle in an image, and selecting at least one group of key point pairs from each body key point; carrying out perspective transformation on the at least one group of key point pairs, and determining respective corresponding ground point pairs;
for each group of key point pairs in the at least one group of key point pairs, determining a spatial distance between different physical space points in a physical space point pair corresponding to the key point pair, and determining coordinate information of the physical space point pair according to coordinate information of a ground point pair corresponding to the key point pair, the spatial distance and optical center coordinate information of a camera which is calibrated in advance and collects the image in a ground plane coordinate system;
and determining the ground projection area of the target vehicle according to the coordinate information of the physical space point pair corresponding to each group of key point pairs.
Further, said selecting at least one set of keypoint pairs from said individual body keypoints comprises:
selecting at least one group of key point pairs belonging to a set from the key points of the vehicle body according to the pre-stored set formed by the key point pairs meeting the preset conditions; and the included angle between the connecting line of any key point pair meeting the preset condition and the ground is smaller than a set angle threshold value.
Further, the determining the spatial distance between different physical space points in the pair of physical space points corresponding to the key point pair includes:
and determining the vehicle type information of the target vehicle, and determining the spatial distance between different physical space points in the physical space point pair corresponding to the key point pair according to the vehicle type information.
Further, determining the coordinate information of the physical space point pair according to the coordinate information of the ground point pair corresponding to the key point pair, the spatial distance, and the optical center coordinate information of a camera which is calibrated in advance and collects the image in a ground plane coordinate system includes:
determining the corresponding relation between the optical center coordinate information and the coordinate information of different ground points in the ground point pair corresponding to the key point pair according to the optical center coordinate information of a camera which is calibrated in advance and acquires the image in a ground plane coordinate system and the connecting lines of the different ground points;
determining the corresponding relation between the coordinate information of different physical space points in the physical space point pair corresponding to the key point pair and the space distance between the different physical space points;
and determining the coordinate information of the physical space point pair according to the corresponding relation between the optical center coordinate information and the coordinate information of the different surface points, the corresponding relation between the coordinate information of the different physical space points and the spatial distance between the different physical space points.
Further, the determining the ground projection area of the target vehicle according to the coordinate information of the physical space point pair corresponding to each group of key point pairs includes:
determining the coordinate information of the physical projection point pairs corresponding to each group of the physical space point pairs according to the coordinate information of the physical space point pairs corresponding to each group of the key point pairs;
determining a minimum circumscribed rectangular area of each group of physical projection point pairs according to the coordinate information of each group of physical projection point pairs, and taking the minimum circumscribed rectangular area as a ground projection area of the target vehicle; or determining the locomotive orientation information of the target vehicle, and determining the coordinate information of the shielded ground projection points according to the locomotive orientation information, the prestored vehicle body attribute information of the target vehicle and the coordinate information of each group of physical projection point pairs; and determining the ground projection area of the target vehicle according to the coordinate information of each group of physical projection point pairs and the coordinate information of the shielded ground projection points.
Further, the determining the heading information of the target vehicle comprises:
acquiring a region image containing the target vehicle, inputting the region image into a pre-trained vehicle head orientation recognition model, and determining vehicle head orientation information of the target vehicle in the region image based on the vehicle head orientation recognition model; the vehicle head orientation recognition model is obtained by training based on a sample image marked with vehicle head orientation information.
Further, after determining the ground projection area of the target vehicle, the method further comprises:
and performing reverse homography transformation on the ground projection area of the target vehicle to obtain the ground projection area in the image, and performing violation detection on the target vehicle based on the ground projection area in the image.
In another aspect, an embodiment of the present invention provides an apparatus for determining a ground projection of a vehicle, where the apparatus includes:
the first determining module is used for acquiring each body key point of the target vehicle in the image and selecting at least one group of key point pairs from each body key point; carrying out perspective transformation on the at least one group of key point pairs, and determining respective corresponding ground point pairs;
a second determining module, configured to determine, for each key point pair of the at least one group of key point pairs, a spatial distance between different physical space points in a physical space point pair corresponding to the key point pair, and determine coordinate information of the physical space point pair according to coordinate information of a ground point pair corresponding to the key point pair, the spatial distance, and pre-calibrated optical center coordinate information of a camera acquiring the image in a ground plane coordinate system;
and the third determining module is used for determining the ground projection area of the target vehicle according to the coordinate information of the physical space point pairs corresponding to each group of key point pairs.
Further, the first determining module is specifically configured to select at least one group of key point pairs belonging to a set from the key point pairs of the vehicle body according to the set formed by pre-stored key point pairs meeting preset conditions; and the included angle between the connecting line of any key point pair meeting the preset condition and the ground is smaller than a set angle threshold value.
Further, the second determining module is specifically configured to determine vehicle type information of the target vehicle, and determine, according to the vehicle type information, a spatial distance between different physical space points in the physical space point pair corresponding to the key point pair.
Further, the second determining module is specifically configured to determine, according to connection lines between optical center coordinate information of a camera, which is calibrated in advance and acquires the image, in a ground plane coordinate system and different ground points in the ground point pair corresponding to the key point pair, correspondence between the optical center coordinate information and coordinate information of the different ground points; determining the corresponding relation between the coordinate information of different physical space points in the physical space point pair corresponding to the key point pair and the space distance between the different physical space points; and determining the coordinate information of the physical space point pair according to the corresponding relation between the optical center coordinate information and the coordinate information of the different surface points, the corresponding relation between the coordinate information of the different physical space points and the spatial distance between the different physical space points.
Further, the third determining module is specifically configured to determine, according to the coordinate information of the physical space point pair corresponding to each group of key point pairs, the coordinate information of the physical projection point pair corresponding to each group of physical space point pairs;
determining a minimum circumscribed rectangular area of each group of physical projection point pairs according to the coordinate information of each group of physical projection point pairs, and taking the minimum circumscribed rectangular area as a ground projection area of the target vehicle; or determining the locomotive orientation information of the target vehicle, and determining the coordinate information of the shielded ground projection points according to the locomotive orientation information, the prestored vehicle body attribute information of the target vehicle and the coordinate information of each group of physical projection point pairs; and determining the ground projection area of the target vehicle according to the coordinate information of each group of physical projection point pairs and the coordinate information of the shielded ground projection points.
Further, the third determining module is specifically configured to acquire an area image including the target vehicle, input the area image into a vehicle heading identification model trained in advance, and determine vehicle heading information of the target vehicle in the area image based on the vehicle heading identification model; the vehicle head orientation recognition model is obtained by training based on a sample image marked with vehicle head orientation information.
Further, the apparatus further comprises:
and the violation detection module is used for performing reverse homography transformation on the ground projection area of the target vehicle to obtain the ground projection area in the image, and performing violation detection on the target vehicle based on the ground projection area in the image.
On the other hand, the embodiment of the invention provides electronic equipment, which comprises a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory complete mutual communication through the communication bus;
a memory for storing a computer program;
a processor for implementing any of the above method steps when executing a program stored in the memory.
In another aspect, an embodiment of the present invention provides a computer-readable storage medium, in which a computer program is stored, and the computer program, when executed by a processor, implements the method steps of any one of the above.
The embodiment of the invention provides a method, a device, equipment and a medium for determining vehicle ground projection, wherein the method comprises the following steps: obtaining each body key point of a target vehicle in an image, and selecting at least one group of key point pairs from each body key point; carrying out perspective transformation on the at least one group of key point pairs, and determining respective corresponding ground point pairs; for each group of key point pairs in the at least one group of key point pairs, determining a spatial distance between different physical space points in a physical space point pair corresponding to the key point pair, and determining coordinate information of the physical space point pair according to coordinate information of a ground point pair corresponding to the key point pair, the spatial distance and optical center coordinate information of a camera which is calibrated in advance and collects the image in a ground plane coordinate system; and determining the ground projection area of the target vehicle according to the coordinate information of the physical space point pair corresponding to each group of key point pairs.
The technical scheme has the following advantages or beneficial effects:
in the embodiment of the invention, each body key point of a target vehicle is acquired in an image, at least one group of key point pairs is selected, a ground point pair is obtained through perspective transformation, the spatial distance between different physical space points in a physical space point pair corresponding to the key point pair is determined, the coordinate information of the physical space point pair is determined by combining the coordinate information of the ground point pair corresponding to the key point pair, the spatial distance and the optical center coordinate information of a camera in a ground plane coordinate system, and then the ground projection area of the target vehicle is determined according to the coordinate information of the physical space point pair. Even if the target vehicle in the image is blocked or cut off, although the wheel attachment point cannot be acquired, the vehicle body key point of the target vehicle can be acquired, and the ground projection area of the target vehicle can be finally obtained based on the vehicle body key point. The problem of among the prior art, when the wheel of vehicle is pasted the place and is lacked, can't the accurate projected vehicle ground that determines is solved.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings needed to be used in the description of the embodiments will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.
Fig. 1 is a schematic diagram of a wheel mounting position missing due to a shooting angle according to an embodiment of the present invention;
fig. 2 is a schematic diagram illustrating a wheel pasting position missing due to image edge truncation according to an embodiment of the present invention;
FIG. 3 is a schematic diagram of a process for determining a ground projection of a vehicle according to an embodiment of the present invention;
FIG. 4 is a schematic diagram of key points of a vehicle body according to an embodiment of the present invention;
FIG. 5 is a diagram illustrating ground projection points of a tail lamp on an estimated image according to an embodiment of the present invention;
FIG. 6 is a schematic diagram illustrating a ground projection area of a target vehicle according to an embodiment of the present invention;
FIG. 7 is a schematic diagram of another embodiment of the present invention for determining a ground projection area of a target vehicle;
FIG. 8 is a schematic structural diagram of an apparatus for determining a ground projection of a vehicle according to an embodiment of the present invention;
fig. 9 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.
Detailed Description
The present invention will be described in further detail with reference to the attached drawings, and it should be understood that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
In a traffic monitoring scene, in some cases, a wheel attachment point is missing due to a shooting angle or image edge truncation, so that the related art cannot determine a vehicle ground projection area. Fig. 1 is a schematic diagram of a wheel attachment point missing due to a shooting angle, and fig. 2 is a schematic diagram of a wheel attachment point missing due to image edge truncation. In order to solve the problem that the projection area of the vehicle ground cannot be determined under the condition that the wheel sticking points are missing in the related art, the embodiment of the invention provides a technical scheme for determining the projection of the vehicle ground.
Fig. 3 is a schematic diagram of a process for determining a ground projection of a vehicle according to an embodiment of the present invention, where the process includes the following steps:
s101: obtaining each body key point of a target vehicle in an image, and selecting at least one group of key point pairs from each body key point; and carrying out perspective transformation on the at least one group of key point pairs, and determining the ground point pairs corresponding to each key point pair.
S102: and determining the spatial distance between different physical space points in the physical space point pairs corresponding to the key point pairs aiming at each key point pair in the at least one group of key point pairs, and determining the coordinate information of the physical space point pairs according to the coordinate information of the ground point pairs corresponding to the key point pairs, the spatial distance and the optical center coordinate information of a camera which is calibrated in advance and collects the image under a ground plane coordinate system.
S103: and determining the ground projection area of the target vehicle according to the coordinate information of the physical space point pair corresponding to each group of key point pairs.
The method for determining the vehicle ground projection provided by the embodiment of the invention is applied to electronic equipment, and the electronic equipment can be equipment such as a PC (personal computer), a tablet computer, a server and the like, and can also be intelligent image acquisition equipment. If the electronic equipment is intelligent image acquisition equipment, after the intelligent image acquisition equipment acquires the image containing the target vehicle, determining the area of the target vehicle in the image through a target detection algorithm, acquiring each body key point of the target vehicle in the image, and performing the subsequent process of determining the vehicle ground projection. If the electronic equipment is a PC, a tablet computer, a server and the like, after the image acquisition equipment acquires the image containing the target vehicle, the image is sent to the electronic equipment, the electronic equipment determines the area of the target vehicle in the image through a target detection algorithm, acquires each vehicle body key point of the target vehicle in the image, and then determines the process of vehicle ground projection.
After the electronic equipment acquires the image, each body key point of the target vehicle in the image can be acquired through a key point detection algorithm. As shown in fig. 4, the vehicle body key points include a left tail light key point, a right tail light key point, a left headlamp key point, a right headlamp key point, a left back coaming key point, a right back coaming key point, a left front coaming key point, a right front coaming key point, a left back roof key point, a right back roof key point, a left front roof key point, and a right front roof key point. In addition, the key points of the vehicle body can also comprise a left front wheel grounding point, a left rear wheel grounding point and the like. If the grounding point of the left front wheel or the grounding point of the left rear wheel of the target vehicle is obtained, the coordinate information of the ground projection in the image of the grounding point of the left front wheel or the grounding point of the left rear wheel can be directly obtained and can be directly used for subsequent violation detection.
After each body key point of the target vehicle in the image is obtained, at least one group of key point pairs are selected from each body key point. Selecting at least one group of key point pairs belonging to a set from each key point of the vehicle body according to the set formed by each pre-stored key point pair meeting the preset condition; and the connecting line of any key point pair meeting the preset condition is parallel or approximately parallel to the ground. That is, the included angle between the connecting line of any key point pair meeting the preset condition and the ground is smaller than the set angle threshold, and the set angle threshold is, for example, a smaller angle value such as 3 degrees, 5 degrees, and the like.
The pre-stored key point pairs meeting the preset conditions are tail lamp key point pairs, headlamp key point pairs, back panel key point pairs, front panel key point pairs, rear roof key point pairs and front roof key point pairs. For example, the body key points of the target vehicle in the acquired image are a left tail lamp key point, a right tail lamp key point, a left back panel key point, a left rear roof key point and a right rear roof key point. And determining at least one group of key point pairs as a tail lamp key point pair and a roof key point pair according to a set formed by all the key point pairs which are stored in advance and meet preset conditions.
And then, carrying out perspective transformation on at least one group of key point pairs, and determining respective corresponding ground point pairs, namely respectively obtaining the ground point pairs corresponding to the tail lamp key point pairs and the roof key point pairs.
The intersection point of the camera optical center, the key point and the connection line of the physical space points corresponding to the key point and the ground is a ground point.
For each group of key point pairs in at least one group of key point pairs, firstly determining the spatial distance between different physical space points in the physical space point pairs corresponding to the key point pairs, then determining the coordinate information of the physical space point pairs according to the coordinate information of the ground point pairs corresponding to the key point pairs, the spatial distance and the optical center coordinate information of a camera which is calibrated in advance and used for collecting the image in a ground plane coordinate system, and further determining the ground projection area of the target vehicle according to the coordinate information of each group of physical space point pairs.
Wherein determining the spatial distance between different physical space points in the pair of physical space points corresponding to the key point pair comprises: and determining the vehicle type information of the target vehicle, and determining the spatial distance between different physical space points in the physical space point pair corresponding to the key point pair according to the vehicle type information.
When determining the model information of the target vehicle, the model information recognition model may be trained in advance, and corresponding model information is labeled on sample images in a training set adopted by the model information recognition model, where the model information is, for example, a car, a minibus, a pick-up truck, an SUV, and the like. In order to determine that the spatial distance between different physical space points in the physical space point pair corresponding to the key point pair is more accurate, when the model information of the vehicle in the sample image is labeled, the labeling granularity can be smaller, for example, the model information is maiteng of the popular series, karya of the toyota series, and attic of the honda series. After the training of the vehicle type information recognition model is completed, the image area to which the target vehicle belongs can be input into the vehicle type information recognition model, and the vehicle type information of the target vehicle is determined based on the vehicle type information recognition model.
When the spatial distance between different physical space points in the physical space point pair corresponding to the key point pair is determined according to the vehicle type information, the spatial distance may be determined by a vehicle type information recognition model, for example, when the vehicle type information recognition model is trained, the output result is the vehicle type information of the vehicle and the spatial distance between different physical space points in the physical space point pair corresponding to each group of key point pairs, and the spatial distance between different physical space points in the physical space point pair corresponding to each group of key point pairs is determined directly based on the vehicle type information recognition model. Alternatively, the electronic device may store a database, where the database stores vehicle type information and spatial distances between different physical spatial points in the pair of physical spatial points corresponding to each group of key point pairs of the vehicle type. After the model information of the target vehicle is determined based on the model information recognition model, the database is searched, and the spatial distance between different physical space points in the physical space point pair corresponding to the key point pair corresponding to the model information is determined.
After the spatial distance between different physical space points in a physical space point pair corresponding to a key point pair is determined, the coordinate information of the physical space point pair is determined according to the coordinate information of a ground point pair corresponding to the key point pair, the spatial distance and the optical center coordinate information of a camera which is calibrated in advance and used for collecting the image in a ground plane coordinate system.
Specifically, according to connection lines between optical center coordinate information of a camera which collects the image and different ground points in a ground point pair corresponding to the key point pair respectively in a ground plane coordinate system which is calibrated in advance, determining the corresponding relation between the optical center coordinate information and the coordinate information of the different ground points respectively; determining the corresponding relation between the coordinate information of different physical space points in the physical space point pair corresponding to the key point pair and the space distance between the different physical space points; and determining the coordinate information of the physical space point pair according to the corresponding relation between the optical center coordinate information and the coordinate information of the different geographical points, the corresponding relation between the coordinate information of the different physical space points and the spatial distance between the different physical space points.
The following description will be given taking an example of estimating a tail light ground projected point on an image when there is a tail light key point pair.
As shown in fig. 5, first, coordinates l, r of a tail light key point pair on an image are obtained. And obtaining ground points L 'and R' of the tail lamp key point pair after perspective transformation through perspective transformation, wherein the obtained ground points after perspective transformation deviate from the actual ground projection points L 'and R' of the tail lamp due to the loss of height information. Taking a car as an example, the vehicle width is usually about 2m, and it is assumed that the spatial distance between different physical spatial points in the pair of physical spatial points corresponding to the tail lamp key point pair is 2 m. Fig. 5 shows an optical center o, a left rear tail lamp key point L on an image, a right rear tail lamp key point R on the image, a ground projection point L 'on the image corresponding to the key point L, a ground projection point R' on the image corresponding to the key point R, a physical space point L corresponding to the key point L, a physical space point R corresponding to the key point R, a physical ground projection point L 'corresponding to the physical space point L, a physical ground projection point R' corresponding to the physical space point R, a perspective-transformed ground point L ″ corresponding to the key point L, and a perspective-transformed ground point R ″ corresponding to the key point R.
Obtaining the external reference of the camera relative to the world coordinate system through calibration, obtaining the coordinate of the optical center on the world coordinate system, and obtaining a straight line l passing through key points on the image 1 ，l 2 Expressions in the world coordinate system can also be obtained. Since the tail light line is approximately parallel to the ground, the L, R coordinates are equal in the Z direction, thus according to L ″ on the straight line L 1 Above, R' is on a straight line R 2 In the above, and the euclidean distance between L and R is the assumed vehicle body width, 5 equations can be constructed, and the unknowns are L, the x, y components of R and the equal z component, and are also 5, so the coordinates of L and R are calculable.
After obtaining the coordinates of L and R, the coordinates of L 'and R' can be obtained. The coordinates of L and R are the same as the xy directions of the coordinates of L 'and R', and the z-direction coordinate of L 'and R' is 0. And obtaining a complete ground projection by combining other key points to the projection coordinates or the orientation information plus the set vehicle length. The coordinates L ', R' of the ground projection point on the image can be obtained by inverse homography transformation L ', R'. A complete ground projection is also obtained according to this process. The orientation information can be directly given by the model or obtained by the spatial relationship of key points.
The perspective transformation is homographic transformation from an image plane to a ground plane, and can be completed through a homographic transformation matrix H obtained by calibration:
the linear equation l 1 ，l 2 After the coordinates that can be obtained by the known camera optical center coordinates, L ", and R":
the Euclidean distance between L and R is equal to the assumed vehicle body width d, and can be expressed as:
the five unknowns are respectively L x ，R x ，L y ，R y ，L z ；
Order:
solving the equation to obtain:
after the coordinate information of each group of physical space point pairs is determined, the ground projection area of the target vehicle is determined according to the coordinate information of each group of physical space point pairs.
Specifically, determining the ground projection area of the target vehicle according to the coordinate information of the physical space point pair corresponding to each group of key point pairs includes:
determining the coordinate information of the physical projection point pairs corresponding to each group of the physical space point pairs according to the coordinate information of the physical space point pairs corresponding to each group of the key point pairs;
determining a minimum circumscribed rectangular area of each group of physical projection point pairs according to the coordinate information of each group of physical projection point pairs, and taking the minimum circumscribed rectangular area as a ground projection area of the target vehicle; or determining the locomotive orientation information of the target vehicle, and determining the coordinate information of the shielded ground projection points according to the locomotive orientation information, the prestored vehicle body attribute information of the target vehicle and the coordinate information of each group of physical projection point pairs; and determining the ground projection area of the target vehicle according to the coordinate information of each group of physical projection point pairs and the coordinate information of the shielded ground projection points.
The manner of determining the ground projection area of the target vehicle may be:
first, as shown in fig. 6, after coordinate information of each set of physical projection point pairs is determined, a minimum bounding rectangle region of each set of physical projection point pairs is determined, and then the minimum bounding rectangle is used as a ground projection region of the target vehicle.
Secondly, as shown in fig. 7, coordinate information of the shielded ground projection point is determined according to the heading information of the vehicle head, pre-stored vehicle body attribute information of the target vehicle, and coordinate information of each group of physical projection point pairs, and then an area surrounded by the coordinate information of each group of physical projection point pairs and the coordinate information of the shielded ground projection point is determined as a ground projection area of the target vehicle.
When the locomotive orientation recognition model is trained, the locomotive orientation information of the vehicle in the sample image is marked aiming at the sample image in the training set. When the vehicle head orientation information is marked, the vehicle head orientation information can be marked according to a connecting line of the wheel sticking points of the vehicle in the sample image. After a vehicle head orientation recognition model is obtained based on sample images in a training set through training, an area image containing a target vehicle is obtained, the area image is input into the vehicle head orientation recognition model which is trained in advance, and vehicle head orientation information of the target vehicle in the area image is determined based on the vehicle head orientation recognition model.
In order to facilitate the electronic device to perform violation detection on the target vehicle, in the embodiment of the invention, after the ground projection area of the target vehicle is determined, the ground projection area of the target vehicle is subjected to reverse homography transformation to obtain the ground projection area in the image, and the violation detection on the target vehicle is performed based on the ground projection area in the image.
The inverse homography transform, which is the inverse operation of the perspective transform from ground plane back to image plane, is the inverse of the H matrix of the perspective transform.
Fig. 8 is a schematic structural diagram of an apparatus for determining a ground projection of a vehicle according to an embodiment of the present invention, where the apparatus includes:
the first determining module 81 is configured to obtain each body key point of the target vehicle in the image, and select at least one group of key point pairs from the body key points; carrying out perspective transformation on the at least one group of key point pairs, and determining respective corresponding ground point pairs;
a second determining module 82, configured to determine, for each key point pair of the at least one group of key point pairs, a spatial distance between different physical space points in a physical space point pair corresponding to the key point pair, and determine coordinate information of the physical space point pair according to coordinate information of a ground point pair corresponding to the key point pair, the spatial distance, and optical center coordinate information of a camera, which is calibrated in advance, that collects the image in a ground plane coordinate system;
and a third determining module 83, configured to determine a ground projection area of the target vehicle according to the coordinate information of the physical space point pair corresponding to each group of key point pairs.
The first determining module 81 is specifically configured to select, according to a set formed by pre-stored key point pairs meeting preset conditions, at least one group of key point pairs belonging to the set from the key point pairs of the vehicle body; and the included angle between the connecting line of any key point pair meeting the preset condition and the ground is smaller than a set angle threshold value.
The second determining module 82 is specifically configured to determine vehicle type information of the target vehicle, and determine a spatial distance between different physical space points in a physical space point pair corresponding to the key point pair according to the vehicle type information.
The second determining module 82 is specifically configured to determine, according to connection lines between optical center coordinate information of a camera, which is calibrated in advance and acquires the image, in a ground plane coordinate system and different ground points in the ground point pair corresponding to the key point pair, correspondence between the optical center coordinate information and coordinate information of the different ground points; determining the corresponding relation between the coordinate information of different physical space points in the physical space point pair corresponding to the key point pair and the space distance between the different physical space points; and determining the coordinate information of the physical space point pair according to the corresponding relation between the optical center coordinate information and the coordinate information of the different surface points, the corresponding relation between the coordinate information of the different physical space points and the spatial distance between the different physical space points.
The third determining module 83 is specifically configured to determine, according to the coordinate information of the physical space point pair corresponding to each group of key point pairs, the coordinate information of the physical projection point pair corresponding to each group of physical space point pairs;
determining a minimum circumscribed rectangular area of each group of physical projection point pairs according to the coordinate information of each group of physical projection point pairs, and taking the minimum circumscribed rectangular area as a ground projection area of the target vehicle; or determining the locomotive orientation information of the target vehicle, and determining the coordinate information of the shielded ground projection points according to the locomotive orientation information, the prestored vehicle body attribute information of the target vehicle and the coordinate information of each group of physical projection point pairs; and determining the ground projection area of the target vehicle according to the coordinate information of each group of physical projection point pairs and the coordinate information of the shielded ground projection points.
The third determining module 83 is specifically configured to obtain an area image including the target vehicle, input the area image into a vehicle heading identification model which is trained in advance, and determine vehicle heading information of the target vehicle in the area image based on the vehicle heading identification model; the vehicle head orientation recognition model is obtained by training based on a sample image marked with vehicle head orientation information.
The device further comprises:
and the violation detection module 84 is configured to perform inverse homography transformation on the ground projection area of the target vehicle to obtain a ground projection area in the image, and perform violation detection on the target vehicle based on the ground projection area in the image.
An embodiment of the present invention further provides an electronic device, as shown in fig. 9, including: the system comprises a processor 301, a communication interface 302, a memory 303 and a communication bus 304, wherein the processor 301, the communication interface 302 and the memory 303 complete mutual communication through the communication bus 304;
the memory 303 has stored therein a computer program which, when executed by the processor 301, causes the processor 301 to perform the steps of:
obtaining each body key point of a target vehicle in an image, and selecting at least one group of key point pairs from each body key point; carrying out perspective transformation on the at least one group of key point pairs, and determining respective corresponding ground point pairs;
for each group of key point pairs in the at least one group of key point pairs, determining a spatial distance between different physical space points in a physical space point pair corresponding to the key point pair, and determining coordinate information of the physical space point pair according to coordinate information of a ground point pair corresponding to the key point pair, the spatial distance and optical center coordinate information of a camera which is calibrated in advance and collects the image in a ground plane coordinate system;
and determining the ground projection area of the target vehicle according to the coordinate information of the physical space point pair corresponding to each group of key point pairs.
Based on the same inventive concept, the embodiment of the invention also provides an electronic device, and as the principle of solving the problems of the electronic device is similar to the method for determining the vehicle ground projection, the implementation of the electronic device can refer to the implementation of the method, and repeated details are omitted.
The electronic device provided by the embodiment of the invention can be a desktop computer, a portable computer, a smart phone, a tablet computer, a Personal Digital Assistant (PDA), a network side device and the like.
The communication bus mentioned in the electronic device may be a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, or the like. The communication bus may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is shown, but this does not mean that there is only one bus or one type of bus.
The communication interface 302 is used for communication between the above-described electronic apparatus and other apparatuses.
The Memory may include a Random Access Memory (RAM) or a Non-Volatile Memory (NVM), such as at least one disk Memory. Alternatively, the memory may be at least one memory device located remotely from the processor.
The Processor may be a general-purpose Processor, including a central processing unit, a Network Processor (NP), and the like; but may also be a Digital Signal Processor (DSP), an application specific integrated circuit, a field programmable gate array or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or the like.
An embodiment of the present invention further provides a computer storage readable storage medium, in which a computer program executable by an electronic device is stored, and when the program runs on the electronic device, the electronic device is caused to execute the following steps:
obtaining each body key point of a target vehicle in an image, and selecting at least one group of key point pairs from each body key point; carrying out perspective transformation on the at least one group of key point pairs, and determining respective corresponding ground point pairs;
for each group of key point pairs in the at least one group of key point pairs, determining a spatial distance between different physical space points in a physical space point pair corresponding to the key point pair, and determining coordinate information of the physical space point pair according to coordinate information of a ground point pair corresponding to the key point pair, the spatial distance and optical center coordinate information of a camera which is calibrated in advance and collects the image in a ground plane coordinate system;
and determining the ground projection area of the target vehicle according to the coordinate information of the physical space point pair corresponding to each group of key point pairs.
Based on the same inventive concept, embodiments of the present invention further provide a computer-readable storage medium, and since the principle of solving the problem when the processor executes the computer program stored on the computer-readable storage medium is similar to the method for determining the vehicle ground projection, the implementation of the computer program stored on the computer-readable storage medium by the processor may refer to the implementation of the method, and repeated details are not repeated.
The computer readable storage medium may be any available medium or data storage device that can be accessed by a processor in an electronic device, including but not limited to magnetic memory such as floppy disks, hard disks, magnetic tape, magneto-optical disks (MOs), etc., optical memory such as CDs, DVDs, BDs, HVDs, etc., and semiconductor memory such as ROMs, EPROMs, EEPROMs, non-volatile memory (NAND FLASH), Solid State Disks (SSDs), etc.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
While preferred embodiments of the present invention have been described, additional variations and modifications in those embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. Therefore, it is intended that the appended claims be interpreted as including preferred embodiments and all such alterations and modifications as fall within the scope of the invention.
It will be apparent to those skilled in the art that various changes and modifications may be made in the present invention without departing from the spirit and scope of the invention. Thus, if such modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include such modifications and variations.
Claims (10)
1. A method of determining a vehicle ground projection, the method comprising:
obtaining each body key point of a target vehicle in an image, and selecting at least one group of key point pairs from each body key point; carrying out perspective transformation on the at least one group of key point pairs, and determining respective corresponding ground point pairs;
for each group of key point pairs in the at least one group of key point pairs, determining a spatial distance between different physical space points in a physical space point pair corresponding to the key point pair, and determining coordinate information of the physical space point pair according to coordinate information of a ground point pair corresponding to the key point pair, the spatial distance and optical center coordinate information of a camera which is calibrated in advance and collects the image in a ground plane coordinate system;
and determining the ground projection area of the target vehicle according to the coordinate information of the physical space point pair corresponding to each group of key point pairs.
2. The method of claim 1, wherein said selecting at least one set of keypoint pairs from said respective body keypoints comprises:
selecting at least one group of key point pairs belonging to a set from the key points of each vehicle body according to the set formed by all the key point pairs which are preserved in advance and meet preset conditions; and the included angle between the connecting line of any key point pair meeting the preset condition and the ground is smaller than a set angle threshold value.
3. The method of claim 1, wherein said determining the spatial distance between different ones of the pair of physical spatial points to which the key point pair corresponds comprises:
and determining the vehicle type information of the target vehicle, and determining the spatial distance between different physical space points in the physical space point pair corresponding to the key point pair according to the vehicle type information.
4. The method of claim 1, wherein the determining the coordinate information of the physical space point pair according to the coordinate information of the ground point pair corresponding to the key point pair, the spatial distance, and the optical center coordinate information of a camera which is calibrated in advance and collects the image in a ground plane coordinate system comprises:
determining the corresponding relation between the optical center coordinate information and the coordinate information of different ground points in the ground point pair corresponding to the key point pair according to the optical center coordinate information of a camera which is calibrated in advance and acquires the image in a ground plane coordinate system and the connecting lines of the different ground points;
determining the corresponding relation between the coordinate information of different physical space points in the physical space point pair corresponding to the key point pair and the space distance between the different physical space points;
and determining the coordinate information of the physical space point pair according to the corresponding relation between the optical center coordinate information and the coordinate information of the different surface points, the corresponding relation between the coordinate information of the different physical space points and the spatial distance between the different physical space points.
5. The method of claim 1, wherein determining the ground projection area of the target vehicle based on the coordinate information of the physical-space point pairs to which the respective sets of key point pairs correspond comprises:
determining the coordinate information of the physical projection point pairs corresponding to each group of the physical space point pairs according to the coordinate information of the physical space point pairs corresponding to each group of the key point pairs;
determining a minimum circumscribed rectangular area of each group of physical projection point pairs according to the coordinate information of each group of physical projection point pairs, and taking the minimum circumscribed rectangular area as a ground projection area of the target vehicle; or determining the locomotive orientation information of the target vehicle, and determining the coordinate information of the shielded ground projection points according to the locomotive orientation information, the prestored vehicle body attribute information of the target vehicle and the coordinate information of each group of physical projection point pairs; and determining the ground projection area of the target vehicle according to the coordinate information of each group of physical projection point pairs and the coordinate information of the shielded ground projection points.
6. The method of claim 5, wherein the determining the heading information of the target vehicle comprises:
acquiring a region image containing the target vehicle, inputting the region image into a pre-trained vehicle head orientation recognition model, and determining vehicle head orientation information of the target vehicle in the region image based on the vehicle head orientation recognition model; the vehicle head orientation recognition model is obtained by training based on a sample image marked with vehicle head orientation information.
7. The method of claim 1 or 5, wherein after determining the ground projection area of the target vehicle, the method further comprises:
and performing reverse homography transformation on the ground projection area of the target vehicle to obtain the ground projection area in the image, and performing violation detection on the target vehicle based on the ground projection area in the image.
8. An apparatus for determining a ground projection of a vehicle, the apparatus comprising:
the first determining module is used for acquiring each vehicle body key point of a target vehicle in an image and selecting at least one group of key point pairs from each vehicle body key point; carrying out perspective transformation on the at least one group of key point pairs, and determining respective corresponding ground point pairs;
a second determining module, configured to determine, for each key point pair of the at least one group of key point pairs, a spatial distance between different physical space points in a physical space point pair corresponding to the key point pair, and determine coordinate information of the physical space point pair according to coordinate information of a ground point pair corresponding to the key point pair, the spatial distance, and pre-calibrated optical center coordinate information of a camera acquiring the image in a ground plane coordinate system;
and the third determining module is used for determining the ground projection area of the target vehicle according to the coordinate information of the physical space point pairs corresponding to each group of key point pairs.
9. An electronic device is characterized by comprising a processor, a communication interface, a memory and a communication bus, wherein the processor and the communication interface are used for realizing mutual communication by the memory through the communication bus;
a memory for storing a computer program;
a processor for implementing the method steps of any one of claims 1 to 7 when executing a program stored in the memory.
10. A computer-readable storage medium, characterized in that a computer program is stored in the computer-readable storage medium, which computer program, when being executed by a processor, carries out the method steps of any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210807361.1A CN114897987B (en) | 2022-07-11 | 2022-07-11 | Method, device, equipment and medium for determining vehicle ground projection |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210807361.1A CN114897987B (en) | 2022-07-11 | 2022-07-11 | Method, device, equipment and medium for determining vehicle ground projection |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114897987A true CN114897987A (en) | 2022-08-12 |
| CN114897987B CN114897987B (en) | 2022-10-28 |
Family
ID=82729321
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210807361.1A Active CN114897987B (en) | 2022-07-11 | 2022-07-11 | Method, device, equipment and medium for determining vehicle ground projection |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114897987B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116597012A (en) * | 2023-06-12 | 2023-08-15 | 深圳海星智驾科技有限公司 | Vehicle pose recognition method and device, computer equipment and storage medium |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104376552A (en) * | 2014-09-19 | 2015-02-25 | 四川大学 | Virtual-real registering algorithm of 3D model and two-dimensional image |
| CN105096317A (en) * | 2015-07-03 | 2015-11-25 | 吴晓军 | Fully automatic calibration method for high performance camera under complicated background |
| US20150378361A1 (en) * | 2014-06-30 | 2015-12-31 | Collin Walker | Systems and methods for controlling vehicle position and orientation |
| CN106991703A (en) * | 2017-03-23 | 2017-07-28 | 长沙冰眼电子科技有限公司 | Vehicle monitoring camera-shooting area obtaining method and system |
| CN107437264A (en) * | 2017-08-29 | 2017-12-05 | 重庆邮电大学 | In-vehicle camera external parameter automatic detection and bearing calibration |
| CN110285793A (en) * | 2019-07-08 | 2019-09-27 | 中原工学院 | A kind of Vehicular intelligent survey track approach based on Binocular Stereo Vision System |
-
2022
- 2022-07-11 CN CN202210807361.1A patent/CN114897987B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20150378361A1 (en) * | 2014-06-30 | 2015-12-31 | Collin Walker | Systems and methods for controlling vehicle position and orientation |
| CN104376552A (en) * | 2014-09-19 | 2015-02-25 | 四川大学 | Virtual-real registering algorithm of 3D model and two-dimensional image |
| CN105096317A (en) * | 2015-07-03 | 2015-11-25 | 吴晓军 | Fully automatic calibration method for high performance camera under complicated background |
| CN106991703A (en) * | 2017-03-23 | 2017-07-28 | 长沙冰眼电子科技有限公司 | Vehicle monitoring camera-shooting area obtaining method and system |
| CN107437264A (en) * | 2017-08-29 | 2017-12-05 | 重庆邮电大学 | In-vehicle camera external parameter automatic detection and bearing calibration |
| CN110285793A (en) * | 2019-07-08 | 2019-09-27 | 中原工学院 | A kind of Vehicular intelligent survey track approach based on Binocular Stereo Vision System |
Non-Patent Citations (1)
| Title |
|---|
| 徐晓娟等: "基于单目序列图像的车辆三维信息的获取", 《电子设计工程》 * |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116597012A (en) * | 2023-06-12 | 2023-08-15 | 深圳海星智驾科技有限公司 | Vehicle pose recognition method and device, computer equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114897987B (en) | 2022-10-28 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109188457B (en) | Object detection frame generation method, device, equipment, storage medium and vehicle | |
| CN110758246B (en) | Automatic parking method and device | |
| US20230215187A1 (en) | Target detection method based on monocular image | |
| KR20180056685A (en) | System and method for non-obstacle area detection | |
| CN112507862B (en) | Vehicle orientation detection method and system based on multitasking convolutional neural network | |
| CN109741241B (en) | Fisheye image processing method, device, equipment and storage medium | |
| CN114705121B (en) | Vehicle pose measurement method and device, electronic equipment and storage medium | |
| CN111915657A (en) | Point cloud registration method and device, electronic equipment and storage medium | |
| CN115526990A (en) | Target visualization method and device for digital twins and electronic equipment | |
| CN114897987B (en) | Method, device, equipment and medium for determining vehicle ground projection | |
| CN112198878A (en) | Instant map construction method and device, robot and storage medium | |
| CN111709377B (en) | Feature extraction method, target re-identification method and device and electronic equipment | |
| CN116863124B (en) | Vehicle attitude determination method, controller and storage medium | |
| CN113902047B (en) | Image element matching method, device, equipment and storage medium | |
| CN116245915A (en) | Target tracking method based on video | |
| Lai et al. | Vision based ADAS for Forward Vehicle Detection using Convolutional Neural Networks and Motion Tracking. | |
| CN111462243A (en) | Vehicle-mounted streaming media rearview mirror calibration method, system and device | |
| CN114708571A (en) | Parking space marking method and device for automatic parking based on domain controller platform | |
| CN114648639A (en) | Target vehicle detection method, system and device | |
| CN114066958A (en) | Method and device for predicting depth information of target, electronic device and storage medium | |
| CN113408568A (en) | Method, device and equipment for correlating detection model training of object key points | |
| WO2020227933A1 (en) | Six-degree-of-freedom attitude estimation method and apparatus, and computer-readable storage medium | |
| CN118015047B (en) | Multi-target tracking method, device, equipment and storage medium | |
| CN110647890B (en) | High-performance image feature extraction and matching method, system and storage medium | |
| CN115761597A (en) | Muck truck risk early warning method, system, equipment and medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |