CN111582256A - Parking management method and device based on radar and visual information - Google Patents
Parking management method and device based on radar and visual information Download PDFInfo
- Publication number
- CN111582256A CN111582256A CN202010572561.4A CN202010572561A CN111582256A CN 111582256 A CN111582256 A CN 111582256A CN 202010572561 A CN202010572561 A CN 202010572561A CN 111582256 A CN111582256 A CN 111582256A
- Authority
- CN
- China
- Prior art keywords
- radar
- target
- image
- coordinate system
- determining
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000007726 management method Methods 0.000 title claims abstract description 49
- 230000000007 visual effect Effects 0.000 title claims abstract description 15
- 238000001514 detection method Methods 0.000 claims abstract description 52
- 238000012544 monitoring process Methods 0.000 claims abstract description 50
- 230000001360 synchronised effect Effects 0.000 claims abstract description 19
- 238000013507 mapping Methods 0.000 claims description 39
- 238000004364 calculation method Methods 0.000 claims description 25
- 230000009466 transformation Effects 0.000 claims description 23
- 238000000034 method Methods 0.000 claims description 19
- 238000004458 analytical method Methods 0.000 claims description 16
- 230000001131 transforming effect Effects 0.000 claims description 13
- 238000004422 calculation algorithm Methods 0.000 claims description 12
- 230000006399 behavior Effects 0.000 description 13
- 238000012806 monitoring device Methods 0.000 description 13
- 230000006870 function Effects 0.000 description 4
- 238000013461 design Methods 0.000 description 3
- 239000011159 matrix material Substances 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 230000009286 beneficial effect Effects 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 230000018109 developmental process Effects 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 230000004927 fusion Effects 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- 230000004075 alteration Effects 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000004590 computer program Methods 0.000 description 1
- 238000012937 correction Methods 0.000 description 1
- 238000009795 derivation Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 239000000835 fiber Substances 0.000 description 1
- 238000009434 installation Methods 0.000 description 1
- 238000007781 pre-processing Methods 0.000 description 1
- 230000011218 segmentation Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
- 238000013519 translation Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2218/00—Aspects of pattern recognition specially adapted for signal processing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/25—Fusion techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Engineering & Computer Science (AREA)
- Signal Processing (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Artificial Intelligence (AREA)
- Multimedia (AREA)
- Life Sciences & Earth Sciences (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Traffic Control Systems (AREA)
Abstract
The embodiment of the invention provides a parking management method and a parking management device based on radar and visual information, wherein the parking management method comprises the following steps: acquiring radar data and image data acquired in a preset monitoring area through synchronous acquisition frequency; determining a radar target in the radar data to obtain a radar target frame of the radar target on an image two-dimensional coordinate system; determining whether the image data contains a vehicle target, and if so, determining an image detection frame of the vehicle target on an image two-dimensional coordinate system; calculating the intersection ratio of the radar target frame and the image detection frame, and determining a candidate target based on the intersection ratio; and calculating the confidence degree of the candidate target, and if the confidence degree of the candidate target is greater than a preset confidence degree threshold value, performing parking management on the vehicles in a preset monitoring area. According to the invention, the accuracy rate of vehicle target identification is improved; the probability of misjudgment of the parking event can be obviously reduced, and therefore accurate identification of the vehicle target is achieved.
Description
Technical Field
The invention relates to the technical field of intelligent parking management, in particular to a parking management method and device based on radar and visual information.
Background
At present of economic rapid development, the living standard and income of people are continuously improved, the quantity of the preserved urban motor vehicles is rapidly increased, but along with the increase of gaps of urban parking spaces, the parking spaces can not meet huge parking demands, contradictions between the parking spaces and the parking demands are increasingly sharp, especially on two sides of urban roads, because the shortages of roadside parking spaces and the traffic safety awareness of more motor vehicle drivers are thin, urban roadside parking and roadside illegal parking become one of the aeipathia of urban management, the problems of traffic jam, disordered parking and the like are brought about to seriously restrict the rapid development of the cities, and meanwhile, the appearance and the living environment of residents of the cities are seriously influenced, so the treatment degree of the roadside parking and the roadside illegal parking of the cities is reached.
In roadside parking management, an image monitoring device is generally used to monitor a target parking area, and then a target detection and recognition algorithm is used to determine a target vehicle, so as to perform parking management on the vehicle. However, the visual information shot by the camera is easily interfered by the environment, and the strong light, the fog, the rain and the snow and the like can not accurately obtain the target characteristics of the vehicle in the video image, so that the parking management precision is low. Thus, other sensors are often introduced to assist in parking management, such as millimeter wave radar and the like. If the millimeter wave radar is used for monitoring vehicles in a target area, although the environment interference resistance is strong, erroneous judgment is easy to occur for the identification and segmentation of multiple vehicle targets; and lack the ability to obtain key identity information for the vehicle.
Disclosure of Invention
The embodiment of the invention provides a parking management method and device based on radar and visual information, which realize accurate identification of a vehicle target.
The embodiment of the invention provides a parking management method based on radar and visual information, which comprises the following steps:
acquiring radar data and image data acquired in a preset monitoring area through synchronous acquisition frequency;
determining a radar target in the radar data, transforming the radar data through a coordinate system to obtain the mapping of the radar target on an image two-dimensional coordinate system, and obtaining a radar target frame of the radar target on the image two-dimensional coordinate system according to the mapping;
determining whether the image data contains a vehicle target, and if so, determining an image detection frame of the vehicle target on an image two-dimensional coordinate system;
calculating the intersection ratio of the radar target frame and the image detection frame, and determining a candidate target based on the intersection ratio;
and performing behavior analysis on the candidate target based on the radar data and the image data acquired in the preset time respectively, calculating the confidence coefficient of the candidate target, and performing parking management on the vehicle in the preset monitoring area if the confidence coefficient of the candidate target is greater than a preset confidence coefficient threshold value.
The embodiment of the invention also provides a parking management device based on radar and visual information, which comprises:
the system comprises a first acquisition module, a second acquisition module and a third acquisition module, wherein the first acquisition module is used for acquiring radar data and image data acquired in a preset monitoring area through synchronous acquisition frequency;
the transformation module is used for determining a radar target in the radar data, transforming the radar data through a coordinate system to obtain the mapping of the radar target on an image two-dimensional coordinate system, and obtaining a radar target frame of the radar target on the image two-dimensional coordinate system according to the mapping;
the first determining module is used for determining whether the image data contains the vehicle target or not, and if so, determining an image detection frame of the vehicle target on an image two-dimensional coordinate system;
the first calculation module is used for calculating the intersection ratio of the radar target frame and the image detection frame and determining a candidate target based on the intersection ratio;
and the second calculation module is used for performing behavior analysis on the candidate target respectively based on the radar data and the image data acquired in the preset time, calculating the confidence coefficient of the candidate target, and performing parking management on the vehicle in the preset monitoring area if the confidence coefficient of the candidate target is greater than a preset confidence coefficient threshold value.
The technical scheme has the following beneficial effects: according to the invention, the position of the vehicle target in the image can be accurately determined based on the fusion of the radar data and the image data acquired in the preset monitoring area by synchronous acquisition frequency; meanwhile, the parking behavior of the vehicle target in the preset monitoring area can be judged in time through the calculation processing of the radar data and the image data, so that the accuracy rate of judging the vehicle parking event is improved; the problem that when a single image monitoring device is used for monitoring a monitoring area in parking management, the vehicle target cannot be accurately identified due to the fact that the vehicle target is easily influenced by the environment is solved, and the misjudgment probability of a parking event is reduced.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.
FIG. 1 is a flowchart of a parking management method based on radar and visual information according to an embodiment of the present invention;
fig. 2 is a schematic structural diagram of a parking management apparatus based on radar and visual information according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
As shown in fig. 1, which is a flowchart of a parking management method based on radar and visual information in an embodiment of the present invention, the parking management method includes:
101. acquiring radar data and image data acquired in a preset monitoring area through synchronous acquisition frequency;
102. determining a radar target in the radar data, transforming the radar data through a coordinate system to obtain the mapping of the radar target on an image two-dimensional coordinate system, and obtaining a radar target frame of the radar target on the image two-dimensional coordinate system according to the mapping;
103. determining whether the image data contains a vehicle target, and if so, determining an image detection frame of the vehicle target on an image two-dimensional coordinate system;
104. calculating the intersection ratio of the radar target frame and the image detection frame, and determining a candidate target based on the intersection ratio;
105. and performing behavior analysis on the candidate target based on the radar data and the image data acquired in the preset time respectively, calculating the confidence coefficient of the candidate target, and performing parking management on the vehicle in the preset monitoring area if the confidence coefficient of the candidate target is greater than a preset confidence coefficient threshold value.
Further, before the step of acquiring the radar data and the image data acquired in the predetermined monitoring area by synchronizing the acquisition frequency, the method includes:
acquiring radar data acquisition frequency and image data acquisition frequency;
determining a reciprocal value of the least common multiple of the reciprocal value of the radar data acquisition frequency and the reciprocal value of the image data acquisition frequency as a synchronous acquisition frequency;
wherein, the acquiring radar data and image data acquired in a predetermined monitoring area by synchronously acquiring frequency comprises:
and synchronizing the radar data and the initial acquisition point of the image data acquisition.
Further, the determining a radar target in the radar data, transforming the radar data through a coordinate system to obtain a mapping of the radar target on an image two-dimensional coordinate system, and obtaining a radar target frame of the radar target on the image two-dimensional coordinate system according to the mapping specifically includes:
determining a radar target in the radar data, and transforming the radar target through a coordinate system to obtain a mapping relation of data between a coordinate system before transformation and a coordinate system after transformation;
obtaining the mapping of the radar target on the image two-dimensional coordinate system according to the mapping relation;
and obtaining a radar target frame of the radar target on the image two-dimensional coordinate system based on the mapping and a preset radar target frame generation algorithm.
Further, the determining whether the image data includes the vehicle target, and if so, determining an image detection frame of the vehicle target on an image two-dimensional coordinate system specifically includes:
and determining whether the image data contains the vehicle target or not based on a target detection algorithm, and if so, determining an image detection frame of the vehicle target on an image two-dimensional coordinate system.
Further, the calculating an intersection ratio of the radar target frame and the image detection frame, and determining a candidate target based on the intersection ratio includes:
calculating the intersection area and the union area of the radar target frame and the image detection frame;
calculating to obtain an intersection ratio of the radar target frame and the image detection frame according to the intersection area and the union area;
and if the intersection ratio is larger than a preset intersection ratio, determining the radar target frame and the target in the image detection frame as candidate targets.
Further, the performing behavior analysis on the candidate target based on the radar data and the image data acquired in the predetermined time, respectively, calculating a confidence of the candidate target, and performing parking management on the vehicle in the predetermined monitoring area if the confidence of the candidate target is greater than a predetermined confidence threshold, specifically including:
performing behavior analysis on the candidate target based on the radar data and the image data acquired within the preset time, and calculating the confidence of the candidate target according to the calculation parameters;
if the confidence of the candidate target is greater than a preset confidence threshold, determining that vehicles exist in a preset monitoring area, and performing parking management on the vehicles;
wherein the calculation parameters comprise at least one of pose, speed and contour matching degree.
As shown in fig. 2, a schematic structural diagram of a parking management apparatus based on radar and visual information in an embodiment of the present invention includes:
a first obtaining module 21, configured to obtain radar data and image data collected in a predetermined monitoring area through a synchronous collection frequency;
the transformation module 22 is configured to determine a radar target in the radar data, transform the radar data through a coordinate system, obtain mapping of the radar target on an image two-dimensional coordinate system, and obtain a radar target frame of the radar target on the image two-dimensional coordinate system according to the mapping;
the first determining module 23 is configured to determine whether the image data includes a vehicle target, and if so, determine an image detection frame of the vehicle target on an image two-dimensional coordinate system;
a first calculating module 24, configured to calculate an intersection ratio of the radar target frame and the image detection frame, and determine a candidate target based on the intersection ratio;
and the second calculation module 25 is configured to perform behavior analysis on the candidate target based on the radar data and the image data acquired in a predetermined time, calculate a confidence level of the candidate target, and perform parking management on the vehicle in the predetermined monitoring area if the confidence level of the candidate target is greater than a predetermined confidence level threshold.
Further, comprising:
the second acquisition module is used for acquiring radar data acquisition frequency and image data acquisition frequency;
the second determining module is used for determining a reciprocal value of the least common multiple of the reciprocal value of the radar data acquisition frequency and the reciprocal value of the image data acquisition frequency as a synchronous acquisition frequency;
wherein the first obtaining module is further used for
And synchronizing the radar data and the initial acquisition point of the image data acquisition.
Further, the transformation module is particularly useful for
Determining a radar target in the radar data, and transforming the radar target through a coordinate system to obtain a mapping relation of data between a coordinate system before transformation and a coordinate system after transformation;
obtaining the mapping of the radar target on the image two-dimensional coordinate system according to the mapping relation;
and obtaining a radar target frame of the radar target on the image two-dimensional coordinate system based on the mapping and a preset radar target frame generation algorithm.
Further, the first determining module is specifically configured to
And determining whether the image data contains the vehicle target or not based on a target detection algorithm, and if so, determining an image detection frame of the vehicle target on an image two-dimensional coordinate system.
Further, the first computing module includes:
the first calculation unit is used for calculating the intersection area and the union area of the radar target frame and the image detection frame;
the second calculation unit is used for calculating and obtaining an intersection ratio of the radar target frame and the image detection frame according to the intersection area and the union area;
and the determining unit is used for determining the radar target frame and the target in the image detection frame as candidate targets if the intersection ratio is greater than a preset intersection ratio.
Further, the second calculation module is specifically configured to
Performing behavior analysis on the candidate target based on the radar data and the image data acquired within the preset time, and calculating the confidence of the candidate target according to the calculation parameters;
if the confidence of the candidate target is greater than a preset confidence threshold, determining that vehicles exist in a preset monitoring area, and performing parking management on the vehicles;
wherein the calculation parameters comprise at least one of pose, speed and contour matching degree.
The technical scheme of the embodiment of the invention has the following beneficial effects: according to the invention, the position of the vehicle target in the image can be accurately determined based on the fusion of the radar data and the image data acquired in the preset monitoring area by synchronous acquisition frequency; meanwhile, the parking behavior of the vehicle target in the preset monitoring area can be judged in time through the calculation processing of the radar data and the image data, so that the accuracy rate of judging the vehicle parking event is improved; the problem that when a single image monitoring device is used for monitoring a monitoring area in parking management, the vehicle target cannot be accurately identified due to the fact that the vehicle target is easily influenced by the environment is solved, and the misjudgment probability of a parking event is reduced.
The above technical solutions of the embodiments of the present invention are described in detail below with reference to application examples:
the application example of the invention aims to accurately identify the vehicle target.
As shown in fig. 1, for example, in a roadside parking management system, at least one image monitoring device and at least one radar monitoring device are configured, and the radar monitoring device generally uses a millimeter wave radar, and first acquires, in real time, radar data acquired by the homogeneous radar monitoring device and image data acquired by the image monitoring device at the same time in a predetermined monitoring area by synchronously acquiring frequencies; when the radar target exists in the radar data, converting the radar target data through a coordinate system to obtain a radar target frame of the radar target on an image two-dimensional coordinate system; then, whether the image data contains the vehicle target is determined, if yes, an image detection frame of the vehicle target on an image two-dimensional coordinate system is determined; and calculating the intersection ratio of the radar target frame and the image detection frame, and determining whether a vehicle enters the preset monitoring area or not based on the intersection ratio.
It should be noted that, in the embodiment of the present invention, the predetermined monitoring area may be all areas of the roadside parking lot to be monitored, or may also be a partial area in the roadside parking lot to be monitored, and if the predetermined monitoring area is a partial area in the roadside parking lot to be monitored, at least one image monitoring device and at least one radar monitoring device are configured in the corresponding predetermined monitoring area.
In a possible implementation manner, before the step of acquiring, in the step 101, radar data and image data acquired in a predetermined monitoring area by synchronizing acquisition frequencies, the method includes: acquiring radar data acquisition frequency and image data acquisition frequency; and determining the reciprocal value of the least common multiple of the reciprocal value of the radar data acquisition frequency and the reciprocal value of the image data acquisition frequency as the synchronous acquisition frequency.
Wherein, the acquiring radar data and image data acquired in a predetermined monitoring area by synchronously acquiring frequency comprises: and synchronizing the radar data and the initial acquisition point of the image data acquisition.
For example, in a roadside parking management system, at least one image monitoring device and at least one radar monitoring device are configured; acquiring a radar data acquisition frequency of 25Hz and an image data acquisition frequency of 30 Hz; determining the reciprocal value of the radar data acquisition frequency, namely 1/25, and the reciprocal value of the image data acquisition frequency, namely the reciprocal value of the least common multiple 1/5 of 1/30, namely 5 as synchronous acquisition frequency, and then the synchronous acquisition frequency is 5 Hz; and then synchronizing the radar data and the initial acquisition point of the image data acquisition, and acquiring the radar data and the image data acquired in a preset monitoring area by synchronous acquisition frequency of 5Hz after synchronizing the initial acquisition point.
According to the embodiment, the synchronization of the radar data and the image data in time acquisition is realized, so that the maximum number of synchronization points can be obtained in unit time, namely whether a vehicle target exists in a preset monitoring area or not can be judged in the shortest time interval, and the efficiency of vehicle target identification is greatly improved.
In a possible implementation manner, the step 102 determines a radar target in the radar data, transforms the radar data through a coordinate system to obtain a mapping of the radar target on an image two-dimensional coordinate system, and obtains a radar target frame of the radar target on the image two-dimensional coordinate system according to the mapping, and specifically includes: determining a radar target in the radar data, and transforming the radar target through a coordinate system to obtain a mapping relation of data between a coordinate system before transformation and a coordinate system after transformation; obtaining the mapping of the radar target on the image two-dimensional coordinate system according to the mapping relation; and obtaining a radar target frame of the radar target on the image two-dimensional coordinate system based on the mapping and a preset radar target frame generation algorithm.
For example, in the above example, in the roadside parking management system, after radar data and image data acquired in a predetermined monitoring area by synchronously acquiring a frequency of 5Hz are acquired, radar target data in the acquired radar data is determined, and the radar target data is transformed by a coordinate system, specifically, a transformation matrix for transforming a radar coordinate system to an image coordinate system is obtained according to installation positions of a radar and an image monitoring device, such as a camera, the camera can obtain an external reference rotation matrix and a translation vector by calibration, and a transformation matrix from the radar coordinate system to the image coordinate system can be obtained by jointly calibrating the radar and the camera, that is:
radar coordinate system P (R, alpha), camera coordinate system P (X, Y, Z),
the transformation relationship between the two coordinate systems is that,
the transformation process for converting the camera coordinate system to the uncorrected image coordinate system P (x, y) is,
obtaining a mapping relation of data between the coordinate system before transformation and the coordinate system after transformation; because the camera has certain pillow-shaped and barrel-shaped distortion, the internal reference and the external reference of the camera are generally measured by a Zhang Zhengyou camera calibration method, and a corrected image coordinate system P (x ', y') can be obtained through calculation, wherein the correction method is a public technology and is not explained in the embodiment of the invention;
from the transformation of the image coordinate system P (x ', y') corrected as described above to the image coordinate point P (u, v),
by the derivation, the transformation from the radar coordinate system to the image coordinate point can be obtained, and finally the radar target frame of the radar target on the image two-dimensional coordinate system is obtained.
Through the embodiment, the radar data and the image data can be accurately fused, and necessary precondition guarantee is provided for accurately judging the vehicle target subsequently.
In one possible implementation, the step of obtaining a radar target frame of the radar target on the image two-dimensional coordinate system is followed by the steps of: and acquiring a radar target frame of the radar target on the image two-dimensional coordinate system by preprocessing radar data and based on a target detection algorithm of the millimeter wave radar.
Through the embodiment, the most accurate radar target frame is obtained according to the angle and the distance of the preset monitoring area to the radar and the camera, so that important precondition guarantee is provided for accurately judging the vehicle target subsequently.
In a possible implementation manner, the step 103 determines whether the image data includes a vehicle target, and if so, determines an image detection frame of the vehicle target on an image two-dimensional coordinate system, specifically including: and acquiring an image detection frame of the vehicle in the image by using the trained YOLOv3 target detection model.
In a possible implementation manner, the calculating an intersection ratio of the radar target frame and the image detection frame, and determining a candidate target based on the intersection ratio includes: calculating the intersection area and the union area of the radar target frame and the image detection frame; calculating to obtain the intersection ratio of the radar target frame and the image detection frame according to the intersection area and the union area; and if the intersection ratio is larger than a preset intersection ratio, determining the radar target frame and the target in the image detection frame as candidate targets.
For example, as described above, in the roadside parking management system, according to the radar target frame bbox2, the intersection area and the union area of the radar target frame bbox2 and the rectangular boundary frame of the original image bbox1 are calculated; according to the intersection area and the union area, calculating and obtaining an intersection ratio IoU (intersection over Union) of the radar target frame bbox2 and the rectangular boundary frame of the original image bbox1, wherein IoU is the intersection area of the radar target frame and the image detection frame of the image two-dimensional coordinate system/the union area of the radar target frame and the boundary frame of the image detection frame of the image two-dimensional coordinate system; and if the intersection ratio IoU is larger than a preset intersection ratio, determining the radar target frame and the target in the image detection frame as candidate targets. Wherein the predetermined intersection ratio is generally determined according to the angle and distance of the predetermined monitored area to the radar and the camera.
By the present embodiment, the position of the candidate object in the image can be accurately determined, thereby ensuring accurate identification of the candidate object.
In a possible implementation manner, step 105 performs behavior analysis on the candidate target based on radar data and image data acquired within a predetermined time, calculates a confidence level of the candidate target, and performs parking management on a vehicle in a predetermined monitoring area if the confidence level of the candidate target is greater than a predetermined confidence level threshold, specifically including: performing behavior analysis on the candidate target based on the radar data and the image data acquired within the preset time, and calculating the confidence of the candidate target according to the calculation parameters; and if the confidence of the candidate target is greater than a preset confidence threshold, determining that the vehicle exists in the preset monitoring area, and performing parking management on the vehicle.
Wherein, the calculation parameters include but are not limited to at least one of pose, speed and contour matching degree.
For example, in a roadside parking management system, behavior analysis is performed on determined candidate targets based on radar data and image data acquired within a predetermined time, and confidence of the candidate targets is calculated according to calculation parameters such as pose, speed, contour matching degree and the like; and if the confidence of the candidate target is greater than a preset confidence threshold, determining that the vehicle exists in the preset monitoring area, and performing parking management on the vehicle.
By the embodiment, the determined candidate targets can be subjected to multi-dimensional behavior tracking analysis through radar data and image data in time, whether the vehicle targets exist in the preset monitoring area or not can be determined through accurate calculation, and therefore accurate recognition of the vehicle targets is achieved.
In one possible implementation, the method further includes: if the vehicle target in the radar data collected in real time is determined to disappear, determining whether the vehicle target in the image data collected according to the synchronous collection frequency together with the radar data disappears; and if the target vehicle disappears, determining that the target vehicle exits the preset monitoring area.
For example, in the roadside parking management system, if it is determined that a vehicle target such as a vehicle car in radar data collected in real time disappears, it is determined whether the vehicle car in image data collected synchronously with the radar data according to a synchronous collection frequency disappears; if it disappears, it is determined that the vehicle car has exited the predetermined monitoring area.
According to the embodiment, the vehicle target is tracked, the condition that the vehicle target is driven out of the preset monitoring area is accurately and timely identified, necessary vehicle target driving-away data are provided for roadside parking management, and further the management efficiency of roadside parking is improved.
The embodiment of the invention provides a parking management device based on radar and visual information, which can realize the method embodiment provided above, and for specific function realization, please refer to the description in the method embodiment, and details are not repeated herein.
It should be understood that the specific order or hierarchy of steps in the processes disclosed is an example of exemplary approaches. Based upon design preferences, it is understood that the specific order or hierarchy of steps in the processes may be rearranged without departing from the scope of the present disclosure. The accompanying method claims present elements of the various steps in a sample order, and are not intended to be limited to the specific order or hierarchy presented.
In the foregoing detailed description, various features are grouped together in a single embodiment for the purpose of streamlining the disclosure. This method of disclosure is not to be interpreted as reflecting an intention that the claimed embodiments of the subject matter require more features than are expressly recited in each claim. Rather, as the following claims reflect, invention lies in less than all features of a single disclosed embodiment. Thus, the following claims are hereby expressly incorporated into the detailed description, with each claim standing on its own as a separate preferred embodiment of the invention.
The previous description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the present invention. To those skilled in the art; various modifications to these embodiments will be readily apparent, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the disclosure. Thus, the present disclosure is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
What has been described above includes examples of one or more embodiments. It is, of course, not possible to describe every conceivable combination of components or methodologies for purposes of describing the aforementioned embodiments, but one of ordinary skill in the art may recognize that many further combinations and permutations of various embodiments are possible. Accordingly, the embodiments described herein are intended to embrace all such alterations, modifications and variations that fall within the scope of the appended claims. Furthermore, to the extent that the term "includes" is used in either the detailed description or the claims, such term is intended to be inclusive in a manner similar to the term "comprising" as "comprising" is interpreted when employed as a transitional word in a claim. Furthermore, any use of the term "or" in the specification of the claims is intended to mean a "non-exclusive or".
Those of skill in the art will further appreciate that the various illustrative logical blocks, units, and steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or combinations of both. To clearly illustrate the interchangeability of hardware and software, various illustrative components, elements, and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as hardware or software depends upon the particular application and design requirements of the overall system. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present embodiments.
The various illustrative logical blocks, or elements, described in connection with the embodiments disclosed herein may be implemented or performed with a general purpose processor, a digital signal processor, an Application Specific Integrated Circuit (ASIC), a field programmable gate array or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination thereof designed to perform the functions described herein. A general-purpose processor may be a microprocessor, but in the alternative, the processor may be any conventional processor, controller, microcontroller, or state machine. A processor may also be implemented as a combination of computing devices, e.g., a digital signal processor and a microprocessor, a plurality of microprocessors, one or more microprocessors in conjunction with a digital signal processor core, or any other similar configuration.
The steps of a method or algorithm described in connection with the embodiments disclosed herein may be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. A software module may be stored in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art. For example, a storage medium may be coupled to the processor such the processor can read information from, and write information to, the storage medium. In the alternative, the storage medium may be integral to the processor. The processor and the storage medium may reside in an ASIC, which may be located in a user terminal. In the alternative, the processor and the storage medium may reside in different components in a user terminal.
In one or more exemplary designs, the functions described above in connection with the embodiments of the invention may be implemented in hardware, software, firmware, or any combination of the three. If implemented in software, the functions may be stored on or transmitted over as one or more instructions or code on a computer-readable medium. Computer-readable media includes both computer storage media and communication media that facilitate transfer of a computer program from one place to another. Storage media may be any available media that can be accessed by a general purpose or special purpose computer. For example, such computer-readable media can include, but is not limited to, RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to carry or store program code in the form of instructions or data structures and which can be read by a general-purpose or special-purpose computer, or a general-purpose or special-purpose processor. Additionally, any connection is properly termed a computer-readable medium, and, thus, is included if the software is transmitted from a website, server, or other remote source via a coaxial cable, fiber optic cable, twisted pair, Digital Subscriber Line (DSL), or wirelessly, e.g., infrared, radio, and microwave. Such discs (disk) and disks (disc) include compact disks, laser disks, optical disks, DVDs, floppy disks and blu-ray disks where disks usually reproduce data magnetically, while disks usually reproduce data optically with lasers. Combinations of the above may also be included in the computer-readable medium.
The above-mentioned embodiments are intended to illustrate the objects, technical solutions and advantages of the present invention in further detail, and it should be understood that the above-mentioned embodiments are merely exemplary embodiments of the present invention, and are not intended to limit the scope of the present invention, and any modifications, equivalent substitutions, improvements and the like made within the spirit and principle of the present invention should be included in the scope of the present invention.
Claims (12)
1. A parking management method based on radar and visual information is characterized by comprising the following steps:
acquiring radar data and image data acquired in a preset monitoring area through synchronous acquisition frequency;
determining a radar target in the radar data, transforming the radar data through a coordinate system to obtain the mapping of the radar target on an image two-dimensional coordinate system, and obtaining a radar target frame of the radar target on the image two-dimensional coordinate system according to the mapping;
determining whether the image data contains a vehicle target, and if so, determining an image detection frame of the vehicle target on an image two-dimensional coordinate system;
calculating the intersection ratio of the radar target frame and the image detection frame, and determining a candidate target based on the intersection ratio;
and performing behavior analysis on the candidate target based on the radar data and the image data acquired in the preset time respectively, calculating the confidence coefficient of the candidate target, and performing parking management on the vehicle in the preset monitoring area if the confidence coefficient of the candidate target is greater than a preset confidence coefficient threshold value.
2. The method of claim 1, wherein prior to the step of acquiring radar data and image data acquired within a predetermined monitored area by synchronizing acquisition frequencies, comprising:
acquiring radar data acquisition frequency and image data acquisition frequency;
determining a reciprocal value of the least common multiple of the reciprocal value of the radar data acquisition frequency and the reciprocal value of the image data acquisition frequency as a synchronous acquisition frequency;
wherein, the acquiring radar data and image data acquired in a predetermined monitoring area by synchronously acquiring frequency comprises:
and synchronizing the radar data and the initial acquisition point of the image data acquisition.
3. The method according to claim 2, wherein the determining the radar target in the radar data, transforming the radar data through a coordinate system to obtain a mapping of the radar target on an image two-dimensional coordinate system, and obtaining a radar target frame of the radar target on the image two-dimensional coordinate system according to the mapping includes:
determining a radar target in the radar data, and transforming the radar target through a coordinate system to obtain a mapping relation of data between a coordinate system before transformation and a coordinate system after transformation;
obtaining the mapping of the radar target on the image two-dimensional coordinate system according to the mapping relation;
and obtaining a radar target frame of the radar target on the image two-dimensional coordinate system based on the mapping and a preset radar target frame generation algorithm.
4. The method according to claim 1, wherein the determining whether the image data includes the vehicle object, and if so, determining an image detection frame of the vehicle object on an image two-dimensional coordinate system specifically includes:
and determining whether the image data contains the vehicle target or not based on a target detection algorithm, and if so, determining an image detection frame of the vehicle target on an image two-dimensional coordinate system.
5. The method of claim 1, wherein calculating an intersection ratio of the radar target frame and the image detection frame, and determining a candidate target based on the intersection ratio comprises:
calculating the intersection area and the union area of the radar target frame and the image detection frame;
calculating to obtain an intersection ratio of the radar target frame and the image detection frame according to the intersection area and the union area;
and if the intersection ratio is larger than a preset intersection ratio, determining the radar target frame and the target in the image detection frame as candidate targets.
6. The method according to any one of claims 1 to 5, wherein the performing behavior analysis on the candidate target based on the radar data and the image data acquired within a predetermined time, respectively, calculates a confidence level of the candidate target, and performs parking management on the vehicle in a predetermined monitoring area if the confidence level of the candidate target is greater than a predetermined confidence level threshold, specifically comprises:
performing behavior analysis on the candidate target based on the radar data and the image data acquired within the preset time, and calculating the confidence of the candidate target according to the calculation parameters;
if the confidence of the candidate target is greater than a preset confidence threshold, determining that vehicles exist in a preset monitoring area, and performing parking management on the vehicles;
wherein the calculation parameters comprise at least one of pose, speed and contour matching degree.
7. A parking management device based on radar and visual information, comprising:
the system comprises a first acquisition module, a second acquisition module and a third acquisition module, wherein the first acquisition module is used for acquiring radar data and image data acquired in a preset monitoring area through synchronous acquisition frequency;
the transformation module is used for determining a radar target in the radar data, transforming the radar data through a coordinate system to obtain the mapping of the radar target on an image two-dimensional coordinate system, and obtaining a radar target frame of the radar target on the image two-dimensional coordinate system according to the mapping;
the first determining module is used for determining whether the image data contains the vehicle target or not, and if so, determining an image detection frame of the vehicle target on an image two-dimensional coordinate system;
the first calculation module is used for calculating the intersection ratio of the radar target frame and the image detection frame and determining a candidate target based on the intersection ratio;
and the second calculation module is used for performing behavior analysis on the candidate target respectively based on the radar data and the image data acquired in the preset time, calculating the confidence coefficient of the candidate target, and performing parking management on the vehicle in the preset monitoring area if the confidence coefficient of the candidate target is greater than a preset confidence coefficient threshold value.
8. The apparatus of claim 7, comprising:
the second acquisition module is used for acquiring radar data acquisition frequency and image data acquisition frequency;
the second determining module is used for determining a reciprocal value of the least common multiple of the reciprocal value of the radar data acquisition frequency and the reciprocal value of the image data acquisition frequency as a synchronous acquisition frequency;
wherein the first obtaining module is further used for
And synchronizing the radar data and the initial acquisition point of the image data acquisition.
9. Device according to claim 8, characterized in that said transformation module is particularly adapted to
Determining a radar target in the radar data, and transforming the radar target through a coordinate system to obtain a mapping relation of data between a coordinate system before transformation and a coordinate system after transformation;
obtaining the mapping of the radar target on the image two-dimensional coordinate system according to the mapping relation;
and obtaining a radar target frame of the radar target on the image two-dimensional coordinate system based on the mapping and a preset radar target frame generation algorithm.
10. The apparatus according to claim 7, wherein the first determining means is specifically configured to determine the first threshold value
And determining whether the image data contains the vehicle target or not based on a target detection algorithm, and if so, determining an image detection frame of the vehicle target on an image two-dimensional coordinate system.
11. The apparatus of claim 7, wherein the first computing module comprises:
the first calculation unit is used for calculating the intersection area and the union area of the radar target frame and the image detection frame;
the second calculation unit is used for calculating and obtaining an intersection ratio of the radar target frame and the image detection frame according to the intersection area and the union area;
and the determining unit is used for determining the radar target frame and the target in the image detection frame as candidate targets if the intersection ratio is greater than a preset intersection ratio.
12. The apparatus according to any of claims 7-11, wherein the second computing module is specifically configured to compute the second value
Performing behavior analysis on the candidate target based on the radar data and the image data acquired within the preset time, and calculating the confidence of the candidate target according to the calculation parameters;
if the confidence of the candidate target is greater than a preset confidence threshold, determining that vehicles exist in a preset monitoring area, and performing parking management on the vehicles;
wherein the calculation parameters comprise at least one of pose, speed and contour matching degree.
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2020103376676 | 2020-04-26 | ||
| CN202010337667 | 2020-04-26 |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111582256A true CN111582256A (en) | 2020-08-25 |
| CN111582256B CN111582256B (en) | 2024-05-14 |
Family
ID=72122060
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010572561.4A Active CN111582256B (en) | 2020-04-26 | 2020-06-22 | Parking management method and device based on radar and visual information |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111582256B (en) |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112528771A (en) * | 2020-11-27 | 2021-03-19 | 深兰科技（上海）有限公司 | Obstacle detection method, obstacle detection device, electronic device, and storage medium |
| CN112738394A (en) * | 2020-12-25 | 2021-04-30 | 浙江大华技术股份有限公司 | Linkage method and device of radar and camera equipment and storage medium |
| CN112859006A (en) * | 2021-01-11 | 2021-05-28 | 成都圭目机器人有限公司 | Method for detecting metal curved cylindrical structure in multi-channel ground penetrating radar data |
| CN112863195A (en) * | 2021-03-18 | 2021-05-28 | 浙江大华技术股份有限公司 | Vehicle state determination method and device |
| CN113469130A (en) * | 2021-07-23 | 2021-10-01 | 浙江大华技术股份有限公司 | Shielded target detection method and device, storage medium and electronic device |
| CN113486795A (en) * | 2021-07-06 | 2021-10-08 | 广州小鹏自动驾驶科技有限公司 | Visual identification performance test method, device, system and equipment |
| CN113850995A (en) * | 2021-09-14 | 2021-12-28 | 华设设计集团股份有限公司 | Event detection method, device and system based on tunnel radar vision data fusion |
| CN116030556A (en) * | 2023-02-09 | 2023-04-28 | 深圳妙月科技有限公司 | Parking space ground lock system and parking space sharing method |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104637059A (en) * | 2015-02-09 | 2015-05-20 | 吉林大学 | Night preceding vehicle detection method based on millimeter-wave radar and machine vision |
| CN109085829A (en) * | 2018-08-09 | 2018-12-25 | 北京智行者科技有限公司 | A kind of sound state target identification method |
| CN110456363A (en) * | 2019-06-17 | 2019-11-15 | 北京理工大学 | The target detection and localization method of three-dimensional laser radar point cloud and infrared image fusion |
| CN110532896A (en) * | 2019-08-06 | 2019-12-03 | 北京航空航天大学 | A kind of road vehicle detection method merged based on trackside millimetre-wave radar and machine vision |
| CN110533923A (en) * | 2019-08-29 | 2019-12-03 | 北京精英路通科技有限公司 | Parking management method, device, computer equipment and storage medium |
-
2020
- 2020-06-22 CN CN202010572561.4A patent/CN111582256B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104637059A (en) * | 2015-02-09 | 2015-05-20 | 吉林大学 | Night preceding vehicle detection method based on millimeter-wave radar and machine vision |
| CN109085829A (en) * | 2018-08-09 | 2018-12-25 | 北京智行者科技有限公司 | A kind of sound state target identification method |
| CN110456363A (en) * | 2019-06-17 | 2019-11-15 | 北京理工大学 | The target detection and localization method of three-dimensional laser radar point cloud and infrared image fusion |
| CN110532896A (en) * | 2019-08-06 | 2019-12-03 | 北京航空航天大学 | A kind of road vehicle detection method merged based on trackside millimetre-wave radar and machine vision |
| CN110533923A (en) * | 2019-08-29 | 2019-12-03 | 北京精英路通科技有限公司 | Parking management method, device, computer equipment and storage medium |
Cited By (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112528771A (en) * | 2020-11-27 | 2021-03-19 | 深兰科技（上海）有限公司 | Obstacle detection method, obstacle detection device, electronic device, and storage medium |
| CN112738394A (en) * | 2020-12-25 | 2021-04-30 | 浙江大华技术股份有限公司 | Linkage method and device of radar and camera equipment and storage medium |
| CN112738394B (en) * | 2020-12-25 | 2023-04-18 | 浙江大华技术股份有限公司 | Linkage method and device of radar and camera equipment and storage medium |
| CN112859006A (en) * | 2021-01-11 | 2021-05-28 | 成都圭目机器人有限公司 | Method for detecting metal curved cylindrical structure in multi-channel ground penetrating radar data |
| CN112859006B (en) * | 2021-01-11 | 2023-08-29 | 成都圭目机器人有限公司 | Method for detecting metal bending cylindrical structure in multichannel ground penetrating radar data |
| CN112863195A (en) * | 2021-03-18 | 2021-05-28 | 浙江大华技术股份有限公司 | Vehicle state determination method and device |
| CN113486795A (en) * | 2021-07-06 | 2021-10-08 | 广州小鹏自动驾驶科技有限公司 | Visual identification performance test method, device, system and equipment |
| CN113469130A (en) * | 2021-07-23 | 2021-10-01 | 浙江大华技术股份有限公司 | Shielded target detection method and device, storage medium and electronic device |
| CN113850995A (en) * | 2021-09-14 | 2021-12-28 | 华设设计集团股份有限公司 | Event detection method, device and system based on tunnel radar vision data fusion |
| CN113850995B (en) * | 2021-09-14 | 2022-12-27 | 华设设计集团股份有限公司 | Event detection method, device and system based on tunnel radar vision data fusion |
| CN116030556A (en) * | 2023-02-09 | 2023-04-28 | 深圳妙月科技有限公司 | Parking space ground lock system and parking space sharing method |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111582256B (en) | 2024-05-14 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111582256B (en) | Parking management method and device based on radar and visual information | |
| CN107738612B (en) | Automatic parking space detection and identification system based on panoramic vision auxiliary system | |
| US10229332B2 (en) | Method and apparatus for recognizing obstacle of vehicle | |
| CN112215306B (en) | Target detection method based on fusion of monocular vision and millimeter wave radar | |
| CN111739338A (en) | Parking management method and system based on multiple types of sensors | |
| CN108859952B (en) | Vehicle lane change early warning method and device and radar | |
| CN110688902B (en) | Method and device for detecting vehicle area in parking space | |
| CN113205691B (en) | Method and device for identifying vehicle position | |
| CN113066306B (en) | Management method and device for roadside parking | |
| CN111931673B (en) | Method and device for checking vehicle detection information based on vision difference | |
| CN112836699A (en) | Long-time multi-target tracking-based berth entrance and exit event analysis method | |
| CN113505638A (en) | Traffic flow monitoring method, traffic flow monitoring device and computer-readable storage medium | |
| CN111693998A (en) | Method and device for detecting vehicle position based on radar and image data | |
| CN112801024B (en) | Detection information processing method and device | |
| CN113052141A (en) | Method and device for detecting parking position of vehicle | |
| CN115457780B (en) | Vehicle flow and velocity automatic measuring and calculating method and system based on priori knowledge set | |
| CN116434056A (en) | Target identification method and system based on radar fusion and electronic equipment | |
| CN116504068A (en) | Statistical method, device, computer equipment and storage medium for lane-level traffic flow | |
| CN113449624B (en) | Method and device for determining vehicle behavior based on pedestrian re-identification | |
| CN116110230A (en) | Vehicle lane crossing line identification method and system based on vehicle-mounted camera | |
| CN113408514B (en) | Method and device for detecting berths of roadside parking lot based on deep learning | |
| CN115457488A (en) | Roadside parking management method and system based on binocular stereo vision | |
| CN114895274A (en) | Guardrail identification method | |
| CN116363623B (en) | Vehicle detection method based on millimeter wave radar and vision fusion | |
| CN113075668B (en) | Dynamic obstacle object identification method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information |
Address after: 075000 ten building, phase 1, airport economic and Technological Development Zone, Zhangjiakou, Hebei Applicant after: Smart intercommunication Technology Co.,Ltd. Address before: 075000 ten building, phase 1, airport economic and Technological Development Zone, Zhangjiakou, Hebei Applicant before: INTELLIGENT INTER CONNECTION TECHNOLOGY Co.,Ltd. |
|
| CB02 | Change of applicant information | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |