CN113066306B - Management method and device for roadside parking - Google Patents
Management method and device for roadside parking Download PDFInfo
- Publication number
- CN113066306B CN113066306B CN202110317344.5A CN202110317344A CN113066306B CN 113066306 B CN113066306 B CN 113066306B CN 202110317344 A CN202110317344 A CN 202110317344A CN 113066306 B CN113066306 B CN 113066306B
- Authority
- CN
- China
- Prior art keywords
- motion vector
- area
- image
- vehicle
- determining
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G08—SIGNALLING
- G08G—TRAFFIC CONTROL SYSTEMS
- G08G1/00—Traffic control systems for road vehicles
- G08G1/14—Traffic control systems for road vehicles indicating individual free spaces in parking areas
- G08G1/145—Traffic control systems for road vehicles indicating individual free spaces in parking areas where the indication depends on the parking areas
- G08G1/148—Management of a network of parking areas
-
- G—PHYSICS
- G08—SIGNALLING
- G08G—TRAFFIC CONTROL SYSTEMS
- G08G1/00—Traffic control systems for road vehicles
- G08G1/01—Detecting movement of traffic to be counted or controlled
- G08G1/017—Detecting movement of traffic to be counted or controlled identifying vehicles
- G08G1/0175—Detecting movement of traffic to be counted or controlled identifying vehicles by photographing vehicles, e.g. when violating traffic rules
Landscapes
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Image Analysis (AREA)
- Traffic Control Systems (AREA)
Abstract
The embodiment of the invention provides a method and a device for managing roadside parking, wherein the method comprises the following steps: acquiring video image information acquired by a plurality of cameras in real time, and determining a berth area in each video frame image; expanding the berthage area in each video frame image to obtain the interesting area calculated by the dense optical flow of each video frame; performing dense optical flow calculation on the interested area to obtain the motion vector direction and the motion vector distance of the interested area of each frame of image; calculating the motion vector direction average value and the motion vector distance average value of the interested area of each frame image in a statistical interval of a preset duration through dense optical flow; determining vehicles suspected of having exit/entrance events in the berth area; determining whether a departure/entry event of the vehicle has occurred. The invention greatly improves the utilization rate of the computing power of the host, thereby realizing the analysis and the processing of the parking events of the multi-rod position and multi-path video cameras with high efficiency and accuracy.
Description
Technical Field
The invention relates to the technical field of intelligent traffic, in particular to a method and a device for managing roadside parking.
Background
With the development of the times, the management of smart parking at the side of the city road is an extremely important part in the management of smart traffic in the city, and the period of rapid development and application begins to be entered. At present, Artificial Intelligence, the internet of things, cloud computing, big data and other new-generation information technologies are rapidly developed, and AI (Artificial Intelligence) such as edge-side GPU (Central Processing Unit), NPU (Neural-network Processing Units, network processor) and the like accelerate the computing power of a computing hardware platform to be rapidly improved, so that an important technical basis is provided for deep fusion of urban roadside intelligent parking and Artificial Intelligence technologies.
At present, commercial road side parking management technologies on the ground in partial cities mainly comprise geomagnetic modes, low-level road video piles and high-level videos, and the high-level videos are the road side parking management modes preferentially developed in all large cities due to the fact that the high-level videos are rich in evidence obtaining information, complete in evidence chain, high in installation position and not prone to damage, strong in scene adaptability and high in reliability.
The roadside parking management scheme based on high-order video mainly has the following on the technical scheme:
the first technology is as follows: video stream real-time detection tracking mode
According to the method, vehicle target detection and tracking are continuously carried out on each frame of real-time image collected by a video camera, and license plate recognition is carried out on each frame, so that related information of parking events and related information of vehicles are obtained. However, the parking management device needs to continuously track and process each frame of image of the video camera in real time, however, the parking behavior itself is an intermittent behavior, and there is a behavior that the vehicle has no entrance or exit in a certain time period, and since it is not possible to predict when the vehicle enters or exits, the method can only continuously analyze the video image, but the process of analyzing the video image needs to use a deep learning target detection and tracking algorithm, and consumes a large amount of computing power of an edge AI computing host, so that a large number of video camera images cannot be processed in parallel, and the method cannot manage more parking spaces simultaneously.
The second technology is as follows: perception mode based on image difference algorithm
The method carries out differential comparison on each frame of image collected by the video camera, and carries out deeper video image analysis processing on the parking process when the parking target area is found to have the change exceeding the image differential threshold value. The method uses the differential detection algorithm, so that the calculation power consumption of the differential algorithm is low, the consumption waste of uninterruptedly carrying out vehicle target detection and real-time tracking calculation power on the real-time image of each frame of the video camera in a video stream real-time detection and tracking mode is avoided, and the number of the video cameras which can be processed in parallel by a single edge AI calculation host is increased to a certain extent. However, the difference algorithm can only detect the size change of the image pixel value, the moving direction of the object cannot be determined, interference factors such as pedestrians and passing vehicles can cause a large amount of difference misjudgments, and a large error is generated on the estimation of the time interval of the parking event, so that the subsequent video image analysis is seriously influenced.
The third technology: detection is assisted by using radar equidistant sensors and video image analysis is combined
In this way, a distance sensor (such as a radar sensor, an ultrasonic sensor, etc.) needs to be installed on the rod position, and when a vehicle is present in the parking target area and the distance sensor detects the distance change of the target area, the monitoring device is informed to capture the image and identify the license plate information of the vehicle, so as to manage the vehicle at the parking position. However, in this management method, a distance sensor needs to be added, so that the management cost is high, and in an environment such as rain and snow, the distance precision error measured by the distance sensor is extremely large, the anti-interference capability is poor, and the detection precision is greatly reduced.
Therefore, a roadside parking management method which can efficiently and accurately detect the parked vehicles and has low management cost is urgently needed.
Disclosure of Invention
The embodiment of the invention provides a method and a device for managing roadside parking, which greatly improve the computational power utilization rate of a host on the premise of ensuring accurate analysis of parking events, thereby realizing efficient and accurate analysis and processing of parking events of multiple-pole-position and multi-path video cameras.
In one aspect, an embodiment of the present invention provides a method for managing roadside parking, including:
acquiring video image information acquired by a plurality of cameras in real time, and determining a berthing area in each video frame image;
expanding the berthage area in each video frame image to obtain an interested area calculated by the dense optical flow of each video frame;
carrying out dense optical flow calculation on the interested area to obtain the motion vector direction and the motion vector distance of the interested area of each frame of image;
calculating the motion vector direction average value and the motion vector distance average value of the interested area of each frame image in a statistical interval of preset duration through dense optical flow according to the obtained motion vector direction and motion vector distance of the interested area of each frame image;
determining vehicles suspected to have exit/entrance events in the parking area according to the motion vector direction average value and the motion vector distance average value;
and identifying vehicle information of a vehicle suspected to have a departure/entrance event based on the video image information, and determining whether the vehicle has a departure/entrance event according to the vehicle information.
Further, before the step of acquiring information of video images collected by a plurality of cameras in real time and determining a berthing area in each video frame image, the method comprises the following steps:
determining coordinates of each vertex of the vehicle berth in a preset image acquisition area in each image;
and determining the berthing area in the video frame image acquired by each video camera according to the coordinates.
Further, the expanding the parking area in each video frame image to obtain the region of interest of dense optical flow calculation of each video frame includes:
in a two-dimensional coordinate system of each video frame image, determining coordinate information of a berthage area in each video frame image, and extending the long side of a rectangular frame of the berthage area for a preset length along the direction of gradually reducing the Y-axis value to obtain an interesting area calculated by the dense optical flow of each video frame.
Further, the performing dense optical flow calculation on the region of interest to obtain the motion vector direction and the motion vector distance of the region of interest of each frame of image includes:
and decoding each frame of image, and performing dense optical flow calculation on the interested area in each decoded frame of image to obtain the motion vector direction and the motion vector distance of the interested area of each frame of image.
Further, the performing dense optical flow calculation on the regions of interest in each decoded frame image to obtain the motion vector direction and the motion vector distance of the regions of interest in each frame image includes:
for each frame image, averagely dividing the region of interest of the decoded current frame into a predetermined number of calculation regions;
carrying out dense optical flow calculation on each calculation area to obtain the motion vector direction and the motion vector distance of each calculation area;
and averaging the motion vector direction and the motion vector distance of each calculation area to obtain the motion vector direction and the motion vector distance of the current frame region of interest.
Further, the determining, according to the motion vector direction average value and the motion vector distance average value, a vehicle suspected of having a departure/entry event in the parking area includes:
judging whether the motion vector distance average value is larger than a preset pixel threshold value or not, and if so, determining that vehicles suspected of generating entrance events exist in the berthing area;
determining the motion direction of the average value of the motion vector directions in a two-dimensional coordinate system of any video frame image of each video image;
if the motion direction of the motion vector direction average value points to a parking area, determining that the vehicle is suspected to have an entrance event;
and if the motion vector direction average value motion direction points to the outside of the parking area, determining that the vehicle is suspected to have a departure event.
Further, the identifying, based on the video image information, vehicle information of a vehicle suspected of having a departure/entry event, and determining whether the vehicle has a departure/entry event according to the vehicle information includes:
determining the starting and ending time of the suspected departure/entrance event of the vehicle, and analyzing the vehicle information of the vehicle in the video image information to be identified within the starting and ending time;
and detecting and tracking the vehicle in the video image information to be identified according to the vehicle information, and determining whether the vehicle has a departure/entrance event.
In another aspect, an embodiment of the present invention provides a roadside parking management device, including:
the acquisition and determination module is used for acquiring video image information acquired by a plurality of cameras in real time and determining a berthing area in each video frame image;
the expansion module is used for expanding the berth area in each video frame image to obtain the interesting area calculated by the dense optical flow of each video frame;
the first calculation module is used for carrying out dense optical flow calculation on the interested area to obtain the motion vector direction and the motion vector distance of the interested area of each frame of image;
the second calculation module is used for calculating the motion vector direction average value and the motion vector distance average value of each frame of image interesting region in a statistical interval of preset duration through dense optical flow according to the obtained motion vector direction and motion vector distance of each frame of image interesting region;
the first determining module is used for determining vehicles suspected to have exit/entrance events in the berth area according to the motion vector direction average value and the motion vector distance average value;
and the identification and determination module is used for identifying the vehicle information of the vehicle suspected to have the exit/entrance event based on the video image information and determining whether the vehicle has the exit/entrance event according to the vehicle information.
Further, comprising:
the second determining module is used for determining the coordinates of each vertex of the vehicle berth in the preset image acquisition area in each image;
and the third determining module is used for determining the berthing area in the video frame image acquired by each camera according to the coordinates.
Further, the expansion module is particularly used for
In a two-dimensional coordinate system of each video frame image, determining coordinate information of a berthage area in each video frame image, and extending the long side of a rectangular frame of the berthage area for a preset length along the direction of gradually reducing the Y-axis value to obtain an interesting area calculated by the dense optical flow of each video frame.
Further, the first computing module includes:
and the decoding unit is used for decoding each frame of image and carrying out dense optical flow calculation on the interested area in each frame of image after decoding to obtain the motion vector direction and the motion vector distance of the interested area of each frame of image.
Further, the decoding unit is specifically used for
For each frame image, averagely dividing the region of interest of the decoded current frame into a predetermined number of calculation regions;
carrying out dense optical flow calculation on each calculation area to obtain the motion vector direction and the motion vector distance of each calculation area;
and averaging the motion vector direction and the motion vector distance of each calculation area to obtain the motion vector direction and the motion vector distance of the current frame region of interest.
Further, the first determining module is specifically configured to
Judging whether the motion vector distance average value is larger than a preset pixel threshold value or not, and if so, determining that vehicles suspected of generating entrance events exist in the berthing area;
determining the motion direction of the average value of the motion vector directions in a two-dimensional coordinate system of any video frame image of each video image;
if the motion direction of the motion vector direction average value points to a parking area, determining that the vehicle is suspected to have an entrance event;
and if the motion vector direction average value motion direction points to the outside of the parking area, determining that the vehicle is suspected to have a departure event.
Further, the identification and determination module is specifically configured for
Determining the starting and ending time of the suspected departure/entrance event of the vehicle, and analyzing the vehicle information of the vehicle in the video image information to be identified within the starting and ending time;
and detecting and tracking the vehicle in the video image information to be identified according to the vehicle information, and determining whether the vehicle has a departure/entrance event.
The technical scheme has the following beneficial effects: by the method, the image analysis and calculation can be simultaneously carried out on the video information acquired by a plurality of paths of video equipment, the dense optical flow calculation is carried out on the region of interest of the dense optical flow calculation in the video information, the motion vector direction and the motion vector distance of each frame of image can be accurately determined, important precondition guarantee is provided for subsequently determining the vehicles suspected to have the exit/entrance events in the parking area, a large number of video frame images which are not necessarily calculated are filtered by determining the vehicles suspected to have the exit/entrance events in the parking area, the calculation amount for subsequently determining whether the vehicles have the exit/entrance events through the video image analysis is greatly reduced, meanwhile, the requirements on vehicle detection and tracking and high real-time performance of a license plate algorithm are greatly reduced, and therefore, the method for efficiently and accurately detecting and detecting the vehicles at a plurality of rods, Analyzing and processing the parking events of the multiple paths of video cameras; on the premise of ensuring accurate analysis of parking events, the utilization rate of the calculation power of the host is greatly improved, and further, the cost of parking management is greatly reduced.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.
FIG. 1 is a flow chart of a method for managing roadside parking in an embodiment of the present invention;
fig. 2 is a schematic structural diagram of a roadside parking management device according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The technical scheme of the embodiment of the invention has the following beneficial effects: by the method, the image analysis and calculation can be simultaneously carried out on the video information acquired by a plurality of paths of video equipment, the dense optical flow calculation is carried out on the region of interest of the dense optical flow calculation in the video information, the motion vector direction and the motion vector distance of each frame of image can be accurately determined, important precondition guarantee is provided for subsequently determining the vehicles suspected to have the exit/entrance events in the parking area, a large number of video frame images which are not necessarily calculated are filtered by determining the vehicles suspected to have the exit/entrance events in the parking area, the calculation amount for subsequently determining whether the vehicles have the exit/entrance events through the video image analysis is greatly reduced, meanwhile, the requirements on vehicle detection and tracking and high real-time performance of a license plate algorithm are greatly reduced, and therefore, the method for efficiently and accurately detecting and detecting the vehicles at a plurality of rods, Analyzing and processing the parking events of the multiple paths of video cameras; on the premise of ensuring accurate analysis of parking events, the utilization rate of the calculation power of the host is greatly improved, and further, the cost of parking management is greatly reduced.
The above technical solutions of the embodiments of the present invention are described in detail below with reference to application examples:
the application example of the invention aims to greatly improve the utilization rate of the computing power of the host computer on the premise of ensuring the accurate analysis of the parking event, thereby realizing the efficient and accurate analysis and processing of the parking event of a multi-pole position and multi-path video camera.
In one possible implementation, as in the roadside parking management system S, the system includes a video array camera, an edge AI computation host C, a switch, a router, and a cloud server; the video array cameras are used for collecting roadside berth video information, the video camera arrays of a plurality of rod positions are connected to the same edge AI computing host C in a cascade mode through a switch, the switch is used for interconnecting the edge AI computing host C with each group of video arrays, and the router is used for uploading parking evidence obtaining data between the edge AI computing host C and the cloud server; and the cloud server is used for receiving and processing the parking evidence obtaining information and generating a parking evidence obtaining record. Firstly, acquiring video image information of a monitoring area in real time through a plurality of video cameras, such as 16 video cameras, of a video array camera, and determining a berthing area in each video frame image; subsequently, expanding the berthage area in each video frame image to obtain an interested area calculated by the dense optical flow of each video frame; carrying out dense optical flow calculation on the interested area of each video frame to obtain the motion vector direction and the motion vector distance of the interested area of each frame image; calculating the motion vector direction average value and the motion vector distance average value of the interested area of each frame image in a statistical interval of preset duration through dense optical flow according to the obtained motion vector direction and motion vector distance of the interested area of each frame image; determining vehicles suspected to have a departure/entry event in the parking area according to the motion vector direction average value and the motion vector distance average value; and finally, identifying the vehicle information of the vehicle suspected to have the exit/entrance event based on the video image information, and determining whether the vehicle has the exit/entrance event according to the vehicle information.
The edge AI computation host C in the embodiment of the present invention has a core composition of an embedded artificial intelligence computation platform, and a GPU of the platform includes 384 CUDA cores (an arithmetic logic unit) and 48 Tensor cores (a novel processing core, which performs a special matrix mathematical operation and is suitable for deep learning), and also includes 2 DLAs (Data Link Access, Data Link Address), and the total computation power reaches 21T, and the platform is the edge side embedded artificial intelligence platform with the highest computation power at present. It should be noted that, although the specific edge AI calculation host is taken as an example in the embodiment of the present invention, the present invention is not limited thereto.
In a possible implementation manner, before the step of acquiring, in real time, video image information acquired by a plurality of cameras in step 101 and determining a berthing area in each video frame image, the method includes: determining coordinates of each vertex of the vehicle berth in a preset image acquisition area in each image; and determining the berthing area in the video frame image acquired by each video camera according to the coordinates.
For example, in the roadside parking management system S, coordinates of respective vertexes of a vehicle parking position within a predetermined image capture area in respective images captured by the video array camera are determined, and then a parking position area in a video frame image captured by each video camera is determined according to the coordinates of the respective vertexes in the respective images; the berth area can be determined by debugging and determining when the camera is installed on the upright rod, the video image identification is not needed to be determined again in real time, and it needs to be explained that the berth area can be manually adjusted and determined again according to the requirements of the actual scene, and the method is not limited herein.
In one possible implementation, the step 102 of expanding the berthing regions in the images of each video frame to obtain the regions of interest of dense optical flow calculation of each video frame includes: in a two-dimensional coordinate system of each video frame image, determining coordinate information of a berthage area in each video frame image, and extending the long side of a rectangular frame of the berthage area for a preset length along the direction of gradually reducing the Y-axis value to obtain an interesting area calculated by the dense optical flow of each video frame.
Wherein, the dense optical flow calculation is carried out on the interested area to obtain the motion vector direction and the motion vector distance of the interested area of each frame of image, and the method comprises the following steps: and decoding each frame of image, and performing dense optical flow calculation on the interested area in each decoded frame of image to obtain the motion vector direction and the motion vector distance of the interested area of each frame of image.
For example, in the roadside parking management system S, a parking area in each camera video capture video frame image is predetermined; the method comprises the steps that video image information of a monitored area is obtained in real time through a plurality of video cameras of a video array camera, an AI intelligent host C obtains h264 code stream data collected by the plurality of video cameras in a video array in real time through an rtsp protocol, wherein the video resolution is 1080 × 1920, and the frame rate is 15 frames; then, based on the predetermined berth area in each video frame image of video acquisition of each video camera, the berth area in each video frame image of real-time acquisition is determined; determining coordinate information of a berthage area in each video frame image in a two-dimensional coordinate system of each video frame image, wherein in the two-dimensional coordinate system of each video frame image, a vertex at the upper left corner of each video frame image is an origin of the two-dimensional coordinate system of each video frame image, the vertex is a positive direction of an X axis from the origin to the right, and the vertex is a positive direction of a Y axis from the origin to the bottom; and then, extending the long side of the rectangular frame of the berthing area along the direction of gradually reducing the Y-axis value by a preset length, and combining the determined closed area formed by the width of the berthing area to obtain the region of interest of dense optical flow calculation of each video frame. The preset length of the long-edge expansion of the rectangular frame of the berth area can be in a field calibration mode, namely a 1.6-meter rod is erected on two vertexes of a short berth line at a position far away from the rectangular frame of the berth at a position far away from the shooting angle direction of the camera, meanwhile, a projection point of the top end of the rod body on an image is recorded, and the distance from the vertex of the berth line on the image to the projection point of the top end of the rod body is the expanded length. Then, the AI intelligent host C decodes each frame of image, and carries out dense optical flow calculation on the interested area in each decoded frame of image to obtain the motion vector direction and the motion vector distance of the interested area of each frame of image, wherein the AI intelligent host C adopts a self-contained multi-channel parallel hardware decoder to carry out real-time hardware decoding on the acquired h264 (digital video compression format) code stream data of each channel of video to obtain a decoded image frame; the AI intelligent host C decodes the h264 code stream data, and simultaneously circularly covers and caches each undecoded video frame image at a preset storage position according to a preset cache frequency; if the data is cached once according to the frequency of 30 minutes, caching the undecoded h264 original video stream data on a local memory of the AI intelligent host C in a file form, and circularly covering the content cached at the previous time when caching again; it should be noted that, although the predetermined length of the extension may be determined in a specific manner in the embodiment of the present invention, the predetermined length of the extension may also be determined according to actual working experience, and is not limited herein.
By the method and the device, the region of interest most suitable for calculation can be determined, so that important precondition guarantee is provided for subsequent image analysis and calculation, and further, the region of interest most suitable for calculation can be determined, so that the calculation capability of the intelligent host can be exerted to the maximum extent, and unnecessary calculation is reduced.
In a possible implementation manner, performing dense optical flow calculation on the region of interest in each decoded frame image to obtain the motion vector direction and the motion vector distance of the region of interest in each frame image, includes: for each frame image, averagely dividing the region of interest of the decoded current frame into a predetermined number of calculation regions; carrying out dense optical flow calculation on each calculation area to obtain the motion vector direction and the motion vector distance of each calculation area; and averaging the motion vector direction and the motion vector distance of each calculation area to obtain the motion vector direction and the motion vector distance of the current frame region of interest.
For example, in the roadside parking management system S, first, video image information of a monitored area is acquired in real time by a plurality of video cameras of a video array camera and the AI intelligent host C, and a parking area in each video frame image is determined; subsequently, expanding the berthage area in each video frame image to obtain an interested area calculated by the dense optical flow of each video frame; then, for each frame image, equally dividing the Region Of Interest Of the decoded current frame into a predetermined number Of calculation regions, for example, dividing the Region Of Interest Of the current video frame into a plurality Of calculation regions by 4 × 4 pixel blocks, and performing dense optical flow calculation on the calculation Region Of each 4 × 4 pixel block to obtain a motion vector direction and a motion vector distance Of each calculation Region Of the current video frame, wherein each video frame includes a motion vector (u, v) Of each 4 × 4 pixel block calculation Region in a Region Of Interest (ROI), where u represents a horizontal component Of the vector and v represents a vertical component Of the vector; then, the motion vector direction and the motion vector distance of each calculation area are averaged, and according to the motion vector (U, V) of each block in the optical flow vector frame, the average motion vector (U ', V') of the current frame interesting area is calculated, the motion vector distance average of the current frame interesting area is the module of the vector (U ', V'), and the motion direction of the motion vector of the current frame interesting area is the included angle between the vector (U ', V') and the image x direction.
By the method and the device, the motion vector direction and the motion vector distance of the current frame can be accurately obtained, so that necessary preconditions are provided for subsequently determining the suspected vehicle of the departure event.
In a possible implementation manner, the step 105 of determining, according to the motion vector direction average value and the motion vector distance average value, a vehicle suspected of having an exit/entry event in the parking area includes: judging whether the motion vector distance average value is larger than a preset pixel threshold value or not, and if so, determining that vehicles suspected of generating entrance events exist in the berthing area; determining the motion direction of the motion vector direction average value in a two-dimensional coordinate system of any video frame image of each video image; if the motion direction of the motion vector direction average value points to a parking area, determining that the vehicle is suspected to have an entrance event; and if the motion vector direction average value motion direction points to the outside of the parking area, determining that the vehicle is suspected to have a departure event.
For example, as shown in the above example, in the roadside parking management system S, according to the obtained motion vector direction average value and motion vector distance average value of each frame, the motion vector direction average value and motion vector distance average value of the region of interest of each frame image are calculated in a statistical interval of a predetermined duration through dense optical flow, for example, 20 seconds is a statistical interval, and then, whether the motion vector distance average value is greater than a predetermined pixel threshold value is judged, and if so, it is determined that a vehicle suspected of generating an entry event exists in the parking area; then, in a two-dimensional coordinate system of any video frame image of each video image, determining the motion direction of the motion vector direction average value; and if the motion direction of the motion vector direction average value points to the parking area, determining that the suspected vehicle is suspected to have an entrance event.
By the embodiment, a large number of video frame images which are not necessarily calculated are filtered, and the calculation amount for subsequently analyzing the video images to determine whether the suspected vehicle has the event of departure/entry is greatly reduced, so that the calculation capacity utilization rate of the intelligent host is improved, necessary preconditions are provided for simultaneously processing multiple paths of video information, the situation that identification errors are caused by errors of a small number of frames is avoided, and the identification accuracy is further improved.
In a possible implementation manner, the step 106, based on the video image information, of identifying vehicle information of a vehicle suspected to have a departure/entry event, and determining whether the vehicle has a departure/entry event according to the vehicle information, includes: determining the starting and ending time of the suspected departure/entrance event of the vehicle, and analyzing the vehicle information of the vehicle in the video image information to be identified within the starting and ending time; and detecting and tracking the vehicle in the video image information to be identified according to the vehicle information, and determining whether the vehicle has a departure/entrance event.
For example, in the roadside parking management system S, after it is determined that an entrance event is suspected to occur in a vehicle, a video clip of the start-stop time of the exit/entrance event is intercepted from video information cached in a predetermined storage location, and then vehicle information of the vehicle in the video clip within the start-stop time, including vehicle information such as a license plate number, a vehicle type, and a vehicle body color, is analyzed, and the vehicle is detected and tracked according to the vehicle information of the vehicle, so as to determine whether the exit/entrance event occurs in the vehicle. And then, the relevant information of the vehicle in/out event is sent to a cloud server, the cloud server generates parking evidence obtaining information comprising vehicle feature pictures, license plate numbers, in/out time and the like, and the parking evidence obtaining information is uploaded to a service platform, so that the service platform can more closely finish the processing of the whole parking service.
Through the embodiment, the requirements on vehicle detection tracking and high real-time performance of a license plate algorithm are greatly reduced, meanwhile, the entrance and exit events of the vehicles can be efficiently and accurately determined based on the filtered video frame images, and the efficiency of parking management is greatly improved.
The embodiment of the present invention provides a device for managing roadside parking, which can implement the method embodiment provided above, and for specific function implementation, please refer to the description in the method embodiment, which is not described herein again.
It should be understood that the specific order or hierarchy of steps in the processes disclosed is an example of exemplary approaches. Based upon design preferences, it is understood that the specific order or hierarchy of steps in the processes may be rearranged without departing from the scope of the present disclosure. The accompanying method claims present elements of the various steps in a sample order, and are not intended to be limited to the specific order or hierarchy presented.
In the foregoing detailed description, various features are grouped together in a single embodiment for the purpose of streamlining the disclosure. This method of disclosure is not to be interpreted as reflecting an intention that the claimed embodiments of the subject matter require more features than are expressly recited in each claim. Rather, as the following claims reflect, invention lies in less than all features of a single disclosed embodiment. Thus, the following claims are hereby expressly incorporated into the detailed description, with each claim standing on its own as a separate preferred embodiment of the invention.
The previous description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the present invention. To those skilled in the art; various modifications to these embodiments will be readily apparent, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the disclosure. Thus, the present disclosure is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
What has been described above includes examples of one or more embodiments. It is, of course, not possible to describe every conceivable combination of components or methodologies for purposes of describing the aforementioned embodiments, but one of ordinary skill in the art may recognize that many further combinations and permutations of various embodiments are possible. Accordingly, the embodiments described herein are intended to embrace all such alterations, modifications and variations that fall within the scope of the appended claims. Furthermore, to the extent that the term "includes" is used in either the detailed description or the claims, such term is intended to be inclusive in a manner similar to the term "comprising" as "comprising" is interpreted when employed as a transitional word in a claim. Furthermore, any use of the term "or" in the specification of the claims is intended to mean a "non-exclusive or".
Those of skill in the art will also appreciate that the various illustrative logical blocks, elements, and steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or combinations of both. To clearly illustrate the interchangeability of hardware and software, various illustrative components, elements, and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as hardware or software depends upon the particular application and design requirements of the overall system. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present embodiments.
The various illustrative logical blocks, or elements, described in connection with the embodiments disclosed herein may be implemented or performed with a general purpose processor, a digital signal processor, an Application Specific Integrated Circuit (ASIC), a field programmable gate array or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination thereof designed to perform the functions described herein. A general-purpose processor may be a microprocessor, but in the alternative, the processor may be any conventional processor, controller, microcontroller, or state machine. A processor may also be implemented as a combination of computing devices, e.g., a digital signal processor and a microprocessor, a plurality of microprocessors, one or more microprocessors in conjunction with a digital signal processor core, or any other similar configuration.
The steps of a method or algorithm described in connection with the embodiments disclosed herein may be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. A software module may be stored in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art. For example, a storage medium may be coupled to the processor such the processor can read information from, and write information to, the storage medium. In the alternative, the storage medium may be integral to the processor. The processor and the storage medium may reside in an ASIC, which may be located in a user terminal. In the alternative, the processor and the storage medium may reside in different components in a user terminal.
In one or more exemplary designs, the functions described above in connection with the embodiments of the invention may be implemented in hardware, software, firmware, or any combination of the three. If implemented in software, the functions may be stored on or transmitted over as one or more instructions or code on a computer-readable medium. Computer-readable media includes both computer storage media and communication media that facilitate transfer of a computer program from one place to another. Storage media may be any available media that can be accessed by a general purpose or special purpose computer. For example, such computer-readable media can include, but is not limited to, RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to carry or store program code in the form of instructions or data structures and which can be read by a general-purpose or special-purpose computer, or a general-purpose or special-purpose processor. Additionally, any connection is properly termed a computer-readable medium, and, thus, is included if the software is transmitted from a website, server, or other remote source via a coaxial cable, fiber optic cable, twisted pair, Digital Subscriber Line (DSL), or wirelessly, e.g., infrared, radio, and microwave. Such discs (disk) and disks (disc) include compact disks, laser disks, optical disks, DVDs, floppy disks and blu-ray disks where disks usually reproduce data magnetically, while disks usually reproduce data optically with lasers. Combinations of the above may also be included in the computer-readable medium.
The above-mentioned embodiments, objects, technical solutions and advantages of the present invention are further described in detail, it should be understood that the above-mentioned embodiments are only examples of the present invention, and are not intended to limit the scope of the present invention, and any modifications, equivalent substitutions, improvements and the like made within the spirit and principle of the present invention should be included in the scope of the present invention.
Claims (12)
1. A method of managing roadside parking, comprising:
acquiring video image information acquired by a plurality of cameras in real time, and determining a berthing area in each video frame image;
expanding the berthage area in each video frame image to obtain the interesting area calculated by the dense optical flow of each video frame;
carrying out dense optical flow calculation on the interested area to obtain the motion vector direction and the motion vector distance of the interested area of each frame of image;
calculating the motion vector direction average value and the motion vector distance average value of the interested area of each frame image in a statistical interval of preset duration through dense optical flow according to the obtained motion vector direction and motion vector distance of the interested area of each frame image;
judging whether the motion vector distance average value is larger than a preset pixel threshold value or not, and if so, determining that vehicles suspected of generating entrance events exist in the berth area; determining the motion direction of the average value of the motion vector directions in a two-dimensional coordinate system of any video frame image of each video image; if the motion direction of the motion vector direction average value points to a parking area, determining that the vehicle is suspected to have an entrance event; if the motion vector direction average value motion direction points to the outside of the parking area, determining that the vehicle is suspected to have a departure event;
and identifying vehicle information of a vehicle suspected to have a departure/entrance event based on the video image information, and determining whether the vehicle has a departure/entrance event according to the vehicle information.
2. The method of claim 1, wherein prior to the step of obtaining video image information captured by a plurality of cameras in real time and determining the berthing area in each video frame image, the method comprises:
determining coordinates of each vertex of the vehicle berth in a preset image acquisition area in each image;
and determining the berthing area in the video frame image acquired by each video camera according to the coordinates.
3. The method according to claim 1 or 2, wherein the expanding the berthing areas in the images of each video frame to obtain the interest areas of the dense optical flow calculation of each video frame comprises:
in a two-dimensional coordinate system of each video frame image, determining coordinate information of a berthage area in each video frame image, and extending the long side of a rectangular frame of the berthage area for a preset length along the direction of gradually reducing the Y-axis value to obtain an interesting area calculated by the dense optical flow of each video frame.
4. The method of claim 3, wherein said performing dense optical flow calculations on said region of interest to obtain motion vector direction and motion vector distance for each frame of image region of interest comprises:
and decoding each frame of image, and performing dense optical flow calculation on the interested area in each decoded frame of image to obtain the motion vector direction and the motion vector distance of the interested area of each frame of image.
5. The method according to claim 4, wherein the performing dense optical flow calculation on the regions of interest in the decoded images to obtain the motion vector direction and the motion vector distance of the regions of interest in each image frame comprises:
for each frame image, averagely dividing the region of interest of the decoded current frame into a predetermined number of calculation regions;
carrying out dense optical flow calculation on each calculation area to obtain the motion vector direction and the motion vector distance of each calculation area;
and averaging the motion vector direction and the motion vector distance of each calculation area to obtain the motion vector direction and the motion vector distance of the current frame region of interest.
6. The method of claim 1, wherein the identifying vehicle information of a vehicle suspected of having a departure/entry event based on the video image information and determining whether the vehicle has a departure/entry event according to the vehicle information comprises:
determining the starting and ending time of the suspected departure/entrance event of the vehicle, and analyzing the vehicle information of the vehicle in the video image information to be identified within the starting and ending time;
and detecting and tracking the vehicle in the video image information to be identified according to the vehicle information, and determining whether the vehicle has a departure/entry event.
7. A roadside parking management device, comprising:
the acquisition and determination module is used for acquiring video image information acquired by a plurality of cameras in real time and determining a berthing area in each video frame image;
the expansion module is used for expanding the berthage area in each video frame image to obtain an interested area calculated by the dense optical flow of each video frame;
the first calculation module is used for carrying out dense optical flow calculation on the interested area to obtain the motion vector direction and the motion vector distance of the interested area of each frame of image;
the second calculation module is used for calculating the motion vector direction average value and the motion vector distance average value of each frame of image interesting region in a statistical interval of preset duration through dense optical flow according to the obtained motion vector direction and motion vector distance of each frame of image interesting region;
the first determining module is used for judging whether the motion vector distance average value is larger than a preset pixel threshold value or not, and if so, determining that a vehicle suspected of having an entrance event exists in the berth area; determining the motion direction of the average value of the motion vector directions in a two-dimensional coordinate system of any video frame image of each video image; if the motion direction of the motion vector direction average value points to a parking area, determining that the vehicle is suspected to have an entrance event; if the motion vector direction average value motion direction points to the outside of the parking area, determining that the vehicle is suspected to have a departure event;
and the identification and determination module is used for identifying the vehicle information of the vehicle suspected to have the exit/entrance event based on the video image information and determining whether the vehicle has the exit/entrance event according to the vehicle information.
8. The apparatus of claim 7, comprising:
the second determining module is used for determining the coordinates of each vertex of the vehicle berth in the preset image acquisition area in each image;
and the third determining module is used for determining the berthing area in the video frame image acquired by each camera according to the coordinates.
9. Device according to claim 7 or 8, characterized in that said expansion module is, in particular, intended for
In a two-dimensional coordinate system of each video frame image, determining coordinate information of a berthage area in each video frame image, and extending the long side of a rectangular frame of the berthage area for a preset length along the direction of gradually reducing the Y-axis value to obtain an interesting area calculated by the dense optical flow of each video frame.
10. The apparatus of claim 9, wherein the first computing module comprises:
and the decoding unit is used for decoding each frame of image and carrying out dense optical flow calculation on the interested area in each frame of image after decoding to obtain the motion vector direction and the motion vector distance of the interested area of each frame of image.
11. Device according to claim 10, wherein the decoding unit is specifically configured to
For each frame image, averagely dividing the region of interest of the decoded current frame into a predetermined number of calculation regions;
carrying out dense optical flow calculation on each calculation area to obtain the motion vector direction and the motion vector distance of each calculation area;
and averaging the motion vector direction and the motion vector distance of each calculation area to obtain the motion vector direction and the motion vector distance of the current frame region of interest.
12. Device according to claim 7, characterized in that said identification and determination module is particularly adapted to
Determining the starting and ending time of the suspected departure/entrance event of the vehicle, and analyzing the vehicle information of the vehicle in the video image information to be identified within the starting and ending time;
and detecting and tracking the vehicle in the video image information to be identified according to the vehicle information, and determining whether the vehicle has a departure/entrance event.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110317344.5A CN113066306B (en) | 2021-03-23 | 2021-03-23 | Management method and device for roadside parking |
| PCT/CN2021/112931 WO2022198897A1 (en) | 2021-03-23 | 2021-08-17 | Management method and device for on-street parking |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110317344.5A CN113066306B (en) | 2021-03-23 | 2021-03-23 | Management method and device for roadside parking |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113066306A CN113066306A (en) | 2021-07-02 |
| CN113066306B true CN113066306B (en) | 2022-07-08 |
Family
ID=76561865
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110317344.5A Active CN113066306B (en) | 2021-03-23 | 2021-03-23 | Management method and device for roadside parking |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN113066306B (en) |
| WO (1) | WO2022198897A1 (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113066306B (en) * | 2021-03-23 | 2022-07-08 | 超级视线科技有限公司 | Management method and device for roadside parking |
| CN114038232B (en) * | 2021-10-28 | 2022-09-20 | 超级视线科技有限公司 | Roadside parking management method and system based on edge end calculation and storage combination |
| CN114005068A (en) * | 2021-11-08 | 2022-02-01 | 支付宝(杭州)信息技术有限公司 | Method and device for monitoring movement of goods |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107730975A (en) * | 2017-09-13 | 2018-02-23 | 浙江大学 | Supermarket's stopping guide reverse car seeking and the system and method for the guiding that appears on the scene |
| EP3561775A1 (en) * | 2018-04-26 | 2019-10-30 | Volvo Car Corporation | Methods and systems for semi-automated image segmentation and annotation |
Family Cites Families (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9129524B2 (en) * | 2012-03-29 | 2015-09-08 | Xerox Corporation | Method of determining parking lot occupancy from digital camera images |
| JP2014110028A (en) * | 2012-12-04 | 2014-06-12 | Sony Corp | Image processing apparatus, image processing method, and program |
| CN104658249A (en) * | 2013-11-22 | 2015-05-27 | 上海宝康电子控制工程有限公司 | Method for rapidly detecting vehicle based on frame difference and light stream |
| CN104376741B (en) * | 2014-12-02 | 2017-03-22 | 深圳市捷顺科技实业股份有限公司 | Parking lot state detection method and system |
| CN107767673B (en) * | 2017-11-16 | 2019-09-27 | 智慧互通科技有限公司 | A kind of Roadside Parking management method based on multiple-camera, apparatus and system |
| CN109684996B (en) * | 2018-12-22 | 2020-12-04 | 北京工业大学 | Real-time vehicle access identification method based on video |
| CN112037265B (en) * | 2020-11-04 | 2021-02-02 | 天津天瞳威势电子科技有限公司 | Library bit tracking method and device |
| CN113066306B (en) * | 2021-03-23 | 2022-07-08 | 超级视线科技有限公司 | Management method and device for roadside parking |
-
2021
- 2021-03-23 CN CN202110317344.5A patent/CN113066306B/en active Active
- 2021-08-17 WO PCT/CN2021/112931 patent/WO2022198897A1/en unknown
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107730975A (en) * | 2017-09-13 | 2018-02-23 | 浙江大学 | Supermarket's stopping guide reverse car seeking and the system and method for the guiding that appears on the scene |
| EP3561775A1 (en) * | 2018-04-26 | 2019-10-30 | Volvo Car Corporation | Methods and systems for semi-automated image segmentation and annotation |
Non-Patent Citations (2)
| Title |
|---|
| 基于运动检测的多车辆跟踪方法研究;单玉刚等;《计算机测量与控制》;20170325(第03期);全文 * |
| 稀疏光流快速计算的动态目标检测与跟踪;陈添丁等;《中国图象图形学报》;20131216(第12期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| WO2022198897A1 (en) | 2022-09-29 |
| CN113066306A (en) | 2021-07-02 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113066306B (en) | Management method and device for roadside parking | |
| CN111368687B (en) | Sidewalk vehicle illegal parking detection method based on target detection and semantic segmentation | |
| Grassi et al. | Parkmaster: An in-vehicle, edge-based video analytics service for detecting open parking spaces in urban environments | |
| CN110491168B (en) | Method and device for detecting vehicle parking state based on wheel landing position | |
| CN111739335B (en) | Parking detection method and device based on visual difference | |
| GB2507395A (en) | Estimating vehicle speed from video stream motion vectors | |
| KR102253989B1 (en) | object tracking method for CCTV video by use of Deep Learning object detector | |
| CN111259868B (en) | Reverse vehicle detection method, system and medium based on convolutional neural network | |
| Laureshyn et al. | Application of automated video analysis for behavioural studies: concept and experience | |
| JP6915219B2 (en) | Computer implementation methods, imaging systems, and image processing systems | |
| CN112861773A (en) | Multi-level-based berthing state detection method and system | |
| CN113205691A (en) | Method and device for identifying vehicle position | |
| CN110880205B (en) | Parking charging method and device | |
| CN113450575B (en) | Management method and device for roadside parking | |
| Li et al. | Intelligent transportation video tracking technology based on computer and image processing technology | |
| CN113869258A (en) | Traffic incident detection method and device, electronic equipment and readable storage medium | |
| Mossi et al. | Real-time traffic analysis at night-time | |
| Zheng et al. | Detecting cycle failures at signalized intersections using video image processing | |
| KR101161557B1 (en) | The apparatus and method of moving object tracking with shadow removal moudule in camera position and time | |
| Bravo et al. | Outdoor vacant parking space detector for improving mobility in smart cities | |
| Yao et al. | Embedded technology and algorithm for video-based vehicle queue length detection | |
| CN115174889A (en) | Position deviation detection method for camera, electronic device, and storage medium | |
| CN113808408A (en) | Roadside parking detection method, device and equipment and readable storage medium | |
| CN114463976A (en) | Vehicle behavior state determination method and system based on 3D vehicle track | |
| CN113449624A (en) | Method and device for determining vehicle behavior based on pedestrian re-recognition |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |