WO2022206974A1 - Roadside sensing unit-based static and non-static object point cloud recognition method - Google Patents

Roadside sensing unit-based static and non-static object point cloud recognition method Download PDF

Info

Publication number
WO2022206974A1
WO2022206974A1 PCT/CN2022/084912 CN2022084912W WO2022206974A1 WO 2022206974 A1 WO2022206974 A1 WO 2022206974A1 CN 2022084912 W CN2022084912 W CN 2022084912W WO 2022206974 A1 WO2022206974 A1 WO 2022206974A1
Authority
WO
WIPO (PCT)
Prior art keywords
static
point cloud
area
data
objects
Prior art date
Application number
PCT/CN2022/084912
Other languages
French (fr)
Chinese (zh)
Inventor
许军
赵聪
魏斯瑀
陆日琪
Original Assignee
许军
马儒争
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 许军, 马儒争 filed Critical 许军
Priority to CN202280026656.3A priority Critical patent/CN117836667A/en
Publication of WO2022206974A1 publication Critical patent/WO2022206974A1/en

Links

Images

Classifications

    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S7/00—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00
    • G01S7/02—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00 of systems according to group G01S13/00
    • G01S7/40—Means for monitoring or calibrating
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S17/00—Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
    • G01S17/87—Combinations of systems using electromagnetic waves other than radio waves
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S17/00—Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
    • G01S17/88—Lidar systems specially adapted for specific applications
    • G01S17/89—Lidar systems specially adapted for specific applications for mapping or imaging
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S17/00—Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
    • G01S17/88—Lidar systems specially adapted for specific applications
    • G01S17/93—Lidar systems specially adapted for specific applications for anti-collision purposes
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S17/00—Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
    • G01S17/88—Lidar systems specially adapted for specific applications
    • G01S17/93—Lidar systems specially adapted for specific applications for anti-collision purposes
    • G01S17/931—Lidar systems specially adapted for specific applications for anti-collision purposes of land vehicles
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S7/00—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00
    • G01S7/003—Transmission of data between radar, sonar or lidar systems and remote stations
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S7/00—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00
    • G01S7/48—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00 of systems according to group G01S17/00
    • G01S7/4808—Evaluating distance, position or velocity data
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S7/00—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00
    • G01S7/48—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00 of systems according to group G01S17/00
    • G01S7/497—Means for monitoring or calibrating
    • G01S7/4972—Alignment of sensor
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
    • G—PHYSICS
    • G08—SIGNALLING
    • G08G—TRAFFIC CONTROL SYSTEMS
    • G08G1/00—Traffic control systems for road vehicles
    • G08G1/01—Detecting movement of traffic to be counted or controlled
    • G08G1/0104—Measuring and analyzing of parameters relative to traffic conditions
    • G08G1/0108—Measuring and analyzing of parameters relative to traffic conditions based on the source of data
    • G08G1/0116—Measuring and analyzing of parameters relative to traffic conditions based on the source of data from roadside infrastructure, e.g. beacons
    • G—PHYSICS
    • G08—SIGNALLING
    • G08G—TRAFFIC CONTROL SYSTEMS
    • G08G1/00—Traffic control systems for road vehicles
    • G08G1/01—Detecting movement of traffic to be counted or controlled
    • G08G1/0104—Measuring and analyzing of parameters relative to traffic conditions
    • G08G1/0125—Traffic data processing
    • G08G1/0133—Traffic data processing for classifying traffic situation
    • G—PHYSICS
    • G08—SIGNALLING
    • G08G—TRAFFIC CONTROL SYSTEMS
    • G08G1/00—Traffic control systems for road vehicles
    • G08G1/01—Detecting movement of traffic to be counted or controlled
    • G08G1/0104—Measuring and analyzing of parameters relative to traffic conditions
    • G08G1/0137—Measuring and analyzing of parameters relative to traffic conditions for specific applications
    • G08G1/0141—Measuring and analyzing of parameters relative to traffic conditions for specific applications for traffic information dissemination
    • G—PHYSICS
    • G08—SIGNALLING
    • G08G—TRAFFIC CONTROL SYSTEMS
    • G08G1/00—Traffic control systems for road vehicles
    • G08G1/01—Detecting movement of traffic to be counted or controlled
    • G08G1/0104—Measuring and analyzing of parameters relative to traffic conditions
    • G08G1/0137—Measuring and analyzing of parameters relative to traffic conditions for specific applications
    • G08G1/0145—Measuring and analyzing of parameters relative to traffic conditions for specific applications for active traffic flow control
    • G—PHYSICS
    • G08—SIGNALLING
    • G08G—TRAFFIC CONTROL SYSTEMS
    • G08G1/00—Traffic control systems for road vehicles
    • G08G1/01—Detecting movement of traffic to be counted or controlled
    • G08G1/04—Detecting movement of traffic to be counted or controlled using optical or ultrasonic detectors
    • G—PHYSICS
    • G08—SIGNALLING
    • G08G—TRAFFIC CONTROL SYSTEMS
    • G08G1/00—Traffic control systems for road vehicles
    • G08G1/01—Detecting movement of traffic to be counted or controlled
    • G08G1/048—Detecting movement of traffic to be counted or controlled with provision for compensation of environmental or other condition, e.g. snow, vehicle stopped at detector
    • G—PHYSICS
    • G08—SIGNALLING
    • G08G—TRAFFIC CONTROL SYSTEMS
    • G08G1/00—Traffic control systems for road vehicles
    • G08G1/16—Anti-collision systems
    • G08G1/164—Centralised systems, e.g. external to vehicles
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44—Arrangements for executing specific programs
    • G06F9/451—Execution arrangements for user interfaces
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/194—Segmentation; Edge detection involving foreground-background segmentation

Definitions

  • the invention belongs to the technical field of data processing, and in particular relates to a point cloud recognition method for static and non-static objects based on a roadside perception unit, which is mainly oriented to target perception on the infrastructure side in a vehicle-road coordination environment.
  • the car As the carrier of software and hardware, through the on-board sensors, decision-making units and actuators and other electronic equipment, it provides intelligent support for the car, so that the car can make driving behavior decisions based on the surrounding environment, so as to avoid the traffic caused by the uneven personal quality of drivers. risk, to achieve the purpose of improving vehicle safety.
  • the pace of autonomous driving and unmanned driving technology from laboratory to practical application is accelerating.
  • the basic idea of the vehicle-road coordination system is to use the method of multi-disciplinary crossover and integration, use advanced wireless communication technology and sensing technology to obtain real-time vehicle and road information, and realize vehicle-to-vehicle communication through vehicle-vehicle communication and vehicle-road communication.
  • Information exchange and sharing between vehicles and intelligent roadside facilities on the road to achieve intelligent collaboration between vehicles and vehicles, and between vehicles and roads, thereby improving road traffic safety, road traffic efficiency, and road traffic system resource utilization.
  • the vehicle-road coordination system can be divided into two subsystems: the intelligent roadside system and the intelligent vehicle-mounted system.
  • the intelligent roadside system is mainly responsible for the acquisition and release of traffic flow information, roadside equipment control, vehicle-road communication, traffic management and control; Vehicle-to-vehicle communication/vehicle-to-road communication, safety warning and vehicle auxiliary control tasks.
  • the intelligent roadside system and the intelligent in-vehicle system transmit and share information between the two parties through vehicle-road communication, thereby realizing data interaction, broadening the perception range and data volume of autonomous driving vehicles, enhancing the decision-making basis of autonomous driving vehicles, and improving driving safety. .
  • Roadside sensing devices are more generous than in-vehicle sensing devices, their installation positions are relatively free, and hard conditions such as energy supply are more tolerant.
  • Common roadside sensing devices include: 1. toroidal coil; 2. millimeter-wave radar; 3. UWB technology; 4. visual method; 5. lidar, etc.
  • the technologies that can be used as engineering solutions are mainly visual detection methods and lidar technology. Both have the characteristics of simple and easy-to-understand data form, and relatively mature target detection technology, but in comparison, if roadside visual data is to serve the vehicle end, the transmitted data must be the detection result, because the viewing angle is quite different.
  • the pre-fusion method even if the pre-fusion method is more favorable, it still has limitations.
  • the biggest limitation of its application is that the amount of data transmission is larger, because the transmitted data is raw data, and the size of one frame of general point cloud data is in Between a few megabytes and a dozen megabytes, that is, the size of the data transmitted one-to-one per second can be as high as several hundred megabytes.
  • the total data transmission volume per second can even be as high as several gigabytes. The size of the transmitted data must be reduced.
  • An effective method suitable for automatic driving scenarios is to extract the key parts of the data.
  • the surrounding vehicles, non- Objects such as motor vehicles, pedestrians, and roadblocks are far more important than objects such as roads, green plants, and buildings on both sides. Therefore, low-value objects can be identified and screened in the original data to achieve the purpose of reducing the amount of data. .
  • the use value of the remaining objects will naturally increase again, because after eliminating the interference factors, the existence of important objects is more prominent and the influence of low-value data is avoided.
  • Roadside perception unit In the vehicle-road collaboration scenario, the roadside pole or gantry is used as the installation base, and the environment perception device is arranged around the road.
  • the roadside sensing unit refers specifically to the lidar system, and the two should be regarded as the same description.
  • Point cloud data a set of vectors in a three-dimensional coordinate system, these vectors usually include at least X, Y, Z three-dimensional coordinate data, used to represent the outer surface shape of the object, and sometimes other information can be obtained depending on the device .
  • the symbol D is usually used to represent the point cloud data and the processed partial data. Any content expressed by the symbol D should be understood as point cloud data.
  • Static objects D s mainly the road surface and its ancillary facilities, buildings and roadside green plants, etc., without considering reconstruction and expansion, frequent road maintenance, such objects are in a state of long-term fixed position and no change in appearance, it can be considered that Objects with no obvious changes in position and appearance within one month are static objects.
  • the basis for judging whether it is an obvious change is that the centroid deviation of the horizontal projection of the object exceeds 1 meter, which is regarded as a significant change, or the change in the side length or volume of the outer silhouette exceeds 5% of the original data, which is regarded as a significant change.
  • Short-term static objects D st mainly include temporary parking, standing pedestrians, etc. Such objects are in a short-term position and state without change, but the possibility of their next moment movement is not excluded. In the present invention, it is considered that they do not belong to static objects and Objects that do not have obvious position changes and appearance changes within 5 frames are short-term static objects.
  • the basis for judging the obvious change is: the centroid deviation of the horizontal projection of the object exceeds 0.3m, which is regarded as a significant change, or the change in the side length or volume of the outer silhouette exceeds 5% of the original data, which is regarded as a significant change.
  • Dynamic objects D d mainly include moving vehicles, walking pedestrians, etc. Such objects are in motion when they are observed, and can be considered not static objects and have obvious position changes or appearance changes in 2 consecutive frames. dynamic objects.
  • the basis for judging the obvious change is: the centroid deviation of the horizontal projection of the object exceeds 0.3m, which is regarded as a significant change, or the change in the side length or volume of the outer silhouette exceeds 5% of the original data, which is regarded as a significant change.
  • Non-static objects D ns the sum of short-term static objects and dynamic objects.
  • Original data D 0 the point cloud data set used in the preprocessing part of the present invention, generally contains about 1000 frames of point cloud data, and is required to include most common traffic scenes in the road section where the road test sensing unit is installed.
  • Point cloud data to be identified different from the aforementioned original data D 0 , it is the point cloud data frame actually used for identification during the use of the present invention, and the identification result can be used to support subsequent point cloud target detection and other work, in the description Usually recorded as the i-th frame data D i .
  • the area including the road surface that is emphatically identified by the method of the present invention is usually a road area that can be clearly identified and is the main traffic route within the scanning range of the lidar.
  • Data value When point cloud data is used by autonomous vehicles, how much influence it has on driving decisions. In the present invention, the judgment is based on the danger that the object may cause to the autonomous vehicle. Without considering the ability of the autonomous driving system to understand data, it is generally considered that the data value of dynamic objects is the greatest, followed by short-term static objects, and static objects. Object is the smallest.
  • Invalid data also known as invalid point cloud data; it refers to point cloud data with almost no data value, usually including buildings, slopes and open spaces on both sides of the road, generally point cloud data 5m away from the edge of the road.
  • the division range of invalid data can be adjusted according to identification needs. For example, in road scenes with roadside guardrails such as urban expressways, the point cloud data outside the road guardrail can be regarded as invalid data.
  • Valid data also known as valid point cloud data; the point cloud data remaining after the invalid data is eliminated from the point cloud data, in the expression of the present invention, the operation of extracting valid data is indicated by adding a single quotation mark (').
  • Boundary equation system E b The function boundary used to separate invalid data and valid data. After projecting the point cloud data into a bird's-eye view, the boundary points are manually selected and established by least squares fitting.
  • Static point cloud background B a purely static background space that does not contain any short-term static objects and dynamic objects. For the vehicle-road collaboration scene targeted by the present invention, it does not contain any non-permanently parked vehicles, non-motor vehicles, pedestrians and other traffic Participant's traffic environment.
  • Valid point cloud data to be identified After cutting the point cloud data to be identified using the boundary equation set Eb, the obtained point cloud data set is the valid point cloud data to be identified, which is denoted as valid i-th frame data D i ' in the description.
  • the point cloud data has the characteristics of being dense in the vicinity and sparse in the distance. In order to avoid this characteristic affecting the subsequent identification work, the point cloud data is divided into multiple statistical spaces, and the superposition and downsampling operations are used to make the point clouds everywhere. The densities are roughly equal.
  • Point cloud density an indicator used to describe the density of point clouds, which is represented by the number of point clouds per unit volume. The specific calculation method and method parameters can be determined according to the equipment conditions.
  • Scanning distance L refers to the distance between the point cloud and the center of the lidar, which can be represented by the plane distance after projecting the point cloud data into a bird's-eye view.
  • Point cloud target detection algorithm refers to the target detection algorithm used to detect point cloud data as a specific category (such as large vehicles, small vehicles, pedestrians, etc.), and its function is different from the target recognition method described in this paper. The methods described above only identify specific point cloud datasets without detecting their specific categories.
  • Detection confidence P Input the point cloud data that can characterize an object into the point cloud target detection algorithm, and get the confidence of the output result. In particular, if the point cloud target detection algorithm does not detect the result, the detection confidence is regarded as 0.
  • Recognition Trigger Distance Threshold DT The perceived distance range that enables most point cloud object detection algorithms to perform well. In the present invention, good performance of the point cloud target detection algorithm is defined as all non-static objects in the area can be detected, and the detection confidence is not lower than 75%.
  • Point cloud data for identification the result of trimming the point cloud data by using the identification trigger distance threshold, in the description of the present invention, double quotation marks (") are used to indicate the operation of extracting point cloud data for identification.
  • Recognition point cloud data to be recognized After cutting the valid point cloud data to be recognized by the recognition trigger distance threshold DT, the obtained point cloud data set is the point cloud data to be recognized for recognition, which is recorded as the first point cloud data for recognition in the description. i-frame data Di".
  • Voxel Abbreviation for Volume Pixel, similar to the definition of pixel in two-dimensional space, it is the smallest unit in three-dimensional space division, and it is expressed in the form of a space cube. The side length of the cube can be established manually.
  • the described models vary in granularity.
  • Voxelization The operation of converting point cloud data into voxels, which is represented by the subscript v in the expression of the present invention.
  • Voxelized point cloud data to be recognized After the point cloud data for identification to be identified is subjected to voxelization operation, the obtained point cloud data set is the voxelized point cloud data to be identified, which is recorded as voxelization in the description The i-th frame data Dv".
  • Origin of point cloud coordinate system point cloud data is usually represented in the form of three-dimensional coordinates.
  • the origin of the coordinate system of the point cloud data is recorded as the origin of the point cloud coordinate system.
  • Circular spare identification area In order to avoid incomplete static or non-static objects due to the aforementioned cropping operation, a circular spare identification area is added to the outside of the recognition trigger distance threshold, and is divided into multiple sub-areas at a fixed angle; Identify the position of the trigger distance threshold edge and identify the non-static area, record the horizontal angle between the non-static area and the X-axis of the point cloud coordinate system, and add the annular spare identification sub-area corresponding to the angle to the non-static area.
  • Sliding window method Use a fixed-size data screening box to continuously filter out part of the sub-data from the original data along a certain direction, so that the operation is only applied to these sub-data, reducing the amount of data processing and strengthening the identification of local features .
  • Background difference method usually refers to a method of detecting non-static objects by comparing the current frame in the image sequence with the background reference model, which is applied to 3D point cloud data in the present invention.
  • Static area A s a spatial area containing multiple voxels, when the number, location, distribution and other characteristics of voxels in it are smaller than the determination threshold compared with the static point cloud background, it can be considered that the spatial area is within the The object or scene does not change, that is, the static area. Essentially a subset of static objects.
  • Non-static area A ns a spatial area containing multiple voxels, when the number, position, distribution and other characteristics of the voxels are compared with the static point cloud background, and the variation range is greater than the judgment threshold, it can be considered that the Objects or scenes in a spatial area have changed, that is, a non-static area.
  • Temporary static area A st a spatial area containing multiple voxels, which is formed by saving the non-static area at a fixed frequency.
  • the short-term static area is used to determine the dynamic object and the short-term static object, and the part after the dynamic object is separated is the short-term static object.
  • Dynamic area A d a spatial area containing multiple voxels, belonging to a non-static area, when the number, location, distribution and other characteristics of voxels in it are compared with the short-term static area, the variation range is larger than the judgment threshold , it can be considered that the object or scene in the space area has changed, that is, the dynamic area. Essentially a subset of dynamic objects.
  • the invention provides a method for fast identification of dynamic and static objects and point cloud segmentation based on a roadside perception unit, facing the real traffic environment, based on the relative fixed position of the roadside laser radar, and using the point cloud background data collected in the previous stage as a priori
  • the applicable scene of the present invention is not limited to a single radar environment, but can also be applied to a multi-radar network.
  • the extracted to-be-identified area can also be fused with other data formats to improve detection accuracy.
  • the collected point cloud data requires full coverage and no occlusion, and should include static objects, mainly the road surface and its ancillary facilities, buildings and roadside green plants, etc., and should also include enough non-static objects, such as pedestrians, non-motor vehicles and Vehicles, etc., can be identified manually, and it is recommended that the total number should not be less than 300.
  • the roadside lidar scene built should ensure that there is no large-area occlusion within the scanning range, that is, all key road areas within the scanning range should be clearly visible.
  • the left picture of Fig. 3 shows the bad distribution points that lose half-side road width data due to the influence of the central divider, and the right picture of Fig. 3 is a relatively good example picture.
  • the format of the point cloud data collected by the present invention is shown in the following table.
  • the point cloud data includes three-dimensional coordinate data X, Y, Z, reflection intensity value Intensity, three-channel color value RGB, and return number of echoes.
  • the present invention only uses the three-dimensional coordinate data X, Y, and Z as the point cloud data extraction basis, and selects the three-dimensional coordinate data, reflection intensity value and three-channel color value to be transmitted together when transmitting the screening result, so as to avoid the failure of the vehicle end due to the lack of information. Use this data.
  • the applicable point cloud data format of the present invention is not limited to the above examples, and any point cloud data including three-dimensional coordinate data X, Y, and Z can be regarded as the applicable scope of the present invention.
  • the data collection described in the present invention is divided into two stages: one is the data collection stage serving the preprocessing work: the collection work that provides the data source for the preprocessing work, and the data collected in this stage should conform to the items described below.
  • the purpose is to comprehensively reflect the normal road conditions of the installation scene of the drive test sensing unit; the second is the data collection stage for daily use: this stage has no detailed requirements for data collection, and only needs to ensure the normal operation of the roadside sensing unit.
  • the point cloud data sets collected in the preprocessing stage are collectively referred to as original data D 0
  • the point cloud data collected in the use stage are named D 1 , D 2 and so on in chronological order (number of frames). The following focuses on the data collection in the preprocessing stage.
  • the need for post-processing it is generally required to collect no less than 1000 frames of data, and the specific number of frames to be collected can be adjusted appropriately according to the layout scene of the roadside sensing unit.
  • the influence of environmental factors should be taken into account, such as the double decrease of road point cloud density and quality caused by the reduction of the number of lidar echoes caused by the wet ground in rainy days, or the high-brightness high beam of vehicles at night, which interferes with the normal operation of lidar. Echoes, resulting in abnormal point cloud quality in some areas, etc.
  • the intensity of visible light has little effect on lidar, for the sake of rigor, the collected data should be appropriately dispersed over multiple time periods.
  • the original data collection scheme proposed by the present invention is:
  • Applicable data collection schemes of the present invention include but are not limited to the above cases.
  • data collection arrangements should be made in combination with the traffic conditions within the scanning range of the road test lidar, and it should be ensured that the collected data contains no less than 50% of the single-frame data with fewer than 2 vehicles or pedestrians, and The total number of samples of non-static objects such as the number of vehicles or pedestrians is not less than 300.
  • the boundary equation system E b is established, and the invalid data is eliminated from the original data.
  • the linear relationship between point cloud density and scanning distance is obtained by sampling the point cloud data at equal intervals.
  • the point cloud target detection algorithm to detect the non-static objects to establish the distribution curve of detection confidence and scanning distance, and select the ones whose confidence is higher than the threshold.
  • the scanning distance range is used as the recognition trigger distance threshold.
  • use the recognition trigger distance threshold to crop all static objects, and build a static point cloud background B v by stacking and properly sampling the cropped multi-frame recognition static objects.
  • the flowchart of the preprocessing stage is shown in Figure x.
  • the scanning range of roadside lidar is usually relatively wide.
  • the farthest scanning distance of high-level equipment is more than 100 meters, and its effective scanning distance can generally reach more than 50 meters. Therefore, the scanned data must contain a large number of various objects, such as surrounding buildings and green plants. Compared with elements such as vehicles, pedestrians, and road surfaces, such objects have very low data value for traffic environment detection and can be directly eliminated. Therefore, in the present invention, point cloud data that is 5m away from the road edge and has almost no data value is defined as invalid data, which usually includes areas such as buildings, slopes, and open spaces on both sides of the road. Static objects, etc. are defined as valid data. Invalid data can be eliminated before subsequent calculation work, reducing the amount of data processing.
  • the method for eliminating invalid data proposed by the present invention is:
  • the culling boundary is to clearly divide the boundary between invalid data and valid data.
  • 3D Reshaper 3D Reshaper
  • the number of selected points should not be less than 30 points, and the distance between the two points should not be less than 50cm. If the length of the boundary cannot meet the above requirements, the integration between adjacent boundaries can be considered. Considering the amount of calculation, it is recommended that the number of boundary equations should not exceed 6.
  • boundary equation group E b 4Finally unify all boundary equations into boundary equation group E b , and record the direction of data elimination, which is used as the screening condition as the data screening condition in the actual identification process later.
  • the processing method is the same as that proposed in the present invention; If it has invaded the road space, as shown in Figure 5, the tree crown has covered part of the pavement area, the bottom data should be filtered out according to the height threshold before projecting to the horizontal plane.
  • the height threshold is the distance between the bottom of the tree crown and the key road area. Any value between the maximum vehicle heights of common vehicles, which satisfies the removal of tree canopy areas while retaining vehicle data.
  • a fixed value can be selected for the height threshold.
  • the height threshold can be distributed in a stepwise manner or constructed with a spatial plane equation. Stepwise distribution means that for the road section whose plane coordinates are located in a certain area, the same fixed value is selected for the height threshold, and its expression is as follows:
  • X, Y correspond to the X and Y values of the point cloud data
  • x 1 , x 2 , x 3 , x 4 and y 1 , y 2 , y 3 , y 4 represent the upper and lower area thresholds in the X and Y directions, respectively
  • H represents Height threshold
  • h 1 , h 2 represent the height thresholds selected for different plane coordinate regions.
  • the space plane equation is to fit the plane equation through the X, Y, and Z coordinates of the road points, and then translate it upward to make the plane meet the segmentation condition of the height threshold. Its manifestation is:
  • A, B, C, and D are the coefficients of the plane equation, and x, y, and z correspond to the X, Y, and Z coordinates of the point cloud data, respectively.
  • x, y, and z correspond to the X, Y, and Z coordinates of the point cloud data, respectively.
  • the Z value can be calculated according to the X and Y coordinates of the point cloud, and the obtained Z value is the height threshold of the current area.
  • the subsequent processing method is the same as the invalid data elimination method proposed by the present invention.
  • the method for eliminating invalid data applicable to the present invention is not limited to the above method, and other methods with the same function can be used as one of the variant schemes.
  • the invalid data elimination method can be placed after other steps.
  • the present invention only considers it as a pre-step to obtain the linear relationship between the point cloud density and the scanning distance for the sake of reducing the amount of data calculation.
  • the sequential technical solutions should all be regarded as variants of the present invention. After the above steps are processed, a set D 0 ' of valid data is obtained.
  • the point cloud data has the characteristics that the closer it is to the interior, the denser it is, and the closer it is to the outside, the sparser it is, that is, the point cloud density in the point cloud data is closely related to its scanning distance.
  • the point cloud target recognition algorithm is closely related to the point cloud density of the target.
  • objects with high point cloud density are easier to be identified. Therefore, in order to improve the accuracy of subsequent point cloud target recognition, it is necessary to first establish the relationship between the point cloud density and the scanning distance. It can be seen from the scanning principle of lidar that the distance between two points on the same ring line is linear with the distance between the ring line and the center, so it is inferred that the point cloud density and the scanning distance are also linear.
  • sample points are collected from the inside to the outside at equal intervals of each scanning ring line, and the number of points within a radius of 10 cm with each sampling point as the center is recorded as the point cloud density. 30 sampling points were selected on the ring line, and the statistical results were filled in the following table.
  • ⁇ is the point cloud density
  • L is the scanning distance
  • k is the linear function parameter
  • each frame of point cloud data needs to be divided into two parts by manual extraction, namely the static object D s and the non-static object D ns .
  • the so-called static objects refer to the road surface and its ancillary facilities, buildings and roadside green plants, etc., such objects are in a long-term position and state without any change in the case of reconstruction and expansion and frequent road maintenance.
  • objects with no obvious changes in position and appearance within a month can be regarded as static objects.
  • the non-static object is the sum of objects other than static objects in the point cloud data, and is further divided into dynamic objects D d and short-term static objects D st .
  • Dynamic objects refer to moving vehicles, walking pedestrians, etc. These objects are in motion when they are observed.
  • Short-term static objects refer to temporary parking, standing pedestrians, etc. Such objects are in a short-term position and state without change, but the possibility of their next moment movement is not excluded. Objects that do not have obvious position changes and appearance changes are short-term static objects.
  • the static object D s is used to extract the static point cloud background B.
  • the so-called static point cloud background refers to a pure static background space that does not contain any short-term static objects and dynamic objects.
  • the vehicle-road collaboration scene for the present invention that is, it does not contain any non-permanently parked vehicles, pedestrians and other traffic participants. traffic environment, and its effect is shown in Figure 4.
  • the non-static object D ns is used to obtain the recognition trigger distance threshold DT.
  • the so-called recognition trigger distance threshold refers to the perceptual distance range that enables most point cloud target detection methods to perform well. Because the point cloud data has the phenomenon of outer sparseness and inner density, distant non-static objects may only be described by one or two scan lines, and such a sparse number of point clouds is difficult to be detected by most point cloud target detection algorithms. detected. Therefore, in order to meet the detection requirements of most point cloud detection algorithms, it is necessary to establish an appropriate recognition trigger distance threshold, which is used to represent the trigger distance of subsequent methods.
  • point cloud data outside the recognition trigger distance threshold will be regarded as low-value data and eliminated.
  • the recognition trigger distance threshold DT it is necessary to establish the relationship between the scanning distance L and the detection confidence P first.
  • the final result of the present invention will be provided to the vehicle for use.
  • the built-in point cloud target detection algorithms of various autonomous vehicle manufacturers are different. Therefore, for practical reasons, the present invention selects several common point cloud target detection methods.
  • the algorithm is used as a test algorithm in the preprocessing stage, including VoxelNet, PIXOR, and PointRCNN. in:
  • VoxelNet is a typical voxelized point cloud processing method. It divides the 3D point cloud into a certain number of voxels. After random sampling and normalization of points, several voxel feature encoding layers are used for each non-empty voxel. Perform local feature extraction to obtain Voxel-wise features, and then further abstract features through three-dimensional convolution operation, and increase the receptive field and learn geometric space representation in the process, and finally use Region Proposal Network to classify objects and perform position regression. .
  • PIXOR is a typical image point cloud processing method. By projecting the point cloud, a two-dimensional bird's-eye view with height and reflectivity channels is obtained, and then the fine-tuned RetinaNet is used for object detection and localization. The overall process is more similar to traditional image object detection methods.
  • PointRCNN is a typical point cloud processing method that utilizes the original point cloud data structure.
  • the whole framework includes two stages: the first stage uses bottom-up 3D proposal generation, and the second stage is used to modify the proposal in canonical coordinates to obtain the final Test results.
  • the stage 1 sub-network is generated directly from the point cloud in a bottom-up manner by segmenting the point cloud of the entire scene into foreground and background points. A small number of high-quality 3D samples.
  • the stage 2 sub-network transforms the pooled points of each example into canonical coordinates to better learn local spatial features, this process is combined with the learning of global semantic features of each point in stage 1 for Box optimization and confidence Degree prediction.
  • the above three methods are typical representatives of the most mainstream three types of point cloud target detection algorithms, which can better simulate the intelligent perception of autonomous vehicles. It should be understood that the above three algorithms selected by the present invention cannot fully represent all point cloud target detection algorithms. Therefore, it is reasonable to use other point cloud target detection algorithms as the test algorithms in the preprocessing stage, and should be regarded as variants. one of the options.
  • the present invention also adopts the random sampling method to obtain the average point cloud density of non-static objects, but the sampling method here is slightly different from the above point cloud sampling, as described below.
  • the ratio of point cloud sampling needs to be established with reference to hardware equipment parameters and the actual total number of point clouds of the target.
  • the random sampling method proposed by the present invention is:
  • the calculation method of the point cloud density is the same as the above, that is, with each sampling point as the center, the number of points within a radius of 10cm is used as the point cloud density.
  • point cloud sampling methods for obtaining the point cloud density of non-static objects can be regarded as one of the variants of the sampling methods applicable to the present invention.
  • the following variants of different sampling ratios can be used for different types of objects:
  • each non-static object is input into different back-end algorithms for detection, and each detection result and detection confidence P are obtained. As shown in Figure 8, draw the distribution curve of the scanning distance L and the detection confidence P, using the following formula:
  • j, i represent the upper and lower limits of the recognition trigger distance threshold DT
  • i is the shortest distance threshold
  • j is the farthest distance threshold
  • n i represents the total number of dynamic targets at distances i and j from the origin
  • p>75% means the total number of non-static targets with detection confidence greater than 75% in the range i and j. It should be understood that 75% is only the determination threshold recommended by the present invention, and its actual value can be adjusted according to the scene in which the roadside sensing unit is installed.
  • lidar devices generally have vertical scanning angle parameters, that is, when the physical distance to lidar is too close and the height is lower than lidar, it cannot be scanned. arrived. At this time, it may happen that some vehicles are very close to the interior, but only half of the vehicle body is scanned. Therefore, it is necessary to establish a lower limit of the distance to ensure that the complete vehicle body can be obtained.
  • the initial values of i and j can be established by directly observing the image distribution. Based on the initial values, with 0.5m as the offset, the upper and lower limits of the range are repeatedly changed, and an area with the largest range is established as the final i and j values. Finally, it is necessary to convert the i and j values into the form of boundary equations, generally using the equation expression of a circle. Therefore, the expression form of the identification trigger distance threshold DT should be an annular interval, which is composed of the boundary equations of two circles.
  • the recognition trigger distance threshold DT After the recognition trigger distance threshold DT is obtained, it is used to crop the aforementioned static object D s , which is the same as the elimination of invalid data, that is, the point cloud data other than the recognition trigger distance threshold DT is eliminated by analogous linear programming, and the static object for recognition is obtained. D′′ s .
  • the static object for recognition needs to be converted into a static point cloud background B.
  • a single frame of data can only reflect the current situation of the scene at a certain moment, so it is unquestionable to use multiple frames of data to superimpose and integrate into a point cloud background that meets the vast majority of situations in the scene.
  • simple superposition can easily lead to denser areas, and sparse parts are still sparse in comparison.
  • the present invention adopts the superposition method with sampling between partitions to avoid the above problems.
  • the working principle of lidar is rotary scanning
  • the scanned data is circular distribution.
  • Figure 6 for each frame of point cloud data, it is divided into n statistical spaces with gradually increasing spacing from the innermost to the outer.
  • the specific division interval needs to refer to the scanning range and point cloud density parameters of the hardware device.
  • the interval proposed in the present invention is divided as follows:
  • r is the width of the inner ring
  • l is the length of the grid
  • R is the distance of the inner ring from the origin.
  • ⁇ is the point cloud density
  • n is the total number of points contained in the statistical space
  • S is the horizontal projected area of the statistical space
  • r, l, and R have the same meanings as above.
  • the point cloud density of a certain statistical space is greater than the preset threshold ⁇ , the point cloud in the space is randomly down-sampled to maintain its point cloud density.
  • the threshold suggested by the present invention is 2000 points/m 2 .
  • the point cloud density in each statistical space should be roughly equal, especially the point cloud density of the outermost statistical space and the innermost statistical space should be checked;
  • the number of statistical spaces should not exceed 500 as far as possible, otherwise the calculation amount may be too large, and should not be less than 100, otherwise a single statistical space is too large, which is not conducive to the establishment of the point cloud density threshold ⁇ , and it is easy to cause the point cloud density
  • the distribution of point clouds in the same statistical space is quite different, which is not conducive to subsequent calculation work.
  • the static point cloud background B is obtained.
  • lidar is limited by its mechanical structure, it cannot guarantee that the laser point emitted in the previous round and the laser point emitted in the next round are in the same position during the scanning process. Position comparison is not only cumbersome and complicated, but also meaningless. To this end, the present invention introduces voxel features as a comparison basis.
  • Voxel or Voxel, short for volume pixel is conceptually similar to a pixel in a two-dimensional space, and a voxel is the smallest unit in a three-dimensional space.
  • Voxelization means that the three-dimensional model is uniformly represented by voxels.
  • Voxelization can be understood as the generalization of two-dimensional pixelation in three-dimensional space.
  • the simplest voxelization is binary voxelization, which means that the voxelization value is either 0 or 1.
  • Voxelization describes 3D scene data, which can express complex 3D scenes and correctly describe vertically overlapping objects.
  • Figure 9 is an example of point cloud voxelization. It can be seen that the amount of voxel expression data is less and the semantic information is less lost.
  • the size of the voxel that is, the side length v of the cube, needs to be established according to the density of the collected point cloud data. If it is too large, it is easy to lose semantic information, and if it is too small, it cannot play the role of voxelization to reduce the amount of data.
  • the general voxel size can be selected to be 1/20 to 1/40 of the statistical space size in the first step, and the effect is better.
  • the voxel size suggested by the present invention is 30cm*30cm*30cm, but the actual value should be changed according to the actual use effect.
  • the present invention proposes to calculate the voxel position to which any point in the point cloud data belongs based on the following formula:
  • x, y, z are the coordinates of any point in the point cloud data; x 0 , y 0 , z 0 represent the origin coordinates of the point cloud data, and they are not directly 0 because there may be multiple radar networks that make the radar center When not (0, 0, 0); v is the side length of the voxel. r, c, and h are the coordinate indices of the voxels.
  • the number of point clouds contained in each voxel is counted. If the number of point clouds is less than 5, it is regarded as an empty voxel, deleted from the voxel list, and finally all non-empty voxels are retained.
  • the point cloud data When actually recognizing a certain frame of point cloud data, the point cloud data is first eliminated invalid data and the recognition area is separated by the recognition trigger threshold, and then it is voxelized into a three-dimensional matrix. If the change rate of a continuous area in the sliding window and the background value is greater than the judgment threshold, the area is marked as a non-static area, otherwise it is marked as a static area.
  • the invention draws on the algorithm idea of separation of foreground and background in image processing, based on the static point cloud background obtained in the first step, and uses the sliding window method to compare it with the point cloud data distribution of the current frame. If it is too large, it can be considered that there are new objects that are not static objects in this area, so as to extract non-static objects.
  • the roadside sensing unit is installed in the scene to be detected, and the above preprocessing process is used to complete the preliminary data processing work, and the following results are obtained: voxelized static point cloud background B v , recognition trigger distance threshold DT, boundary equation set E b .
  • the data collected when the system is started is generally not stable, and it usually takes 3-5 minutes to start using the method of the present invention to identify dynamic and static targets.
  • the first frame of point cloud data at the official start is denoted as D 1 , and the subsequent data are sequentially D 2 , D 3 — Di .
  • each frame of data is cropped for the first time by using the boundary equation set E b obtained by preprocessing to obtain valid data D′ 1 , D′ 2 , ⁇ D′ i .
  • the recognition trigger distance threshold DT to cut the data of each frame for the second time, and obtain the data for recognition D′′ 1 , D′′ 2 , ... D′′ i .
  • perform voxelization operation on the data for recognition to obtain the voxelized data D′′ v1 , D′′ v2 , ... D′′ vi .
  • the voxelized data of the current frame is compared with the voxelized data from the inside to the outside. static background to match.
  • the size of the sliding window suggested by the present invention is 5 times the size of the voxel frame, that is, a sliding window contains at most 125 voxels.
  • centroid position of the voxels in the sliding window Compared with the same area in the static point cloud background, if the position offset does not exceed 2 voxel side lengths, the area where the window is located is marked as a static area, otherwise it is marked as a non-static area .
  • the centroid is calculated as follows:
  • x, y, z represent the coordinates of the centroid
  • x i , y i , and zi represent the coordinate indices of each voxel
  • n is the total number of voxels included in the sliding window.
  • the set of all static regions is the static object D s in the final result, and the remaining non-static objects are reclassified through the dynamic object recognition described below.
  • the recognition trigger distance threshold may only contain half of the vehicle. For example, if a vehicle enters the lidar scanning range from outside the scanning range, it will also be identified due to the change in the distribution of point clouds in the area. However, it may not be detected by the point cloud target detection algorithm or be detected incorrectly due to the incomplete data. Since the recognition trigger distance threshold is usually smaller than the actual scanning range of the lidar, the incomplete vehicle correspondence at this time is actually complete in the original data.
  • the present invention on the basis of identifying the triggering distance threshold, will add the identification redundancy value, that is, the annular standby identification area with a width of 1.5m is expanded from the boundary where the identification triggering distance threshold is located to the outside, and then the scanning angle (the present invention adopts 10°) is divided into sub-regions, as shown in Figure 10. If a dynamic area is identified in the edge area, the outer spare area closest to the dynamic area is added to the dynamic area. Calculating the nearest outer spare area can directly calculate the distance between each spare area and the centroid of the dynamic area.
  • the danger of short-term static objects is greater than that of static objects but less than that of dynamic objects, which is the secondary identification target of the present invention, so its transmission frequency can also be between the two.
  • Static objects are non-transmission or minute-level low-frequency transmission, and dynamic data is real-time high-frequency transmission. For short-term static objects, it is transmitted at a second-level intermediate frequency.
  • the identification of short-term static objects is also slightly different from that of non-static objects. Since the position of the short-term static object does not move, the difference in point cloud distribution between the two frames is almost negligible. , the frame before and after can be considered as the same object. Based on this idea, the following methods are used for identification.
  • every fixed frequency interval all non-static areas Ans in the identification frame are recorded as temporary static areas A st for secondary matching.
  • the first frame of data is used as the starting frame, and its non-static area has no temporary static area for comparison, so all its non-static areas are recorded as temporary static areas, but the result output is all regarded as dynamic objects.
  • the adopted fixed frequency interval can generally be selected from 1/2 to 1/5 of the collection frequency of the lidar, and the frequency selected in the present invention is recorded once every 5 frames.
  • each individual point cloud space is used as a matching object.
  • the corresponding relationship table can be established in the previous matching step, or the Euler distance can be used for clustering and separation into sub-regions after unified recording.
  • the present invention proposes to adopt the former.
  • the sub-regions of the two are denoted as Ans -i and Ast -j, respectively.
  • Ans -i can be regarded as A st -j, that is, the same object represented by Ans -i and A st -j is marked as a short-term static object, otherwise all non-static sub-regions that do not meet the conditions are still marked as dynamic area.
  • the dynamic object is actually recorded in the temporary static area every time the temporary static area is recorded, through the comparison of the previous and previous frames, if there is a sub-area A st -j in the temporary static area, the non-static area of the subsequent frame data is made. If there is no sub-region that can be matched with it, it is considered that the sub-region A st -j is not a short-term static object, so it can be eliminated from the temporary static region to reduce the comparison amount of the temporary static region.
  • the set of all dynamic regions is the dynamic object D d .
  • the remaining part is the short-term static object D st , which is also used for the next comparison.
  • Temporary static area A st is also used for the next comparison.
  • a counter is assigned to the temporary static area. If there is a short-term static object after the comparison, the counter value is increased by 1.
  • the system can manually set the transmission frequency of short-term static objects. If the counter is set to send every 3 increments, the corresponding transmission frequency of short-term static objects is 1/3 of the transmission frequency of dynamic objects.
  • FIG. 7 Schematic diagram of point cloud sampling method
  • FIG. 10 Schematic illustration of the supplementary description of the edge area
  • the roadside sensing unit is arranged according to the description of the patented invention.
  • the case adopts the pole-type installation method.
  • the installation height of the lidar is about 5 meters, and the scanning range covers a two-way two-lane road, as well as surrounding buildings, green trees and other objects.
  • the farthest scanning distance of the actual identification data is about 120 meters, the data collection frequency is 10HZ, and the number of point clouds per frame is more than 100,000.
  • a sample visualization is shown in Figure 11.
  • step 1 collect and process data. Since it was all sunny during this test, the lidar scanning data in rainy days could not be collected, and the road surface was sprayed with water in a large area instead, and about 720 frames of corresponding point cloud data were obtained. About 600 frames of data were collected at around 8:00 in the morning, around 13:00 at noon, and around 21:00 at night, and in order to avoid chance, the data were collected continuously for 3 days. And at night, drive the experimental vehicle to the scanning area, turn on the high beams to obtain point cloud data of the environment illuminated by strong light, about 250 frames. The total number of data frames is close to 6000 frames. After artificial screening, about 2000 frames of point cloud data are obtained for extracting static point cloud background.
  • the boundary equation system is established based on artificial means, and is completed by using common point cloud data visualization processing tools.
  • the present invention selects a domestic point cloud processing software as a visual operation tool. Since the selected example road segment has obvious curbs as the road boundary, the road area and the non-road area can be clearly divided.
  • the point cloud sampling was carried out in the curb area.
  • the sampling method was manual sampling along the extension direction of the curb with an interval of 50 cm to obtain point cloud samples for fitting the road boundary. Based on the above sampling points, the least squares method is used to fit the plane equation of the road boundary, and finally the plane equation of the left and right boundaries of the road is obtained, and the direction of data removal is recorded.
  • the first data filter condition is used to fit the plane equation of the road boundary, and finally the plane equation of the left and right boundaries of the road is obtained, and the direction of data removal is recorded.
  • sample points are collected from the inside to the outside at equal intervals of each scanning ring line, and the number of points within a radius of 10 cm with each sampling point as the center is recorded as the point cloud density. 30 sampling points, some examples of case sampling results are shown in the table below.
  • ⁇ is the point cloud density
  • L is the scanning distance
  • 0.97 is the linear function parameter
  • the static objects in each frame of data are compared with non-static objects, and the static objects are used to obtain the static point cloud background.
  • the point cloud target detection algorithm used in this case only needs to input the original point cloud data, and it is not necessary to extract non-static objects separately for detection. All point cloud target detection algorithms should train a model with better recognition effect before use, and the present invention considers that the algorithm has a better recognition effect if the precision rate of the algorithm is greater than 85%.
  • the label frame obtained by the algorithm can generally be used as the extraction boundary, and all point clouds inside the label frame can be regarded as the detection target.
  • the average point cloud density is obtained by random sampling method.
  • the ratio of point cloud sampling is established with reference to the parameters of the selected lidar device and the actual total number of point clouds of the target.
  • the method used in this case is:
  • the calculation method of the point cloud density is the same as the above, that is, with each sampling point as the center, the number of points within a radius of 10cm is used as the point cloud density.
  • each non-static object is input into the back-end algorithm for detection, and each detection result and detection confidence P are obtained.
  • Draw the distribution curve of scanning distance L and detection confidence P using the following formula:
  • j, i represent the upper and lower limits of the recognition trigger distance threshold
  • i is the shortest distance threshold
  • j is the farthest distance threshold
  • n i , n j represent the total number of non-static targets at distances i and j from the origin
  • (n j ⁇ n i ) p>75% means the total number of non-static targets with built-in reliability greater than 75% in i, j range.
  • i, j values so that P ij is greater than 75%, as the non-static object extraction range in actual recognition.
  • the values of i and j selected by the present invention are 3 and 45, that is, the extraction range of the corresponding non-static object is from a horizontal distance of 4 m from the center of the laser radar to a horizontal distance of 25 m from the center of the laser radar.
  • r is the width of the inner ring
  • l is the length of the grid
  • R is the distance of the inner ring from the origin.
  • the point cloud density of each statistical space is detected.
  • the calculation formula of the point cloud density is:
  • ⁇ is the point cloud density
  • n is the total number of points contained in the statistical space
  • S is the area of the statistical space
  • r is the width of the inner ring
  • l is the length of the grid
  • R is the distance between the inner ring and the origin.
  • the point cloud density of a certain statistical space is greater than the preset threshold of 2000 points/m 2 , the point cloud in the space is randomly down-sampled to maintain its point cloud density, and finally an ideal static point cloud background B is obtained.
  • the static point cloud background is first processed by voxelization.
  • the size of the voxel used in the present invention is 30cm*30cm*30cm. Calculate the voxel position to which any point in the point cloud data belongs based on the following formula:
  • x, y, and z are the coordinates of any point in the point cloud data; x0, y0, and z0 represent the origin coordinates of the point cloud data, and they are not directly 0 because there may be multiple radar networks, so that the radar center is not ( 0, 0, 0); v is the side length of the voxel. r, c, and h are the coordinate indices of the voxels.
  • step 3 voxelize the newly collected data and use the sliding window method to filter out the non-static area.
  • the method is the same as that of static point cloud background processing. After two data screening, the average voxel The number is around 15,000.
  • the voxelized data of the current frame is matched with the voxelized static background from the inside to the outside.
  • the size of the sliding window used in this case is 5 times the size of the voxel box, that is, a sliding window contains at most 125 voxels.
  • centroid position of the voxels in the sliding window Compared with the same area in the static point cloud background, if the position offset does not exceed 2 voxel side lengths, the area where the window is located is marked as a static area, otherwise it is marked as a non-static area .
  • the centroid is calculated as follows:
  • x, y, z represent the coordinates of the centroid
  • x i , y i , and zi represent the coordinate indices of each voxel
  • n is the total number of voxels included in the sliding window.
  • the point cloud data of all static regions are extracted as static objects. Since static objects are continuous point cloud data, it is difficult to compare their recognition rates, so they are converted into comparative non-static objects.
  • the recognition rate of non-static objects in the inner area (the area with a horizontal distance of less than 23m from the center of the lidar) can reach more than 97%
  • the edge area (the area with a horizontal distance from the center of the lidar greater than 23m and less than 25m) due to cross-border
  • the size of the intercepted part of the vehicle is different, the recognition rate is relatively weak, can reach more than 85%, and the average non-static object recognition rate is about 92%.
  • the sub-areas are compared with the sub-areas recorded in the temporary static area in turn.
  • the features and order of comparison are as follows:
  • Ans -i can be regarded as A st -j, that is, the same object represented by Ans -i and A st -j is marked as a short-term static object, otherwise all non-static sub-regions that do not meet the conditions are still marked as dynamic area.
  • the dynamic object is actually recorded in the temporary static area every time the temporary static area is recorded, through the comparison of the previous and previous frames, if there is a sub-area A st -j in the temporary static area, the non-static area of the subsequent frame data is made. If there is no sub-region that can be matched with it, it is considered that the sub-region A st -j is not a short-term static object, so it can be eliminated from the temporary static region to reduce the comparison amount of the temporary static region.
  • the set of all dynamic regions is the dynamic object D d .
  • the remaining part is the short-term static object D st , which is also used for the next comparison.
  • Temporary static area A st is also used for the next comparison.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Remote Sensing (AREA)
  • Theoretical Computer Science (AREA)
  • Electromagnetism (AREA)
  • Data Mining & Analysis (AREA)
  • Analytical Chemistry (AREA)
  • Chemical & Material Sciences (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Evolutionary Biology (AREA)
  • Evolutionary Computation (AREA)
  • General Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Traffic Control Systems (AREA)
  • Radar Systems Or Details Thereof (AREA)
  • Optical Radar Systems And Details Thereof (AREA)

Abstract

A roadside sensing unit-based method for quick recognition of dynamic and static objects, comprising the following steps: 1, constructing a roadside laser radar holographic sensing scene, and collecting a batch of point cloud data as a-priori information; 2, establishing an effective target recognition range, and extracting a static point cloud background for subsequent matching; 3, recognizing a non-static area having larger change variation by comparing the point cloud data of a current frame to be recognized with recorded voxel features of the static point cloud background, and recording the remaining part as a static object; and 4, recording the non-static area as a temporary static area at a fixed frequency, and recognizing a short-term static object and a dynamic object by comparing the non-static area with the recorded temporary static area.

Description

一种基于路侧感知单元的静态和非静态物体点云识别方法A point cloud recognition method for static and non-static objects based on roadside perception unit 技术领域technical field
本发明属于数据处理技术领域，特别涉及一种基于路侧感知单元的静态和非静态物体点云识别方法，主要面向于车路协同环境下基础设施侧的目标感知。The invention belongs to the technical field of data processing, and in particular relates to a point cloud recognition method for static and non-static objects based on a roadside perception unit, which is mainly oriented to target perception on the infrastructure side in a vehicle-road coordination environment.
背景技术Background technique
随着我国国民经济和汽车工业技术的不断提升和发展，汽车已经成为我们日常生活出行以及从事生产不可或缺的交通工具。然而，必须承认的事实是，汽车在极大地改善了我们人类的生产生活方式的同时，也伴随着交通事故、交通拥堵、经济损失和环境污染等问题的产生，而且愈演愈烈。为了改善交通大环境，世界各国政府以及专家学者积极探索能够有效解决上述交通安全问题的途径，自动驾驶技术应运而生。以汽车作为软硬件载体，通过车载的传感器、决策单元和执行器等电子设备，为汽车提供智能化的支持，使得汽车可以基于周边环境进行驾驶行为决策，从而避免驾驶员个人素质良莠不齐导致的交通风险，达到提高车辆安全性的目的。另外，随着专用短程通讯技术、传感器技术、车辆控制技术越来越成熟，自动驾驶和无人驾驶技术从实验室走向实际应用的步伐正在加快。With the continuous improvement and development of my country's national economy and automobile industry technology, automobiles have become an indispensable means of transportation for our daily travel and production. However, the fact that must be acknowledged is that while automobiles have greatly improved the production and lifestyle of human beings, they are also accompanied by problems such as traffic accidents, traffic congestion, economic losses and environmental pollution, which are becoming more and more serious. In order to improve the traffic environment, governments, experts and scholars around the world are actively exploring ways to effectively solve the above-mentioned traffic safety problems, and autonomous driving technology emerges as the times require. Taking the car as the carrier of software and hardware, through the on-board sensors, decision-making units and actuators and other electronic equipment, it provides intelligent support for the car, so that the car can make driving behavior decisions based on the surrounding environment, so as to avoid the traffic caused by the uneven personal quality of drivers. risk, to achieve the purpose of improving vehicle safety. In addition, as dedicated short-range communication technology, sensor technology, and vehicle control technology become more and more mature, the pace of autonomous driving and unmanned driving technology from laboratory to practical application is accelerating.
然而从目前自动驾驶技术的发展态势来看，仅仅依靠单车智能***将难以真正有效解决交通安全风险问题，其原因主要有以下几点：1、车载设备感知能力有限，在某些场景下可能出现感知数据不足引发决策错误的现象；2、车载设备的感知范围有限，受限于安装位置，许多感知设备不能充分利用其理论的感知范围，往往会被自动驾驶车辆周边的其他车辆遮挡而忽视掉可能的危险；3、决策能力还有待提高，当前的自动驾驶决策***的智能化程度还难以应对复杂多变的交通环境。基于上述问题，世界各国开始提出一种新的解决路径，即车路协同***。车路协同***的基本思想是运用多学科交叉与融合的方法，利用先进的无线通信技术和传感技术等实时获取车辆和道路信息，通过车-车通信和车-路通信方式实现车辆与车辆之间、车辆与道路智能路侧设施之间的信息交互和共享，实现车与车、车与路的智能协同，从而提高道路交通安全性、道路通行效率以及道路交通***资源利用率等目的。通常情况下，根据传感器在***中的安装位置，车路协同***可以分为智能路侧***与智能车载***两个子***。其中智能路侧***主要承担交通流信息的获取与发布、路侧设备控制、车-路通信、交通管理与控制等任务；智能车载***主要完成车辆自身的运动状态信息及车辆周围环境信息获取、车-车通信/车-路通信、安全预警及车辆辅助控制等任务。智能路侧***与智能车载***通过车-路通信进行双方信息的传输与共享，从而实现数据的交互，拓宽自动驾驶车辆的感知范围和数据量，增强自动驾驶车辆的决策基础，提升驾驶安全性。However, judging from the current development trend of autonomous driving technology, it is difficult to effectively solve the problem of traffic safety risks only by relying on the intelligent system of bicycles. Insufficient sensing data leads to wrong decision-making; 2. The sensing range of in-vehicle equipment is limited and limited by the installation location. Many sensing devices cannot make full use of their theoretical sensing range, and are often blocked by other vehicles around the autonomous vehicle and ignored. 3. The decision-making ability needs to be improved, and the current level of intelligence of the autonomous driving decision-making system is still difficult to cope with the complex and changeable traffic environment. Based on the above problems, countries around the world have begun to propose a new solution, namely the vehicle-road coordination system. The basic idea of the vehicle-road coordination system is to use the method of multi-disciplinary crossover and integration, use advanced wireless communication technology and sensing technology to obtain real-time vehicle and road information, and realize vehicle-to-vehicle communication through vehicle-vehicle communication and vehicle-road communication. Information exchange and sharing between vehicles and intelligent roadside facilities on the road, to achieve intelligent collaboration between vehicles and vehicles, and between vehicles and roads, thereby improving road traffic safety, road traffic efficiency, and road traffic system resource utilization. Usually, according to the installation position of the sensor in the system, the vehicle-road coordination system can be divided into two subsystems: the intelligent roadside system and the intelligent vehicle-mounted system. Among them, the intelligent roadside system is mainly responsible for the acquisition and release of traffic flow information, roadside equipment control, vehicle-road communication, traffic management and control; Vehicle-to-vehicle communication/vehicle-to-road communication, safety warning and vehicle auxiliary control tasks. The intelligent roadside system and the intelligent in-vehicle system transmit and share information between the two parties through vehicle-road communication, thereby realizing data interaction, broadening the perception range and data volume of autonomous driving vehicles, enhancing the decision-making basis of autonomous driving vehicles, and improving driving safety. .
就发展现状而言，智能路侧***目前的可实现应用场景主要为为自动驾驶车辆提供辅助感知信息。路侧感知设备比车载感知设备的类型更宽裕一些，安装位置也相对更自由，且能源供给等硬性条件更宽容。常见的路侧感知设备包括：1、环形线圈；2、毫米波雷达；3、UWB技术；4、视觉方式；5、激光雷达等。上述感知手段中，可作为工程解决方案使用的技术主要为视觉检测方法和激光雷达技术。二者都具备数据形式简单易理解，且目标检测技术较为成熟的特点，但相比而言，路侧视觉数据若要服务于车端，所传输的数据必须为检测结果，因为视 角差异较大的图像数据几乎无法实现原始数据级别的数据融合，仅可融合其检测结果；而激光雷达数据为点云坐标形式，通过坐标轴转换即可实现数据融合，这种前融合的数据融合方式相比基于检测结果的后融合而言，数据的语义信息丢失更少，对提升检测结果的识别精度更有利。As far as the development status is concerned, the current achievable application scenarios of intelligent roadside systems are mainly to provide auxiliary perception information for autonomous vehicles. Roadside sensing devices are more generous than in-vehicle sensing devices, their installation positions are relatively free, and hard conditions such as energy supply are more tolerant. Common roadside sensing devices include: 1. toroidal coil; 2. millimeter-wave radar; 3. UWB technology; 4. visual method; 5. lidar, etc. Among the above perception methods, the technologies that can be used as engineering solutions are mainly visual detection methods and lidar technology. Both have the characteristics of simple and easy-to-understand data form, and relatively mature target detection technology, but in comparison, if roadside visual data is to serve the vehicle end, the transmitted data must be the detection result, because the viewing angle is quite different. It is almost impossible to achieve the data fusion of the original data level of the image data, only the detection results can be fused; while the lidar data is in the form of point cloud coordinates, and the data fusion can be achieved through the transformation of the coordinate axes. Based on the post-fusion of detection results, the semantic information of the data is less lost, which is more beneficial to improve the recognition accuracy of the detection results.
但需注意的是，即使前融合方式更有利，但其仍存在局限性，其运用的最大局限为数据传输量更大，因为传输的数据为原始数据，而一般点云数据的一帧大小在几M至十几M之间，即每秒一对一传输的数据大小就可高达几百M，而对应于车辆众多的复杂智慧交通环境，每秒总数据传输量甚至可高达数G，因此必须缩减传输的数据大小才。However, it should be noted that even if the pre-fusion method is more favorable, it still has limitations. The biggest limitation of its application is that the amount of data transmission is larger, because the transmitted data is raw data, and the size of one frame of general point cloud data is in Between a few megabytes and a dozen megabytes, that is, the size of the data transmitted one-to-one per second can be as high as several hundred megabytes. Corresponding to the complex intelligent traffic environment with many vehicles, the total data transmission volume per second can even be as high as several gigabytes. The size of the transmitted data must be reduced.
缩减数据大小有诸多方法，如点云稀疏化采样、骨架结构提取等，一种适应于自动驾驶场景的有效方法为提取数据中的重点部分，如针对目标车辆而言，其周边的车辆、非机动车、行人以及路障设施等物体的重要程度就远胜于路面、绿化植物和两侧建筑物等物体，因此可以在原始数据中识别并筛选出对应低价值的物体从而达成缩减数据量的目的。另外，剔除上述低价值物体之后，剩余物体的使用价值将自然再提高，因为排除干扰因素后，使得重要物体的存在更加突出，避免了低价值数据的影响。There are many ways to reduce the data size, such as point cloud sparse sampling, skeleton structure extraction, etc. An effective method suitable for automatic driving scenarios is to extract the key parts of the data. For example, for the target vehicle, the surrounding vehicles, non- Objects such as motor vehicles, pedestrians, and roadblocks are far more important than objects such as roads, green plants, and buildings on both sides. Therefore, low-value objects can be identified and screened in the original data to achieve the purpose of reducing the amount of data. . In addition, after removing the above-mentioned low-value objects, the use value of the remaining objects will naturally increase again, because after eliminating the interference factors, the existence of important objects is more prominent and the influence of low-value data is avoided.
现有技术current technology
CN108152831 ACN108152831 A
CN106846494 ACN106846494 A
CN107133966 ACN107133966 A
CN110443978 ACN110443978 A
术语解释Terminology Explanation
为使本发明的描述更加准确清晰，现对本发明中会出现的各种术语作如下解释：In order to make the description of the present invention more accurate and clear, various terms that will appear in the present invention are now explained as follows:
路侧感知单元：车路协同场景下，以路侧立杆或龙门架为安装基底，布设于道路周边的环境感知设备。在本发明中，路侧感知单元特指激光雷达***，二者应当视为同一描述。Roadside perception unit: In the vehicle-road collaboration scenario, the roadside pole or gantry is used as the installation base, and the environment perception device is arranged around the road. In the present invention, the roadside sensing unit refers specifically to the lidar system, and the two should be regarded as the same description.
点云数据：在一个三维坐标***中的一组向量的集合，这些向量通常至少包括X、Y、Z三维坐标数据，用来代表物体的外表面形状，依据设备的不同有时还能获取其他信息。在本发明中通常使用符号D来表示点云数据及其处理后的部分数据，凡采用符号D来表述的内容均应理解其表现形式为点云数据。Point cloud data: a set of vectors in a three-dimensional coordinate system, these vectors usually include at least X, Y, Z three-dimensional coordinate data, used to represent the outer surface shape of the object, and sometimes other information can be obtained depending on the device . In the present invention, the symbol D is usually used to represent the point cloud data and the processed partial data. Any content expressed by the symbol D should be understood as point cloud data.
静态物体D s：主要为路面及其附属设施、建筑物和路侧绿化植物等，在不考虑改扩建、频繁路面维修的情况下，该类物体处于长期位置固定、外观无变化状态，可认为一月内位置无明显变化、外观无明显变化的物体即为静态物体。其中判断是否属于明显变化的依据为：物体水平投影的形心偏移超过1米即视为明显变化，或外轮廓边长或体积变化超过原数据的5％即视为明显变化。 Static objects D s : mainly the road surface and its ancillary facilities, buildings and roadside green plants, etc., without considering reconstruction and expansion, frequent road maintenance, such objects are in a state of long-term fixed position and no change in appearance, it can be considered that Objects with no obvious changes in position and appearance within one month are static objects. The basis for judging whether it is an obvious change is that the centroid deviation of the horizontal projection of the object exceeds 1 meter, which is regarded as a significant change, or the change in the side length or volume of the outer silhouette exceeds 5% of the original data, which is regarded as a significant change.
短时静态物体D st：主要包括临时停车、伫立的行人等，该类物体处于短期位置、状态无变化状态，但不排除其下一刻运动的可能性，在本发明中认为不属于静态物体且在5帧内未发生明显位置变化和外观变化的物体即为短时静态物体。其中判断明显变化的依据为：物体水平投影的形心偏移超过0.3m即视为明显变化，或外轮廓边长或体积变化超过原数据的5％即视为明显变化。 Short-term static objects D st : mainly include temporary parking, standing pedestrians, etc. Such objects are in a short-term position and state without change, but the possibility of their next moment movement is not excluded. In the present invention, it is considered that they do not belong to static objects and Objects that do not have obvious position changes and appearance changes within 5 frames are short-term static objects. The basis for judging the obvious change is: the centroid deviation of the horizontal projection of the object exceeds 0.3m, which is regarded as a significant change, or the change in the side length or volume of the outer silhouette exceeds 5% of the original data, which is regarded as a significant change.
动态物体D d：主要包括行驶的车辆、走动的行人等，该类物体被观察时即处于运动状态，可认为不属于静态物体且连续2帧内发生较明显位置变化或外观变化的物体即为动态物体。其中判断明显变化的依据为：物体水平投影的形心偏移超过0.3m即视为明显变化，或外轮廓边长或体积变化超过原数据的5％即视为明显变化。 Dynamic objects D d : mainly include moving vehicles, walking pedestrians, etc. Such objects are in motion when they are observed, and can be considered not static objects and have obvious position changes or appearance changes in 2 consecutive frames. dynamic objects. The basis for judging the obvious change is: the centroid deviation of the horizontal projection of the object exceeds 0.3m, which is regarded as a significant change, or the change in the side length or volume of the outer silhouette exceeds 5% of the original data, which is regarded as a significant change.
非静态物体D ns：短时静态物体和动态物体的总和。 Non-static objects D ns : the sum of short-term static objects and dynamic objects.
原始数据D 0：用于本发明预处理部分的点云数据集，一般包含1000帧左右的点云数据，要求应当包含绝大部分路测感知单元安装路段常见交通场景。 Original data D 0 : the point cloud data set used in the preprocessing part of the present invention, generally contains about 1000 frames of point cloud data, and is required to include most common traffic scenes in the road section where the road test sensing unit is installed.
待识别点云数据：区别于前述原始数据D 0，是在本发明使用过程中被实际用于识别的点云数据帧，其识别结果可用于支持后续的点云目标检测等工作，在说明中通常记为第i帧数据D i。 Point cloud data to be identified: different from the aforementioned original data D 0 , it is the point cloud data frame actually used for identification during the use of the present invention, and the identification result can be used to support subsequent point cloud target detection and other work, in the description Usually recorded as the i-th frame data D i .
关键路面区域：利用本发明所述方法着重识别的包含路面的区域，通常为激光雷达扫描范围内可清晰分辨且为主要交通途径的道路区域。Key pavement area: The area including the road surface that is emphatically identified by the method of the present invention is usually a road area that can be clearly identified and is the main traffic route within the scanning range of the lidar.
数据价值：点云数据被自动驾驶车辆利用时，对驾驶决策的影响力大小。本发明中，其判断依据为物体对自动驾驶车辆可能造成的危险性大小，在不考虑自动驾驶***对数据理解能力的条件下，一般认为动态物体的数据价值最大，短时静态物体其次，静态物体最小。Data value: When point cloud data is used by autonomous vehicles, how much influence it has on driving decisions. In the present invention, the judgment is based on the danger that the object may cause to the autonomous vehicle. Without considering the ability of the autonomous driving system to understand data, it is generally considered that the data value of dynamic objects is the greatest, followed by short-term static objects, and static objects. Object is the smallest.
无效数据：又称无效点云数据；指几乎没有数据价值的点云数据，通常包括道路两侧的建筑物、边坡及空地等，一般为距离道路边缘5m以外的点云数据。在实际应用中可根据识别需要调整无效数据的划分范围，如在城市快速路等具有路边护栏的道路场景中，道路护栏外的点云数据均可视作无效数据。Invalid data: also known as invalid point cloud data; it refers to point cloud data with almost no data value, usually including buildings, slopes and open spaces on both sides of the road, generally point cloud data 5m away from the edge of the road. In practical applications, the division range of invalid data can be adjusted according to identification needs. For example, in road scenes with roadside guardrails such as urban expressways, the point cloud data outside the road guardrail can be regarded as invalid data.
有效数据：又称有效点云数据；点云数据中剔除完无效数据后剩下的点云数据，在本发明表述中以加单引号(’)表示提取有效数据操作。Valid data: also known as valid point cloud data; the point cloud data remaining after the invalid data is eliminated from the point cloud data, in the expression of the present invention, the operation of extracting valid data is indicated by adding a single quotation mark (').
边界方程组E b：用以分隔无效数据和有效数据的函数边界，将点云数据投影为鸟瞰图后，由人工选择边界点，再通过最小二乘法拟合确立。 Boundary equation system E b : The function boundary used to separate invalid data and valid data. After projecting the point cloud data into a bird's-eye view, the boundary points are manually selected and established by least squares fitting.
静态点云背景B：不包含任何短时静态物体和动态物体的纯静态背景空间，对本发明面向的车路协同场景而言即不包含任何非永久驻车的车辆、非机动车、行人等交通参与者的交通环境。Static point cloud background B: a purely static background space that does not contain any short-term static objects and dynamic objects. For the vehicle-road collaboration scene targeted by the present invention, it does not contain any non-permanently parked vehicles, non-motor vehicles, pedestrians and other traffic Participant's traffic environment.
待识别有效点云数据：将待识别点云数据利用边界方程组Eb裁剪后，得到的点云数据集即为待识别有效点云数据，在说明中记为有效第i帧数据D i’。 Valid point cloud data to be identified: After cutting the point cloud data to be identified using the boundary equation set Eb, the obtained point cloud data set is the valid point cloud data to be identified, which is denoted as valid i-th frame data D i ' in the description.
统计空间：点云数据具有近处密集，远处稀疏的特性，为了避免该特性影响后续的识别工作，将点云数据划分为多个统计空间并利用叠加和下采样操作使得各处的点云密度大致相等。Statistical space: The point cloud data has the characteristics of being dense in the vicinity and sparse in the distance. In order to avoid this characteristic affecting the subsequent identification work, the point cloud data is divided into multiple statistical spaces, and the superposition and downsampling operations are used to make the point clouds everywhere. The densities are roughly equal.
点云密度：用以描述点云密集程度的指标，采用单位体积内点云的数量来表征。具体计算方法及方法参数可依据设备情况来决定。Point cloud density: an indicator used to describe the density of point clouds, which is represented by the number of point clouds per unit volume. The specific calculation method and method parameters can be determined according to the equipment conditions.
扫描距离L：指点云距离激光雷达中心的距离，可以用将点云数据投影为鸟瞰图后的平面距离来表征。Scanning distance L: refers to the distance between the point cloud and the center of the lidar, which can be represented by the plane distance after projecting the point cloud data into a bird's-eye view.
点云目标检测算法：指用以将点云数据检测判定为某一具体类别(如大型车辆、小型车辆、行人等)的目标检测算法，其作用与本文所述的目标识别方法不同，本文所述方法仅识别特定的点云数据集，而不检测其具体类别。Point cloud target detection algorithm: refers to the target detection algorithm used to detect point cloud data as a specific category (such as large vehicles, small vehicles, pedestrians, etc.), and its function is different from the target recognition method described in this paper. The methods described above only identify specific point cloud datasets without detecting their specific categories.
检测置信度P：将可以表征某物体的点云数据输入点云目标检测算法中，得到的输出结果的置信度。特别地，若点云目标检测算法并未检测出结果，则视其检测置信度为0。Detection confidence P: Input the point cloud data that can characterize an object into the point cloud target detection algorithm, and get the confidence of the output result. In particular, if the point cloud target detection algorithm does not detect the result, the detection confidence is regarded as 0.
识别触发距离阈值DT：使得大多数点云目标检测算法能表现良好的感知距离范围。本发明中将点云目标检测算法表现良好定义为区域内所有非静态物体都能被检测出，且检测置信度均不低于75％。Recognition Trigger Distance Threshold DT: The perceived distance range that enables most point cloud object detection algorithms to perform well. In the present invention, good performance of the point cloud target detection algorithm is defined as all non-static objects in the area can be detected, and the detection confidence is not lower than 75%.
识别用点云数据：利用识别触发距离阈值对点云数据进行裁剪后的结果，在本发明表述中以加双引号(”)表示提取识别用点云数据操作。Point cloud data for identification: the result of trimming the point cloud data by using the identification trigger distance threshold, in the description of the present invention, double quotation marks (") are used to indicate the operation of extracting point cloud data for identification.
待识别的识别用点云数据：将待识别有效点云数据利用识别触发距离阈值DT裁剪后，得到的点云数据集即为待识别的识别用点云数据，在说明中记为识别用第i帧数据Di”。Recognition point cloud data to be recognized: After cutting the valid point cloud data to be recognized by the recognition trigger distance threshold DT, the obtained point cloud data set is the point cloud data to be recognized for recognition, which is recorded as the first point cloud data for recognition in the description. i-frame data Di".
体素：体积元素(Volume Pixel)的简称，类似像素于二维空间的定义，是三维空间分割上的最小单位，表现形式为空间立方体，立方体的边长大小可由人为确立，不同大小的体素描述的模型精细度不同。Voxel: Abbreviation for Volume Pixel, similar to the definition of pixel in two-dimensional space, it is the smallest unit in three-dimensional space division, and it is expressed in the form of a space cube. The side length of the cube can be established manually. The described models vary in granularity.
体素化：将点云数据转化为体素的操作，在本发明的表述中以下标 v表示。 Voxelization: The operation of converting point cloud data into voxels, which is represented by the subscript v in the expression of the present invention.
待识别体素化点云数据：将待识别的识别用点云数据进行体素化操作后，得到的点云数据集即为待识别体素化点云数据，在说明中记为体素化第i帧数据Dv”。Voxelized point cloud data to be recognized: After the point cloud data for identification to be identified is subjected to voxelization operation, the obtained point cloud data set is the voxelized point cloud data to be identified, which is recorded as voxelization in the description The i-th frame data Dv".
点云坐标系原点：点云数据通常以三维坐标形式表示，在本发明中将点云数据的坐标系原点记为点云坐标系原点。Origin of point cloud coordinate system: point cloud data is usually represented in the form of three-dimensional coordinates. In the present invention, the origin of the coordinate system of the point cloud data is recorded as the origin of the point cloud coordinate system.
环形备用识别区域：为避免因前述裁剪操作使得部分静态或非静态物体不完整，在识别触发距离阈值的外侧添加一个环形的备用识别区域，并按固定的角度分割为多个子区域；当在靠近识别触发距离阈值边缘的位置识别出非静态区域，则记录非静态区域与点云坐标系X轴的水平夹角，在非静态区域上添加该夹角对应的环形备用识别子区域。Circular spare identification area: In order to avoid incomplete static or non-static objects due to the aforementioned cropping operation, a circular spare identification area is added to the outside of the recognition trigger distance threshold, and is divided into multiple sub-areas at a fixed angle; Identify the position of the trigger distance threshold edge and identify the non-static area, record the horizontal angle between the non-static area and the X-axis of the point cloud coordinate system, and add the annular spare identification sub-area corresponding to the angle to the non-static area.
滑动窗口法：利用固定大小的数据筛选框，沿着某一方向连续地从原数据中筛选出部分子数据，使得操作只应用于这些子数据，减小数据处理量并加强对局部特征的识别。Sliding window method: Use a fixed-size data screening box to continuously filter out part of the sub-data from the original data along a certain direction, so that the operation is only applied to these sub-data, reducing the amount of data processing and strengthening the identification of local features .
背景差法：通常指采用图像序列中的当前帧和背景参考模型比较来检测非静态物体的一种方法，在本发明中将其应用于三维点云数据中。Background difference method: usually refers to a method of detecting non-static objects by comparing the current frame in the image sequence with the background reference model, which is applied to 3D point cloud data in the present invention.
静态区域A s：某一包含多个体素的空间区域，当其中的体素的数量、位置、分布等特征与静态点云背景相比，变化幅度均小于判定阈值时，可认为该空间区域内的物体或场景未发生变化，即为静态区域。实质为静态物体的子集。 Static area A s : a spatial area containing multiple voxels, when the number, location, distribution and other characteristics of voxels in it are smaller than the determination threshold compared with the static point cloud background, it can be considered that the spatial area is within the The object or scene does not change, that is, the static area. Essentially a subset of static objects.
非静态区域A ns：某一包含多个体素的空间区域，当其中的体素的数量、位置、分布等特征与静态点云背景相比，变化幅度存在大于判定阈值的情况时，可认为该空间区域内的物体或场景已发生变化，即为非静态区域。 Non-static area A ns : a spatial area containing multiple voxels, when the number, position, distribution and other characteristics of the voxels are compared with the static point cloud background, and the variation range is greater than the judgment threshold, it can be considered that the Objects or scenes in a spatial area have changed, that is, a non-static area.
临时静态区域A st：某一包含多个体素的空间区域，由固定频率将非静态区域保存而成。短时静态区域用于判定动态物体与短时静态物体，分离出动态物体后的部分即为短时静态物体。 Temporary static area A st : a spatial area containing multiple voxels, which is formed by saving the non-static area at a fixed frequency. The short-term static area is used to determine the dynamic object and the short-term static object, and the part after the dynamic object is separated is the short-term static object.
动态区域A d：某一包含多个体素的空间区域，属于非静态区域，当其中的体素的数量、位置、分布等特征与短时静态区域相比，变化幅度存在大于判定阈值的情况时，可认为该空间区域内的物体或场景已发生变化，即为动态区域。实质为动态物体的子集。 Dynamic area A d : a spatial area containing multiple voxels, belonging to a non-static area, when the number, location, distribution and other characteristics of voxels in it are compared with the short-term static area, the variation range is larger than the judgment threshold , it can be considered that the object or scene in the space area has changed, that is, the dynamic area. Essentially a subset of dynamic objects.
发明内容SUMMARY OF THE INVENTION
本发明提供了一种基于路侧感知单元的动静态物体快速识别及点云分割方法，面向真实交通环境，以路侧激光雷达位置相对固定为基础，将前期采集的点云背景数据作为先验信息，通过比对待识别帧与背景数据，快速筛选出有明显变化的待识别区域，大大缩减无效点云数据，减小数据传输量。并且本发明的适用场景不限于单雷达环境，对多雷达组网亦可适用，同时提取出的待识别区域还可与其他数据格式进行融合处理，提升检测精度。The invention provides a method for fast identification of dynamic and static objects and point cloud segmentation based on a roadside perception unit, facing the real traffic environment, based on the relative fixed position of the roadside laser radar, and using the point cloud background data collected in the previous stage as a priori By comparing the frame to be identified and the background data, the area to be identified that has obvious changes can be quickly screened out, which greatly reduces the invalid point cloud data and reduces the amount of data transmission. Moreover, the applicable scene of the present invention is not limited to a single radar environment, but can also be applied to a multi-radar network. At the same time, the extracted to-be-identified area can also be fused with other data formats to improve detection accuracy.
本发明的流程图如图1和图2所示，其特征包括如下步骤：The flow chart of the present invention is shown in Figure 1 and Figure 2, and its features include the following steps:
(一)数据采集(1) Data collection
面向车路协同环境，搭建路侧激光雷达感知场景，并采集一批点云数据用于预处理阶段。采集的点云数据要求覆盖全面无遮挡，应当包含静态物体，主要为路面及其附属设施、建筑物和路侧绿化植物等，且还应当包含足够的非静态物体，如行人、非机动车和车辆等，可人工分辨出，建议总数不应低于300个。For the vehicle-road collaborative environment, build a roadside lidar perception scene, and collect a batch of point cloud data for the preprocessing stage. The collected point cloud data requires full coverage and no occlusion, and should include static objects, mainly the road surface and its ancillary facilities, buildings and roadside green plants, etc., and should also include enough non-static objects, such as pedestrians, non-motor vehicles and Vehicles, etc., can be identified manually, and it is recommended that the total number should not be less than 300.
所搭建的路侧激光雷达场景应当保障扫描范围内不存在大面积遮挡现象，即扫描范围内所有关键路面区域应当清晰可见。图3左图所示为受中央分隔带的影响而失去半侧路幅数据的不良布点，图3右图为较为良好的示例图。The roadside lidar scene built should ensure that there is no large-area occlusion within the scanning range, that is, all key road areas within the scanning range should be clearly visible. The left picture of Fig. 3 shows the bad distribution points that lose half-side road width data due to the influence of the central divider, and the right picture of Fig. 3 is a relatively good example picture.
本发明采集的点云数据格式如下表所示。The format of the point cloud data collected by the present invention is shown in the following table.
Figure PCTCN2022084912-appb-000001
Figure PCTCN2022084912-appb-000001
点云数据包含三维坐标数据X、Y、Z，反射强度值Intensity，三通道颜色数值RGB以及回波次数Return Number。本发明仅使用其中的三维坐标数据X、Y、Z作为点云数据提取依据，而传输筛选结果时则选择三维坐标数据、反射强度值和三通道颜色数值共同传递，避免信息缺失导致车端无法利用该数据。应当理解的是，本发明适用的点云数据格式不止于上述示例，凡包含三维坐标数据X、Y、Z的点云数据均可作为本发明适用范围之内。The point cloud data includes three-dimensional coordinate data X, Y, Z, reflection intensity value Intensity, three-channel color value RGB, and return number of echoes. The present invention only uses the three-dimensional coordinate data X, Y, and Z as the point cloud data extraction basis, and selects the three-dimensional coordinate data, reflection intensity value and three-channel color value to be transmitted together when transmitting the screening result, so as to avoid the failure of the vehicle end due to the lack of information. Use this data. It should be understood that the applicable point cloud data format of the present invention is not limited to the above examples, and any point cloud data including three-dimensional coordinate data X, Y, and Z can be regarded as the applicable scope of the present invention.
本发明所述数据采集分为两个阶段：其一为服务于预处理工作的数据采集阶段：为预处理工作提供数据源的采集工作，该阶段要求采集的数据应当符合下文所述的各项要求，旨在综合反映路测感知单元安装场景的正常路况；其二为服务于日常使用的数据采集阶段：该阶段对数据采集无详细要求，仅需保障路侧感知单元正常工作即可。在本发明中，将预处理阶段采集获 得的点云数据集统称为原始数据D 0，将使用阶段采集获得的点云数据按时间顺序(帧数)依次命名为D 1、D 2等。以下重点阐述预处理阶段的数据采集工作。 The data collection described in the present invention is divided into two stages: one is the data collection stage serving the preprocessing work: the collection work that provides the data source for the preprocessing work, and the data collected in this stage should conform to the items described below. The purpose is to comprehensively reflect the normal road conditions of the installation scene of the drive test sensing unit; the second is the data collection stage for daily use: this stage has no detailed requirements for data collection, and only needs to ensure the normal operation of the roadside sensing unit. In the present invention, the point cloud data sets collected in the preprocessing stage are collectively referred to as original data D 0 , and the point cloud data collected in the use stage are named D 1 , D 2 and so on in chronological order (number of frames). The following focuses on the data collection in the preprocessing stage.
考虑到后期预处理需要，一般要求采集不少于1000帧的数据，具体采集帧数可按照路侧感知单元的布设场景适当调整。采集过程中要考虑到环境因素的影响，如雨天地面潮湿导致的激光雷达回波数缩减造成的路面点云密度和质量的双重下降，或夜间车辆开启高亮度的远光灯，干扰激光雷达的正常回波，从而造成的部分区域点云质量异常化等。另外，虽然一般认为可见光的强度对激光雷达的影响很小，但出于严谨性考虑，采集的数据应当适当分散于多个时间段。基于上述需求，本发明建议采取的原始数据采集方案为：Considering the need for post-processing, it is generally required to collect no less than 1000 frames of data, and the specific number of frames to be collected can be adjusted appropriately according to the layout scene of the roadside sensing unit. In the acquisition process, the influence of environmental factors should be taken into account, such as the double decrease of road point cloud density and quality caused by the reduction of the number of lidar echoes caused by the wet ground in rainy days, or the high-brightness high beam of vehicles at night, which interferes with the normal operation of lidar. Echoes, resulting in abnormal point cloud quality in some areas, etc. In addition, although it is generally believed that the intensity of visible light has little effect on lidar, for the sake of rigor, the collected data should be appropriately dispersed over multiple time periods. Based on the above requirements, the original data collection scheme proposed by the present invention is:
①条件允许时，在大雨天气或通过路面大面积大量洒水模拟雨天天气，运行路侧激光雷达采集200帧以上数据；①When conditions permit, run the roadside lidar to collect more than 200 frames of data in heavy rain or by sprinkling a large area of the road to simulate rainy weather;
②在夜间利用强光发生设备照射路面，尤其应当覆盖由高反射率材料制成的路面标志标识，采集200帧以上数据；②Use strong light generating equipment to illuminate the road at night, especially the road signs made of high reflectivity materials should be covered, and more than 200 frames of data should be collected;
③在日常环境中，分别采集早中晚三个时间段的数据，每个时间段各采集200帧以上数据。③In the daily environment, the data of three time periods of morning, middle and evening are collected respectively, and more than 200 frames of data are collected in each time period.
应当理解的是，除上述采集方案外，其他根据实际场景进行调整的采集方案应当均属于本发明适用的采集方案变种之一，如下述举例的变种方案。It should be understood that, in addition to the above-mentioned collection scheme, other collection schemes adjusted according to actual scenarios should all belong to one of the collection scheme variants applicable to the present invention, such as the following exemplified variant schemes.
面向全年少雨但风沙较多的地区可采用如下变种采集方案：For areas with little rainfall throughout the year but more sandstorms, the following variant collection schemes can be used:
①在沙尘或大风等视线不好的天气时，运行路侧激光雷达采集200帧以上数据；①In the weather with poor visibility such as sand or strong wind, run the roadside lidar to collect more than 200 frames of data;
②在夜间利用强光发生设备照射路面，尤其应当覆盖由高反射率材料制成的路面标志标识，采集200帧以上数据；②Use strong light generating equipment to illuminate the road at night, especially the road signs made of high reflectivity materials should be covered, and more than 200 frames of data should be collected;
③在晴朗环境中，分别采集早中晚三个时间段的数据，每个时间段各采集200帧以上数据。③ In a sunny environment, data were collected in three time periods of morning, middle and evening, and more than 200 frames of data were collected in each time period.
面向北方冬季漫长、冰雪覆盖路面情况较严重的区域可采用如下变种采集方案：For areas facing the north with long winters and serious snow-covered road conditions, the following variant collection schemes can be used:
①在大雪天气或路面被冰雪覆盖时，运行路侧激光雷达采集200帧以上数据；①In heavy snow or when the road is covered with ice and snow, run the roadside lidar to collect more than 200 frames of data;
②在夜间利用强光发生设备照射路面，尤其应当覆盖由高反射率材料制成的路面标志标识，采集200帧以上数据；②Use strong light generating equipment to illuminate the road at night, especially the road signs made of high reflectivity materials should be covered, and more than 200 frames of data should be collected;
③在晴朗环境中，分别采集早中晚三个时间段的数据，每个时间段各采集200帧以上数据。③ In a sunny environment, data were collected in three time periods of morning, middle and evening, and more than 200 frames of data were collected in each time period.
本发明适用的数据采集方案包括但不限于上述案例。Applicable data collection schemes of the present invention include but are not limited to the above cases.
在上述环境下，应当结合路测激光雷达扫描范围的交通情况进行数据采集安排，应保障采集的数据中，包含车辆或行人数量低于2个的单帧数据占比不低于50％，且车辆或行人数量等非静态物体的样本总数不低于300个。同时，不应存在某一可视区域被长期遮挡的情况，即需保障关键路面区域清晰可见的帧数占比不低于数据总数的90％。若采集的数据无法达到以上条件需重新选择时段再次采集。In the above environment, data collection arrangements should be made in combination with the traffic conditions within the scanning range of the road test lidar, and it should be ensured that the collected data contains no less than 50% of the single-frame data with fewer than 2 vehicles or pedestrians, and The total number of samples of non-static objects such as the number of vehicles or pedestrians is not less than 300. At the same time, there should not be a situation where a certain visible area is blocked for a long time, that is, it is necessary to ensure that the number of frames that are clearly visible in key road areas should not be less than 90% of the total data. If the collected data cannot meet the above conditions, you need to re-select the time period to collect again.
(二)预处理(2) Preprocessing
首先确立边界方程组E b，将无效数据从原始数据中剔除。再通过对点云数据等间距采样获得点云密度-扫描距离的线性关系。再将每帧点云数据中的静态物体与非静态物体分离，其中利用点云目标检测算法对非静态物体检测以建立检测置信度与扫描距离的分布曲线，并选取 置信度均高于阈值的扫描距离范围作为识别触发距离阈值。再利用识别触发距离阈值对所有静态物体进行裁剪，将裁剪后的多帧识别用静态物体通过叠加并适当采样的方式建立静态点云背景B v。最后，对静态点云背景执行体素化操作。预处理阶段的流程图如图x所示。 First, the boundary equation system E b is established, and the invalid data is eliminated from the original data. Then, the linear relationship between point cloud density and scanning distance is obtained by sampling the point cloud data at equal intervals. Then separate the static objects and non-static objects in each frame of point cloud data, and use the point cloud target detection algorithm to detect the non-static objects to establish the distribution curve of detection confidence and scanning distance, and select the ones whose confidence is higher than the threshold. The scanning distance range is used as the recognition trigger distance threshold. Then use the recognition trigger distance threshold to crop all static objects, and build a static point cloud background B v by stacking and properly sampling the cropped multi-frame recognition static objects. Finally, perform a voxelization operation on the static point cloud background. The flowchart of the preprocessing stage is shown in Figure x.
首先，路侧激光雷达的扫描范围通常比较广阔，高水准的设备的最远扫描距离都在百米以上，而其有效扫描距离一般也能达到50米以上。因此，扫描得到的数据中必然包含了数量众多的各类物体，如周边建筑物、绿化植物等。该类物体相比于车辆、行人和路面等要素，对于交通环境的检测而言，数据价值非常低，可以直接将其剔除。故此，在本发明中，将距离道路边缘5m以外且几乎没有数据价值的点云数据定义为无效数据，通常包括道路两侧的建筑物、边坡及空地等区域，与之相对的道路和非静态物体等则定义为有效数据。无效数据可在后续计算工作前加以剔除，减少数据处理量。本发明建议的无效数据的剔除方法为：First of all, the scanning range of roadside lidar is usually relatively wide. The farthest scanning distance of high-level equipment is more than 100 meters, and its effective scanning distance can generally reach more than 50 meters. Therefore, the scanned data must contain a large number of various objects, such as surrounding buildings and green plants. Compared with elements such as vehicles, pedestrians, and road surfaces, such objects have very low data value for traffic environment detection and can be directly eliminated. Therefore, in the present invention, point cloud data that is 5m away from the road edge and has almost no data value is defined as invalid data, which usually includes areas such as buildings, slopes, and open spaces on both sides of the road. Static objects, etc. are defined as valid data. Invalid data can be eliminated before subsequent calculation work, reducing the amount of data processing. The method for eliminating invalid data proposed by the present invention is:
①先将点云数据投影至水平面，即仅考虑点云数据中的X、Y值，形成鸟瞰图。① Project the point cloud data to the horizontal plane first, that is, only consider the X and Y values in the point cloud data to form a bird's-eye view.
②基于人工手段确立剔除边界，可利用一些常见的点云数据可视化处理工具来完成，如3D Reshaper等，剔除边界即为明确分割无效数据与有效数据的界限，实际选取中建议采取保守策略，如无明显道路边界时，人工较难分辨某区域为道路或非道路，则可认为其为道路区域，即视为有效数据。② Establish the culling boundary based on artificial means, which can be done by using some common point cloud data visualization processing tools, such as 3D Reshaper, etc. The culling boundary is to clearly divide the boundary between invalid data and valid data. When there is no obvious road boundary, it is difficult to manually distinguish an area as a road or a non-road, and it can be regarded as a road area, which is regarded as valid data.
③基于边界上的点构建边界方程，依据前面选择的边界，在每条边界上选取适当的点数，利用最小二乘法等方式拟合出边界的平面方程。一般选取的点数不应少于30个点，且两点间距不应小于50cm。若边界的长度无法满足上述要求，则可考虑邻近边界之间的整合。出于计算量考虑，建议边界方程数量不大于6条。③Construct the boundary equation based on the points on the boundary, select the appropriate number of points on each boundary according to the previously selected boundary, and use the least squares method to fit the boundary plane equation. Generally, the number of selected points should not be less than 30 points, and the distance between the two points should not be less than 50cm. If the length of the boundary cannot meet the above requirements, the integration between adjacent boundaries can be considered. Considering the amount of calculation, it is recommended that the number of boundary equations should not exceed 6.
④最后统合所有边界方程为边界方程组E b，并记录数据剔除的方向，以其为筛选条件作为之后实际识别过程中的数据筛选条件。 ④Finally unify all boundary equations into boundary equation group E b , and record the direction of data elimination, which is used as the screening condition as the data screening condition in the actual identification process later.
应当理解的是，由于路侧感知单元的安装场景存在差异，对于无效数据的定义也会合理地存在偏差，如针对城市快速路或高架路，道路两侧理应不存在干扰交通流的物体，则凡是护栏两侧外的点云数据均可剔除。针对上述场景可采取如下变种无效数据剔除方法：It should be understood that due to differences in the installation scenarios of roadside sensing units, there will be reasonable deviations in the definition of invalid data. For example, for urban expressways or elevated roads, there should be no objects that interfere with traffic flow on both sides of the road. The point cloud data on both sides of the guardrail can be eliminated. For the above scenarios, the following variants of invalid data removal methods can be adopted:
若点云数据投影至水平面上后，绿化植物、路侧设施等物体并未侵入道路空间，即不存在上述物体的点云数据覆盖了道路数据的情况，则处理方法与本发明建议方法相同；若其已侵入道路空间，如图5所示，树冠已覆盖了部分路面区域，则应在投影至水平面之前先依高度阈值筛选出底部数据，所述高度阈值为树冠底部与关键路面区域通行的常见车辆的最大车高之间的任一值，即满足剔除树冠区域的同时保留车辆数据。If the objects such as green plants and roadside facilities do not invade the road space after the point cloud data is projected on the horizontal plane, that is, there is no situation that the point cloud data of the above objects covers the road data, the processing method is the same as that proposed in the present invention; If it has invaded the road space, as shown in Figure 5, the tree crown has covered part of the pavement area, the bottom data should be filtered out according to the height threshold before projecting to the horizontal plane. The height threshold is the distance between the bottom of the tree crown and the key road area. Any value between the maximum vehicle heights of common vehicles, which satisfies the removal of tree canopy areas while retaining vehicle data.
对于平缓路段，即道路纵坡不大于3％的路段，高度阈值可选择一固定值。对于陡坡路段，即道路坡度大于3％的路段，高度阈值可采用阶梯式分布或构建空间平面方程。阶梯式分布即指对于平面坐标位于某区域内的路段，高度阈值选择同一固定值，其表现形式为：For a gentle road section, that is, a road section with a road longitudinal slope not greater than 3%, a fixed value can be selected for the height threshold. For steep-slope sections, that is, sections with a road gradient greater than 3%, the height threshold can be distributed in a stepwise manner or constructed with a spatial plane equation. Stepwise distribution means that for the road section whose plane coordinates are located in a certain area, the same fixed value is selected for the height threshold, and its expression is as follows:
Figure PCTCN2022084912-appb-000002
Figure PCTCN2022084912-appb-000002
其中X，Y对应点云数据的X、Y值，x 1,x 2,x 3,x 4和y 1,y 2,y 3,y 4分别表示X、Y方向的上下区域阈值，H表示高度阈值，h 1,h 2表示不同平面坐标区域所选取的高度阈值。 Where X, Y correspond to the X and Y values of the point cloud data, x 1 , x 2 , x 3 , x 4 and y 1 , y 2 , y 3 , y 4 represent the upper and lower area thresholds in the X and Y directions, respectively, and H represents Height threshold, h 1 , h 2 represent the height thresholds selected for different plane coordinate regions.
空间平面方程即通过路面点的X、Y、Z坐标拟合平面方程，再使之平移向上以使得平面满足高度阈值的分割条件。其表现形式为：The space plane equation is to fit the plane equation through the X, Y, and Z coordinates of the road points, and then translate it upward to make the plane meet the segmentation condition of the height threshold. Its manifestation is:
Ax+By+Cz+D＝0Ax+By+Cz+D=0
其中A、B、C、D均为平面方程的系数，x、y、z分别对应点云数据的X、Y、Z坐标。拟合时需要对路段数据进行随机采样，采样点数不少于100点，利用最小二乘法拟合平面方程。使用时，可依据点云X、Y坐标计算Z值，所得Z值即为当前区域的高度阈值。Among them, A, B, C, and D are the coefficients of the plane equation, and x, y, and z correspond to the X, Y, and Z coordinates of the point cloud data, respectively. During fitting, random sampling of road section data is required, and the number of sampling points is not less than 100 points, and the least squares method is used to fit the plane equation. When in use, the Z value can be calculated according to the X and Y coordinates of the point cloud, and the obtained Z value is the height threshold of the current area.
上述处理完成后，后续的处理方法与前述本发明建议的无效数据剔除方法相同。After the above processing is completed, the subsequent processing method is the same as the invalid data elimination method proposed by the present invention.
本发明适用的无效数据剔除方法不限于上述方法，其他起相同作用的方法均可作为变种方案之一。另外，无效数据剔除方法可置于其他步骤之后，本发明仅出于减小数据计算量考虑而将其作为获得点云密度与扫描距离的线性关系的前置步骤，凡其他变更本发明步骤先后顺序的技术方案应均当视为本发明的变种方案。经上述步骤处理后，得到有效数据的集合D 0′。 The method for eliminating invalid data applicable to the present invention is not limited to the above method, and other methods with the same function can be used as one of the variant schemes. In addition, the invalid data elimination method can be placed after other steps. The present invention only considers it as a pre-step to obtain the linear relationship between the point cloud density and the scanning distance for the sake of reducing the amount of data calculation. The sequential technical solutions should all be regarded as variants of the present invention. After the above steps are processed, a set D 0 ' of valid data is obtained.
其次，由于受硬件自身的物理采集能力限制，点云数据具有越靠近内部越稠密，越靠近外部越稀疏的特性，即点云数据中点云密度与其扫描距离息息相关。而点云目标识别算法与目标的点云密度密切相关，一般点云密度大的物体更容易被识别出。故为了提升后续点云目标识别的准确率，需首先确立点云密度与扫描距离的关系。由激光雷达的扫描原理可知，同一环线上的两点间距与环线距中心的距离呈线性关系，故推断点云密度与扫描距离之间同为线性关系。Secondly, due to the limitation of the physical acquisition capability of the hardware itself, the point cloud data has the characteristics that the closer it is to the interior, the denser it is, and the closer it is to the outside, the sparser it is, that is, the point cloud density in the point cloud data is closely related to its scanning distance. The point cloud target recognition algorithm is closely related to the point cloud density of the target. Generally, objects with high point cloud density are easier to be identified. Therefore, in order to improve the accuracy of subsequent point cloud target recognition, it is necessary to first establish the relationship between the point cloud density and the scanning distance. It can be seen from the scanning principle of lidar that the distance between two points on the same ring line is linear with the distance between the ring line and the center, so it is inferred that the point cloud density and the scanning distance are also linear.
如图7所示，以0.5米为采样间隔，由内至外在各扫描环线等间距处采集样本点，记录以每个采样点为中心，半径10cm范围内的点数作为点云密度，平均每条环线上选取30个采样点，统计结果填入下表。As shown in Figure 7, with a sampling interval of 0.5 meters, sample points are collected from the inside to the outside at equal intervals of each scanning ring line, and the number of points within a radius of 10 cm with each sampling point as the center is recorded as the point cloud density. 30 sampling points were selected on the ring line, and the statistical results were filled in the following table.
点云密度point cloud density 第1采样点1st sampling point 第2采样点2nd sampling point ……... 第30采样点30th sampling point 点云密度均值Mean point cloud density
第1环线1st loop               
第2环线2nd Ring Road               
第3环线3rd Ring Road               
……...               
统计每条环线的点云密度均值，将点云密度均值与其对应的环线距离分别作为x、y值，利用最小二乘法拟合，即可确立点云密度-扫描距离的线性关系，表示为：Calculate the mean point cloud density of each loop line, take the mean point cloud density and its corresponding loop line distance as the x and y values respectively, and use the least squares method to fit the linear relationship between the point cloud density and the scanning distance, which is expressed as:
ρ＝k·Lρ=k·L
其中ρ为点云密度，L为扫描距离，k为线性函数参数。where ρ is the point cloud density, L is the scanning distance, and k is the linear function parameter.
完成上述步骤后，需通过人工提取的方式将每帧点云数据分割为两个部分，即静态物体D s，和非静态物体D ns。所谓静态物体，是指路面及其附属设施、建筑物和路侧绿化植物等，在不考虑改扩建、频繁路面维修的情况下，该类物体处于长期位置、状态无变化状态。通常一月内位置、外观无明显变化的物体即可视为静态物体。而非静态物体则是点云数据中除去静态物体以外的物体的总和，又分为动态物体D d和短时静态物体D st。动态物体是指行驶的车辆、走动的行人等，该类物体被观察时即处于运动状态，可认为不属于静态物体且连续2帧内发生 较明显位置变化或外观变化的物体即为动态物体。短时静态物体是指临时停车、伫立的行人等，该类物体处于短期位置、状态无变化状态，但不排除其下一刻运动的可能性，在本发明中认为不属于静态物体且在5帧内未发生明显位置变化和外观变化的物体即为短时静态物体。 After the above steps are completed, each frame of point cloud data needs to be divided into two parts by manual extraction, namely the static object D s and the non-static object D ns . The so-called static objects refer to the road surface and its ancillary facilities, buildings and roadside green plants, etc., such objects are in a long-term position and state without any change in the case of reconstruction and expansion and frequent road maintenance. Usually, objects with no obvious changes in position and appearance within a month can be regarded as static objects. The non-static object is the sum of objects other than static objects in the point cloud data, and is further divided into dynamic objects D d and short-term static objects D st . Dynamic objects refer to moving vehicles, walking pedestrians, etc. These objects are in motion when they are observed. It can be considered that objects that are not static objects and have obvious position changes or appearance changes within 2 consecutive frames are dynamic objects. Short-term static objects refer to temporary parking, standing pedestrians, etc. Such objects are in a short-term position and state without change, but the possibility of their next moment movement is not excluded. Objects that do not have obvious position changes and appearance changes are short-term static objects.
静态物体D s用于提取静态点云背景B。所谓静态点云背景是指不包含任何短时静态物体和动态物体的纯静态背景空间，对本发明面向的车路协同场景而言，即不包含任何非永久驻车的车辆、行人等交通参与者的交通环境，其效果如图4所示。 The static object D s is used to extract the static point cloud background B. The so-called static point cloud background refers to a pure static background space that does not contain any short-term static objects and dynamic objects. For the vehicle-road collaboration scene for the present invention, that is, it does not contain any non-permanently parked vehicles, pedestrians and other traffic participants. traffic environment, and its effect is shown in Figure 4.
而非静态物体D ns则用来获取识别触发距离阈值DT。所谓识别触发距离阈值是指使得大多数点云目标检测方法能表现良好的感知距离范围。因为点云数据有外疏内密现象的存在，使得较远处的非静态物体可能只有一或两条扫描线来描述，而如此稀少的点云数是很难被大多数点云目标检测算法所检测的。因此为了满足大多数点云检测算法的检测需求，有必要确立一个合适的识别触发距离阈值，该阈值用于表示后续方法的触发距离。对于位于识别触发距离阈值以外部分的点云，即使存在非静态物体，其或者无法被检测出，或者检测结果的置信度过低，传输给车端后反而可能导致车端决策失误。故位于识别触发距离阈值以外的点云数据将被视为低价值数据而剔除。 The non-static object D ns is used to obtain the recognition trigger distance threshold DT. The so-called recognition trigger distance threshold refers to the perceptual distance range that enables most point cloud target detection methods to perform well. Because the point cloud data has the phenomenon of outer sparseness and inner density, distant non-static objects may only be described by one or two scan lines, and such a sparse number of point clouds is difficult to be detected by most point cloud target detection algorithms. detected. Therefore, in order to meet the detection requirements of most point cloud detection algorithms, it is necessary to establish an appropriate recognition trigger distance threshold, which is used to represent the trigger distance of subsequent methods. For the point cloud located outside the recognition trigger distance threshold, even if there is a non-static object, it either cannot be detected, or the confidence of the detection result is too low, which may lead to a decision error on the vehicle side after being transmitted to the vehicle. Therefore, point cloud data outside the recognition trigger distance threshold will be regarded as low-value data and eliminated.
要获得识别触发距离阈值DT，需先确立扫描距离L与检测置信度P之间的关系。本发明的最终结果将提供给车端使用，而目前各自动驾驶车辆生产厂家内置的点云目标检测算法各不相同，故出于实用性考虑，本发明选取几种较常见的点云目标检测算法作为预处理阶段的测试算法，包括VoxelNet、PIXOR、PointRCNN三种。其中：To obtain the recognition trigger distance threshold DT, it is necessary to establish the relationship between the scanning distance L and the detection confidence P first. The final result of the present invention will be provided to the vehicle for use. At present, the built-in point cloud target detection algorithms of various autonomous vehicle manufacturers are different. Therefore, for practical reasons, the present invention selects several common point cloud target detection methods. The algorithm is used as a test algorithm in the preprocessing stage, including VoxelNet, PIXOR, and PointRCNN. in:
VoxelNet是典型的体素化点云处理方法，其将三维点云划分为一定数量的体素，经过点的随机采样以及归一化后，对每一个非空体素使用若干个体素特征编码层进行局部特征提取，得到Voxel-wise特征，然后经过三维卷积化操作进一步抽象特征，并在此过程中增大感受域并学习几何空间表示，最后使用Region Proposal Network对物体进行分类检测与位置回归。VoxelNet is a typical voxelized point cloud processing method. It divides the 3D point cloud into a certain number of voxels. After random sampling and normalization of points, several voxel feature encoding layers are used for each non-empty voxel. Perform local feature extraction to obtain Voxel-wise features, and then further abstract features through three-dimensional convolution operation, and increase the receptive field and learn geometric space representation in the process, and finally use Region Proposal Network to classify objects and perform position regression. .
PIXOR是典型的图像化点云处理方法，通过将点云投影得到以高度和反射率为通道的二维鸟瞰图，然后使用结构微调的RetinaNet进行物体检测与定位。总体处理过程更类似于传统的图像目标检测方法。PIXOR is a typical image point cloud processing method. By projecting the point cloud, a two-dimensional bird's-eye view with height and reflectivity channels is obtained, and then the fine-tuned RetinaNet is used for object detection and localization. The overall process is more similar to traditional image object detection methods.
PointRCNN是典型的利用原始点云数据结构的点云处理方法，整个框架包括两个阶段：第一阶段使用自下而上的3D提案产生，第二阶段用于在规范坐标中修改提案获得最终的检测结果。阶段1子网络不是从RGB图像或者将点云投影到鸟瞰图或者体素中，而是通过将整个场景的点云分割为前景点和背景点，以自下而上的方式直接从点云生成少量高质量的三维样例。阶段2子网络将每个样例的池化的点转换为规范坐标，更好地学习局部空间特征，这个过程与阶段1中学习每个点的全局语义特征相结合，用于Box优化和置信度预测。PointRCNN is a typical point cloud processing method that utilizes the original point cloud data structure. The whole framework includes two stages: the first stage uses bottom-up 3D proposal generation, and the second stage is used to modify the proposal in canonical coordinates to obtain the final Test results. Instead of projecting the point cloud into a bird's-eye view or voxels from an RGB image, the stage 1 sub-network is generated directly from the point cloud in a bottom-up manner by segmenting the point cloud of the entire scene into foreground and background points. A small number of high-quality 3D samples. The stage 2 sub-network transforms the pooled points of each example into canonical coordinates to better learn local spatial features, this process is combined with the learning of global semantic features of each point in stage 1 for Box optimization and confidence Degree prediction.
上述三种方法是目前最主流的三类点云目标检测算法中的典型代表，可以较好的模拟自动驾驶车辆的智能感知。应当理解的是，本发明所选择的上述三种算法无法完全代表所有的点云目标检测算法，因此，若采用其他点云目标检测算法作为预处理阶段的测试算法也是合理的，应当视为变种方案之一。The above three methods are typical representatives of the most mainstream three types of point cloud target detection algorithms, which can better simulate the intelligent perception of autonomous vehicles. It should be understood that the above three algorithms selected by the present invention cannot fully represent all point cloud target detection algorithms. Therefore, it is reasonable to use other point cloud target detection algorithms as the test algorithms in the preprocessing stage, and should be regarded as variants. one of the options.
由于常见车辆的长度一般在3.5米至4.5之间，一般会跨越了多条扫描线，可能造成车辆前部点云密集，后部点云稀疏，乃至于被遮挡而几乎无点云数据。因此，需要明确以车辆为代表的非静态物体的平均点云密度。Since the length of common vehicles is generally between 3.5 meters and 4.5 meters, they generally span multiple scan lines, which may cause dense point clouds in the front of the vehicle and sparse point clouds in the rear, or even occluded and almost no point cloud data. Therefore, the average point cloud density of non-static objects represented by vehicles needs to be specified.
本发明对非静态物体同样采用随机采样法获取其平均点云密度，但此处的采样方法与上述点云采样略有不同，具体如下所述。点云采样的比例需参考硬件设备参数和目标实际的点云总数确立，本发明建议采用的随机采样法为：The present invention also adopts the random sampling method to obtain the average point cloud density of non-static objects, but the sampling method here is slightly different from the above point cloud sampling, as described below. The ratio of point cloud sampling needs to be established with reference to hardware equipment parameters and the actual total number of point clouds of the target. The random sampling method proposed by the present invention is:
①对于点云总数大于3000的非静态物体，随机采样3次，每次采样300点，最终计算3次采样的平均点云密度；① For non-static objects with a total number of point clouds greater than 3000, randomly sample 3 times with 300 points each time, and finally calculate the average point cloud density of 3 samplings;
②对于点云总数大于1000的非静态物体，随机采样3次，每次采样100点，最终计算3次采样的平均点云密度；② For non-static objects with a total number of point clouds greater than 1000, randomly sample 3 times, each time sampling 100 points, and finally calculate the average point cloud density of 3 samplings;
③对于点云总数大于500的非静态物体，随机采样3次，每次采样75点，最终计算3次采样的平均点云密度；③ For non-static objects with a total number of point clouds greater than 500, randomly sample 3 times, each sampling 75 points, and finally calculate the average point cloud density of the 3 samplings;
④对于点云总数小于500的非静态物体，随机采样100点，计算平均点云密度。④ For non-static objects with a total number of point clouds less than 500, randomly sample 100 points and calculate the average point cloud density.
上诉采样方法中，点云密度的计算方法与前述一致，即以每个采样点为中心，半径10cm范围内的点数作为点云密度。In the sampling method appealed, the calculation method of the point cloud density is the same as the above, that is, with each sampling point as the center, the number of points within a radius of 10cm is used as the point cloud density.
除上述采样方法外，其他用于获取非静态物体点云密度的点云采样方法均可视为本发明所适用的采样方法的变种之一。例如下述针对不同类型的物体可采用不同采样比例的变种方案：In addition to the above sampling methods, other point cloud sampling methods for obtaining the point cloud density of non-static objects can be regarded as one of the variants of the sampling methods applicable to the present invention. For example, the following variants of different sampling ratios can be used for different types of objects:
①对于非静态物体中的行人，随机采样2次，每次采样100点，不足则全采，最终计算2次采样的平均点云密度；①For pedestrians in non-static objects, randomly sample 2 times with 100 points each time, if not enough, all samples are collected, and finally the average point cloud density of 2 times of sampling is calculated;
②对于非静态物体中的非机动车，随机采样3次，每次采样100点，不足则全采，最终计算3次采样的平均点云密度；②For non-motor vehicles in non-static objects, randomly sample 3 times with 100 points each time, if not enough, all samples are collected, and the average point cloud density of 3 times of sampling is finally calculated;
③对于非静态物体中的小型汽车，随机采样3次，每次采样200点，不足则全采，最终计算3次采样的平均点云密度；③ For small cars in non-static objects, randomly sample 3 times, 200 points each time, if not enough, all samples are collected, and finally the average point cloud density of 3 times of sampling is calculated;
④对于非静态物体中的大型汽车，随机采样3次，每次采样300点，不足则全采，最终计算3次采样的平均点云密度。④ For large cars in non-static objects, randomly sample 3 times with 300 points each time, if not enough, all samples are collected, and finally the average point cloud density of 3 times of sampling is calculated.
确立每个非静态物体的平均点云密度后，将各非静态物体输入不同的后端算法进行检测，得到各检测结果与检测置信度P。如图8所示，画出扫描距离L与检测置信度P的分布曲线图，利用如下公式：After the average point cloud density of each non-static object is established, each non-static object is input into different back-end algorithms for detection, and each detection result and detection confidence P are obtained. As shown in Figure 8, draw the distribution curve of the scanning distance L and the detection confidence P, using the following formula:
Figure PCTCN2022084912-appb-000003
Figure PCTCN2022084912-appb-000003
其中，j、i表示识别触发距离阈值DT的上下限，i为最近距离阈值，j为最远距离阈值，n i,n j分别表示距离原点i、j处的动态目标总数，(n j-n i) p>75％表示i、j范围内检测置信度大于75％的非静态目标总数。应当理解的是，75％仅为本发明推荐使用的判定阈值，其实际取值可依据路侧感知单元所安装的场景进行调整。 Among them, j, i represent the upper and lower limits of the recognition trigger distance threshold DT, i is the shortest distance threshold, j is the farthest distance threshold, n i , n j represent the total number of dynamic targets at distances i and j from the origin, (n j − n i ) p>75% means the total number of non-static targets with detection confidence greater than 75% in the range i and j. It should be understood that 75% is only the determination threshold recommended by the present invention, and its actual value can be adjusted according to the scene in which the roadside sensing unit is installed.
需要注意的是，此处有识别触发距离阈值的下限的原因是，激光雷达设备一般都有竖直扫描角参数，即当物理距离激光雷达过近且高度低于激光雷达时，是无法被扫描到的。此时可能出现部分车辆虽然非常靠近内部但只被扫描出半截车身的情况，因此需设立保障能获取完全车身的距离下限值。It should be noted that the reason for the lower limit of the identification trigger distance threshold is that lidar devices generally have vertical scanning angle parameters, that is, when the physical distance to lidar is too close and the height is lower than lidar, it cannot be scanned. arrived. At this time, it may happen that some vehicles are very close to the interior, but only half of the vehicle body is scanned. Therefore, it is necessary to establish a lower limit of the distance to ensure that the complete vehicle body can be obtained.
选择合适的i、j值使得P ij大于75％，作为实际识别时的目标提取范围。一般i、j初始值可通过直接观察图像分布确立，基于初始值，以0.5m为偏移量，反复更改范围上下限值，确立一个范围最大的区域即为最终的i、j值。最终，需要将i、j值转化为边界方程形式，一般利用圆的方程表达式即可。故识别触发距离阈值DT的表现形式应当为一个环形区间，由两个圆的边界方程共同构成。 Choose appropriate i and j values so that P ij is greater than 75%, as the target extraction range in actual recognition. Generally, the initial values of i and j can be established by directly observing the image distribution. Based on the initial values, with 0.5m as the offset, the upper and lower limits of the range are repeatedly changed, and an area with the largest range is established as the final i and j values. Finally, it is necessary to convert the i and j values into the form of boundary equations, generally using the equation expression of a circle. Therefore, the expression form of the identification trigger distance threshold DT should be an annular interval, which is composed of the boundary equations of two circles.
获得识别触发距离阈值DT后，将其用于裁剪前述静态物体D s，与剔除无效数据相同，即将识别触发距离阈值DT以外的点云数据利用类比线性规划的方式剔除掉，得到识别用静态物体D″ s。 After the recognition trigger distance threshold DT is obtained, it is used to crop the aforementioned static object D s , which is the same as the elimination of invalid data, that is, the point cloud data other than the recognition trigger distance threshold DT is eliminated by analogous linear programming, and the static object for recognition is obtained. D″ s .
然后，需要将识别用静态物体转化为静态点云背景B。单帧数据仅能反映某一时刻的场景现状，故利用多帧数据叠加整合为满足场景绝大多数状况的点云背景是毋庸置疑的。但由于点云数据具有外疏内密的特性，单纯的叠加容易导致密集的地方越发密集，稀疏的部分相比而言仍然稀疏。因此，可能出现外部因点云过于稀疏，从而被当作噪声数据，而使得***难以分辨点云分布变化的情况，或是内部因点云过于稠密而使得对点云分布变化过于敏感，即使是自身抖动产生的点云变化也被识别为物体运动的特征。故本发明采用分区间有采样的叠加方法来避免上述问题。Then, the static object for recognition needs to be converted into a static point cloud background B. A single frame of data can only reflect the current situation of the scene at a certain moment, so it is unquestionable to use multiple frames of data to superimpose and integrate into a point cloud background that meets the vast majority of situations in the scene. However, due to the characteristics of point cloud data that are sparse outside and dense inside, simple superposition can easily lead to denser areas, and sparse parts are still sparse in comparison. Therefore, it may happen that the external point cloud is too sparse, which is regarded as noise data, making it difficult for the system to distinguish the change of the point cloud distribution, or the internal point cloud is too dense and it is too sensitive to the change of the point cloud distribution, even if the point cloud is too dense. The point cloud changes produced by self-jitter are also identified as features of object motion. Therefore, the present invention adopts the superposition method with sampling between partitions to avoid the above problems.
考虑到激光雷达工作原理为旋转式扫描，故可认为扫描的数据为环形分布。如图6所示，对每帧点云数据，从最内侧向外侧以逐渐递增的间距将其划分为n个统计空间。具体划分间隔需参考硬件设备的扫描范围和点云密度参数，本发明建议采用的间距划分为：Considering that the working principle of lidar is rotary scanning, it can be considered that the scanned data is circular distribution. As shown in Figure 6, for each frame of point cloud data, it is divided into n statistical spaces with gradually increasing spacing from the innermost to the outer. The specific division interval needs to refer to the scanning range and point cloud density parameters of the hardware device. The interval proposed in the present invention is divided as follows:
Figure PCTCN2022084912-appb-000004
Figure PCTCN2022084912-appb-000004
r表示内环宽度，l表示方格长度，R表示内环距原点的距离。r is the width of the inner ring, l is the length of the grid, and R is the distance of the inner ring from the origin.
从初始帧开始，依次叠加下一帧识别用静态物体，每次叠加时，需统计每个统计空间的点云密度，此时的点云密度的计算公式为：Starting from the initial frame, superimpose the next frame of static objects for identification in turn. When superimposing each time, the point cloud density of each statistical space needs to be counted. The calculation formula of the point cloud density at this time is:
Figure PCTCN2022084912-appb-000005
Figure PCTCN2022084912-appb-000005
其中ρ为点云密度，n为统计空间包含的总点数，S为统计空间的水平投影面积，r、l、R含义同上。where ρ is the point cloud density, n is the total number of points contained in the statistical space, S is the horizontal projected area of the statistical space, and r, l, and R have the same meanings as above.
若某统计空间的点云密度大于预设阈值α，则对该空间内的点云进行随机下采样以保持其点云密度。本发明建议采用的阈值为2000点/m 2。 If the point cloud density of a certain statistical space is greater than the preset threshold α, the point cloud in the space is randomly down-sampled to maintain its point cloud density. The threshold suggested by the present invention is 2000 points/m 2 .
应当理解的是，上述参数取值均仅为参考方案，实际取值应当按路侧感知单元的实际性能来确立，其确立的主要依据为：It should be understood that the above parameter values are only reference solutions, and the actual values should be established according to the actual performance of the roadside sensing unit. The main basis for its establishment is:
依各参数划分出的统计空间，在后续处理完成后，每个统计空间内的点云密度应大致相等，尤其需要检查最外侧统计空间与最内侧统计空间的点云密度；According to the statistical space divided by each parameter, after the subsequent processing is completed, the point cloud density in each statistical space should be roughly equal, especially the point cloud density of the outermost statistical space and the innermost statistical space should be checked;
统计空间的数量应尽量不超过500个，否则可能导致计算量过大，且不应少于100个，否则单个统计空间过大，不利于点云密度阈值α的确立，也容易导致点云密度相同的统计空间中点云分布差异较大，不利于后续计算工作。The number of statistical spaces should not exceed 500 as far as possible, otherwise the calculation amount may be too large, and should not be less than 100, otherwise a single statistical space is too large, which is not conducive to the establishment of the point cloud density threshold α, and it is easy to cause the point cloud density The distribution of point clouds in the same statistical space is quite different, which is not conducive to subsequent calculation work.
凡满足上述需求的参数均可作为实际参数用于计算。All parameters that meet the above requirements can be used as actual parameters for calculation.
叠加、下采样完成后，得到静态点云背景B。最后，为了使其在之后的匹配中具有可比性，需将其进行体素化操作。因为激光雷达受限于其机械构造，在扫描过程中是无法保障前一轮发射出的激光点与下一轮发射出的激光点是打在同一位置上的，换言之，点与点之间的位置比对不仅繁琐复杂，而且是无意义的。为此，本发明引入体素特征来作为比对依据。After the superposition and downsampling are completed, the static point cloud background B is obtained. Finally, to make it comparable in subsequent matching, it needs to be voxelized. Because lidar is limited by its mechanical structure, it cannot guarantee that the laser point emitted in the previous round and the laser point emitted in the next round are in the same position during the scanning process. Position comparison is not only cumbersome and complicated, but also meaningless. To this end, the present invention introduces voxel features as a comparison basis.
体素或立体像素，是体积像素(volume pixel)的简称，概念上类似二维空间的像素，体素是在三维空间中的最小单位。体素化即将三维模型用体素统一表示。体素化可以理解为二维像素化在三维空间的推广。最简单的体素化为二值体素化，这表明体素化的值非0即1。体素化描述的是三维场景数据，可以表达复杂的三维场景，正确地描述垂直重叠的对象。图9为点云体素化示例，可以看出体素表达数据量更少且语义信息丢失较少。Voxel or Voxel, short for volume pixel, is conceptually similar to a pixel in a two-dimensional space, and a voxel is the smallest unit in a three-dimensional space. Voxelization means that the three-dimensional model is uniformly represented by voxels. Voxelization can be understood as the generalization of two-dimensional pixelation in three-dimensional space. The simplest voxelization is binary voxelization, which means that the voxelization value is either 0 or 1. Voxelization describes 3D scene data, which can express complex 3D scenes and correctly describe vertically overlapping objects. Figure 9 is an example of point cloud voxelization. It can be seen that the amount of voxel expression data is less and the semantic information is less lost.
确立体素的大小，即立方体的边长v，需根据采集到的点云数据密度而确立，过大则容易丢失语义信息，过小则无法发挥体素化缩减数据量的作用。经测试，一般体素大小可选择为第一步中统计空间大小的1/20至1/40，效果较好。本发明建议采用的体素大小为30cm*30cm*30cm，但实际取值应按照实际使用效果变更。The size of the voxel, that is, the side length v of the cube, needs to be established according to the density of the collected point cloud data. If it is too large, it is easy to lose semantic information, and if it is too small, it cannot play the role of voxelization to reduce the amount of data. After testing, the general voxel size can be selected to be 1/20 to 1/40 of the statistical space size in the first step, and the effect is better. The voxel size suggested by the present invention is 30cm*30cm*30cm, but the actual value should be changed according to the actual use effect.
本发明建议基于下述公式计算点云数据中任意一点所属的体素位置：The present invention proposes to calculate the voxel position to which any point in the point cloud data belongs based on the following formula:
Figure PCTCN2022084912-appb-000006
Figure PCTCN2022084912-appb-000006
其中x、y、z为点云数据中任一一点的坐标；x 0、y 0、z 0表示点云数据的原点坐标值，不直接为0是因为可能存在多雷达组网使得雷达中心不为(0，0，0)的情况；v为体素的边长。r、c、h为体素的的坐标索引。 Among them, x, y, z are the coordinates of any point in the point cloud data; x 0 , y 0 , z 0 represent the origin coordinates of the point cloud data, and they are not directly 0 because there may be multiple radar networks that make the radar center When not (0, 0, 0); v is the side length of the voxel. r, c, and h are the coordinate indices of the voxels.
然后依据体素的坐标索引排序，统计每个体素所包含的点云数量，若点云数量小于5，则视其为空体素，从体素列表中删除，最终保留所有非空体素。Then, according to the coordinate index of the voxels, the number of point clouds contained in each voxel is counted. If the number of point clouds is less than 5, it is regarded as an empty voxel, deleted from the voxel list, and finally all non-empty voxels are retained.
除上述方法，其他体素化理论上方法均可适用于本发明，如利用PCL库中的函数等，理应都被视为本发明所用方法的变种之一。In addition to the above-mentioned methods, other theoretical voxelization methods can be applied to the present invention, such as using functions in the PCL library, etc., and should be regarded as one of the variants of the methods used in the present invention.
(三)静态物体识别(3) Static object recognition
实际识别某一帧点云数据时，首先将点云数据剔除无效数据并利用识别触发阈值分离出识别区域，再将其体素化为三维矩阵，利用滑动窗口法将结果与同样体素化后的静态点云背景进 行比对，若滑动窗口内某一连续区域与背景值的变化率大于判定阈值，则该区域被标记为非静态区域，否则标记为静态区域。When actually recognizing a certain frame of point cloud data, the point cloud data is first eliminated invalid data and the recognition area is separated by the recognition trigger threshold, and then it is voxelized into a three-dimensional matrix. If the change rate of a continuous area in the sliding window and the background value is greater than the judgment threshold, the area is marked as a non-static area, otherwise it is marked as a static area.
本发明借鉴图像处理中前后景分离的算法思路，基于第一步中获得的静态点云背景，利用滑动窗口法将其与当前帧的点云数据分布进行比对，若发现部分区域的分布变化过大，可认为该区域中存在着不属于静态物体的新增物体，借此来提取出非静态物体。The invention draws on the algorithm idea of separation of foreground and background in image processing, based on the static point cloud background obtained in the first step, and uses the sliding window method to compare it with the point cloud data distribution of the current frame. If it is too large, it can be considered that there are new objects that are not static objects in this area, so as to extract non-static objects.
首先将路侧感知单元安装于需要检测的场景中，并采用上述预处理流程完成前期数据处理工作，得到以下结果：体素化静态点云背景B v、识别触发距离阈值DT、边界方程组E b。 First, the roadside sensing unit is installed in the scene to be detected, and the above preprocessing process is used to complete the preliminary data processing work, and the following results are obtained: voxelized static point cloud background B v , recognition trigger distance threshold DT, boundary equation set E b .
将路侧感知单元用于实际使用。***启动时采集的数据一般还不稳定，通常需等待3-5分钟之后开始使用本发明所述方法用于动静态目标的识别。将正式开始时的第一帧点云数据记为D 1，之后的数据依次即为D 2、D 3……D i。 Use the roadside perception unit for practical use. The data collected when the system is started is generally not stable, and it usually takes 3-5 minutes to start using the method of the present invention to identify dynamic and static targets. The first frame of point cloud data at the official start is denoted as D 1 , and the subsequent data are sequentially D 2 , D 3 ...... Di .
之后，利用预处理获得的边界方程组E b对每帧数据进行第一次裁剪，得到有效数据D′ 1、D′ 2、……D′ i。再利用识别触发距离阈值DT对每帧数据进行第二次裁剪，得到识别用数据D″ 1、D″ 2、……D″ i。最后将识别用数据执行体素化操作得到体素化数据D″ v1、D″ v2、……D″ vi。 After that, each frame of data is cropped for the first time by using the boundary equation set E b obtained by preprocessing to obtain valid data D′ 1 , D′ 2 , ··· D′ i . Then use the recognition trigger distance threshold DT to cut the data of each frame for the second time, and obtain the data for recognition D″ 1 , D″ 2 , ... D″ i . Finally, perform voxelization operation on the data for recognition to obtain the voxelized data D″ v1 , D″ v2 , ... D″ vi .
比较单个体素的有无是无意义的，无法反映物体的语义信息，因此借鉴图像处理方法中的滑动窗口法和背景差法，由内至外将当前帧的体素化数据与体素化的静态背景进行匹配。本发明建议采用的滑动窗口的大小为体素框大小的5倍，即一个滑动窗口最多包含125个体素。应当理解的是，上述参数取值均仅为参考方案，实际取值应当按方法运行时的实际性能来确立。It is meaningless to compare the presence or absence of a single voxel and cannot reflect the semantic information of the object. Therefore, borrowing from the sliding window method and the background difference method in the image processing method, the voxelized data of the current frame is compared with the voxelized data from the inside to the outside. static background to match. The size of the sliding window suggested by the present invention is 5 times the size of the voxel frame, that is, a sliding window contains at most 125 voxels. It should be understood that the above parameter values are only reference solutions, and the actual values should be established according to the actual performance when the method is running.
由于环境震动可能导致扫描结果波动，出现静态物体前后两帧的点云分布情况不同，因此，需要设定判定阈值来避免。若窗口内体素分布相差超过判定阈值β，则滑动窗口所包含的所有体素标记为非静态区域A ns，否则标记为静态区域A s。本发明中建议采用的比对流程为： Since the environmental vibration may cause the scanning result to fluctuate, the point cloud distribution of the two frames before and after the static object appears is different. Therefore, it is necessary to set a judgment threshold to avoid it. If the difference of voxel distribution in the window exceeds the judgment threshold β, all voxels included in the sliding window are marked as non-static area Ans , otherwise, it is marked as static area As . The comparison process suggested to adopt in the present invention is:
①若滑动窗口内的体素Z轴最高值与静态点云背景中同一区域相比，变化率未超过20％，则窗口所在区域标注为静态区域，否则进行下一步比对；①If the highest value of the voxel Z-axis in the sliding window is compared with the same area in the static point cloud background, and the change rate does not exceed 20%, the area where the window is located is marked as a static area, otherwise, the next step is compared;
②若滑动窗口内的体素总数与静态点云背景中同一区域相比，变化率未超过20％，则窗口所在区域标注为静态区域，否则进行下一步比对；② If the total number of voxels in the sliding window is compared with the same area in the static point cloud background, and the change rate does not exceed 20%, the area where the window is located is marked as a static area, otherwise, the next step is compared;
③计算滑动窗口内体素的的形心位置，与静态点云背景中同一区域相比，若位置偏移未超过2体素边长则窗口所在区域标注为静态区域，否则标记为非静态区域。形心计算方法如下：③ Calculate the centroid position of the voxels in the sliding window. Compared with the same area in the static point cloud background, if the position offset does not exceed 2 voxel side lengths, the area where the window is located is marked as a static area, otherwise it is marked as a non-static area . The centroid is calculated as follows:
Figure PCTCN2022084912-appb-000007
Figure PCTCN2022084912-appb-000007
其中x、y、z表示形心的坐标，x i、y i、z i表示各体素的坐标索引，n为滑动窗口包含的体素总数。 Where x, y, z represent the coordinates of the centroid, x i , y i , and zi represent the coordinate indices of each voxel, and n is the total number of voxels included in the sliding window.
④当再次识别出非静态区域时，若其已与已知非静态区域A ns-1相邻，则同样标记为A ns-1，否则标记为非静态区域A ns-2，以此类推。 ④ When the non-static area is identified again, if it is adjacent to the known non-static area An ns -1, it is also marked as An ns -1, otherwise it is marked as a non-static area An ns -2, and so on.
上述比对流程结束后，所有静态区域的集合即为最终结果中的静态物体D s，其余非静态物体通过下文所述的动态物体识别进行再次分类。 After the above comparison process is completed, the set of all static regions is the static object D s in the final result, and the remaining non-static objects are reclassified through the dynamic object recognition described below.
应当理解的是，上述比对流程仅为本发明建议的方案之一，其他用以比较滑动窗口内点云与静态点云背景的方法均可适用于本发明，且理应被视为本方案的变种。It should be understood that the above comparison process is only one of the solutions proposed by the present invention, and other methods for comparing the point cloud in the sliding window and the background of the static point cloud are applicable to the present invention, and should be regarded as part of this solution. variant.
对于识别触发距离阈值所在的边缘区域，可能出现仅包含半截车辆的情况，如某车辆从扫描范围以外驶入激光雷达扫描范围以内，其同样会因区域内点云分布发生变化而被识别出，但有可能因其数据不完整而导致无法被点云目标检测算法检测出或被检测错误。由于识别触发距离阈值通常小于激光雷达实际扫描范围，故此时不完整的车辆对应在原始数据中实际是完整的。For the edge area where the recognition trigger distance threshold is located, it may only contain half of the vehicle. For example, if a vehicle enters the lidar scanning range from outside the scanning range, it will also be identified due to the change in the distribution of point clouds in the area. However, it may not be detected by the point cloud target detection algorithm or be detected incorrectly due to the incomplete data. Since the recognition trigger distance threshold is usually smaller than the actual scanning range of the lidar, the incomplete vehicle correspondence at this time is actually complete in the original data.
为此，本发明在识别触发距离阈值的基础上，会添加识别冗余值，即从识别触发距离阈值所在边界向外部扩展宽度为1.5m的环形备用识别区域，再以扫描角(本发明采用10°)划分为各子区域，如图10所示。若在边缘区域识别到动态区域，则将动态区域最临近的外部备用区域加入动态区域中。计算最邻近的外部备用区域可直接计算各备用区域与动态区域形心的间距。For this reason, the present invention, on the basis of identifying the triggering distance threshold, will add the identification redundancy value, that is, the annular standby identification area with a width of 1.5m is expanded from the boundary where the identification triggering distance threshold is located to the outside, and then the scanning angle (the present invention adopts 10°) is divided into sub-regions, as shown in Figure 10. If a dynamic area is identified in the edge area, the outer spare area closest to the dynamic area is added to the dynamic area. Calculating the nearest outer spare area can directly calculate the distance between each spare area and the centroid of the dynamic area.
(四)动态物体识别(4) Dynamic object recognition
上述识别过程中，依照一定频率，将某帧数据中所有的待识别区域记录为临时静态区域A st。当后续帧进行识别时，将待识别区域与临时静态区域进行匹配，若识别到存在两区域大小及位置等特征均无变化时即可认为其为短时静态物体，否则为动态物体。最终遍历完整个识别区域，将识别结果按不同频率分发给各个车辆。 In the above identification process, according to a certain frequency, all areas to be identified in a certain frame of data are recorded as temporary static areas A st . When the subsequent frames are recognized, the to-be-recognized area is matched with the temporary static area. If it is recognized that there are no changes in the size and position of the two areas, it can be considered as a short-term static object, otherwise it is a dynamic object. Finally, the entire identification area is traversed, and the identification results are distributed to each vehicle at different frequencies.
基于前文的分析，短时静态物体的危险性大于静态物体但小于动态物体，为本发明的次要识别目标，因此其传输频率亦可介于二者之间。静态物体为不传输或分钟级低频传输，动态数据为实时高频传输。而短时静态物体则为秒级中频传输。Based on the foregoing analysis, the danger of short-term static objects is greater than that of static objects but less than that of dynamic objects, which is the secondary identification target of the present invention, so its transmission frequency can also be between the two. Static objects are non-transmission or minute-level low-frequency transmission, and dynamic data is real-time high-frequency transmission. For short-term static objects, it is transmitted at a second-level intermediate frequency.
短时静态物体的识别方式与非静态物体也略有不同。由于短时静态物体的位置并未发生移动，故前后两帧之间的点云分布差异也几乎可忽略，换言之，若识别到属于非静态物体但前后两帧间特征几乎无变化的两个物体，即可认为前后帧为同一物体。基于该思路，采用如下方法进行识别。The identification of short-term static objects is also slightly different from that of non-static objects. Since the position of the short-term static object does not move, the difference in point cloud distribution between the two frames is almost negligible. , the frame before and after can be considered as the same object. Based on this idea, the following methods are used for identification.
实际识别过程中，每隔固定频率间隔，记录下识别帧中所有非静态区域A ns作为临时静态区域A st用于二次匹配。特别的，第1帧数据作为起始帧，其非静态区域无可用以比对的临时静态区域，故将其所有非静态区域记录为临时静态区域，但结果输出则全视为动态物体。所采用的固定频率间隔一般可选取激光雷达采集频率的1/2至1/5，本发明中选取的频率为每5帧记录一次。 In the actual identification process, every fixed frequency interval, all non-static areas Ans in the identification frame are recorded as temporary static areas A st for secondary matching. In particular, the first frame of data is used as the starting frame, and its non-static area has no temporary static area for comparison, so all its non-static areas are recorded as temporary static areas, but the result output is all regarded as dynamic objects. The adopted fixed frequency interval can generally be selected from 1/2 to 1/5 of the collection frequency of the lidar, and the frequency selected in the present invention is recorded once every 5 frames.
在待识别帧通过步骤(3)提取出所有非静态区域后，由于非静态区域明显为非连续点云数据，因此其中每个单独的点云空间作为匹配的对象。可在上一步匹配中即建立对应的关系 表，或统一记录后再利用欧拉距离进行聚类分离为子区域，本发明建议采用前者。二者的子区域分别记为A ns-i和A st-j。 After all non-static regions are extracted from the frame to be identified through step (3), since the non-static regions are obviously discontinuous point cloud data, each individual point cloud space is used as a matching object. The corresponding relationship table can be established in the previous matching step, or the Euler distance can be used for clustering and separation into sub-regions after unified recording. The present invention proposes to adopt the former. The sub-regions of the two are denoted as Ans -i and Ast -j, respectively.
依次比较非静态区域A ns与临时静态区域A st中的各个子区域，本发明建议采用如下方式进行比对： Comparing each sub-area in the non-static area Ans and the temporary static area A st in turn, the present invention proposes the comparison in the following manner:
①非静态区域A ns和临时静态区域A st中的子区域均依形心的扫描距离排序，依次比较二者中子区域的形心位置，若存在子区域A ns-i、A st-j满足二者形心距离不大于0.3米，且1米范围内无其他匹配对象，则进入下一步，否则将所有不满足条件的非静态区域子区域标记为动态区域； ①The sub-areas in the non-static area An ns and the temporary static area A st are sorted according to the scanning distance of the centroid, and the centroid positions of the sub-areas in the two are compared in turn. If there are sub-areas An ns -i, A st -j If the distance between the centroids of the two is not greater than 0.3 meters, and there is no other matching object within 1 meter, then go to the next step, otherwise, mark all non-static sub-regions that do not meet the conditions as dynamic regions;
②A ns-i、A st-j两区域的水平投影大小相比，若两区域位于边缘区域且变化率在15％之内，或两区域位于内部区域且变化率低于5％，则进入下一步，否则将所有不满足条件的非静态区域子区域标记为动态区域； ②Comparing the horizontal projection sizes of the two areas of Ans -i and A st -j, if the two areas are located in the edge area and the change rate is within 15%, or the two areas are located in the inner area and the change rate is less than 5%, enter the next step One step, otherwise, all non-static sub-regions that do not meet the conditions are marked as dynamic regions;
③A ns-i、A st-j两区域的体素Z轴最高值相比，若两区域位于边缘区域且变化率在15％之内，或两区域位于内部区域且变化率低于5％，则认为A ns-i可视为A st-j，即将A ns-i、A st-j表征的同一物体标记为短时静态物体，否则将所有不满足条件的非静态区域子区域仍标记为动态区域。 ③Comparing the highest value of the voxel Z-axis of the two regions of Ans -i and Ast -j, if the two regions are located in the edge region and the change rate is within 15%, or the two regions are located in the inner region and the change rate is less than 5%, Then it is considered that Ans -i can be regarded as A st -j, that is, the same object represented by Ans -i and A st -j is marked as a short-term static object, otherwise all non-static sub-regions that do not meet the conditions are still marked as dynamic area.
④由于每次记录临时静态区域时实际将动态物体也记录进临时静态区域中，故通过前后帧比对，若存在临时静态区域中的子区域A st-j，使得后帧数据的非静态区域中未有任何子区域可以与其匹配，则认为子区域A st-j并非短时静态物体，故可将其从临时静态区域中剔除，减少之后临时静态区域的比对量。 ④Since the dynamic object is actually recorded in the temporary static area every time the temporary static area is recorded, through the comparison of the previous and previous frames, if there is a sub-area A st -j in the temporary static area, the non-static area of the subsequent frame data is made. If there is no sub-region that can be matched with it, it is considered that the sub-region A st -j is not a short-term static object, so it can be eliminated from the temporary static region to reduce the comparison amount of the temporary static region.
最终，上述比对完成后，所有动态区域的集合即为动态物体D d，临时静态区域在剔除了动态物体后，剩余的部分即为短时静态物体D st，同时也是下一次比对时的临时静态区域A st。 Finally, after the above comparison is completed, the set of all dynamic regions is the dynamic object D d . After the dynamic objects are removed from the temporary static region, the remaining part is the short-term static object D st , which is also used for the next comparison. Temporary static area A st .
应当理解的是，上述比对流程仅为本发明建议的方案之一，其他用以比较非静态区域内子区域与临时静态区域子区域的方法均可适用于本发明，且理应被视为本方案的变种。It should be understood that the above comparison process is only one of the solutions proposed by the present invention, and other methods for comparing the sub-regions in the non-static area and the sub-regions in the temporary static area are applicable to the present invention, and should be regarded as this solution. variant.
在记录临时静态区域时，为临时静态区域赋加一个计数器，若比对结束后存在短时静态物体，则计数器数值加1。***可人为设定短时静态物体的传输频率，如若计数器设置为每加3则发送，则对应为短时静态物体的传输频率为动态物体传输频率的1/3。When recording the temporary static area, a counter is assigned to the temporary static area. If there is a short-term static object after the comparison, the counter value is increased by 1. The system can manually set the transmission frequency of short-term static objects. If the counter is set to send every 3 increments, the corresponding transmission frequency of short-term static objects is 1/3 of the transmission frequency of dynamic objects.
附图简要说明Brief Description of Drawings
图1预处理阶段流程图Figure 1 Flowchart of the preprocessing stage
图2识别阶段流程图Figure 2 Flowchart of the identification stage
图3不良数据样本示例和可用数据样本示例Figure 3 Examples of bad data samples and examples of available data samples
图4静态点云背景数据可视化示例Figure 4 Example of static point cloud background data visualization
图5路侧绿化带侵入道路空间数据可视化示例Figure 5. Example of spatial data visualization of roadside green belt intrusion
图6统计空间划分示意Figure 6 Schematic diagram of statistical space division
图7点云采样方法示意Figure 7 Schematic diagram of point cloud sampling method
图8距离L与置信度P的分布曲线图Figure 8 Distribution curve of distance L and confidence P
图9点云体素化示例Figure 9 Example of point cloud voxelization
图10边缘区域补充说明示意Figure 10 Schematic illustration of the supplementary description of the edge area
图11测试用例单帧数据可视化示意Figure 11 Visualization of a single frame of data in a test case
具体实施方式Detailed ways
依照专利发明内容所述，布置路侧感知单元。案例采用立杆式安装方法，激光雷达的安装高度约为5米，扫描范围覆盖了一段双向双车道道路，以及周边的建筑物、绿化树木等物体。实际识别数据的最远扫描距离为120米左右，数据采集频率为10HZ，每帧点云数量达十万以上。可视化样例如图11所示。The roadside sensing unit is arranged according to the description of the patented invention. The case adopts the pole-type installation method. The installation height of the lidar is about 5 meters, and the scanning range covers a two-way two-lane road, as well as surrounding buildings, green trees and other objects. The farthest scanning distance of the actual identification data is about 120 meters, the data collection frequency is 10HZ, and the number of point clouds per frame is more than 100,000. A sample visualization is shown in Figure 11.
首先依据步骤一，采集与处理数据。由于本次测试过程中全为晴天，故未能采集到雨天的激光雷达扫描数据，采用路面大范围洒水的方式代替，获取相应的点云数据约720帧。依次在早晨8：00左右、中午13：00左右及夜间21：00左右各采集约600帧数据，且为了避免偶然性，连续采集3天。并在夜间，驾驶实验车辆至扫描区域，开启远光灯来获取强光照射环境的点云数据，约250帧。总数据帧数接近6000帧，通过人为筛选后，得到约2000帧点云数据用于提取静态点云背景。First, according to step 1, collect and process data. Since it was all sunny during this test, the lidar scanning data in rainy days could not be collected, and the road surface was sprayed with water in a large area instead, and about 720 frames of corresponding point cloud data were obtained. About 600 frames of data were collected at around 8:00 in the morning, around 13:00 at noon, and around 21:00 at night, and in order to avoid chance, the data were collected continuously for 3 days. And at night, drive the experimental vehicle to the scanning area, turn on the high beams to obtain point cloud data of the environment illuminated by strong light, about 250 frames. The total number of data frames is close to 6000 frames. After artificial screening, about 2000 frames of point cloud data are obtained for extracting static point cloud background.
基于人工手段确立边界方程组，利用常见的点云数据可视化处理工具来完成。本发明选取某国产点云处理软件作为可视化操作工具。由于所选示例路段具有明显的路缘石作为道路边界，故可清晰划分出道路区域与非道路区域。在路缘石区域进行点云采样，采样方法为沿路缘石沿伸方向以50cm为间距进行人工采样，获得用于拟合道路边界的点云样本。基于上述采样点，利用最小二乘法等方式拟合出道路边界的平面方程，最终得到道路左右边界的平面方程，并记录数据剔除的方向，以其为无效数据剔除边界作为之后实际识别过程中的第一次数据筛选条件。The boundary equation system is established based on artificial means, and is completed by using common point cloud data visualization processing tools. The present invention selects a domestic point cloud processing software as a visual operation tool. Since the selected example road segment has obvious curbs as the road boundary, the road area and the non-road area can be clearly divided. The point cloud sampling was carried out in the curb area. The sampling method was manual sampling along the extension direction of the curb with an interval of 50 cm to obtain point cloud samples for fitting the road boundary. Based on the above sampling points, the least squares method is used to fit the plane equation of the road boundary, and finally the plane equation of the left and right boundaries of the road is obtained, and the direction of data removal is recorded. The first data filter condition.
接下来以0.5米为采样间隔，由内至外在各扫描环线等间距处采集样本点，记录以每个采样点为中心，半径10cm范围内的点数作为点云密度，平均每条环线上选取30个采样点，案例采样结果的部分示例如下表所示。Next, with a sampling interval of 0.5 meters, sample points are collected from the inside to the outside at equal intervals of each scanning ring line, and the number of points within a radius of 10 cm with each sampling point as the center is recorded as the point cloud density. 30 sampling points, some examples of case sampling results are shown in the table below.
点云密度point cloud density 第1采样点1st sampling point 第2采样点2nd sampling point ……... 第30采样点30th sampling point 点云密度均值Mean point cloud density
第1环线1st loop 2828 23twenty three    2626 2727
第2环线 2nd Ring Road 2525 24twenty four    2727 2626
第3环线3rd Ring Road 24twenty four 2525    23twenty three 24twenty four
……...               
第29环线Ring 29 77 88    66 99
第30环线 Ring 30 99 66    77 99
第31环线Ring 31 99 77    88 88
……...               
第58环线Ring 58 33 22    22 22
第59环线Ring 59 22 22    11 11
第60环线 Loop 60 22 00    11 11
计算每条环线的点云密度均值，将点云密度均值与其对应的环线距离分别作为x、y值，利用最小二乘法拟合，即可确立点云密度-扫描距离的线性关系，本案例结果表示为：Calculate the mean point cloud density of each loop line, take the mean point cloud density and its corresponding loop line distance as the x and y values respectively, and use the least squares method to fit the linear relationship between the point cloud density and the scanning distance. The results of this case Expressed as:
ρ＝0.97·Lρ=0.97·L
其中ρ为点云密度，L为扫描距离，0.97为线性函数参数。where ρ is the point cloud density, L is the scanning distance, and 0.97 is the linear function parameter.
再基于人工手段，将每帧数据中的静态物体与非静态物体进行，其中静态物体用于获得静态点云背景。但与发明内容所述不同的是，本案例中采用的点云目标检测算法只需输入原始点云数据即可，无需单独提取出非静态物体用于检测。所有点云目标检测算法应当在使用前训练出识别效果较好的模型，本发明认为算法的查准率大于85％即可认为算法的识别效果较好。对检测出的目标进行点云采样时，一般可利用算法得出的标注框作为提取边界，视标注框内部的所有点云均属于检测目标即可。Based on artificial means, the static objects in each frame of data are compared with non-static objects, and the static objects are used to obtain the static point cloud background. However, unlike what is described in the summary of the invention, the point cloud target detection algorithm used in this case only needs to input the original point cloud data, and it is not necessary to extract non-static objects separately for detection. All point cloud target detection algorithms should train a model with better recognition effect before use, and the present invention considers that the algorithm has a better recognition effect if the precision rate of the algorithm is greater than 85%. When sampling the point cloud of the detected target, the label frame obtained by the algorithm can generally be used as the extraction boundary, and all point clouds inside the label frame can be regarded as the detection target.
采用随机采样法获取其平均点云密度。点云采样的比例参考所选激光雷达设备的参数和目标实际的点云总数确立，本案例采用的方式为：The average point cloud density is obtained by random sampling method. The ratio of point cloud sampling is established with reference to the parameters of the selected lidar device and the actual total number of point clouds of the target. The method used in this case is:
对于点云总数大于3000的非静态物体，随机采样3次，每次采样300点，最终计算3次采样的平均点云密度；For non-static objects with a total number of point clouds greater than 3000, randomly sample 3 times with 300 points each time, and finally calculate the average point cloud density of the 3 samplings;
对于点云总数大于1000的非静态物体，随机采样3次，每次采样100点，最终计算3次采样的平均点云密度；For non-static objects with a total number of point clouds greater than 1000, randomly sample 3 times with 100 points each time, and finally calculate the average point cloud density of 3 samplings;
对于点云总数大于500的非静态物体，随机采样3次，每次采样75点，最终计算3次采样的平均点云密度；For non-static objects with a total number of point clouds greater than 500, randomly sample 3 times, 75 points each time, and finally calculate the average point cloud density of 3 times of sampling;
对于点云总数小于500的非静态物体，随机采样100点，计算平均点云密度。For non-static objects with a total number of point clouds less than 500, 100 points are randomly sampled and the average point cloud density is calculated.
上诉采样方法中，点云密度的计算方法与前述一致，即以每个采样点为中心，半径10cm范围内的点数作为点云密度。In the sampling method appealed, the calculation method of the point cloud density is the same as the above, that is, with each sampling point as the center, the number of points within a radius of 10cm is used as the point cloud density.
确立每个非静态物体的平均点云密度后，将各非静态物体输入后端算法进行检测，得到各检测结果与检测置信度P。画出扫描距离L与检测置信度P的分布曲线图，利用如下公式：After the average point cloud density of each non-static object is established, each non-static object is input into the back-end algorithm for detection, and each detection result and detection confidence P are obtained. Draw the distribution curve of scanning distance L and detection confidence P, using the following formula:
Figure PCTCN2022084912-appb-000008
Figure PCTCN2022084912-appb-000008
其中，j、i表示识别触发距离阈值的上下限，i为最近距离阈值，j为最远距离阈值，n i,n j分别表示距离原点i、j处的非静态目标总数，(n j-n i) p>75％表示i、j范围内置信度大于75％的非静态目标总数。 Among them, j, i represent the upper and lower limits of the recognition trigger distance threshold, i is the shortest distance threshold, j is the farthest distance threshold, n i , n j represent the total number of non-static targets at distances i and j from the origin, (n j − n i ) p>75% means the total number of non-static targets with built-in reliability greater than 75% in i, j range.
选择合适的i、j值使得P ij大于75％，作为实际识别时的非静态物体提取范围。本发明选取的i、j值为3和45，即对应非静态物体提取范围为从距激光雷达中心水平距离4m处至距激光雷达中心水平距离25m处。 Choose appropriate i, j values so that P ij is greater than 75%, as the non-static object extraction range in actual recognition. The values of i and j selected by the present invention are 3 and 45, that is, the extraction range of the corresponding non-static object is from a horizontal distance of 4 m from the center of the laser radar to a horizontal distance of 25 m from the center of the laser radar.
再对每帧点云数据，参考所用激光雷达设备的扫描范围和点云密度参数，从最内侧向外侧以逐渐递增的间距将其划分为93个统计空间。本案例所采用的间距划分为：Then, for each frame of point cloud data, referring to the scanning range and point cloud density parameters of the lidar equipment used, it is divided into 93 statistical spaces with gradually increasing spacing from the innermost to the outer. The spacing used in this case is divided into:
Figure PCTCN2022084912-appb-000009
Figure PCTCN2022084912-appb-000009
其中，r表示内环宽度，l表示方格长度，R表示内环距原点的距离。Among them, r is the width of the inner ring, l is the length of the grid, and R is the distance of the inner ring from the origin.
从初始帧开始，依次叠加下一帧点云数据，每次叠加时，检测每个统计空间的点云密度，点云密度的计算公式为：Starting from the initial frame, the next frame of point cloud data is superimposed in turn. Each time the superposition is performed, the point cloud density of each statistical space is detected. The calculation formula of the point cloud density is:
Figure PCTCN2022084912-appb-000010
Figure PCTCN2022084912-appb-000010
其中ρ为点云密度，n为统计空间包含的总点数，S为统计空间的面积，r表示内环宽度，l表示方格长度，R表示内环距原点的距离。where ρ is the point cloud density, n is the total number of points contained in the statistical space, S is the area of the statistical space, r is the width of the inner ring, l is the length of the grid, and R is the distance between the inner ring and the origin.
若某统计空间的点云密度大于预设阈值2000点/m 2，则对该空间内的点云进行随机下采样以保持其点云密度，最终得到较为理想的静态点云背景B。 If the point cloud density of a certain statistical space is greater than the preset threshold of 2000 points/m 2 , the point cloud in the space is randomly down-sampled to maintain its point cloud density, and finally an ideal static point cloud background B is obtained.
为方便后续计算，先对静态点云背景进行体素化处理。本发明采用的体素大小为30cm*30cm*30cm。基于下述公式计算点云数据中任意一点所属的体素位置：In order to facilitate subsequent calculations, the static point cloud background is first processed by voxelization. The size of the voxel used in the present invention is 30cm*30cm*30cm. Calculate the voxel position to which any point in the point cloud data belongs based on the following formula:
Figure PCTCN2022084912-appb-000011
Figure PCTCN2022084912-appb-000011
其中x、y、z为点云数据中任一一点的坐标；x0、y0、z0表示点云数据的原点坐标值，不直接为0是因为可能存在多雷达组网使得雷达中心不为(0，0，0)的情况；v为体素的边长。r、c、h为体素的的坐标索引。Among them, x, y, and z are the coordinates of any point in the point cloud data; x0, y0, and z0 represent the origin coordinates of the point cloud data, and they are not directly 0 because there may be multiple radar networks, so that the radar center is not ( 0, 0, 0); v is the side length of the voxel. r, c, and h are the coordinate indices of the voxels.
依据体素的坐标索引排序，统计每个体素所包含的点云数量，若点云数量小于5，则视其为空体素，从体素列表中删除，最终保留所有非空体素，即为体素化静态点云背景B v。 Sort according to the coordinate index of the voxel, and count the number of point clouds contained in each voxel. If the number of point clouds is less than 5, it is regarded as an empty voxel, deleted from the voxel list, and finally all non-empty voxels are retained, that is is the voxelized static point cloud background B v .
再进行步骤三，对新采集的数据进行体素化并利用滑动窗口法筛选出非静态区域。首先需对每帧点云数据依次进行无效数据剔除、分离识别区域以及体素化操作，方法同静态点云背景处理时一样，在经过两次数据筛选之后，每帧点云数据的平均体素数量为15000左右。Then proceed to step 3, voxelize the newly collected data and use the sliding window method to filter out the non-static area. First of all, it is necessary to perform invalid data removal, separation and identification area and voxelization operations on each frame of point cloud data in turn. The method is the same as that of static point cloud background processing. After two data screening, the average voxel The number is around 15,000.
借鉴图像处理方法中的滑动窗口法和背景差法，由内至外将当前帧的体素化数据与体素化的静态背景进行匹配。本案例采用的滑动窗口的大小为体素框大小的5倍，即一个滑动窗口最多包含125个体素。Drawing on the sliding window method and the background difference method in the image processing method, the voxelized data of the current frame is matched with the voxelized static background from the inside to the outside. The size of the sliding window used in this case is 5 times the size of the voxel box, that is, a sliding window contains at most 125 voxels.
由于环境震动可能导致扫描结果波动，出现静态物体前后两帧的点云分布情况不同，因此，需要设定触发阈值来避免。若窗口内体素分布相差超过固定阈值，则滑动窗口所包含的所有体素标记为非静态区域A ns，否则标记为静态区域A s。具体比对流程为： Since environmental vibrations may cause fluctuations in the scanning results, the distribution of point clouds in the two frames before and after the appearance of static objects is different. Therefore, it is necessary to set a trigger threshold to avoid it. If the difference of voxel distribution in the window exceeds a fixed threshold, all voxels included in the sliding window are marked as non-static area Ans , otherwise, marked as static area As . The specific comparison process is as follows:
⑤若滑动窗口内的体素Z轴最高值与静态点云背景中同一区域相比，变化率未超过20％，则窗口所在区域标注为静态区域，否则进行下一步比对；⑤ If the highest value of the voxel Z-axis in the sliding window is compared with the same area in the static point cloud background, and the change rate does not exceed 20%, the area where the window is located is marked as a static area, otherwise, the next step is compared;
⑥若滑动窗口内的体素总数与静态点云背景中同一区域相比，变化率未超过20％，则窗口所在区域标注为静态区域，否则进行下一步比对；⑥ If the total number of voxels in the sliding window is compared with the same area in the static point cloud background, and the change rate does not exceed 20%, the area where the window is located is marked as a static area, otherwise, the next step is compared;
⑦计算滑动窗口内体素的的形心位置，与静态点云背景中同一区域相比，若位置偏移未超过2体素边长则窗口所在区域标注为静态区域，否则标为非静态区域。形心计算方法如下：⑦ Calculate the centroid position of the voxels in the sliding window. Compared with the same area in the static point cloud background, if the position offset does not exceed 2 voxel side lengths, the area where the window is located is marked as a static area, otherwise it is marked as a non-static area . The centroid is calculated as follows:
Figure PCTCN2022084912-appb-000012
Figure PCTCN2022084912-appb-000012
其中x、y、z表示形心的坐标，x i、y i、z i表示各体素的坐标索引，n为滑动窗口包含的体素总数。 Where x, y, z represent the coordinates of the centroid, x i , y i , and zi represent the coordinate indices of each voxel, and n is the total number of voxels included in the sliding window.
当再次识别出非静态区域时，若其已与已知非静态区域A ns-1相邻，则同样标记为A ns-1，否则标记为非静态区域A ns-2，以此类推。 When the non-static area is identified again, if it is adjacent to the known non-static area An ns -1, it is also marked as An ns -1, otherwise it is marked as a non-static area An ns -2, and so on.
最后，提取所有静态区域的点云数据作为静态物体。由于静态物体为连续点云数据，难以比较其识别率，故转化为比较非静态物体。本案例中，内部区域(距激光雷达中心水平距离小于23m的区域)的非静态物体识别率可以达到97％以上，边缘区域(距激光雷达中心水平距离大于23m且小于25m的区域)由于跨边界车辆截取部分的大小不一，识别率相对较弱，可以达到85％以上，平均非静态物体识别率为92％左右。Finally, the point cloud data of all static regions are extracted as static objects. Since static objects are continuous point cloud data, it is difficult to compare their recognition rates, so they are converted into comparative non-static objects. In this case, the recognition rate of non-static objects in the inner area (the area with a horizontal distance of less than 23m from the center of the lidar) can reach more than 97%, and the edge area (the area with a horizontal distance from the center of the lidar greater than 23m and less than 25m) due to cross-border The size of the intercepted part of the vehicle is different, the recognition rate is relatively weak, can reach more than 85%, and the average non-static object recognition rate is about 92%.
利用实验车辆模拟路侧停车行为，用以检测步骤四的方法。每隔5帧数据，记录下检测帧中所有非静态区域A ns作为临时静态区域A st用于二次匹配。记录的特征包括非静态区域A各子区域的的水平投影大小、形心所在位置以及Z轴最高值。 Use the experimental vehicle to simulate the roadside parking behavior to detect the method of step 4. Every 5 frames of data, all non-static areas Ans in the detection frame are recorded as temporary static areas A st for secondary matching. The recorded features include the horizontal projection size of each sub-region of the non-static region A, the location of the centroid, and the highest value of the Z-axis.
在待识别帧通过步骤三提取出所有非静态区域后，依次比较其子区域与临时静态区域中记录的各子区域。比较的特征及顺序如下：After all the non-static areas are extracted from the frame to be identified through step 3, the sub-areas are compared with the sub-areas recorded in the temporary static area in turn. The features and order of comparison are as follows:
①非静态区域A ns和临时静态区域A st中的子区域均依形心的扫描距离排序，依次比较二者中子区域的形心位置，若存在子区域A ns-i、A st-j满足二者形心距离不大于0.3米，且1米范围内无其他匹配对象，则进入下一步，否则将所有不满足条件的非静态区域子区域标记为动态区域； ①The sub-areas in the non-static area An ns and the temporary static area A st are sorted according to the scanning distance of the centroid, and the centroid positions of the sub-areas in the two are compared in turn. If there are sub-areas An ns -i, A st -j If the distance between the centroids of the two is not greater than 0.3 meters, and there is no other matching object within 1 meter, then go to the next step, otherwise, mark all non-static sub-regions that do not meet the conditions as dynamic regions;
②A ns-i、A st-j两区域的水平投影大小相比，若两区域位于边缘区域且变化率在15％之内，或两区域位于内部区域且变化率低于5％，则进入下一步，否则将所有不满足条件的非静态区域子区域标记为动态区域； ②Comparing the horizontal projection sizes of the two areas of Ans -i and A st -j, if the two areas are located in the edge area and the change rate is within 15%, or the two areas are located in the inner area and the change rate is less than 5%, enter the next step One step, otherwise, all non-static sub-regions that do not meet the conditions are marked as dynamic regions;
③A ns-i、A st-j两区域的体素Z轴最高值相比，若两区域位于边缘区域且变化率在15％之内，或两区域位于内部区域且变化率低于5％，则认为A ns-i可视为A st-j，即将A ns-i、A st-j表征的同一物体标记为短时静态物体，否则将所有不满足条件的非静态区域子区域仍标记为动态区域。 ③Comparing the highest value of the voxel Z-axis of the two regions of Ans -i and Ast -j, if the two regions are located in the edge region and the change rate is within 15%, or the two regions are located in the inner region and the change rate is less than 5%, Then it is considered that Ans -i can be regarded as A st -j, that is, the same object represented by Ans -i and A st -j is marked as a short-term static object, otherwise all non-static sub-regions that do not meet the conditions are still marked as dynamic area.
④由于每次记录临时静态区域时实际将动态物体也记录进临时静态区域中，故通过前后帧比对，若存在临时静态区域中的子区域A st-j，使得后帧数据的非静态区域中未有任何子区域可以与其匹配，则认为子区域A st-j并非短时静态物体，故可将其从临时静态区域中剔除，减少之后临时静态区域的比对量。 ④Since the dynamic object is actually recorded in the temporary static area every time the temporary static area is recorded, through the comparison of the previous and previous frames, if there is a sub-area A st -j in the temporary static area, the non-static area of the subsequent frame data is made. If there is no sub-region that can be matched with it, it is considered that the sub-region A st -j is not a short-term static object, so it can be eliminated from the temporary static region to reduce the comparison amount of the temporary static region.
最终，上述比对完成后，所有动态区域的集合即为动态物体D d，临时静态区域在剔除了动态物体后，剩余的部分即为短时静态物体D st，同时也是下一次比对时的临时静态区域A st。 Finally, after the above comparison is completed, the set of all dynamic regions is the dynamic object D d . After the dynamic objects are removed from the temporary static region, the remaining part is the short-term static object D st , which is also used for the next comparison. Temporary static area A st .

Claims (11)

  1. 一种基于路侧感知单元的静态和非静态物体点云识别方法，包括如下步骤：A point cloud recognition method for static and non-static objects based on a roadside perception unit, comprising the following steps:
    (一)数据采集(1) Data collection
    面向车路协同环境，搭建路侧激光雷达感知场景，并采集点云原始数据D 0用于预处理； Facing the vehicle-road collaborative environment, build a roadside lidar perception scene, and collect point cloud raw data D 0 for preprocessing;
    (二)预处理(2) Preprocessing
    2.1)首先确立边界方程组E b，从所述原始数据D 0中剔除无效数据，得到有效点云数据； 2.1) First establish boundary equations E b , remove invalid data from the original data D 0 to obtain valid point cloud data;
    2.2)对有效点云数据进行等间距采样，建立点云密度与扫描距离的线性关系；2.2) Sampling the valid point cloud data at equal intervals to establish a linear relationship between the point cloud density and the scanning distance;
    2.3)识别每帧有效点云数据中的静态物体D s，建立静态点云背景B； 2.3) Identify the static object D s in the valid point cloud data of each frame, and establish the static point cloud background B;
    2.4)对静态点云背景执行体素化操作，得到体素化静态点云背景B v； 2.4) perform voxelization operation to static point cloud background, obtain voxelization static point cloud background B v ;
    (三)静态物体识别(3) Static object recognition
    3.1)从有效点云数据中利用识别触发阈值分离出识别用区域；3.1) Use the recognition trigger threshold to separate the recognition area from the valid point cloud data;
    3.2)将识别用区域体素化为三维矩阵，利用滑动窗口法将所述三维矩阵与体素化后的静态点云背景B v进行比对，若滑动窗口内某一连续区域与体素化后的静态点云背景B v中处于同一位置的区域之间的变化率大于静态区域判定阈值，则该连续区域被标记为非静态区域，否则标记为静态区域；所有静态区域的并集即为所述有效点云数据中的静态物体。 3.2) Voxelize the area for identification into a three-dimensional matrix, and use the sliding window method to compare the three-dimensional matrix with the voxelized static point cloud background B v . If a continuous area in the sliding window is compared with the voxelized The rate of change between the regions at the same position in the static point cloud background B v is greater than the static region determination threshold, then the continuous region is marked as a non-static region, otherwise it is marked as a static region; the union of all static regions is Static objects in the valid point cloud data.
  2. 如权利要求1所述的方法，其特征在于，还包括如下步骤：The method of claim 1, further comprising the steps of:
    (四)非静态物体识别(4) Non-static object recognition
    4.1)上述静态物体识别过程中，依照固定频率间隔，将某帧体素化点云数据中所有的静态区域记录为临时静态区域A st； 4.1) In the above-mentioned static object recognition process, according to a fixed frequency interval, all static areas in a certain frame of voxelized point cloud data are recorded as temporary static areas A st ;
    4.2)当识别后续帧时，将待识别体素化点云数据的非静态区域与所述临时静态区域A st进行匹配，若该两区域的大小及位置的变化率均小于短时静态物体判定阈值时即可认为为待识别体素化点云数据的非静态区域为短时静态物体，否则为动态物体； 4.2) When identifying subsequent frames, match the non-static area of the voxelized point cloud data to be identified with the temporary static area A st , if the rate of change of the size and position of the two areas is smaller than the short-term static object judgment When the threshold is reached, it can be considered that the non-static area of the voxelized point cloud data to be recognized is a short-term static object, otherwise it is a dynamic object;
    4.3)重复4.1)和4.2)直至遍历完整个有效点云数据。4.3) Repeat 4.1) and 4.2) until the entire valid point cloud data is traversed.
  3. 如权利要求1所述的方法，其特征在于，采集的原始点云数据包含静态物体和非静态物体。The method of claim 1, wherein the collected raw point cloud data includes static objects and non-static objects.
  4. 如权利要求1所述的方法，其特征在于，所述静态物体包括路面及其附属设施、建筑物和路侧绿化植物；所述非静态物体包括行人、非机动车和/或车辆。The method of claim 1, wherein the static objects include road surfaces and their ancillary facilities, buildings and roadside green plants; and the non-static objects include pedestrians, non-motor vehicles and/or vehicles.
  5. 如权利要求1所述的方法，其特征在于，所述点云密度与扫描距离的线性关系的建立方法：The method of claim 1, wherein the method for establishing the linear relationship between the point cloud density and the scanning distance:
    以0.5米为采样间隔，由内至外在各扫描环线等间距采集样本点，记录以每个采样点为中心半径10cm范围内的点数作为点云密度，统计每条环线的平均点云密度，即可确立点云密度与扫描距离的线性关系。With a sampling interval of 0.5 meters, sample points are collected at equal intervals in each scanning loop from the inside to the outside, and the number of points within a radius of 10 cm with each sampling point as the center is recorded as the point cloud density, and the average point cloud density of each loop line is counted. The linear relationship between point cloud density and scan distance can be established.
  6. 如权利要求1所述的方法，其特征在于，所述静态点云背景B按照如下方法建立：The method of claim 1, wherein the static point cloud background B is established according to the following method:
    2.3.1)利用点云目标检测算法对非静态物体检测以建立检测置信度与扫描距离的分布曲线，并选取置信度均高于阈值的扫描距离范围作为识别触发距离阈值；2.3.1) Use the point cloud target detection algorithm to detect non-static objects to establish the distribution curve of detection confidence and scanning distance, and select the scanning distance range whose confidence is higher than the threshold as the recognition trigger distance threshold;
    2.3.2)利用识别触发距离阈值对所有静态物体进行裁剪，将裁剪后的多帧识别用静态物体通过叠加并适当采样的方式建立静态点云背景B。2.3.2) Use the recognition trigger distance threshold to crop all static objects, and create a static point cloud background B by stacking and properly sampling the cropped static objects for multi-frame recognition.
  7. 如权利要求6所述的方法，其特征在于，所述识别触发距离阈值DT的确立方法为：The method of claim 6, wherein the method for establishing the identification trigger distance threshold DT is:
    提取各类非静态物体，对每个非静态物体按一定比例进行点云采样并统计其平均点云密度，确定其所对应的扫描距离L；Extract all kinds of non-static objects, sample the point cloud of each non-static object according to a certain proportion and count the average point cloud density to determine its corresponding scanning distance L;
    将各检测目标输入点云目标检测算法进行检测，得到检测结果与检测置信度P；Input each detection target into the point cloud target detection algorithm for detection, and obtain the detection result and detection confidence P;
    画出距离L与置信度P的分布曲线图，利用如下公式：Draw the distribution curve of distance L and confidence P, using the following formula:
    Figure PCTCN2022084912-appb-100001
    Figure PCTCN2022084912-appb-100001
    其中，n i,n j分别表示距离原点i、j处的采样目标总数，(n j-n i) p>75％表示i、j范围内，置信度大于75％的采样目标总数；以0.5米为上下限采样间隔，选择合适的i、j值使得P ij大于75％，且使得i与j的差值最大，则该I,j值转化的边界方程即为别触发距离阈值DT。 Among them, n i , n j represent the total number of sampling targets at distances i and j, respectively, (n j -n i ) p>75% represents the total number of sampling targets within the range of i and j with a confidence greater than 75%; take 0.5 m is the upper and lower sampling interval, select the appropriate i and j values so that P ij is greater than 75%, and make the difference between i and j the largest, then the boundary equation transformed by the I, j values is the trigger distance threshold DT.
  8. 如权利要求6所述的方法，其特征在于，所述体素化静态点云背景B v的构建方法为： The method of claim 6, wherein the construction method of the voxelized static point cloud background B v is:
    基于人工提取方法，将每一帧有效点云数据中的静态物体与非静态物体进行分离，对于静态物体利用识别触发距离阈值DT分离出识别用区域；Based on the manual extraction method, the static objects and non-static objects in each frame of valid point cloud data are separated, and the recognition trigger distance threshold DT is used to separate the recognition area for static objects;
    再从点云坐标系原定向外侧以逐渐递增的间距将其划分为n个统计空间；从初始帧开始，依次叠加下一帧有效点云数据；每次叠加时，检测每个统计空间的点云密度，若其大于阈值α，则对该空间内的有效点云数据进行随机采样以保持其密度；Then, from the outside of the original orientation of the point cloud coordinate system, it is divided into n statistical spaces with gradually increasing spacing; starting from the initial frame, the next frame of valid point cloud data is superimposed; each time the point cloud is superimposed, the points in each statistical space are detected. The cloud density, if it is greater than the threshold α, randomly sample the valid point cloud data in the space to maintain its density;
    最终得到点云密度适中的静态点云背景B；Finally, a static point cloud background B with moderate point cloud density is obtained;
    最后，对静态点云背景执行体素化操作，得到体素化静态点云背景B v。 Finally, a voxelization operation is performed on the static point cloud background to obtain the voxelized static point cloud background B v .
  9. 如权利要求1所述的方法，其特征在于所述静态区域和非静态区域的标记方法为：The method according to claim 1, wherein the marking method of the static area and the non-static area is:
    将待识别点云数据帧先利用边界方程组E d、识别触发距离阈值DT裁剪，再按统计空间大小的1/20至1/40的密度进行体素化处理，得到待识别体素化点云数据Dv”； The point cloud data frame to be identified is first cropped by using the boundary equation set E d and the identification trigger distance threshold DT, and then voxelized at a density of 1/20 to 1/40 of the statistical space size to obtain the voxelized point to be identified. Cloud Data Dv";
    由外至内采用滑动窗口和背景差法将待识别体素化点云数据中某一连续区域与体素化静态点云背景中同一位置处的区域进行匹配；From outside to inside, the sliding window and background difference method are used to match a continuous area in the voxelized point cloud data to be identified with the area at the same position in the voxelized static point cloud background;
    若上述二者之间的变化率相差超过静态区域判定阈值，则标记为非静态区域A ns，否则标记为静态区域A s； If the difference in the rate of change between the above two exceeds the static area determination threshold, it is marked as a non-static area An s , otherwise it is marked as a static area A s ;
    当再次检测出动态区域时，若其与已知非静态区域A ns-1相邻，则同样标记为A ns-1，否则标记为非静态区域A ns-2，以此类推。 When a dynamic area is detected again, if it is adjacent to the known non-static area An ns -1, it is also marked as An ns -1, otherwise, it is marked as a non-static area An ns -2, and so on.
  10. 如权利要求7所述的方法，其特征在于，对边缘区域，向外部扩展宽度为1.5m的环形备用识别区域，再以扫描角划分为若干子区域；若在边缘区域检测到非静态区域，则以非静态区域所在的扫描角将扫描角范围内的环形备用识别区域划入非静态区域中。The method according to claim 7, characterized in that, for the edge area, a ring-shaped spare identification area with a width of 1.5m is expanded to the outside, and then divided into several sub-areas by scanning angle; if a non-static area is detected in the edge area, Then, the annular spare identification area within the scanning angle range is divided into the non-static area by the scanning angle where the non-static area is located.
  11. 如权利要求2所述方法，其特征在于，在短时静态物体的识别过程中，每隔固定频率间隔记录下非静态区域A ns作为临时静态区域A st用于二次匹配；在待识别体素化点云数据通过滑动窗口和背景差法提取出所有非静态区域后，依次比较隶属于待识别非静态区域的各子区域与临时静态区域中的各子区域，若二者间存在两子区域A ns-i、A st-j使得二者之间的位置、形态特征差异小于判定阈值，则视二者为同一物体，标记为短时静态物体，否则标记为动态物体。 The method according to claim 2, characterized in that, during the identification process of the short-term static object, the non-static area Ans is recorded every fixed frequency interval as the temporary static area A st for secondary matching; After the pixelized point cloud data is extracted all non-static regions by sliding window and background difference method, the sub-regions belonging to the non-static region to be identified and the sub-regions in the temporary static region are compared in turn. If there are two sub-regions between the two In the regions Ans -i and Ast -j, the difference in position and morphological characteristics between the two is less than the judgment threshold, then the two are regarded as the same object, and marked as a short-term static object, otherwise, it is marked as a dynamic object.
PCT/CN2022/084912 2021-01-01 2022-04-01 Roadside sensing unit-based static and non-static object point cloud recognition method WO2022206974A1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202280026656.3A CN117836667A (en) 2021-01-01 2022-04-01 Static and non-static object point cloud identification method based on road side sensing unit

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
CN202110000327 2021-01-01
CN202110228419 2021-03-01
PCT/CN2021/085147 WO2022141911A1 (en) 2021-01-01 2021-04-01 Roadside sensing unit-based method for quick recognition of dynamic target point cloud and point cloud segmentation
CNPCT/CN2021/085147 2021-04-01

Publications (1)

Publication Number Publication Date
WO2022206974A1 true WO2022206974A1 (en) 2022-10-06

Family

ID=82260124

Family Applications (9)

Application Number Title Priority Date Filing Date
PCT/CN2021/085148 WO2022141912A1 (en) 2021-01-01 2021-04-01 Vehicle-road collaboration-oriented sensing information fusion representation and target detection method
PCT/CN2021/085147 WO2022141911A1 (en) 2021-01-01 2021-04-01 Roadside sensing unit-based method for quick recognition of dynamic target point cloud and point cloud segmentation
PCT/CN2021/085150 WO2022141914A1 (en) 2021-01-01 2021-04-01 Multi-target vehicle detection and re-identification method based on radar and video fusion
PCT/CN2021/085146 WO2022141910A1 (en) 2021-01-01 2021-04-01 Vehicle-road laser radar point cloud dynamic segmentation and fusion method based on driving safety risk field
PCT/CN2021/085149 WO2022141913A1 (en) 2021-01-01 2021-04-01 On-board positioning device-based roadside millimeter-wave radar calibration method
PCT/CN2022/084912 WO2022206974A1 (en) 2021-01-01 2022-04-01 Roadside sensing unit-based static and non-static object point cloud recognition method
PCT/CN2022/084925 WO2022206977A1 (en) 2021-01-01 2022-04-01 Cooperative-vehicle-infrastructure-oriented sensing information fusion representation and target detection method
PCT/CN2022/084929 WO2022206978A1 (en) 2021-01-01 2022-04-01 Roadside millimeter-wave radar calibration method based on vehicle-mounted positioning apparatus
PCT/CN2022/084738 WO2022206942A1 (en) 2021-01-01 2022-04-01 Laser radar point cloud dynamic segmentation and fusion method based on driving safety risk field

Family Applications Before (5)

Application Number Title Priority Date Filing Date
PCT/CN2021/085148 WO2022141912A1 (en) 2021-01-01 2021-04-01 Vehicle-road collaboration-oriented sensing information fusion representation and target detection method
PCT/CN2021/085147 WO2022141911A1 (en) 2021-01-01 2021-04-01 Roadside sensing unit-based method for quick recognition of dynamic target point cloud and point cloud segmentation
PCT/CN2021/085150 WO2022141914A1 (en) 2021-01-01 2021-04-01 Multi-target vehicle detection and re-identification method based on radar and video fusion
PCT/CN2021/085146 WO2022141910A1 (en) 2021-01-01 2021-04-01 Vehicle-road laser radar point cloud dynamic segmentation and fusion method based on driving safety risk field
PCT/CN2021/085149 WO2022141913A1 (en) 2021-01-01 2021-04-01 On-board positioning device-based roadside millimeter-wave radar calibration method

Family Applications After (3)

Application Number Title Priority Date Filing Date
PCT/CN2022/084925 WO2022206977A1 (en) 2021-01-01 2022-04-01 Cooperative-vehicle-infrastructure-oriented sensing information fusion representation and target detection method
PCT/CN2022/084929 WO2022206978A1 (en) 2021-01-01 2022-04-01 Roadside millimeter-wave radar calibration method based on vehicle-mounted positioning apparatus
PCT/CN2022/084738 WO2022206942A1 (en) 2021-01-01 2022-04-01 Laser radar point cloud dynamic segmentation and fusion method based on driving safety risk field

Country Status (3)

Country Link
CN (5) CN116685873A (en)
GB (2) GB2620877A (en)
WO (9) WO2022141912A1 (en)

Families Citing this family (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114724362B (en) * 2022-03-23 2022-12-27 中交信息技术国家工程实验室有限公司 Vehicle track data processing method
CN115358530A (en) * 2022-07-26 2022-11-18 上海交通大学 Vehicle-road cooperative sensing roadside test data quality evaluation method
CN115236628B (en) * 2022-07-26 2024-05-31 中国矿业大学 Method for detecting residual cargoes in carriage based on laser radar
CN115113157B (en) * 2022-08-29 2022-11-22 成都瑞达物联科技有限公司 Beam pointing calibration method based on vehicle-road cooperative radar
CN115166721B (en) * 2022-09-05 2023-04-07 湖南众天云科技有限公司 Radar and GNSS information calibration fusion method and device in roadside sensing equipment
CN115480243B (en) * 2022-09-05 2024-02-09 江苏中科西北星信息科技有限公司 Multi-millimeter wave radar end-edge cloud fusion calculation integration and application method thereof
CN115272493B (en) * 2022-09-20 2022-12-27 之江实验室 Abnormal target detection method and device based on continuous time sequence point cloud superposition
CN115235478B (en) * 2022-09-23 2023-04-07 武汉理工大学 Intelligent automobile positioning method and system based on visual label and laser SLAM
CN115830860B (en) * 2022-11-17 2023-12-15 西部科学城智能网联汽车创新中心(重庆)有限公司 Traffic accident prediction method and device
CN115966084B (en) * 2023-03-17 2023-06-09 江西昂然信息技术有限公司 Holographic intersection millimeter wave radar data processing method and device and computer equipment
CN116189116B (en) * 2023-04-24 2024-02-23 江西方兴科技股份有限公司 Traffic state sensing method and system
CN117452392B (en) * 2023-12-26 2024-03-08 哈尔滨工业大学(深圳)(哈尔滨工业大学深圳科技创新研究院) Radar data processing system and method for vehicle-mounted auxiliary driving system
CN117471461B (en) * 2023-12-26 2024-03-08 哈尔滨工业大学(深圳)(哈尔滨工业大学深圳科技创新研究院) Road side radar service device and method for vehicle-mounted auxiliary driving system
CN117961915B (en) * 2024-03-28 2024-06-04 太原理工大学 Intelligent auxiliary decision-making method of coal mine tunneling robot

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107133966A (en) * 2017-03-30 2017-09-05 浙江大学 A kind of three-dimensional sonar image background dividing method based on sampling consistency algorithm
CN108152831A (en) * 2017-12-06 2018-06-12 中国农业大学 A kind of laser radar obstacle recognition method and system
CN108629231A (en) * 2017-03-16 2018-10-09 百度在线网络技术（北京）有限公司 Obstacle detection method, device, equipment and storage medium
WO2018224634A1 (en) * 2017-06-08 2018-12-13 Renault S.A.S Method and system for identifying at least one moving object
CN109509260A (en) * 2017-09-14 2019-03-22 百度在线网络技术（北京）有限公司 Mask method, equipment and the readable medium of dynamic disorder object point cloud
CN110458112A (en) * 2019-08-14 2019-11-15 上海眼控科技股份有限公司 Vehicle checking method, device, computer equipment and readable storage medium storing program for executing
CN111880191A (en) * 2020-06-16 2020-11-03 北京大学 Map generation method based on multi-agent laser radar and visual information fusion
US10839530B1 (en) * 2018-09-04 2020-11-17 Apple Inc. Moving point detection

Family Cites Families (39)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6661370B2 (en) * 2001-12-11 2003-12-09 Fujitsu Ten Limited Radar data processing apparatus and data processing method
EP2923174A2 (en) * 2012-11-22 2015-09-30 GeoSim Systems Ltd. Point-cloud fusion
KR101655606B1 (en) * 2014-12-11 2016-09-07 현대자동차주식회사 Apparatus for tracking multi object using lidar and method thereof
TWI597513B (en) * 2016-06-02 2017-09-01 財團法人工業技術研究院 Positioning system, onboard positioning device and positioning method thereof
CN105892471B (en) * 2016-07-01 2019-01-29 北京智行者科技有限公司 Automatic driving method and apparatus
WO2018126248A1 (en) * 2017-01-02 2018-07-05 Okeeffe James Micromirror array for feedback-based image resolution enhancement
KR102056147B1 (en) * 2016-12-09 2019-12-17 (주)엠아이테크 Registration method of distance data and 3D scan data for autonomous vehicle and method thereof
CN106846494A (en) * 2017-01-16 2017-06-13 青岛海大新星软件咨询有限公司 Oblique photograph three-dimensional building thing model automatic single-body algorithm
US10281920B2 (en) * 2017-03-07 2019-05-07 nuTonomy Inc. Planning for unknown objects by an autonomous vehicle
CN108932462B (en) * 2017-05-27 2021-07-16 华为技术有限公司 Driving intention determining method and device
CN107609522B (en) * 2017-09-19 2021-04-13 东华大学 Information fusion vehicle detection system based on laser radar and machine vision
CN108010360A (en) * 2017-12-27 2018-05-08 中电海康集团有限公司 A kind of automatic Pilot context aware systems based on bus or train route collaboration
CN108639059B (en) * 2018-05-08 2019-02-19 清华大学 Driver based on least action principle manipulates behavior quantization method and device
CN109188379B (en) * 2018-06-11 2023-10-13 深圳市保途者科技有限公司 Automatic calibration method for driving auxiliary radar working angle
CN112368598A (en) * 2018-07-02 2021-02-12 索尼半导体解决方案公司 Information processing apparatus, information processing method, computer program, and mobile apparatus
CN109297510B (en) * 2018-09-27 2021-01-01 百度在线网络技术（北京）有限公司 Relative pose calibration method, device, equipment and medium
CN111429739A (en) * 2018-12-20 2020-07-17 阿里巴巴集团控股有限公司 Driving assisting method and system
JP7217577B2 (en) * 2019-03-20 2023-02-03 フォルシアクラリオン・エレクトロニクス株式会社 CALIBRATION DEVICE, CALIBRATION METHOD
CN110296713B (en) * 2019-06-17 2024-06-04 广州卡尔动力科技有限公司 Roadside automatic driving vehicle positioning navigation system and single/multiple vehicle positioning navigation method
CN110220529B (en) * 2019-06-17 2023-05-23 深圳数翔科技有限公司 Positioning method for automatic driving vehicle at road side
CN110532896B (en) * 2019-08-06 2022-04-08 北京航空航天大学 Road vehicle detection method based on fusion of road side millimeter wave radar and machine vision
CN110443978B (en) * 2019-08-08 2021-06-18 南京联舜科技有限公司 Tumble alarm device and method
CN110850378B (en) * 2019-11-22 2021-11-19 深圳成谷科技有限公司 Automatic calibration method and device for roadside radar equipment
CN110850431A (en) * 2019-11-25 2020-02-28 盟识(上海)科技有限公司 System and method for measuring trailer deflection angle
CN110906939A (en) * 2019-11-28 2020-03-24 安徽江淮汽车集团股份有限公司 Automatic driving positioning method and device, electronic equipment, storage medium and automobile
CN111121849B (en) * 2020-01-02 2021-08-20 大陆投资(中国)有限公司 Automatic calibration method for orientation parameters of sensor, edge calculation unit and roadside sensing system
CN111999741B (en) * 2020-01-17 2023-03-14 青岛慧拓智能机器有限公司 Method and device for detecting roadside laser radar target
CN111157965B (en) * 2020-02-18 2021-11-23 北京理工大学重庆创新中心 Vehicle-mounted millimeter wave radar installation angle self-calibration method and device and storage medium
CN111476822B (en) * 2020-04-08 2023-04-18 浙江大学 Laser radar target detection and motion tracking method based on scene flow
CN111554088B (en) * 2020-04-13 2022-03-22 重庆邮电大学 Multifunctional V2X intelligent roadside base station system
CN111192295B (en) * 2020-04-14 2020-07-03 中智行科技有限公司 Target detection and tracking method, apparatus, and computer-readable storage medium
CN111537966B (en) * 2020-04-28 2022-06-10 东南大学 Array antenna error correction method suitable for millimeter wave vehicle-mounted radar field
CN111766608A (en) * 2020-06-12 2020-10-13 苏州泛像汽车技术有限公司 Environmental perception system based on laser radar
CN111880174A (en) * 2020-07-03 2020-11-03 芜湖雄狮汽车科技有限公司 Roadside service system for supporting automatic driving control decision and control method thereof
CN111914664A (en) * 2020-07-06 2020-11-10 同济大学 Vehicle multi-target detection and track tracking method based on re-identification
CN111985322B (en) * 2020-07-14 2024-02-06 西安理工大学 Road environment element sensing method based on laser radar
CN111862157B (en) * 2020-07-20 2023-10-10 重庆大学 Multi-vehicle target tracking method integrating machine vision and millimeter wave radar
CN112019997A (en) * 2020-08-05 2020-12-01 锐捷网络股份有限公司 Vehicle positioning method and device
CN112509333A (en) * 2020-10-20 2021-03-16 智慧互通科技股份有限公司 Roadside parking vehicle track identification method and system based on multi-sensor sensing

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108629231A (en) * 2017-03-16 2018-10-09 百度在线网络技术（北京）有限公司 Obstacle detection method, device, equipment and storage medium
CN107133966A (en) * 2017-03-30 2017-09-05 浙江大学 A kind of three-dimensional sonar image background dividing method based on sampling consistency algorithm
WO2018224634A1 (en) * 2017-06-08 2018-12-13 Renault S.A.S Method and system for identifying at least one moving object
CN109509260A (en) * 2017-09-14 2019-03-22 百度在线网络技术（北京）有限公司 Mask method, equipment and the readable medium of dynamic disorder object point cloud
CN108152831A (en) * 2017-12-06 2018-06-12 中国农业大学 A kind of laser radar obstacle recognition method and system
US10839530B1 (en) * 2018-09-04 2020-11-17 Apple Inc. Moving point detection
CN110458112A (en) * 2019-08-14 2019-11-15 上海眼控科技股份有限公司 Vehicle checking method, device, computer equipment and readable storage medium storing program for executing
CN111880191A (en) * 2020-06-16 2020-11-03 北京大学 Map generation method based on multi-agent laser radar and visual information fusion

Also Published As

Publication number Publication date
WO2022141912A1 (en) 2022-07-07
CN117836667A (en) 2024-04-05
WO2022141911A1 (en) 2022-07-07
GB202313215D0 (en) 2023-10-11
CN117836653A (en) 2024-04-05
CN117441197A (en) 2024-01-23
WO2022206978A1 (en) 2022-10-06
CN117441113A (en) 2024-01-23
WO2022206942A1 (en) 2022-10-06
WO2022141910A1 (en) 2022-07-07
GB2618936A (en) 2023-11-22
CN116685873A (en) 2023-09-01
WO2022141913A1 (en) 2022-07-07
GB202316625D0 (en) 2023-12-13
WO2022141914A1 (en) 2022-07-07
WO2022206977A1 (en) 2022-10-06
GB2620877A (en) 2024-01-24

Similar Documents

Publication Publication Date Title
WO2022206974A1 (en) Roadside sensing unit-based static and non-static object point cloud recognition method
Han et al. Research on road environmental sense method of intelligent vehicle based on tracking check
CN108710875B (en) A kind of take photo by plane road vehicle method of counting and device based on deep learning
WO2018068653A1 (en) Point cloud data processing method and apparatus, and storage medium
CN115605777A (en) Dynamic target point cloud rapid identification and point cloud segmentation method based on road side sensing unit
CN110210389B (en) Multi-target identification tracking method for road traffic scene
WO2018133851A1 (en) Point cloud data processing method and apparatus, and computer storage medium
CN108920481B (en) Road network reconstruction method and system based on mobile phone positioning data
CN102779280B (en) Traffic information extraction method based on laser sensor
CN106199558A (en) Barrier method for quick
Chen et al. Architecture of vehicle trajectories extraction with roadside LiDAR serving connected vehicles
CN113569915B (en) Multi-strategy rail transit obstacle recognition method based on laser radar
CN111461088B (en) Rail transit obstacle avoidance system based on image processing and target recognition
CN106570454A (en) Pedestrian traffic parameter extraction method based on mobile laser scanning
CN105160309A (en) Three-lane detection method based on image morphological segmentation and region growing
CN106683530A (en) Computerized judging system and method based on three-dimensional laser vision and high-precision lane model
CN102779410B (en) Parallel implementation method of multi-source heterogeneous traffic data fusion
CN102855759A (en) Automatic collecting method of high-resolution satellite remote sensing traffic flow information
CN105608505A (en) Cellular signaling data based track traffic travel mode identification method for resident
CN103500329B (en) Street lamp automatic extraction method based on vehicle-mounted mobile laser scanning point cloud
CN116685874A (en) Camera-laser radar fusion object detection system and method
CN114898296A (en) Bus lane occupation detection method based on millimeter wave radar and vision fusion
CN103794050A (en) Real-time transport vehicle detecting and tracking method
CN116830164A (en) LiDAR decorrelated object detection system and method
CN110210384A (en) A kind of road global information extract real-time and indicate system

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22779154

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 202280026656.3

Country of ref document: CN

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 22779154

Country of ref document: EP

Kind code of ref document: A1