CN113900119A - Laser radar vehicle detection method, system, storage medium and equipment - Google Patents
Laser radar vehicle detection method, system, storage medium and equipment Download PDFInfo
- Publication number
- CN113900119A CN113900119A CN202111147855.3A CN202111147855A CN113900119A CN 113900119 A CN113900119 A CN 113900119A CN 202111147855 A CN202111147855 A CN 202111147855A CN 113900119 A CN113900119 A CN 113900119A
- Authority
- CN
- China
- Prior art keywords
- neural network
- sample
- data
- detected vehicle
- sample blocks
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 75
- 230000007246 mechanism Effects 0.000 claims abstract description 66
- 238000003062 neural network model Methods 0.000 claims abstract description 39
- 238000013528 artificial neural network Methods 0.000 claims abstract description 38
- 238000004364 calculation method Methods 0.000 claims abstract description 27
- 238000000605 extraction Methods 0.000 claims abstract description 22
- 238000012545 processing Methods 0.000 claims abstract description 15
- 238000000034 method Methods 0.000 claims description 61
- 238000004590 computer program Methods 0.000 claims description 9
- 238000013461 design Methods 0.000 claims description 9
- 230000000750 progressive effect Effects 0.000 claims description 8
- 230000000903 blocking effect Effects 0.000 claims description 7
- 238000012360 testing method Methods 0.000 claims description 7
- 238000007781 pre-processing Methods 0.000 claims description 6
- 230000009467 reduction Effects 0.000 claims description 5
- 238000010586 diagram Methods 0.000 description 14
- 238000003384 imaging method Methods 0.000 description 5
- 230000002411 adverse Effects 0.000 description 4
- 238000009826 distribution Methods 0.000 description 4
- 230000000694 effects Effects 0.000 description 4
- 230000008569 process Effects 0.000 description 4
- 238000004422 calculation algorithm Methods 0.000 description 3
- 230000006870 function Effects 0.000 description 3
- 230000006872 improvement Effects 0.000 description 3
- 238000012544 monitoring process Methods 0.000 description 3
- 230000001360 synchronised effect Effects 0.000 description 3
- 230000003247 decreasing effect Effects 0.000 description 2
- 230000014509 gene expression Effects 0.000 description 2
- 238000005259 measurement Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000002787 reinforcement Effects 0.000 description 2
- 238000012795 verification Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 230000001094 effect on targets Effects 0.000 description 1
- 238000005286 illumination Methods 0.000 description 1
- 230000010365 information processing Effects 0.000 description 1
- 238000010295 mobile communication Methods 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000012549 training Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S17/00—Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
- G01S17/88—Lidar systems specially adapted for specific applications
- G01S17/93—Lidar systems specially adapted for specific applications for anti-collision purposes
- G01S17/931—Lidar systems specially adapted for specific applications for anti-collision purposes of land vehicles
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S17/00—Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
- G01S17/66—Tracking systems using electromagnetic waves other than radio waves
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S17/00—Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
- G01S17/88—Lidar systems specially adapted for specific applications
- G01S17/89—Lidar systems specially adapted for specific applications for mapping or imaging
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Electromagnetism (AREA)
- Theoretical Computer Science (AREA)
- Remote Sensing (AREA)
- Radar, Positioning & Navigation (AREA)
- Computer Networks & Wireless Communication (AREA)
- Biophysics (AREA)
- Mathematical Physics (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Software Systems (AREA)
- Computational Linguistics (AREA)
- Biomedical Technology (AREA)
- Artificial Intelligence (AREA)
- Life Sciences & Earth Sciences (AREA)
- Health & Medical Sciences (AREA)
- Optical Radar Systems And Details Thereof (AREA)
- Traffic Control Systems (AREA)
Abstract
The invention provides a laser radar vehicle detection method, a laser radar vehicle detection system, a laser radar vehicle detection storage medium and laser radar vehicle detection equipment, wherein the laser radar vehicle detection method comprises the following steps: acquiring a laser radar original point cloud image of a detected vehicle and performing voxelization feature extraction on the original point cloud image so as to form a voxelization grid of the detected vehicle; performing block processing on the data of the voxelized grid of the detected vehicle according to the distance in the direction towards the detected vehicle to obtain a plurality of sample blocks; respectively designing a neural network model for the plurality of sample blocks, wherein the neural network model of the plurality of sample blocks has different neural network dimension enhancement mechanisms and/or data enhancement mechanisms adopted for the plurality of sample blocks; and performing feature extraction and loss calculation of the detected vehicle based on the neural network model of the plurality of sample blocks, wherein the loss calculation comprises three-dimensional contact ratio loss. The invention improves the accuracy of the laser radar detection in the aspect of three-dimensional detection of vehicles.
Description
Technical Field
The invention relates to the technical field of three-dimensional imaging, in particular to the field of target detection and tracking of a three-dimensional imaging laser radar, and specifically relates to a laser radar vehicle detection method, a laser radar vehicle detection system, a laser radar vehicle detection storage medium and laser radar vehicle detection equipment.
Background
In recent years, target detection and tracking based on three-dimensional imaging lidar has become an important issue in the field of computer vision. Compared with the traditional camera sensor, the three-dimensional imaging laser radar has the advantages that the three-dimensional geometric information of the target can be effectively acquired, and the influence of external illumination change and imaging distance is small. Therefore, the laser radar has been widely used in the field of unmanned driving.
In the field of unmanned driving, stable traveling of a vehicle must be ensured, and safety principles cannot be violated. For this reason, lidar, and in particular three-dimensional lidar, are sensing devices for wide-ranging applications on unmanned vehicles. Three-dimensional (3D) point cloud data of the surrounding environment, including the associated target vehicle, may be obtained by lidar.
With the continuous improvement of the measurement precision of the laser radar sensing device, the possibility of improving the detection performance is brought by the increase of the measurement dimension and the improvement of the resolution ratio. However, a truly improved performance requires a more intelligent, more robust information processing algorithm.
In addition, the detected target is usually placed in a certain background environment, even merged with the background. In the laser radar detection process, due to the view angle, the background and the like, the target may be shielded, so that the target is difficult to detect and segment.
In order to solve the above problems, the existing technical solution mainly performs some conventional three-dimensional sparse convolution and conventional two-dimensional feature extraction after point cloud voxelization, and finally performs conventional loss regression such as direction and three-dimensional frame.
However, the existing algorithm has low accuracy in the aspect of three-dimensional detection of vehicles, has poor detection effect on target vehicles, and can cause great adverse effects on safety and stability of unmanned driving under the condition of complex traffic conditions on roads.
Therefore, it is necessary to provide a detection method, especially for a vehicle, which can improve the accuracy of the three-dimensional detection of the vehicle and reduce the influence on the safety and stability of the unmanned driving as much as possible in view of the above-mentioned disadvantages in the prior art.
Disclosure of Invention
In view of the above, the present invention is directed to a method, system, storage medium and device for three-dimensional detection of a target, especially for vehicle detection, based on a laser radar, so as to solve the problems of low accuracy in three-dimensional detection of a vehicle, adverse influence on safety and stability of unmanned driving, and the like in the prior art.
In view of the above objects, in one aspect, the present invention provides a method for lidar vehicle detection, wherein the method comprises the steps of:
acquiring a laser radar original point cloud image of a detected vehicle and performing voxelization feature extraction on the original point cloud image so as to form a voxelization grid of the detected vehicle;
performing block processing on the data of the voxelized grid of the detected vehicle according to the distance in the direction towards the detected vehicle to obtain a plurality of sample blocks;
respectively designing a neural network model for the plurality of sample blocks, wherein the neural network model of the plurality of sample blocks has different neural network dimension enhancement mechanisms and/or data enhancement mechanisms adopted for the plurality of sample blocks;
and performing feature extraction and loss calculation of the detected vehicle based on the neural network model of the plurality of sample blocks, wherein the loss calculation comprises three-dimensional contact ratio loss.
In some embodiments of the method of lidar vehicle detection according to the present invention, the neural network dimension enhancement mechanism comprises performing convolution dimensionality reduction in two horizontal directions, a direction towards the detected vehicle and a horizontal direction perpendicular to the direction towards the vehicle to be detected, and maintaining a three-dimensional sparse convolution of dimensions in a vertical direction, the vertical direction being a direction perpendicular to the two horizontal directions.
In some embodiments of the method of lidar vehicle detection according to the present disclosure, the data enhancement mechanism includes dividing the sample of the detected vehicle into four pyramids by four diagonals of a point cloud box, forming a new sample based on the pyramids.
In some embodiments of the method of lidar vehicle detection according to the present disclosure, the data enhancement mechanism comprises:
selecting at least one rectangular pyramid from four rectangular pyramids of the detected vehicle, and combining the four selected rectangular pyramids to serve as a new sample; and/or
And deleting a plurality of point cloud data in at least one rectangular pyramid of the sample of the detected vehicle, and taking the sample containing the deleted point cloud data as a new sample.
In some embodiments of the method of lidar vehicle detection according to the present disclosure, the blocking data of the voxelized grid of the detected vehicle by distance in a direction toward the detected vehicle to obtain a plurality of sample blocks further comprises:
and in the direction towards the detected vehicle, the data of the voxelized network of the detected vehicle is subjected to block processing according to the distance from the detected vehicle, so that a near sample block, a middle sample block and a far sample block are obtained.
In some embodiments of the method for lidar vehicle detection according to the present invention, the performing a neural network model design on the plurality of sample blocks, respectively, wherein the neural network model of the plurality of sample blocks has different neural network dimension enhancement mechanisms and/or data enhancement mechanisms adopted for the plurality of sample blocks further comprises:
a progressive neural network dimension enhancement mechanism and/or a data enhancement mechanism is undertaken for the near, intermediate and far sample blocks.
In some embodiments of the method of lidar vehicle detection according to the present invention, the employing a progressive neural network dimension enhancement mechanism and/or a data enhancement mechanism for the near sample block, the intermediate sample block, and the far sample block further comprises:
adopting a neural network dimension enhancement mechanism aiming at the intermediate sample block;
and adopting a neural network dimension enhancement mechanism and a data enhancement mechanism aiming at the far sample block.
In another aspect of the present invention, there is also provided a laser radar vehicle detection system, including:
the system comprises a data preprocessing module, a data processing module and a data processing module, wherein the data preprocessing module is configured to acquire a laser radar original point cloud image of a detected vehicle and perform voxelization feature extraction on the original point cloud image so as to form a voxelization grid of the detected vehicle;
a data blocking module configured to block data of the voxelized grid of the detected vehicle by distance in a direction toward the detected vehicle, thereby obtaining a plurality of sample blocks;
a neural network module configured to perform neural network model design on the plurality of sample blocks, respectively, wherein the neural network models of the plurality of sample blocks have different neural network dimension enhancement mechanisms and/or data enhancement mechanisms adopted for the plurality of sample blocks;
a loss calculation module configured to perform feature extraction of the test vehicle based on the neural network model of the plurality of sample blocks and perform a loss calculation, wherein the loss calculation includes a three-dimensional overlap ratio loss.
In yet another aspect of the present invention, there is also provided a computer readable storage medium storing computer program instructions which, when executed, implement any of the above-described methods for lidar vehicle detection according to the present invention.
In yet another aspect of the present invention, there is also provided a computer apparatus comprising a memory and a processor, the memory having stored therein a computer program which, when executed by the processor, performs any of the above-described methods of lidar vehicle detection according to the present invention.
The invention has at least the following beneficial technical effects: by adopting a targeted neural network dimension enhancement mechanism and/or a data enhancement mechanism for the blocks of the original point cloud data and the samples of the divided blocks, the detection precision of a neural network model is greatly improved, and besides the conventional loss (loss) calculation such as three-dimensional frame regression and classification for feature extraction, the loss (3Diou loss) calculation of three-dimensional overlap ratio is creatively increased, so that the loss of precision in the numerical direction is better made up.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art that other embodiments can be obtained by using the drawings without creative efforts.
In the figure:
FIG. 1 shows a schematic flow diagram of an embodiment of a method of lidar vehicle detection according to the present disclosure;
FIG. 2 shows a schematic block diagram of an embodiment of a method of lidar vehicle detection according to the present disclosure;
FIG. 3 shows a schematic diagram of an embodiment of the calculation of three-dimensional overlap ratio loss according to the method of lidar vehicle detection of the present invention;
FIG. 4 shows a schematic diagram of an embodiment of a data enhancement mechanism of a method of lidar vehicle detection according to the present disclosure;
FIG. 5 shows a schematic diagram of an embodiment of a blocking process of a method of lidar vehicle detection according to the present disclosure;
FIG. 6 shows a schematic block diagram of an embodiment of a system for lidar vehicle detection according to the present disclosure;
FIG. 7 shows a schematic diagram of an embodiment of a computer readable storage medium embodying a method of lidar vehicle detection, in accordance with the present invention;
fig. 8 shows a hardware configuration diagram of an embodiment of a computer device implementing a method of lidar vehicle detection according to the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the following embodiments of the present invention are described in further detail with reference to the accompanying drawings.
It should be noted that all expressions using "first" and "second" in the embodiments of the present invention are used for distinguishing two non-identical entities with the same name or different parameters, and it is understood that "first" and "second" are only used for convenience of expression and should not be construed as limiting the embodiments of the present invention. Furthermore, the terms "comprises" and "comprising," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements does not include all of the other steps or elements inherent in the list.
Based on the above purpose, the invention creatively provides a method for three-dimensional vehicle detection based on laser radar point cloud, wherein a measuring and calculating method is improved on the basis of the existing laser radar detection hardware framework, so that the three-dimensional detection precision of the vehicle is greatly improved. Fig. 1 shows a schematic flow diagram of an embodiment of a method of lidar vehicle detection according to the present invention. In general, the overall concept of the method is mainly based on the following points. Firstly, the voxel characteristic is partitioned according to different distance data distribution. On the basis, different distributions of the data set can be respectively modeled in a targeted manner, so that information can be respectively extracted by using respective neural network models. Therefore, the model has higher pertinence and stronger characteristic capability. And secondly, aiming at the problems of few point clouds, occlusion verification and the like of a distant target point, a special neural network dimension enhancement mechanism and/or a special data enhancement mechanism are designed so as to improve the accuracy of a neural network model. Finally, aiming at the problems that the precision of the bird vision (bev) visual angle is high and the precision of the vertical direction (z direction) is not high enough, a three-dimensional contact ratio loss (3Diou loss) calculation module is designed, and the precision of the vertical direction (z direction) is greatly improved under the condition that the precision of the bird vision (bev) is not influenced.
To this end, in a first aspect of the invention, a method 100 of lidar vehicle detection is provided. Fig. 2 shows a schematic block diagram of an embodiment of a method of lidar vehicle detection according to the present invention. In the embodiment shown in fig. 1, the method comprises:
step S110: acquiring a laser radar original point cloud image of a detected vehicle and performing voxelization feature extraction on the original point cloud image so as to form a voxelization grid of the detected vehicle;
step S120: performing block processing on the data of the voxelized grid of the detected vehicle according to the distance in the direction towards the detected vehicle to obtain a plurality of sample blocks;
step S130: respectively designing a neural network model for the plurality of sample blocks, wherein the neural network model of the plurality of sample blocks has different neural network dimension enhancement mechanisms and/or data enhancement mechanisms adopted for the plurality of sample blocks;
step S140: and performing feature extraction and loss calculation of the detected vehicle based on the neural network model of the plurality of sample blocks, wherein the loss calculation comprises three-dimensional contact ratio loss.
In general, the key to the method according to the invention is data preprocessing, block processing of the preprocessed data, corresponding neural network model design and feature extraction loss calculation. First, step S110 obtains a lidar raw point cloud image of a detected vehicle and performs voxelization feature extraction on the raw point cloud image, thereby forming a voxelization grid of the detected vehicle. Here, a conventional voxel-based feature extraction is performed to form a voxel grid, preferably 1600 × 1408 × 41(x × y × z), as the raw point cloud image input.
Subsequently, step S120 performs block processing on the data of the voxelized grid of the detected vehicle by distance in the direction toward the detected vehicle, thereby obtaining a plurality of sample blocks. Here, the direction toward the detected vehicle is, for example, the y direction, and the voxel-based grid obtained in step S110 is segmented from near to far in the y direction. For example, in the preferred embodiment of forming the voxelized grid of 1600 × 1408 × 41 described above, the blocks may preferably be (0-400), (400-. Other block divisions, numbers of blocks, etc. may be performed, such as (0-600), (601-1408) or (0-300), (300-600), (600-900), (900-1408), etc.
After the block dividing process is completed, the neural network model design for the plurality of sample blocks may be performed in step S130, where the neural network model of the plurality of sample blocks has different neural network dimension enhancement mechanisms and/or data enhancement mechanisms adopted for the plurality of sample blocks. Preferably, a progressive neural network dimension enhancement mechanism and/or data enhancement mechanism is undertaken from near to far for a plurality of sample blocks to progressively improve the accuracy of the neural network model. By the method, the sample block can be pertinently reinforced aiming at the problems of few point clouds of distant target points, shielding verification and the like, so that the precision of the neural network model is improved to different degrees, the precision requirement of vehicle detection is met, the adverse effects on the safety and stability of unmanned driving are avoided as far as possible, and the burdens in the aspects of model operation, data volume, running speed and the like caused by excessive reinforcement are avoided.
After the modeling is completed, step S140 performs feature extraction of the test vehicle based on the neural network model of the plurality of sample blocks and performs loss calculation, wherein the loss calculation includes a three-dimensional overlap ratio loss. Specifically, in addition to conventional 3D frame regression, classification, etc. loss calculations, the method according to the present invention creatively proposes 3Diou loss, mainly by calculating the degree of overlap (IoU) of the predicted frame with the target frame. The greater the degree of coincidence, the more accurate the prediction. Fig. 3 shows a schematic diagram of an embodiment of the calculation of the three-dimensional contact ratio loss according to the method of lidar vehicle detection of the present invention. Unlike the three-dimensional overlap loss (3Diou loss) described in the present invention, the conventional 3D frame loss regression generally regresses 7 variables such as { x, y, z, w, l, h, r }, where (x, y, z) is the center point coordinates (x, y, z) of the 3D target frame, (w, l, h) are the length, width, and height of the 3D frame, respectively, and r is the rotation amount of the 3D frame. Because there are many features on bev bird views, training the regression in the horizontal (x, y) direction is more accurate. However, the regression accuracy in the vertical (z) direction is poor. Therefore, the invention proposes that the loss of accuracy in the vertical (z) direction can be better compensated by introducing 3Diou loss, in addition to the regression { x, y, z, w, l, h, r }, and additionally adding 3Diou loss.
In some embodiments of the method 100 of lidar vehicle detection according to the present disclosure, the neural network dimension enhancement mechanism in step S140 includes performing convolution dimensionality reduction in two horizontal directions, a direction toward the detected vehicle and a horizontal direction perpendicular to the direction toward the vehicle to be detected, and maintaining a three-dimensional sparse convolution of the dimensions in a vertical direction, the vertical direction being a direction perpendicular to the two horizontal directions.
In particular, in the concept of the method according to the invention, the above-mentioned neural network dimension enhancement mechanism is mainly convolution dimension reduction in two horizontal directions, and three-dimensional sparse convolution of dimensions is maintained in the vertical direction. In other words, the feature dimension increase in the vertical (z) direction is preferably made only for the 3D sparse convolution. In particular, a convolution dimensionality reduction of the two horizontal (x, y) directions is performed, for example preferably 1600 to 200, while the vertical (z) direction remains a higher dimension, for example preferably 41 to 11, even 41 to 2 (for closer sample blocks).
In some embodiments of the method 100 of lidar vehicle detection according to the present disclosure, the data enhancement mechanism includes dividing the sample of the detected vehicle into four pyramids by four diagonals of a point cloud box, forming a new sample based on the pyramids, as shown in fig. 4. A large amount of point cloud data is contained in each of these rectangular pyramids, but the amount of point cloud data varies depending on the situation detected by the lidar.
In some embodiments of the method 100 of lidar vehicle detection according to the present invention, the data enhancement mechanism includes (A) selecting at least one of four pyramids of the vehicle under test, combining the four selected pyramids as a new sample; and/or (B) deleting a plurality of point cloud data in at least one rectangular pyramid of the sample of the detected vehicle, and taking the sample containing the deleted point cloud data as a new sample. That is, in the method of lidar vehicle detection according to the present invention, the data enhancement mechanism may include the above two modes a and B, and one or both of the above modes may be selectively employed. Specifically, in the a-mode, at least one rectangular pyramid is selected from four rectangular pyramids of the vehicle to be detected, and the four selected rectangular pyramids are combined to be used as a new sample. For example, preferably, as in the embodiment shown in fig. 4, the pyramid point cloud data of two detected vehicles are randomly combined to form a new sample, for example, taking two pyramids each from the two samples, or taking one pyramid from the first sample and three pyramids from the second sample. In addition, the number of samples can be increased through a B mode, wherein one sample is selected, the point cloud data in at least one rectangular pyramid is randomly deleted, the sample containing the deleted point cloud data is saved as a new sample, and the original sample is reserved. By one of the two modes, a large amount of enhanced data can be formed, and the model detection precision can be greatly improved by the processing.
In some embodiments of the method 100 of lidar vehicle detection according to the present disclosure, the step S120 of blocking data of the voxelized grid of the detected vehicle by distance in a direction toward the detected vehicle, thereby obtaining a plurality of sample blocks further comprises: and in the direction towards the detected vehicle, the data of the voxelized network of the detected vehicle is subjected to block processing according to the distance from the detected vehicle, so that a near sample block, a middle sample block and a far sample block are obtained. Specifically, in step S120, the point cloud data result of the detected vehicle is visually analyzed, so that the detection accuracy rate is found to be in a decreasing trend from near to far, and the distribution is relatively balanced. According to analysis, generally, the test data lump length is 0-70.4m, wherein the accuracy is higher within 20m and lower after 40 m. Further analysis found that the data point clouds within 20m were dense and less occluded. And the positions outside 40m are just opposite, the target point cloud is sparse, and the shielding area is large. In view of the above, the data is partitioned according to the distance, and preferably, the data is partitioned into three blocks of, for example, 0 to 20, 20 to 40, and 40 to 70.4m, and the neural network model is designed for each of the three blocks to perform feature extraction corresponding to the near sample block, the intermediate sample block, and the far sample block.
In some embodiments of the method 100 for lidar vehicle detection according to the present invention, the step S130 of performing neural network model design on the plurality of sample blocks respectively, wherein the neural network model of the plurality of sample blocks has different neural network dimension enhancement mechanisms and/or data enhancement mechanisms adopted for the plurality of sample blocks further comprises: step S131 assumes a progressive neural network dimension enhancement mechanism and/or a data enhancement mechanism for the near sample block, the middle sample block, and the far sample block. Specifically, step S130 is based on the analysis result, that is, the detection accuracy rate is decreased from near to far, and the distribution is relatively balanced. Therefore, for a near sample block, a middle sample block and a far sample block which are divided into three blocks of 0-20, 20-40, 40-70.4m and the like, a progressive neural network dimension enhancement mechanism and/or a data enhancement mechanism are/is adopted to progressively improve the accuracy of the neural network model.
Further, in some embodiments of the method 100 of lidar vehicle detection according to the present invention, the step S131 of employing a progressive neural network dimension enhancement mechanism and/or a data enhancement mechanism for the near, intermediate and far sample blocks further comprises: step S1311, a neural network dimension enhancement mechanism is adopted for the middle sample block; step S1312 adopts a neural network dimension enhancement mechanism and a data enhancement mechanism for the far sample block. That is to say, for a near sample block, because the detection accuracy is high enough, the neural network does not need to perform special processing, and conventional 3D sparse convolution is adopted, and then 2D feature extraction is performed. And aiming at the fact that the detection precision of the middle sample block is relatively high, only a neural network dimension enhancement mechanism is adopted, namely the characteristic dimension increase in the z direction is carried out on the 3D sparse convolution. For the situation that the detection precision of the remote block is poor, a data enhancement mechanism is adopted besides a neural network dimension enhancement mechanism of the middle sample block. Through the mode, the accuracy of the neural network model is improved to different degrees, so that the accuracy requirement of vehicle detection is met, the adverse effects on the safety and stability of unmanned driving are avoided as far as possible, and meanwhile, the burden on the aspects of model operation, data quantity, running speed and the like due to excessive reinforcement is avoided.
In a second aspect of the present invention, a lidar vehicle detection system 200 is also provided. Fig. 6 shows a schematic block diagram of an embodiment of a system 200 for lidar vehicle detection according to the present invention. As shown in fig. 6, the system includes:
a data preprocessing module 210, wherein the data preprocessing module 210 is configured to acquire a lidar raw point cloud image of a detected vehicle and perform voxelization feature extraction on the raw point cloud image, so as to form a voxelization grid of the detected vehicle;
a data blocking module 220, wherein the data blocking module 220 is configured to block data of the voxelized grid of the detected vehicle by distance in a direction toward the detected vehicle, so as to obtain a plurality of sample blocks;
a neural network module 230, wherein the neural network computation module 230 is configured to perform neural network model design on the plurality of sample blocks respectively, wherein the neural network models of the plurality of sample blocks have different neural network dimension enhancement mechanisms and/or data enhancement mechanisms adopted for the plurality of sample blocks;
a loss calculation module 240, the loss calculation module 240 configured to perform feature extraction of the test vehicle based on the neural network model of the plurality of sample blocks and perform a loss calculation, wherein the loss calculation includes a three-dimensional overlap ratio loss.
In a third aspect of the embodiments of the present invention, there is also provided a computer-readable storage medium, and fig. 7 is a schematic diagram of the computer-readable storage medium of the method for laser radar vehicle detection provided by the embodiments of the present invention. As shown in fig. 7, the computer-readable storage medium 300 stores computer program instructions 310, the computer program instructions 310 being executable by a processor. The computer program instructions 310, when executed, implement the method of any of the embodiments described above.
It is to be understood that all embodiments, features and advantages set forth above with respect to the method for lidar vehicle detection according to the present invention apply equally, without conflict with one another, to the system for quantitative model deployment and to the storage medium according to the present invention.
In a fourth aspect of the embodiments of the present invention, there is further provided a computer device 400, comprising a memory 420 and a processor 410, wherein the memory stores a computer program, and the computer program, when executed by the processor, implements the method of any one of the above embodiments.
Fig. 8 is a schematic hardware structure diagram of an embodiment of a computer device for executing the method for laser radar vehicle detection according to the present invention. Taking the computer device 400 shown in fig. 8 as an example, the computer device includes a processor 410 and a memory 420, and may further include: an input device 430 and an output device 440. The processor 410, the memory 420, the input device 430, and the output device 440 may be connected by a bus or other means, as exemplified by the bus connection in fig. 8. Input device 430 may receive input numeric or character information and generate signal inputs related to lidar vehicle detection. The output device 440 may include a display device such as a display screen.
The memory 420 is a non-volatile computer-readable storage medium, and can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules, such as program instructions/modules corresponding to the resource monitoring method in the embodiment of the present application. The memory 420 may include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required for at least one function; the storage data area may store data created by use of the resource monitoring method, and the like. Further, the memory 420 may include high speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other non-volatile solid state storage device. In some embodiments, memory 420 may optionally include memory located remotely from processor 410, which may be connected to local modules via a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The processor 410 executes various functional applications of the server and data processing by executing nonvolatile software programs, instructions and modules stored in the memory 420, that is, implements the resource monitoring method of the above-described method embodiment.
Finally, it should be noted that the computer-readable storage medium (e.g., memory) herein can be either volatile memory or nonvolatile memory, or can include both volatile and nonvolatile memory. By way of example, and not limitation, nonvolatile memory can include Read Only Memory (ROM), Programmable ROM (PROM), Electrically Programmable ROM (EPROM), Electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM), which can act as external cache memory. By way of example and not limitation, RAM is available in a variety of forms such as synchronous RAM (DRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), Enhanced SDRAM (ESDRAM), Synchronous Link DRAM (SLDRAM), and Direct Rambus RAM (DRRAM). The storage devices of the disclosed aspects are intended to comprise, without being limited to, these and other suitable types of memory.
Those of skill would further appreciate that the various illustrative logical blocks, modules, circuits, and algorithm steps described in connection with the disclosure herein may be implemented as electronic hardware, computer software, or combinations of both. To clearly illustrate this interchangeability of hardware and software, various illustrative components, blocks, modules, circuits, and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as software or hardware depends upon the particular application and design constraints imposed on the overall system. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the disclosed embodiments of the present invention.
The various illustrative logical blocks, modules, and circuits described in connection with the disclosure herein may be implemented or performed with the following components designed to perform the functions herein: a general purpose processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination of these components. A general purpose processor may be a microprocessor, but in the alternative, the processor may be any conventional processor, controller, microcontroller, or state machine. A processor may also be implemented as a combination of computing devices, e.g., a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors in conjunction with a DSP, and/or any other such configuration.
The foregoing is an exemplary embodiment of the present disclosure, but it should be noted that various changes and modifications could be made herein without departing from the scope of the present disclosure as defined by the appended claims. The functions, steps and/or actions of the method claims in accordance with the disclosed embodiments described herein need not be performed in any particular order. Furthermore, although elements of the disclosed embodiments of the invention may be described or claimed in the singular, the plural is contemplated unless limitation to the singular is explicitly stated.
It should be understood that, as used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly supports the exception. It should also be understood that "and/or" as used herein is meant to include any and all possible combinations of one or more of the associated listed items. The numbers of the embodiments disclosed in the embodiments of the present invention are merely for description, and do not represent the merits of the embodiments.
Those of ordinary skill in the art will understand that: the discussion of any embodiment above is meant to be exemplary only, and is not intended to intimate that the scope of the disclosure, including the claims, of embodiments of the invention is limited to these examples; within the idea of an embodiment of the invention, also technical features in the above embodiment or in different embodiments may be combined and there are many other variations of the different aspects of the embodiments of the invention as described above, which are not provided in detail for the sake of brevity. Therefore, any omissions, modifications, substitutions, improvements, and the like that may be made without departing from the spirit and principles of the embodiments of the present invention are intended to be included within the scope of the embodiments of the present invention.
Claims (10)
1. A method of lidar vehicle detection comprising the steps of:
acquiring a laser radar original point cloud image of a detected vehicle and performing voxelization feature extraction on the original point cloud image so as to form a voxelization grid of the detected vehicle;
performing block processing on data of the voxelized grid of the detected vehicle according to distance in a direction towards the detected vehicle to obtain a plurality of sample blocks;
respectively performing neural network model design on the plurality of sample blocks, wherein the neural network models of the plurality of sample blocks have different neural network dimension enhancement mechanisms and/or data enhancement mechanisms adopted for the plurality of sample blocks;
feature extraction and loss calculation of the test vehicle are performed based on the neural network model of the plurality of sample blocks, wherein the loss calculation includes a three-dimensional overlap ratio loss.
2. The method according to claim 1, wherein the neural network dimension enhancement mechanism comprises performing convolution dimensionality reduction in two horizontal directions, a direction towards the detected vehicle and a horizontal direction perpendicular to the direction towards the vehicle to be detected, and maintaining a three-dimensional sparse convolution of the dimensions in a vertical direction, the vertical direction being a direction perpendicular to the two horizontal directions.
3. The method of claim 1, wherein the data enhancement mechanism comprises dividing the sample of the detected vehicle into four pyramids by four diagonals of a point cloud box, forming new samples based on the pyramids.
4. The method of claim 3, wherein the data enhancement mechanism comprises:
selecting at least one rectangular pyramid from four rectangular pyramids of the detected vehicle, and combining the four selected rectangular pyramids to serve as a new sample; and/or
And deleting a plurality of point cloud data in at least one rectangular pyramid of the sample of the detected vehicle, and taking the sample containing the deleted point cloud data as a new sample.
5. The method of any one of claims 1 to 4, wherein the block processing data of the voxelized grid of the detected vehicle by distance in a direction toward the detected vehicle to obtain a plurality of sample blocks further comprises:
and in the direction towards the detected vehicle, according to the distance from the detected vehicle, carrying out block processing on the data of the voxelized network of the detected vehicle so as to obtain a near sample block, a middle sample block and a far sample block.
6. The method of claim 5, wherein the designing the neural network model for the plurality of sample blocks respectively, wherein the neural network model for the plurality of sample blocks has different neural network dimension enhancement mechanisms and/or data enhancement mechanisms adopted for the plurality of sample blocks further comprises:
a progressive neural network dimension enhancement mechanism and/or a data enhancement mechanism is undertaken for the near, intermediate and far sample blocks.
7. The method of claim 6, wherein said employing a progressive neural network dimension enhancement mechanism and/or a data enhancement mechanism for the near, intermediate, and far sample blocks further comprises:
adopting a neural network dimension enhancement mechanism for the intermediate sample block;
a neural network dimension enhancement mechanism and a data enhancement mechanism are employed for the distant sample blocks.
8. A system for lidar vehicle detection, comprising:
a data preprocessing module configured to obtain a lidar raw point cloud image of a detected vehicle and perform voxelization feature extraction on the raw point cloud image to form a voxelization grid of the detected vehicle;
a data blocking module configured to block data of the voxelized grid of the detected vehicle by distance in a direction toward the detected vehicle, thereby obtaining a plurality of sample blocks;
a neural network module configured to perform neural network model design on the plurality of sample blocks, respectively, wherein the neural network models of the plurality of sample blocks have different neural network dimension enhancement mechanisms and/or data enhancement mechanisms adopted for the plurality of sample blocks;
a loss calculation module configured to perform feature extraction of the test vehicle based on the neural network model of the plurality of sample blocks and perform a loss calculation, wherein the loss calculation includes a three-dimensional overlap ratio loss.
9. A computer readable storage medium having stored thereon computer program instructions which, when executed, implement a method of lidar vehicle detection as defined in any of claims 1 to 7.
10. A computer device comprising a memory and a processor, characterized in that the memory has stored therein a computer program which, when executed by the processor, performs the method of lidar vehicle detection as defined in any of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111147855.3A CN113900119B (en) | 2021-09-29 | 2021-09-29 | Method, system, storage medium and equipment for laser radar vehicle detection |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111147855.3A CN113900119B (en) | 2021-09-29 | 2021-09-29 | Method, system, storage medium and equipment for laser radar vehicle detection |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113900119A true CN113900119A (en) | 2022-01-07 |
| CN113900119B CN113900119B (en) | 2024-01-30 |
Family
ID=79189148
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111147855.3A Active CN113900119B (en) | 2021-09-29 | 2021-09-29 | Method, system, storage medium and equipment for laser radar vehicle detection |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113900119B (en) |
Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110032949A (en) * | 2019-03-22 | 2019-07-19 | 北京理工大学 | A kind of target detection and localization method based on lightweight convolutional neural networks |
| CN110674829A (en) * | 2019-09-26 | 2020-01-10 | 哈尔滨工程大学 | Three-dimensional target detection method based on graph convolution attention network |
| CN110827202A (en) * | 2019-11-07 | 2020-02-21 | 上海眼控科技股份有限公司 | Target detection method, target detection device, computer equipment and storage medium |
| CN110853069A (en) * | 2019-10-09 | 2020-02-28 | 上海眼控科技股份有限公司 | Neural network model construction method and system for vehicle appearance segmentation |
| CN111340797A (en) * | 2020-03-10 | 2020-06-26 | 山东大学 | Laser radar and binocular camera data fusion detection method and system |
| US20210011164A1 (en) * | 2017-12-14 | 2021-01-14 | COM-IoT Technologies | Mobile lidar platforms for vehicle tracking |
| CN112347987A (en) * | 2020-11-30 | 2021-02-09 | 江南大学 | Multimode data fusion three-dimensional target detection method |
| CN112444784A (en) * | 2019-08-29 | 2021-03-05 | 北京市商汤科技开发有限公司 | Three-dimensional target detection and neural network training method, device and equipment |
| US20210150230A1 (en) * | 2019-11-15 | 2021-05-20 | Nvidia Corporation | Multi-view deep neural network for lidar perception |
| CN113378854A (en) * | 2021-06-11 | 2021-09-10 | 武汉大学 | Point cloud target detection method integrating original point cloud and voxel division |
-
2021
- 2021-09-29 CN CN202111147855.3A patent/CN113900119B/en active Active
Patent Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20210011164A1 (en) * | 2017-12-14 | 2021-01-14 | COM-IoT Technologies | Mobile lidar platforms for vehicle tracking |
| CN110032949A (en) * | 2019-03-22 | 2019-07-19 | 北京理工大学 | A kind of target detection and localization method based on lightweight convolutional neural networks |
| CN112444784A (en) * | 2019-08-29 | 2021-03-05 | 北京市商汤科技开发有限公司 | Three-dimensional target detection and neural network training method, device and equipment |
| CN110674829A (en) * | 2019-09-26 | 2020-01-10 | 哈尔滨工程大学 | Three-dimensional target detection method based on graph convolution attention network |
| CN110853069A (en) * | 2019-10-09 | 2020-02-28 | 上海眼控科技股份有限公司 | Neural network model construction method and system for vehicle appearance segmentation |
| CN110827202A (en) * | 2019-11-07 | 2020-02-21 | 上海眼控科技股份有限公司 | Target detection method, target detection device, computer equipment and storage medium |
| US20210150230A1 (en) * | 2019-11-15 | 2021-05-20 | Nvidia Corporation | Multi-view deep neural network for lidar perception |
| CN111340797A (en) * | 2020-03-10 | 2020-06-26 | 山东大学 | Laser radar and binocular camera data fusion detection method and system |
| CN112347987A (en) * | 2020-11-30 | 2021-02-09 | 江南大学 | Multimode data fusion three-dimensional target detection method |
| CN113378854A (en) * | 2021-06-11 | 2021-09-10 | 武汉大学 | Point cloud target detection method integrating original point cloud and voxel division |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113900119B (en) | 2024-01-30 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| KR102143108B1 (en) | Lane recognition modeling method, device, storage medium and device, and recognition method, device, storage medium and device | |
| WO2022016311A1 (en) | Point cloud-based three-dimensional reconstruction method and apparatus, and computer device | |
| CN113658256B (en) | Target detection method and device based on laser radar and electronic equipment | |
| DE102017116192A1 (en) | Using virtual data to test and train parking lot detection systems | |
| CN112733812A (en) | Three-dimensional lane line detection method, device and storage medium | |
| Yang et al. | Multi-view semantic learning network for point cloud based 3D object detection | |
| CN113970734B (en) | Method, device and equipment for removing snowfall noise points of road side multi-line laser radar | |
| CN115526990A (en) | Target visualization method and device for digital twins and electronic equipment | |
| CN103959306B (en) | Dynamic line detecting system for the processor with limited inner memory | |
| CN108596032B (en) | Detection method, device, equipment and medium for fighting behavior in video | |
| CN115457492A (en) | Target detection method and device, computer equipment and storage medium | |
| CN115482522A (en) | Method, device and equipment for identifying corner characteristics of aircraft structural part and storage medium | |
| CN114495043A (en) | Method and system for detecting up-and-down slope road conditions based on binocular vision system and intelligent terminal | |
| Pan et al. | A novel computer vision‐based monitoring methodology for vehicle‐induced aerodynamic load on noise barrier | |
| CN112034482A (en) | Road boundary real-time extraction and measurement method and device | |
| CN117590362B (en) | Multi-laser radar external parameter calibration method, device and equipment | |
| CN115294538A (en) | Obstacle detection method and device, storage medium and terminal | |
| CN109523570B (en) | Motion parameter calculation method and device | |
| CN113900119B (en) | Method, system, storage medium and equipment for laser radar vehicle detection | |
| CN115542301B (en) | Method, device and equipment for calibrating external parameters of laser radar and storage medium | |
| CN115830073A (en) | Map element reconstruction method, map element reconstruction device, computer equipment and storage medium | |
| CN115619740A (en) | High-precision video speed measuring method and system, electronic equipment and storage medium | |
| CN114415181A (en) | Method and device for generating original echo of synthetic aperture radar | |
| CN114299029A (en) | Ground point cloud determining method and device, storage medium and electronic device | |
| CN113989632A (en) | Bridge detection method and device for remote sensing image, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |