CN112395956B - Method and system for detecting passable area facing complex environment - Google Patents
Method and system for detecting passable area facing complex environment Download PDFInfo
- Publication number
- CN112395956B CN112395956B CN202011164865.3A CN202011164865A CN112395956B CN 112395956 B CN112395956 B CN 112395956B CN 202011164865 A CN202011164865 A CN 202011164865A CN 112395956 B CN112395956 B CN 112395956B
- Authority
- CN
- China
- Prior art keywords
- network
- point cloud
- road
- detection
- deep learning
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S17/00—Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
- G01S17/88—Lidar systems specially adapted for specific applications
- G01S17/93—Lidar systems specially adapted for specific applications for anti-collision purposes
- G01S17/931—Lidar systems specially adapted for specific applications for anti-collision purposes of land vehicles
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S7/00—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00
- G01S7/48—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00 of systems according to group G01S17/00
- G01S7/4808—Evaluating distance, position or velocity data
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Engineering & Computer Science (AREA)
- Computational Linguistics (AREA)
- Computer Networks & Wireless Communication (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Biophysics (AREA)
- Biomedical Technology (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Remote Sensing (AREA)
- Radar, Positioning & Navigation (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Multimedia (AREA)
- Evolutionary Biology (AREA)
- Electromagnetism (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Bioinformatics & Computational Biology (AREA)
- Traffic Control Systems (AREA)
Abstract
The invention discloses a method for detecting a passable area facing a complex environment, which comprises the following steps: step 1, constructing a passable regional point cloud deep learning detection subsystem which comprises a road type identification network and a passable regional detection network; step 2, constructing a trafficable region traditional method detection subsystem which comprises a road type identification module and a trafficable region detection module; step 3, the point cloud deep learning detection subsystem and the traditional method detection subsystem are directly connected through a deep learning framework bottom layer interface; and 4, the newly acquired point cloud data of the vehicle-mounted laser radar is input into the point cloud deep learning detection subsystem preferentially. According to the method for detecting the passable area facing the complex environment, the passable area can be obtained in a training mode through the arrangement of the steps 1 to 4.
Description
Technical Field
The invention relates to the technical field of auxiliary driving and automatic driving of vehicles, in particular to a method and a system for detecting a passable area facing a complex environment.
Background
The passable area refers to an area where vehicles can safely run, and the passable area is required to stably and accurately detect the positioning navigation of the vehicles, the planning of vehicle paths and the like. The three-dimensional laser radar detection effect is not influenced by illumination and shadow, has three-dimensional information, has great technology mining potential, and is a research hotspot in both a related traditional method and a deep learning method. Two problems exist in the method, namely the problem of generalization of the traditional method, wherein the traditional method needs to manually select characteristics and threshold values according to specific environments, and the system has poor adaptability to environmental changes. Secondly, the deep learning method depends on the capacity of the training set and the training set, so that the training set sample size is too small to enable the network to completely fit the objective function, and good effects are obtained in the training set, even errors are zero, but the effects are poor in the test set. Aiming at the problems, the method is mainly selected to be respectively solved in the traditional method and the deep learning method, such as the segmentation threshold is adopted to enhance the adaptability of the traditional method to the environment and the sample label is manually added to expand the sample set, but the effect is still not ideal.
The patent with the publication number of CN110244321 and the publication date of 2019, 9 and 17 refers to a road passable area detection system based on a three-dimensional laser radar, wherein the system eliminates points above the laser radar beyond a certain height to obtain laser point cloud interest points, performs ground segmentation by using a RANSAC algorithm to distinguish the ground point cloud from obstacle point cloud, performs rasterization on the obstacle point cloud, extracts data points closest to vehicles in each grid, and combines the data points to be boundary points of a passable area. However, the characteristics and the threshold value of the characteristics in the whole system are manually extracted and set based on experience and specific data, so that the system lacks generalization, and the system detection error and even detection error are caused by the slightly disturbed data.
The patent with the authority of publication number CN110070059A and the application publication date of 2019, 7 and 30 mentions an unstructured road detection system based on domain migration, which trains an image segmentation network by utilizing artificial synthetic image data and combines the collected label-free unstructured road detection image data to generate a pseudo label. And training the whole network by using a domain migration technology and combining a real image data set and an artificial synthetic image data set which are formed by pseudo labels, and finally taking out an image segmentation network of the domain migration training network as an unstructured road detection network. Because the whole system network is based on the vision sensor, accuracy and robustness cannot be maintained under the conditions of insufficient illumination and shadow, and the system network is complex in redundancy and high in training time consumption.
The patent with the publication number of CN110414418A and the publication date of 2019, 11 and 5 refers to a road detection method for multi-scale fusion of image-laser radar image data. Data fusion increases the amount of data features, but the time cost increases, and image pixel texture information is not a major feature of passable region detection. Insufficient training sets of the network result in a road detection network that cannot fit a complete model and that has undesirable effects in the test set.
According to investigation, the current passable area detection system is complex in model, weak in generalization and poor in robustness, and cannot meet the requirements of vehicle positioning, navigation and control under different environments. Therefore, the system provided by the invention has the advantages of no need of designing a complex passable region detection model, strong generalization and high robustness, and can realize stable passable region detection in different environments. The system specifically comprises a passable area detection system facing to a complex environment by combining a traditional method and a deep learning method, wherein the traditional detection comprises a road type identification module and a passable area detection module, and the deep learning detection comprises a road identification network and a passable area detection network.
Disclosure of Invention
Aiming at the defects of the prior art, the invention aims to provide a passable region detection method facing complex environments by combining a traditional method and a deep learning method, so that the problems of low generalization capability, poor robustness and insufficient sample size are effectively solved, and various scenes can be effectively treated.
In order to achieve the above purpose, the present invention provides the following technical solutions: a method for detecting passable areas facing complex environments comprises the following steps:
step 2, constructing a trafficable region traditional method detection subsystem which comprises a road type identification module and a trafficable region detection module, wherein the road type identification module consists of feature extraction and a traditional machine learning classifier, and the trafficable region detection module consists of feature extraction, roadside extraction and tracking and road surface extraction;
step 3, the point cloud deep learning detection subsystem and the traditional method detection subsystem are directly connected through a deep learning framework bottom layer interface to complete the combination of the two subsystems, so that a complete passable area detection system facing the complex environment is formed;
and 4, the vehicle-mounted laser radar new acquisition point cloud data is preferentially input into the point cloud deep learning detection subsystem, identification analysis is carried out on the new acquisition point cloud data, a passable area is obtained, and if an effective road type result cannot be obtained when the new acquisition point cloud data is input into the point cloud deep learning detection subsystem, the new acquisition point cloud is transmitted to the traditional method detection subsystem, and the passable area is obtained.
As a further improvement of the invention, the specific step of obtaining the passable area in the step 4 is as follows, the vehicle-mounted laser radar newly-acquired point cloud data is preferentially input into the point cloud deep learning detection subsystem, if the road type identification network gives out an effective road type, the road type result and the newly-acquired point cloud data are continuously input into the passable area detection network, if the effective passable area detection result is output, the detection is directly ended, if the effective passable area detection result is not output, the road type result and the newly-acquired point cloud data are transmitted to the passable area detection module of the traditional detection subsystem, the feature extraction, the roadside detection and the road surface detection are sequentially carried out to obtain the passable area, if the new-acquired point cloud data cannot obtain the effective road type result after being input into the road type identification network of the point cloud deep learning subsystem, the new-acquired point cloud is transmitted to the traditional method detection subsystem, the feature extraction and the classifier are sequentially carried out on the road type identification module to obtain the road type identification result, and the road type result and the new-acquired point cloud data are continuously input into the passable area detection module, and the feature extraction, the roadside detection and the roadside detection are sequentially carried out to obtain the passable area. As a further improvement of the present invention, the specific steps for training the above two networks to achieve the detection accuracy in the step 1 are as follows:
step 1.1, utilizing a three-dimensional laser radar point cloud training set R with a passable area label l，road The sample trains a point cloud deep learning segmentation network G, so that the network G can detect the passable area of the point cloud data. Three-dimensional laser radar point cloud training set R with road type label type The sample trains a point cloud deep learning classification network C, so that the network C can detect the road type of the point cloud data.
Step 1.2, generating a generator in an countermeasure network by taking the point cloud deep learning segmentation network G trained in the step 1.1 as a condition; generating a network for generating conditions in the countermeasure network by taking the point cloud deep learning classification network C trained in the step 1.1 as the conditions; and taking the other point cloud deep learning classification network D as a discriminator for generating the countermeasure network.
Step 1.3, firstly, training and learning are carried out on a discriminator network D, and the training and learning are not carried out on a generator network G and a condition network C thereof;
step 1.4: considering that the influence of the output results of the condition network C and the generator network G on the passable region segmentation effect is not mutually independent, synchronously training and learning the condition network C and the generator network G, and fixing the discriminant network D without training and learning;
step 1.5: repeating the steps 1.3 and 1.4 until the condition network C and the generator network G reach a certain precision, taking out the condition network C as a road type identification network of the passable area point cloud deep learning detection subsystem, and taking out the generator network G as a passable area detection network of the passable area point cloud deep learning detection subsystem.
As a further improvement of the present invention, the specific steps of fixing the generator network G and the condition network C thereof in the step 1.3 without training learning are as follows:
step 1.3.1: the newly acquired untagged point cloud data R u，new Inputting a condition network C to obtain a road type, obtaining a passable area by a point cloud deep learning segmentation network according to the road type by a generator network G, adding a passable area label at a corresponding position of point cloud data, and marking as R f，new，road ；
Step 1.3.2: r output by the generator of the step 1.3.1 f，new，road And the training set R originally provided with labels l，road Inputting a discriminator D, and outputting a discrimination result by the discriminator D: d (R) f，new，road ) And D (R) l，road ). If the optimum direction of the discriminator is the correct discrimination data source, D (R l，road ) =1 and D (R f，new，road ) =0 is the arbiter optimization direction;
step 1.3.3: and constructing a loss function of the training discriminator D, and optimizing by using a gradient descent method.
Loss function:
L D ＝min D (CE(1，D(R l，road ))+CE(0，D(R f，new，road )))；
gradient:
gradient iteration:
θ D ＝θ D -μ D ·g D ；
where CE is a cross entropy function, θ D Is the network parameter of network D, μ D Is the network learning rate.
As a further improvement of the present invention, in the step 1.4, the condition network C and the generator network G perform synchronous training learning, and the specific steps of the arbiter network D fixing the training learning not are as follows:
step 1.4.1: training set data R with original labels l，road Inputting the condition network C to obtain the road type R f，l，type The generator network G obtains a passable area from the point cloud deep learning segmentation network according to the road type, adds a passable area label at a corresponding position of the point cloud data, and marks the passable area as R f，l，road ；
Step 1.4.2: r output by the generator of the step 1.4.1 f，l，road Input into a discriminator to obtain a discrimination result D (R f，l，road ) If the condition network optimization direction is infinitely approximate to the road type of the original data set and the generator optimization direction is infinitely approximate to the passable region segmentation result of the original data set, D (R f，l，road ) =1 is the conditional network and generator optimization direction;
step 1.4.3: and constructing a loss function of the training condition network C and the generator G, and optimizing by using a gradient descent method.
Loss function:
L G ＝
min G (CE(y l，type， R f，l，type )+CE(y l，road ，R f，l，road )+CE(1，D(R f，l，road )))；
gradient with respect to conditional network parameters:
gradient iteration:
θ C ＝θ C -μ C ·g C ；
gradient with respect to generator network parameters:
gradient iteration:
θ G ＝θ G -μ G ·g G ；
wherein CE is a cross entropy function, Y l，type Is true value of the original road type with label data, y l，road Is the true value of the division of the passable region of the original data with the tag, theta C Is a network parameter of network C, θ G Is a network parameter, mu, of the network G C Is the learning rate, mu, of the network C G Learning rate of the network G.
In another aspect, the invention provides a system comprising a vehicle terminal and an operation server, wherein the vehicle terminal and the operation server are in communication connection with each other to operate a program carrying the method, and the vehicle terminal comprises a vehicle and a vehicle-mounted laser radar.
The method has the advantages that the passable area point cloud deep learning detection subsystem can be effectively constructed through the arrangement of the step 1, the passable area traditional method detection subsystem can be effectively constructed through the arrangement of the step 2, the two subsystems can be effectively combined to form the passable area detection system through the arrangement of the step 3, new acquired point cloud data can be input into the two subsystems through the arrangement of the step 4, and analysis and judgment are carried out on the basis of road type results to obtain the passable area.
Drawings
FIG. 1 is a block diagram of a system employing the method of the present invention;
FIG. 2 is a schematic diagram of a discriminant training model;
FIG. 3 is a schematic diagram of a conditional network and a generator training model.
Detailed Description
The invention will be further described in detail with reference to examples of embodiments shown in the drawings.
Referring to fig. 1 to 3, a method for detecting a passable area facing a complex environment according to the present embodiment includes the following steps:
step 2, constructing a trafficable region traditional method detection subsystem which comprises a road type identification module and a trafficable region detection module, wherein the road type identification module consists of feature extraction and a traditional machine learning classifier, and the trafficable region detection module consists of feature extraction, roadside extraction and tracking and road surface extraction;
step 3, the point cloud deep learning detection subsystem and the traditional method detection subsystem are directly connected through a deep learning framework bottom layer interface to complete the combination of the two subsystems, so that a complete passable area detection system facing the complex environment is formed;
and 4, the vehicle-mounted laser radar newly-acquired point cloud data is preferentially input into the point cloud deep learning detection subsystem, identification analysis is carried out on the newly-acquired point cloud data to obtain a passable area, if an effective road type result cannot be obtained when the newly-acquired point cloud data is input into the point cloud deep learning detection subsystem, the newly-acquired point cloud data is transmitted to the traditional method detection subsystem to obtain the passable area, in the process of using the method of the embodiment, only the program for running the method from the step 1 to the step 4 is needed to be loaded into the vehicle-mounted system, then the point cloud deep learning detection subsystem and the traditional method detection subsystem can be effectively obtained by sequentially executing the step 1 and the step 2, then the combination of the two subsystems can be effectively realized by executing the step 3, the vehicle-mounted laser radar newly-acquired point cloud data is sequentially input into the point cloud deep learning detection subsystem and the traditional method detection subsystem, and finally the passable area detection subsystem is obtained according to whether the effective road type result is obtained, and finally the passable area can be obtained more accurately.
As an improved specific implementation manner, the specific step of obtaining the passable area in step 4 is as follows, the vehicle-mounted laser radar newly-acquired point cloud data is preferentially input into the point cloud deep learning detection subsystem, if the road type identification network gives an effective road type, the road type result and the newly-acquired point cloud data are continuously input into the passable area detection network, if the effective passable area detection result is output, the detection is directly ended, if the effective passable area detection result is not output, the road type result and the newly-acquired point cloud data are transmitted to a passable area detection module of the traditional detection subsystem, feature extraction, roadside detection and road surface detection are sequentially carried out to obtain the passable area, if the new-acquired point cloud data cannot obtain the effective road type result after the road type identification network of the new-acquired point cloud input point cloud deep learning subsystem, the new-acquired point cloud is transmitted to the traditional method detection subsystem, the road type identification module sequentially carries out feature extraction and the classifier to obtain the road type identification result, the road type result and the new-acquired point cloud data are further continuously input into the passable area detection module, the feature extraction, the roadside detection and the roadside detection is sequentially carried out to obtain the effective road type result, and the passable area can be effectively acquired by the road type is not obtained, and the road type can be effectively acquired through the road type detection system.
As an improved specific embodiment, the specific steps for training the two networks to achieve the detection accuracy in the step 1 are as follows:
step 1.1, utilizing a three-dimensional laser radar point cloud training set R with a passable area label l，road The sample trains a point cloud deep learning segmentation network G, so that the network G can detect the passable area of the point cloud data. Three-dimensional laser radar point cloud training set R with road type label type The sample trains a point cloud deep learning classification network C, so that the network C can detect the road type of the point cloud data.
Step 1.2, generating a generator in an countermeasure network by taking the point cloud deep learning segmentation network G trained in the step 1.1 as a condition; generating a network for generating conditions in the countermeasure network by taking the point cloud deep learning classification network C trained in the step 1.1 as the conditions; and taking the other point cloud deep learning classification network D as a discriminator for generating the countermeasure network.
Step 1.3, firstly, training and learning are carried out on a discriminator network D, and the training and learning are not carried out on a generator network G and a condition network C thereof;
step 1.4: considering that the influence of the output results of the condition network C and the generator network G on the passable region segmentation effect is not mutually independent, synchronously training and learning the condition network C and the generator network G, and fixing the discriminant network D without training and learning;
step 1.5: and repeating the steps 1.3 and 1.4 until the condition network C and the generator network G reach a certain precision, taking out the condition network C as a road type identification network of the passable area point cloud deep learning detection subsystem, taking out the generator network G as a passable area detection network of the passable area point cloud deep learning detection subsystem, and effectively realizing one-step learning training through the arrangement of the steps 1.1 to 1.5, thereby effectively ensuring the precision of the condition network C and the generator network G, and further improving the result precision of the finally judging the passable area.
As an improved specific embodiment, the specific steps of fixing the generator network G and the condition network C thereof in the step 1.3 without training learning are as follows:
step 1.3.1: the newly acquired untagged point cloud data R u，new Inputting a condition network C to obtain a road type, obtaining a passable area by a point cloud deep learning segmentation network according to the road type by a generator network G, adding a passable area label at a corresponding position of point cloud data, and marking as R f，new，road ；
Step 1.3.2: r output by the generator of the step 1.3.1 f，new，road And the training set R originally provided with labels l，road Inputting a discriminator D, and outputting a discrimination result by the discriminator D: d (R) f，new，road ) And D (R) l，road ). If the optimum direction of the discriminator is the correct discrimination data source, D (R l，road ) =1 and D (R f，new，road ) =0 is the arbiter optimization direction;
step 1.3.3: and constructing a loss function of the training discriminator D, and optimizing by using a gradient descent method.
Loss function:
L D ＝min D (CE(1，D(R l，road ))+CE(0，D(R f，new，road )))；
gradient:
gradient iteration:
θ D ＝θ D -μ D ·g D ；
where CE is a cross entropy function, θ D Is the network parameter of network D, μ D The training learning rate is the network learning rate, and training learning of the generator network G and the condition network C can be effectively realized through the setting of the steps.
As a specific implementation manner of improvement, in the step 1.4, the condition network C and the generator network G perform synchronous training learning, and the specific steps of the arbiter network D fixing the training learning not are as follows:
step 1.4.1: training set data R with original labels l，road Inputting the condition network C to obtain the road type R f，l，type The generator network G obtains a passable area from the point cloud deep learning segmentation network according to the road type, adds a passable area label at a corresponding position of the point cloud data, and marks the passable area as R f，l，road ；
Step 1.4.2: r output by the generator of the step 1.4.1 f，l，road Input into a discriminator to obtain a discrimination result D (R f，l，road ) If the condition network optimization direction is infinitely approximate to the road type of the original data set and the generator optimization direction is infinitely approximate to the passable region segmentation result of the original data set, D (R f，l，road ) =1 is the conditional network and generator optimization direction;
step 1.4.3: and constructing a loss function of the training condition network C and the generator G, and optimizing by using a gradient descent method.
Loss function:
L G ＝
min G (CE(y l，type， R f，l，type )+CE(y l，road， R f，l，road )+CE(1，D(R f，l，road )))；
gradient with respect to conditional network parameters:
gradient iteration:
θ C ＝θ C -μ C ·g C ；
gradient with respect to generator network parameters:
gradient iteration:
θ G ＝θ G -μ G ·g G ；
wherein CE is a cross entropy function, Y l，type Is true value of the original road type with label data, y l，road Is the true value of the division of the passable region of the original data with the tag, theta C Is a network parameter of network C, θ G Is a network parameter, mu, of the network G C Is the learning rate, mu, of the network C G The learning rate of the network G can effectively realize training and learning of the discriminator through the arrangement of the steps.
In another aspect, the present embodiment provides a system, including a vehicle terminal and an operation server, where the vehicle terminal and the operation server are connected in communication with each other to operate a program for carrying out the method, and the vehicle terminal includes a vehicle and a vehicle-mounted lidar.
In summary, the method for detecting the passable area facing the complex environment in this embodiment adopts the arrangement of steps 1 to 4, so that the point cloud deep learning detection subsystem and the conventional method detection subsystem can be trained effectively through steps 1 and 2, then through the arrangement of steps 3 and 4, the two subsystems can be combined, and then the passable area is obtained through the two systems combined with each other on the basis of judging the effective road type, thereby greatly increasing the accuracy of the passable area.
The above description is only a preferred embodiment of the present invention, and the protection scope of the present invention is not limited to the above examples, and all technical solutions belonging to the concept of the present invention belong to the protection scope of the present invention. It should be noted that modifications and adaptations to the present invention may occur to one skilled in the art without departing from the principles of the present invention and are intended to be within the scope of the present invention.
Claims (6)
1. A method for detecting a passable area facing a complex environment is characterized by comprising the following steps: the method comprises the following steps:
step 1, constructing a passable area point cloud deep learning detection subsystem, wherein the passable area point cloud deep learning detection subsystem comprises a road type identification network and a passable area detection network, the road type identification network is specifically a point cloud deep learning classification network, the deep learning detection passable area detection network is specifically a point cloud deep learning segmentation network, and an countermeasure learning network is generated through a generator, a generator condition constraint network and a discriminator condition, wherein the generator is the point cloud deep learning segmentation network, the generator condition constraint network is the point cloud deep learning classification network, the discriminator is another point cloud deep learning classification network, and the road type identification network and the passable area detection network are trained through the countermeasure learning network so as to achieve detection accuracy;
step 2, constructing a trafficable region traditional method detection subsystem which comprises a road type identification module and a trafficable region detection module, wherein the road type identification module consists of feature extraction and a traditional machine learning classifier, and the trafficable region detection module consists of feature extraction, roadside extraction and tracking and road surface extraction;
step 3, the point cloud deep learning detection subsystem and the traditional method detection subsystem are directly connected through a deep learning framework bottom layer interface to complete the combination of the two subsystems, so that a complete passable area detection system facing the complex environment is formed;
and 4, the vehicle-mounted laser radar new acquisition point cloud data is preferentially input into the point cloud deep learning detection subsystem, identification analysis is carried out on the new acquisition point cloud data, a passable area is obtained, and if an effective road type result cannot be obtained when the new acquisition point cloud data is input into the point cloud deep learning detection subsystem, the new acquisition point cloud is transmitted to the traditional method detection subsystem, and the passable area is obtained.
2. The complex environment-oriented passable region detection method of claim 1, wherein: the specific step of obtaining the passable area in step 4 is as follows, the vehicle-mounted laser radar newly-acquired point cloud data is preferentially input into the point cloud deep learning detection subsystem, if the road type identification network gives an effective road type, the road type result and the newly-acquired point cloud data are continuously input into the passable area detection network, if the effective passable area detection result is output, the detection is directly ended, if the effective passable area detection result is not output, the road type result and the newly-acquired point cloud data are transmitted to the passable area detection module of the traditional detection subsystem, feature extraction, roadside detection and road surface detection are sequentially carried out to obtain a passable area, if the effective road type result is not obtained after the newly-acquired point cloud is input into the road type identification network of the point cloud deep learning subsystem, the newly-acquired point cloud is transmitted to the traditional method detection subsystem, feature extraction and classifier classification are sequentially carried out on the road type identification module to obtain the road type identification result, and the road type result and the newly-acquired point cloud data are continuously input into the passable area detection module, and feature extraction, roadside detection and road detection are sequentially carried out to obtain the passable area.
3. The complex environment-oriented passable region detection method of claim 2, wherein: the specific steps for training the road type recognition network and the passable area detection network to achieve the detection precision in the step 1 are as follows:
step 1.1, utilizing a three-dimensional laser radar point cloud training set R with a passable area label l，road The sample trains a point cloud deep learning segmentation network G, so that the network G can detect the passable area of the point cloud data, and a three-dimensional laser radar point cloud training set R with road type labels is utilized type Training a point cloud deep learning classification network C by using a sample to enable the network C to detect the road type of the point cloud data;
step 1.2, generating a generator in an countermeasure network by taking the point cloud deep learning segmentation network G trained in the step 1.1 as a condition; generating a network for generating conditions in the countermeasure network by taking the point cloud deep learning classification network C trained in the step 1.1 as the conditions; taking the other point cloud deep learning classification network D as a discriminator for generating an countermeasure network;
step 1.3, firstly, training and learning are carried out on a discriminator network D, and the training and learning are not carried out on a generator network G and a condition network C thereof;
step 1.4: considering that the influence of the output results of the condition network C and the generator network G on the passable region segmentation effect is not mutually independent, synchronously training and learning the condition network C and the generator network G, and fixing the discriminant network D without training and learning;
step 1.5: repeating the steps 1.3 and 1.4 until the condition network C and the generator network G reach a certain precision, taking out the condition network C as a road type identification network of the passable area point cloud deep learning detection subsystem, and taking out the generator network G as a passable area detection network of the passable area point cloud deep learning detection subsystem.
4. A complex environment-oriented passable region detection method as claimed in claim 3 wherein: in the step 1.3, the specific steps of fixing the generator network G and the condition network C thereof without training learning are as follows:
step 1.3.1: the newly acquired untagged point cloud data R u，new Inputting a condition network C to obtain a road type, obtaining a passable area by a point cloud deep learning segmentation network according to the road type by a generator network G, adding a passable area label at a corresponding position of point cloud data, and marking as R f，new，road ；
Step 1.3.2: r output by the generator of the step 1.3.1 f，new，road And the training set R originally provided with labels l，road Inputting a discriminator D, and outputting a discrimination result by the discriminator D: d (R) f，new，road ) And D (R) l，road ) If the optimum direction of the discriminator is the correct discrimination data source, D (R l，road ) =1 and D (R f，new，road ) =0 is the arbiter optimization direction;
step 1.3.3: constructing a loss function of the training discriminator D, and optimizing by using a gradient descent method;
loss function:
L D ＝min D (CE(1，D(R l，road ))+CE(0，D(R f，new，road )))；
gradient:
gradient iteration:
θ D ＝θ D -μ D ·g D ；
where CE is a cross entropy function, θ D Is the network parameter of network D, μ D Is the network learning rate.
5. The complex environment-oriented passable region detection method of claim 3 or 4, wherein: in the step 1.4, the condition network C and the generator network G perform synchronous training learning, and the specific steps of the arbiter network D fixing the training learning not are as follows:
step 1.4.1: training set data R with original labels l，road Inputting the condition network C to obtain the road type R f，l，type The generator network G obtains a passable area from the point cloud deep learning segmentation network according to the road type, adds a passable area label at a corresponding position of the point cloud data, and marks the passable area as R f，l，road ；
Step 1.4.2: r output by the generator of the step 1.4.1 f，l，road Input into a discriminator to obtain a discrimination result D (R f，l，road ) If the condition network optimization direction is infinitely approximate to the road type of the original data set and the generator optimization direction is infinitely approximate to the passable region segmentation result of the original data set, D (R f，l，road ) =1 is the conditional network and generator optimization direction;
step 1.4.3: constructing a loss function of a training condition network C and a generator G, and optimizing by using a gradient descent method;
loss function:
L G ＝min G (CE(y l，type ，R f，l，type )+CE(y l，road ，R f，l，road )+CE(1，D(R f，l，road )))；
gradient with respect to conditional network parameters:
gradient iteration:
θ C ＝θ C -μ C ·g C ；
gradient with respect to generator network parameters:
gradient iteration:
θ G ＝θ G -μ G ·g G ；
wherein CE is a cross entropy function, Y l，type Is true value of the original road type with label data, y l，road Is the true value of the division of the passable region of the original data with the tag, theta C Is a network parameter of network C, θ G Is a network parameter, mu, of the network G C Is the learning rate, mu, of the network C G Learning rate of the network G.
6. A system for applying the method of any one of claims 1 to 5, characterized in that: the method comprises a vehicle terminal and an operation server, wherein the vehicle terminal and the operation server are in communication connection with each other so as to operate a program carrying the method, and the vehicle terminal comprises a vehicle and a vehicle-mounted laser radar.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011164865.3A CN112395956B (en) | 2020-10-27 | 2020-10-27 | Method and system for detecting passable area facing complex environment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011164865.3A CN112395956B (en) | 2020-10-27 | 2020-10-27 | Method and system for detecting passable area facing complex environment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112395956A CN112395956A (en) | 2021-02-23 |
| CN112395956B true CN112395956B (en) | 2023-06-02 |
Family
ID=74597212
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011164865.3A Active CN112395956B (en) | 2020-10-27 | 2020-10-27 | Method and system for detecting passable area facing complex environment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112395956B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115248447B (en) * | 2021-09-29 | 2023-06-02 | 上海仙途智能科技有限公司 | Laser point cloud-based path edge identification method and system |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109800773A (en) * | 2019-02-19 | 2019-05-24 | 北京大学 | Cross-country road extracting method based on three-dimensional laser radar |
| CN110569796A (en) * | 2019-09-09 | 2019-12-13 | 南京东控智能交通研究院有限公司 | Method for dynamically detecting lane line and fitting lane boundary |
| CN110969104A (en) * | 2019-11-22 | 2020-04-07 | 国网安徽省电力有限公司检修分公司 | Method, system and storage medium for detecting travelable area based on binarization network |
| CN111680611A (en) * | 2020-06-03 | 2020-09-18 | 江苏无线电厂有限公司 | Road trafficability detection method, system and equipment |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109271944B (en) * | 2018-09-27 | 2021-03-12 | 百度在线网络技术（北京）有限公司 | Obstacle detection method, obstacle detection device, electronic apparatus, vehicle, and storage medium |
-
2020
- 2020-10-27 CN CN202011164865.3A patent/CN112395956B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109800773A (en) * | 2019-02-19 | 2019-05-24 | 北京大学 | Cross-country road extracting method based on three-dimensional laser radar |
| CN110569796A (en) * | 2019-09-09 | 2019-12-13 | 南京东控智能交通研究院有限公司 | Method for dynamically detecting lane line and fitting lane boundary |
| CN110969104A (en) * | 2019-11-22 | 2020-04-07 | 国网安徽省电力有限公司检修分公司 | Method, system and storage medium for detecting travelable area based on binarization network |
| CN111680611A (en) * | 2020-06-03 | 2020-09-18 | 江苏无线电厂有限公司 | Road trafficability detection method, system and equipment |
Non-Patent Citations (1)
| Title |
|---|
| 基于视觉的车道线检测方法研究进展;吴一全等;《仪器仪表学报》;20191215(第12期);95-112 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112395956A (en) | 2021-02-23 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Zai et al. | 3-D road boundary extraction from mobile laser scanning data via supervoxels and graph cuts | |
| Dhiman et al. | Pothole detection using computer vision and learning | |
| CN107368787B (en) | Traffic sign identification method for deep intelligent driving application | |
| CN105160309B (en) | Three lanes detection method based on morphological image segmentation and region growing | |
| CN110785719A (en) | Method and system for instant object tagging via cross temporal verification in autonomous vehicles | |
| CN109596078A (en) | Multi-information fusion spectrum of road surface roughness real-time testing system and test method | |
| EP3321887A1 (en) | Urban road recognition method, apparatus, storage medium and device based on laser point cloud | |
| CN110869559A (en) | Method and system for integrated global and distributed learning in autonomous vehicles | |
| CN110753953A (en) | Method and system for object-centric stereo vision in autonomous vehicles via cross-modality verification | |
| CN107392103A (en) | The detection method and device of road surface lane line, electronic equipment | |
| CN106127113A (en) | A kind of road track line detecting method based on three-dimensional laser radar | |
| CN111179300A (en) | Method, apparatus, system, device and storage medium for obstacle detection | |
| CN110197173B (en) | Road edge detection method based on binocular vision | |
| CN111832410B (en) | Forward train detection method based on fusion of vision and laser radar | |
| CN114596555B (en) | Obstacle point cloud data screening method and device, electronic equipment and storage medium | |
| Zhang et al. | Vehicle detection method for intelligent vehicle at night time based on video and laser information | |
| CN114359861B (en) | Intelligent vehicle obstacle recognition deep learning method based on vision and laser radar | |
| CN112395956B (en) | Method and system for detecting passable area facing complex environment | |
| CN114155720B (en) | Vehicle detection and track prediction method for roadside laser radar | |
| CN209214563U (en) | Multi-information fusion spectrum of road surface roughness real-time testing system | |
| CN115439621A (en) | Three-dimensional map reconstruction and target detection method for coal mine underground inspection robot | |
| CN113538585A (en) | High-precision multi-target intelligent identification, positioning and tracking method and system based on unmanned aerial vehicle | |
| Dong et al. | Combination of modified U‐Net and domain adaptation for road detection | |
| Gong et al. | Complex lane detection based on dynamic constraint of the double threshold | |
| Jiang et al. | RailSeg: Learning Local-Global Feature Aggregation with Contextual Information for Railway Point Cloud Semantic Segmentation |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |