WO2019179094A1 - 无人驾驶车道保持方法、装置、计算机设备和存储介质 - Google Patents

无人驾驶车道保持方法、装置、计算机设备和存储介质 Download PDF

Info

Publication number
WO2019179094A1
WO2019179094A1 PCT/CN2018/111274 CN2018111274W WO2019179094A1 WO 2019179094 A1 WO2019179094 A1 WO 2019179094A1 CN 2018111274 W CN2018111274 W CN 2018111274W WO 2019179094 A1 WO2019179094 A1 WO 2019179094A1
Authority
WO
WIPO (PCT)
Prior art keywords
real vehicle
training
model
data
training data
Prior art date
Application number
PCT/CN2018/111274
Other languages
English (en)
French (fr)
Inventor
石含飞
何俏君
徐伟
查鸿山
裴锋
谷俊
Original Assignee
广州汽车集团股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 广州汽车集团股份有限公司 filed Critical 广州汽车集团股份有限公司
Priority to US16/339,008 priority Critical patent/US11505187B2/en
Publication of WO2019179094A1 publication Critical patent/WO2019179094A1/zh

Links

Images

Classifications

    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W30/00—Purposes of road vehicle drive control systems not related to the control of a particular sub-unit, e.g. of systems using conjoint control of vehicle sub-units
    • B60W30/10—Path keeping
    • B60W30/12—Lane keeping
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
    • G05B13/00—Adaptive control systems, i.e. systems automatically adjusting themselves to have a performance which is optimum according to some preassigned criterion
    • G05B13/02—Adaptive control systems, i.e. systems automatically adjusting themselves to have a performance which is optimum according to some preassigned criterion electric
    • G05B13/0265—Adaptive control systems, i.e. systems automatically adjusting themselves to have a performance which is optimum according to some preassigned criterion electric the criterion being a learning criterion
    • G05B13/027—Adaptive control systems, i.e. systems automatically adjusting themselves to have a performance which is optimum according to some preassigned criterion electric the criterion being a learning criterion using neural networks only
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
    • G05D1/02—Control of position or course in two dimensions
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/217—Validation; Performance evaluation; Active pattern learning techniques
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/06—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons
    • G06N3/063—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons using electronic means
    • G06N3/065—Analogue means
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/084—Backpropagation, e.g. using gradient descent
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/90—Determination of colour characteristics
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
    • G06V10/443—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components by matching or filtering
    • G06V10/449—Biologically inspired filters, e.g. difference of Gaussians [DoG] or Gabor filters
    • G06V10/451—Biologically inspired filters, e.g. difference of Gaussians [DoG] or Gabor filters with interaction between the filter responses, e.g. cortical complex cells
    • G06V10/454—Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
    • G06V20/588—Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W2420/00—Indexing codes relating to the type of sensors based on the principle of their operation
    • B60W2420/40—Photo, light or radio wave sensitive means, e.g. infrared sensors
    • B60W2420/403—Image sensing, e.g. optical camera
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W2540/00—Input parameters relating to occupants
    • B60W2540/18—Steering angle
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W2710/00—Output or target parameters relating to a particular sub-units
    • B60W2710/20—Steering systems
    • B60W2710/207—Steering angle of wheels
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/047—Probabilistic or stochastic networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20112—Image segmentation details
    • G06T2207/20132—Image cropping

Definitions

  • the present application relates to the field of driverless technology, and in particular, to an unmanned lane keeping method, apparatus, computer device and storage medium.
  • the traditional unmanned middle lane keeps building a lane model based on artificial knowledge.
  • the lane marking is extracted by collecting the road image, and then the lane offset is calculated according to the lane model, and the corner segmentation PID (Proportion Integral Derivative, The proportional integral derivative controller) calculates the steering wheel angle compensation value required to correct the lane departure distance, and then corrects the vehicle lane departure.
  • the traditional unmanned lane keeping method uses the artificial knowledge to establish the corresponding lane model, so the recognition ability of the road is not clear, the curvature of the curve is large, and the road section of the vehicle is insufficient.
  • An unmanned lane keeping method comprising the steps of: receiving a real vehicle image obtained by collecting a vehicle by a data collector; transmitting the real vehicle image to a real vehicle model for processing, and obtaining an image of the real vehicle Corresponding directional angles, wherein the real vehicle model is established by deep neural network learning for characterizing the correspondence between the real vehicle image and the directional angle; and controlling the vehicle to remain in the corresponding lane according to the directional angle.
  • the method before the step of receiving the real vehicle image obtained by the data collector on the vehicle, the method further includes: establishing a corresponding neural network model based on the convolutional neural network; receiving the training data, and according to the training data A real vehicle model is established with the neural network model, the training data including a real vehicle image and a direction angle.
  • the step of receiving training data and establishing a real vehicle model according to the training data and the neural network model comprises: receiving training data, and pre-processing the training data; The processed training data and the neural network model are model-trained to obtain training results; and a real vehicle model is established according to the training results.
  • the receiving the training data, pre-processing the training data comprising: receiving training data, performing random translation, rotation, flipping, and cropping on the real vehicle image in the training data to obtain pre-processing Subsequent real vehicle image; calculating a direction angle corresponding to the real vehicle image after the preprocessing, and obtaining training data after preprocessing.
  • the training data includes training set data
  • the step of training according to the training data after the pre-processing and the neural network model to obtain training results includes: establishing and pre-processing based on Tensorflow The training data corresponding to the network training model; according to the training set data and the neural network model, the network training model is iteratively trained through the training set data to obtain training results.
  • the training data further includes verification set data
  • the step of establishing a real vehicle model according to the training result comprises: establishing a preliminary model according to the training result; and verifying the preliminary model according to the verification set data , get the real car model.
  • the step of controlling the vehicle to remain in the corresponding lane according to the directional angle comprises: transmitting the directional angle to a steering control system, wherein the directional angle is used by the steering control system to control the vehicle Steering to keep the vehicle in the corresponding lane.
  • An unmanned lane keeping device comprising: a real vehicle image receiving module, configured to receive a real vehicle image obtained by the data collector to collect the vehicle; and a real vehicle inference module, configured to transmit the real vehicle image Processing to the real vehicle model to obtain a direction angle corresponding to the real vehicle image, wherein the real vehicle model is established by deep neural network learning, and is used to represent the correspondence between the real vehicle image and the direction rotation angle; the direction rotation angle control And a module for controlling the vehicle to keep driving in the corresponding lane according to the direction rotation angle.
  • a computer apparatus comprising a memory and a processor, the memory storing a computer program, the processor executing the computer program to implement the steps of any of the methods described above.
  • a computer readable storage medium having stored thereon a computer program, the computer program being executed by a processor to implement the steps of the method described in the above.
  • the above-mentioned unmanned lane keeping method, device, computer equipment and storage medium collect a large amount of real vehicle data as training data, and perform deep learning through a deep neural network to establish a corresponding real vehicle inference model, which can be based on the actual driving process.
  • the collected real vehicle image is obtained by the actual vehicle inference model to obtain the corresponding direction rotation angle, thereby controlling the vehicle to keep driving in the corresponding lane. It can complete the depiction of road information without manual knowledge, and through deep learning, it can also learn the characteristic information that the internal knowledge can not be obtained, which has deep internal understanding of the lane, which can realize the unclear route, the curvature of the curve and the vehicle.
  • Lane keeping in a congested road environment has the advantage of strong recognition ability.
  • FIG. 1 is a schematic flow chart of an unmanned lane keeping method in an embodiment
  • Figure 2 is a schematic diagram of direction angle control in an embodiment
  • FIG. 3 is a schematic flow chart of establishing a real vehicle model in an embodiment
  • FIG. 4 is a schematic structural diagram of a neural network in an embodiment
  • FIG. 5 is a schematic flow chart of establishing a real vehicle model in another embodiment
  • FIG. 6 is a schematic flow chart of preprocessing of training data in an embodiment
  • FIG. 7 is a schematic flow chart of model training in an embodiment
  • FIG. 9 is a schematic diagram of a network training structure in an embodiment
  • Figure 10 is a schematic structural view of an unmanned lane keeping device in an embodiment
  • FIG. 11 is a schematic structural view of establishing a real vehicle model in one embodiment
  • FIG. 12 is a schematic structural diagram of establishing a real vehicle model in another embodiment
  • FIG. 13 is a schematic diagram of a pre-processing structure of training data in an embodiment
  • FIG. 14 is a schematic diagram of a model training structure in an embodiment
  • 15 is a schematic diagram of a model verification structure in an embodiment
  • Figure 16 is a diagram showing the internal structure of a computer device in an embodiment.
  • an unmanned lane keeping method includes the steps of:
  • S300 Receive a real vehicle image obtained by the data collector to collect the vehicle.
  • the real-time road information of the vehicle during the running process is collected according to the data collector in real time.
  • the data collector may be a camera. During the running of the vehicle, the camera takes a photo at a certain frequency to obtain a corresponding real vehicle. image.
  • S400 Transfer the real vehicle image to a preset real vehicle model for processing, and obtain a direction rotation angle corresponding to the real vehicle image.
  • the real vehicle model is established through deep neural network learning, which is used to characterize the correspondence between the real vehicle image and the direction rotation angle. Specifically, after the data collector collects the real vehicle image, it transmits to the preset real vehicle model for real vehicle inference, and obtains a direction rotation angle corresponding to the collected real vehicle image.
  • the preset real vehicle model refers to a model that characterizes the relationship between the real vehicle image and the direction rotation angle established by deep neural network learning according to the actual vehicle image and the direction rotation angle in the actual driving process.
  • the real vehicle image is an RGB image
  • the RGB (Red Green Blue, RGB color mode) real vehicle image is split into three channels of R, G, and B, respectively. The corresponding header and header are added to the channel.
  • the verification is performed.
  • the real vehicle model is inferred, and the real vehicle image that fails the verification is discarded.
  • the RGB value of the real vehicle image is normalized from 0 to 255 to [-1, 1].
  • the RGB real vehicle image of each frame is encapsulated into a three-frame socket udp frame.
  • the transmission loss time of one frame of RGB real vehicle image is less than 200 us, which satisfies the requirement of real-time performance.
  • Socket udp is a general-purpose big datagram communication method. It has easy-to-obtain interface functions in C++ and Python. It can avoid complicated debugging caused by c++ and python, easy to find problems, and shorten development time.
  • step S500 the vehicle is controlled to travel in the corresponding lane according to the direction angle. Specifically, after inferring the direction angle corresponding to the acquired real vehicle image according to the real vehicle model, the control vehicle performs steering according to the obtained corner, and keeps driving in an appropriate lane.
  • step S500 includes transmitting a directional angle to the steering control system, the directional angle for the steering control system to control vehicle steering to maintain the vehicle in the corresponding lane.
  • the EPS Electronic Power Steering
  • the data acquisition end sends the direction rotation angle to the EPS (Electric Power Steering). , electronic power steering system), thereby controlling the vehicle to make steering, so that the vehicle keeps driving in the corresponding lane.
  • the socket (socket, used for data exchange between the two programs) is transmitted, and the data collection end passes through the CAN (Controller Area Network). , controller area network)
  • the bus sends the direction corner to the EPS.
  • step S100 and step S200 are further included before step S300.
  • Step S100 establishing a corresponding neural network model based on the convolutional neural network.
  • the corresponding direction angle is obtained according to the input real vehicle image. Therefore, the lane keeping can be regarded as an image processing problem, and the convolutional neural network has a strong advantage in image classification processing. Therefore, a convolutional neural network is used as the main component of the neural network. Specifically, referring to FIG.
  • the convolutional neural network is divided into 14 layers, one input layer, five convolution layers, and the convolutional layer is sparsely connected to implement image local feature extraction;
  • three max_pooling (maximum pooling) pooling layer the main role of the pooling layer is to downsample the feature map, integrate features;
  • 3 fully connected layers its role is to map the "distributed feature representation" to the sample mark space, which can be considered Weighted sum of previously extracted local features;
  • 1 spp_net space pyramid pooled network
  • the softmax classification probability function, which maps the output value to the classification probability
  • output layer of the class outputs a 1-dimensional vector of length n, and the index number of the element with the largest vector value is the steering wheel angle value predicted from the input image;
  • the softmax layer the output of each layer is subjected to an activation function to enhance
  • the acquisition of the training data is also performed before step S100.
  • the real vehicle image and the direction rotation angle in front of the lane keeping are collected in real time at a certain frequency, and the collected real vehicle image and the direction angle are saved. Further, the real vehicle image is captured at a frequency of 30 Hz and a pixel of 1280*1080, and the captured real vehicle image is saved in a video format, and the time stamp of the captured video is recorded and recorded in a txt file.
  • the direction rotation angle is collected at a frequency of 100 Hz, and the acquired direction rotation angle and the corresponding time stamp are saved in a binary bin file.
  • the acquisition frequency of the real vehicle image and the direction angle is not limited to the sampling frequency listed in the embodiment, and the actual vehicle image and the direction rotation angle can be sampled at other frequencies, as long as the sampling frequency of the direction rotation angle is larger than the actual vehicle image.
  • the sampling frequency is sufficient, and the acquisition pixels of the real vehicle image are not limited to 1280*1080, and the storage form of the document is not limited to the embodiment, as long as the information in the training data can be saved.
  • the establishment of the training database is also performed after the acquisition of the training data.
  • the collected data is divided into four categories: straight, curved, left-biased, and right-biased.
  • the straight is mainly used for normal driving, and the other three types are mainly used to correct the vehicle after it deviates from the lane. In the normal driving process, most of them are straight, so the straight data has a large proportion.
  • the down-channel data is downsampled by the down-sampling factor ⁇ (greater than 1), and the other data maintains the original sampling frequency.
  • the acquisition frequency of the direction angle is high, in order to make the data set contain more original information, we take the acquisition time of the direction corner as the reference, and the image that was collected before and the time is the image corresponding to the current steering wheel angle.
  • the real car image and the direction angle are synchronized.
  • the real vehicle image is captured with 1280*1080 pixels, the field of view obtained for the lane keeping is too broad, and the size of the picture input is larger when training, not only the network parameters will increase, but also the lane keeping. The extraneous factor will also increase, and in order to identify the unrelated factor, the amount of data will increase exponentially. Therefore, the road image H*W (H ⁇ 1280, W ⁇ 1080) is taken at the height of H pixels and the length of W pixels.
  • the specific size can be adjusted according to the actual situation. Since the HDF5 file is easier to apply in machine learning and control software, the HDF5 file is selected to store the video and direction corners, and the order of the images in the file is the same as the order of the video frames in the corresponding video. By establishing a corresponding training database, it is convenient to extract training data in the subsequent training process.
  • Step S200 receiving training data, and establishing a real vehicle model according to the training data and the neural network model.
  • the training data includes a real vehicle image and a direction rotation angle, and based on the neural network model, deep learning is performed according to the received training data to establish a real vehicle model.
  • step S200 includes step S210, step S220, and step S230.
  • Step S210 receiving training data, and pre-processing the training data. Specifically, the collected training data is preprocessed to expand the number of training data and increase the diversity of the samples.
  • step S210 includes step S211 and step S212.
  • Step S211 receiving training data, randomly shifting, rotating, flipping, and cropping the real vehicle image in the training data to obtain a real vehicle image after preprocessing.
  • each of the acquired real vehicle images is randomly translated, rotated, and flipped with a certain probability level, and then the H*W image is cropped to the IN_H*IN_W pixel, and the large-sized image is transformed and then cropped to a small size.
  • the image is mainly to prevent the cropped image from appearing in a small range of black frames.
  • cropping H*W images select the appropriate pixels for cropping according to the size of H*W. When cropping, minimize the proportion of other irrelevant information in the real vehicle image, and ensure the proportion of road information in the real vehicle image. .
  • Step S212 calculating a direction rotation angle corresponding to the real vehicle image after the preprocessing, and obtaining training data after the preprocessing.
  • the direction angle corresponding to the real vehicle image after the pre-processing is obtained by calculation. Obtained by the following transformation formula:
  • Steer_out sym_symbol*(steer_init+pix_shift* ⁇ -pix_rotate* ⁇ )
  • ⁇ is a transform coefficient of a corresponding angle of a random translation pixel
  • ⁇ is a transform coefficient of an image rotation corresponding to a steering wheel angle
  • Steer_out is the angle value corresponding to the image after the image is transformed.
  • Sym_symbol is the horizontally symmetric identifier of the image. It is an explicit function. When sym_symbol is -1, it means horizontal symmetry. When sym_symbol is 1, it means no horizontal symmetry.
  • the calculation formula is as follows:
  • f(-T, T) means that a random integer is generated in the [-T, T] closed interval, and T is an integer that is not zero.
  • pix_shift and pix_rotate are similar, and M and K both indicate no. Zero any integer.
  • the benefit of horizontal symmetry of the image is that it can balance the habitual tendency of the vehicle to directional the corners of the vehicle when it is not in the middle of the lane.
  • Steer_init is the original direction of the acquired angle
  • pix_shift is the number of pixels randomly translated in the horizontal direction. The calculation is as follows:
  • Pix_rotate is the rotation angle of the H*W image for rotation transformation. The calculation formula is as follows:
  • the direction angle corresponding to the real vehicle image after the pre-processing can be obtained, thereby obtaining the training data after the pre-processing.
  • Step S220 performing model training according to the training data after the pre-processing and the neural network model, and obtaining the training result. Specifically, the model training is performed based on the training data after a large number of pre-processing based on the neural network model, and the corresponding training result is obtained.
  • step S220 includes step S221 and step S222.
  • Step S221 establishing a network training model corresponding to the training data after the pre-processing based on Tensorflow.
  • Tensorflow is an intelligent learning system that transmits complex data structures to an artificial intelligence neural network for analysis and processing.
  • a network training model corresponding to the training data after pre-processing is established, which facilitates subsequent iterative training of the training data after pre-processing.
  • Step S222 according to the training set data and the neural network model, the network training model is iteratively trained through the training set data, and the training result is obtained.
  • the training set data is randomly scrambled before the training connection is performed, the correlation between the samples is broken, and the reliability of the training result is increased.
  • the training data is loaded in batches due to the large capacity of the obtained training data, and the training data of each batch is not loaded according to the configuration of the server for training.
  • the selection can be made according to the actual situation, and in order to facilitate the expansion, the storage training data and the iterative training can be divided into different servers, and the data transmission between the two servers through the socket. It can be understood that the training data can be loaded into the network training model at one time under the premise of the server configuration, and the storage and iterative training of the training data can also be placed on the same server.
  • Step S230 establishing a real vehicle model according to the training result. Specifically, based on the Tensorflow network training model, the corresponding training is performed according to the received training data, and the training result corresponding to the real vehicle image and the direction angle is obtained and saved, and the corresponding real vehicle model is established according to the training result of the large amount of training data. .
  • the training data further includes verification set data
  • step S230 includes steps S231 and S232.
  • a preliminary model is established according to the training result. Specifically, based on the Tensorflow network training model and the neural network model, the Tensorflow network training model is iteratively trained according to the training set data, and the corresponding relationship between the real vehicle image and the direction rotation angle is obtained, and a preliminary model is established according to the obtained correspondence relationship.
  • model training adopts mini-batch SGD (Stochastic gradient descent) as the optimizer, the initial learning rate is ⁇ , the learning rate is exponentially decayed by the coefficient ⁇ , and the learning rate is attenuated after the training times reach the set value.
  • the set value is determined based on the experience accumulated by the plurality of trainings.
  • Step S232 verifying the preliminary model according to the verification set data, and obtaining a real vehicle model. Specifically, after performing iterative training according to the training set data, a preliminary model for the correspondence between the real vehicle graphics and the direction rotation angle is established according to the training result, and then the obtained preliminary model is evaluated based on the verification set data, and is verified according to the preliminary model. The trend of the loss value or accuracy on the set determines whether to terminate the training. Further, in order to prevent accidental interruption of the training program, the training result of the model is saved once each pair of training data is trained.
  • the training data further includes test set data, and after the initial training is completed according to the training set data, and the verification set is verified by the preliminary model to obtain the real vehicle model, the real vehicle model obtained by the test set data pair is obtained. Model prediction is performed to measure the performance and classification capabilities of the established real vehicle model, and the results are obtained and output.
  • the obtained training data is divided into training set data, verification set data and test set data, which effectively prevents over-fitting of the model and further improves the reliability of the established real vehicle model.
  • the above-mentioned unmanned lane keeping method collects a large amount of real vehicle data as training data, and performs deep learning through a deep neural network to establish a corresponding real vehicle inference model, which can be based on the collected real vehicle image during actual driving.
  • the vehicle inference model obtains a corresponding direction angle, thereby controlling the vehicle to remain in the corresponding lane. It can complete the depiction of road information without manual knowledge, and through deep learning, it can also learn the characteristic information that the internal knowledge can not be obtained, which has deep internal understanding of the lane, which can realize the unclear route, the curvature of the curve and the vehicle.
  • Lane keeping in a congested road environment has the advantage of strong recognition ability.
  • an unmanned lane keeping device includes a real vehicle image receiving module 300, a real vehicle inference module 400, and a direction angle control module 500.
  • the real vehicle image receiving module 300 is configured to receive a real vehicle image obtained by the data collector to collect the vehicle. Specifically, the real-time road information of the vehicle during the running process is collected according to the data collector in real time. Further, the data collector may be a camera. During the running of the vehicle, the camera takes a photo at a certain frequency to obtain a corresponding real vehicle. image.
  • the real vehicle inference module 400 is configured to transmit the real vehicle image to a preset real vehicle model for processing, and obtain a direction rotation angle corresponding to the real vehicle image.
  • the real vehicle model is established through deep neural network learning, which is used to characterize the correspondence between the real vehicle image and the direction rotation angle. Specifically, after the data collector collects the real vehicle image, it transmits to the preset real vehicle model for real vehicle inference, and obtains a direction rotation angle corresponding to the collected real vehicle image.
  • the preset real vehicle model refers to a model that characterizes the relationship between the real vehicle image and the direction rotation angle established by deep neural network learning according to the actual vehicle image and the direction rotation angle in the actual driving process.
  • the real vehicle image is an RGB image
  • the RGB real vehicle image is split into three channels of R, G, and B, and corresponding headers are added to each channel. And the beginning and end of the newspaper.
  • the verification is performed.
  • the real vehicle model is inferred, and the real vehicle image that fails the verification is discarded.
  • the RGB value of the real vehicle image is normalized from 0 to 255 to [-1, 1].
  • the RGB real vehicle image of each frame is encapsulated into a three-frame socket udp frame.
  • the transmission loss time of one frame of RGB real vehicle image is less than 200 us, which satisfies the requirement of real-time performance.
  • Socket udp is a general-purpose big datagram communication method. It has easy-to-obtain interface functions in C++ and Python. It can avoid complicated debugging caused by c++ and python, easy to find problems, and shorten development time.
  • the direction angle control module 500 is configured to control the vehicle to keep driving in the corresponding lane according to the direction angle. Specifically, after inferring the direction angle corresponding to the acquired real vehicle image according to the real vehicle model, the control vehicle performs steering according to the obtained corner, and keeps driving in an appropriate lane.
  • the directional angle control module 500 transmits a directional angle to the steering control system, and the directional angle is used by the steering control system to control vehicle steering to keep the vehicle in the corresponding lane. Specifically, after the actual vehicle model is inferred to obtain the direction rotation angle corresponding to the acquired real vehicle image, the obtained direction rotation angle is sent to the data acquisition end, and then the data acquisition end sends the direction rotation angle to the EPS, thereby controlling the vehicle. Steering is carried out to keep the vehicle in the corresponding lane.
  • the socket (socket, used for data exchange between the two programs) is transmitted, and the data acquisition end turns the direction through the CAN bus. Send to EPS.
  • the closed-loop control of the vehicle steering is completed in the above manner, and has the advantages of high control smoothness and high generalization capability.
  • the driverless lane keeping device further includes a neural network model building module 100 and a real vehicle model building module 200.
  • the neural network model building module 100 is configured to establish a corresponding neural network model based on the convolutional neural network.
  • the corresponding direction angle is obtained according to the input real vehicle image. Therefore, the lane keeping can be regarded as an image processing problem, and the convolutional neural network has a strong advantage in image classification processing. Therefore, a convolutional neural network is used as the main component of the neural network. Specifically, referring to FIG.
  • the convolutional neural network is divided into 14 layers, one input layer, five convolution layers, and the convolutional layer is sparsely connected to implement image local feature extraction; three max_pooling The pooling layer, the main role of the pooling layer is to downsample the feature map and integrate features; three fully connected layers, the role of which is to map the "distributed feature representation" to the sample mark space, which can be considered as the local part extracted previously.
  • Weighted sum of features 1 spp_net layer, adding a spp_net layer between the last convolutional layer and the fully connected layer, adapting the model to multiple size inputs; 1 n class of softmax output layer, output length n of 1
  • the dimension vector, the index number of the element with the largest vector value is the steering wheel angle value predicted from the input image; except for the softmax layer, the output of each layer is subjected to an activation function to enhance the nonlinear expression of the network. Because there is a certain similarity between the samples, each full connection layer is set with dropout during the training process to suppress over-fitting. Droptout is used to invalidate neurons with a certain probability during a training process.
  • the neural network model building module 100 also performs training data acquisition prior to establishing a corresponding neural network model based on the convolutional neural network.
  • the real vehicle image and the direction rotation angle in front of the lane keeping are collected in real time at a certain frequency, and the collected real vehicle image and the direction angle are saved. Further, the real vehicle image is captured at a frequency of 30 Hz and a pixel of 1280*1080, and the captured real vehicle image is saved in a video format, and the time stamp of the captured video is recorded and recorded in a txt file.
  • the direction rotation angle is collected at a frequency of 100 Hz, and the acquired direction rotation angle and the corresponding time stamp are saved in a binary bin file.
  • the acquisition frequency of the real vehicle image and the direction angle is not limited to the sampling frequency listed in the embodiment, and the actual vehicle image and the direction rotation angle can be sampled at other frequencies, as long as the sampling frequency of the direction rotation angle is larger than the actual vehicle image.
  • the sampling frequency is sufficient, and the acquisition pixels of the real vehicle image are not limited to 1280*1080, and the storage form of the document is not limited to the embodiment, as long as the information in the training data can be saved.
  • the establishment of the training database is also performed after the acquisition of the training data.
  • the collected data is divided into four categories: straight, curved, left-biased, and right-biased.
  • the straight is mainly used for normal driving, and the other three types are mainly used to correct the vehicle after it deviates from the lane. In the normal driving process, most of them are straight, so the straight data has a large proportion.
  • the down-channel data is downsampled by the down-sampling factor ⁇ (greater than 1), and the other data maintains the original sampling frequency.
  • the acquisition frequency of the direction angle is high, in order to make the data set contain more original information, we take the acquisition time of the direction corner as the reference, and the image that was collected before and the time is the image corresponding to the current steering wheel angle.
  • the real car image and the direction angle are synchronized.
  • the real vehicle image is captured with 1280*1080 pixels, the field of view obtained for the lane keeping is too broad, and the size of the picture input is larger when training, not only the network parameters will increase, but also the lane keeping. The extraneous factor will also increase, and in order to identify the unrelated factor, the amount of data will increase exponentially. Therefore, the road image H*W (H ⁇ 1280, W ⁇ 1080) is taken at the height of H pixels and the length of W pixels.
  • the specific size can be adjusted according to the actual situation. Since the HDF5 file is easier to apply in machine learning and control software, the HDF5 file is selected to store the video and direction corners, and the order of the images in the file is the same as the order of the video frames in the corresponding video. By establishing a corresponding training database, it is convenient to extract training data in the subsequent training process.
  • the real vehicle model establishing module 200 is configured to receive training data, and establish a real vehicle model according to the training data and the neural network model.
  • the training data includes a real vehicle image and a direction rotation angle, and based on the neural network model, deep learning is performed according to the received training data to establish a real vehicle model.
  • the real vehicle model building module 200 includes a preprocessing module 210, a training module 220, and a model building module 230.
  • the pre-processing module 210 is configured to receive training data and perform pre-processing on the training data. Specifically, the collected training data is preprocessed to expand the number of training data and increase the diversity of the samples.
  • the pre-processing module 210 includes a real vehicle image processing unit 211 and a direction angle calculation unit 212.
  • the real vehicle image processing unit 211 is configured to receive training data, perform random translation, rotation, flipping, and cropping on the real vehicle image in the training data to obtain a real vehicle image after preprocessing. Specifically, each of the acquired real vehicle images is randomly translated, rotated, and flipped with a certain probability level, and then the H*W image is cropped to the IN_H*IN_W pixel, and the large-sized image is transformed and then cropped to a small size.
  • the image is mainly to prevent the cropped image from appearing in a small range of black frames.
  • cropping H*W images select the appropriate pixels for cropping according to the size of H*W. When cropping, minimize the proportion of other irrelevant information in the real vehicle image, and ensure the proportion of road information in the real vehicle image. .
  • the direction angle calculation unit 212 is configured to calculate a direction angle corresponding to the real vehicle image after the pre-processing, and obtain training data after the pre-processing. Specifically, the direction angle corresponding to the real vehicle image after the pre-processing is obtained by calculation. Obtained by the following transformation formula:
  • Steer_out sym_symbol*(steer_init+pix_shift* ⁇ -pix_rotate* ⁇ )
  • ⁇ is a transform coefficient of a corresponding angle of a random translation pixel
  • ⁇ is a transform coefficient of an image rotation corresponding to a steering wheel angle
  • Steer_out is the angle value corresponding to the image after the image is transformed.
  • Sym_symbol is the horizontally symmetric identifier of the image. It is an explicit function. When sym_symbol is -1, it means horizontal symmetry. When sym_symbol is 1, it means no horizontal symmetry.
  • the calculation formula is as follows:
  • f(-T, T) means that a random integer is generated in the [-T, T] closed interval, and T is an integer that is not zero.
  • pix_shift and pix_rotate are similar, and M and K both indicate no. Zero any integer.
  • the benefit of horizontal symmetry of the image is that it can balance the habitual tendency of the vehicle to directional the corners of the vehicle when it is not in the middle of the lane.
  • Steer_init is the original direction of the acquired angle
  • pix_shift is the number of pixels randomly translated in the horizontal direction. The calculation is as follows:
  • Pix_rotate is the rotation angle of the H*W image for rotation transformation. The calculation formula is as follows:
  • Steer_out is the angle value corresponding to the image after the image is transformed.
  • the direction angle corresponding to the real vehicle image after the pre-processing can be obtained, thereby obtaining the training data after the pre-processing.
  • the training module 220 is configured to perform model training according to the training data after the pre-processing and the neural network model to obtain training results. Specifically, the model training is performed based on the training data after a large number of pre-processing based on the neural network model, and the corresponding training result is obtained.
  • the training module 220 includes a network training model establishing unit 221 and an iterative training unit 222 .
  • the network training model establishing unit 221 is configured to establish a network training model based on Tensorflow.
  • Tensorflow is an intelligent learning system that transmits complex data structures to an artificial intelligence neural network for analysis and processing.
  • a network training model corresponding to the training data after pre-processing is established, which facilitates subsequent iterative training of the training data after pre-processing.
  • the iterative training unit 222 is configured to iteratively train the network training model line through the training set data according to the training set data and the neural network model, and obtain the training result.
  • the training set data is randomly scrambled before the training connection is performed, the correlation between the samples is broken, and the reliability of the training result is increased.
  • the training data is loaded in batches due to the large capacity of the obtained training data, and the training data of each batch is not loaded according to the configuration of the server for training.
  • the selection can be made according to the actual situation, and in order to facilitate the expansion, the storage training data and the iterative training can be divided into different servers, and the data transmission between the two servers through the socket. It can be understood that the training data can be loaded into the network training model at one time under the premise of the server configuration, and the storage and iterative training of the training data can also be placed on the same server.
  • the model building module 230 is configured to build a real vehicle model according to the training result. Specifically, based on the Tensorflow network training model, the corresponding training is performed according to the received training data, and the training result corresponding to the real vehicle image and the direction angle is obtained and saved, and the corresponding real vehicle model is established according to the training result of the large amount of training data. .
  • the training data further includes verification set data
  • the model building module 230 includes a preliminary model establishing unit 231 and a preliminary model verifying unit 232.
  • the preliminary model establishing unit 231 is configured to establish a preliminary model according to the training result. Specifically, based on the Tensorflow network training model and the neural network model, the Tensorflow network training model is iteratively trained according to the training set data, and the corresponding relationship between the real vehicle image and the direction rotation angle is obtained, and a preliminary model is established according to the obtained correspondence relationship.
  • the model training adopts mini-batch SGD as the optimizer, the initial learning rate is ⁇ , the learning rate is exponentially attenuated by the coefficient ⁇ , and the learning rate is attenuated after the training times reach the set value. Specifically, the set value is based on The accumulated training experience is determined.
  • the preliminary model verification unit 232 is configured to verify the preliminary model according to the verification set data to obtain a real vehicle model. Specifically, after performing iterative training according to the training set data, a preliminary model for the correspondence between the real vehicle graphics and the direction rotation angle is established according to the training result, and then the obtained preliminary model is evaluated based on the verification set data, and is verified according to the preliminary model. The trend of the loss value or accuracy on the set determines whether to terminate the training. Further, in order to prevent accidental interruption of the training program, the training result of the model is saved once each pair of training data is trained.
  • the training data further includes test set data, and after the initial training is completed according to the training set data, and the verification set is verified by the preliminary model to obtain the real vehicle model, the real vehicle model obtained by the test set data pair is obtained. Model prediction is performed to measure the performance and classification capabilities of the established real vehicle model, and the results are obtained and output.
  • the obtained training data is divided into training set data, verification set data and test set data, which effectively prevents over-fitting of the model and further improves the reliability of the established real vehicle model.
  • the above-mentioned unmanned lane keeping device collects a large amount of real vehicle data as training data, and performs deep learning through a deep neural network to establish a corresponding real vehicle inference model, which can be based on the collected real vehicle image during actual driving.
  • the vehicle inference model obtains a corresponding direction angle, thereby controlling the vehicle to remain in the corresponding lane. It can complete the depiction of road information without manual knowledge, and through deep learning, it can also learn the characteristic information that the internal knowledge can not be obtained, which has deep internal understanding of the lane, which can realize the unclear route, the curvature of the curve and the vehicle.
  • Lane keeping in a congested road environment has the advantage of strong recognition ability.
  • the various modules in the above-described driverless lane keeping device may be implemented in whole or in part by software, hardware, and combinations thereof.
  • Each of the above modules may be embedded in or independent of the processor in the computer device, or may be stored in a memory in the computer device in a software form, so that the processor invokes the operations corresponding to the above modules.
  • a computer device which may be a server, and its internal structure diagram may be as shown in FIG.
  • the computer device includes a processor, memory, network interface, and database connected by a system bus.
  • the processor of the computer device is used to provide computing and control capabilities.
  • the memory of the computer device includes a non-volatile storage medium, an internal memory.
  • the non-volatile storage medium stores an operating system, a computer program, and a database.
  • the internal memory provides an environment for operation of an operating system and computer programs in a non-volatile storage medium.
  • the database of the computer device is used to store data in the driverless lane keeping method.
  • the network interface of the computer device is used to communicate with an external terminal via a network connection.
  • the computer program is executed by the processor to implement an unmanned lane keeping method.
  • FIG. 16 is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation of the computer device to which the solution of the present application is applied.
  • the specific computer device may It includes more or fewer components than those shown in the figures, or some components are combined, or have different component arrangements.
  • a computer apparatus comprising a memory and a processor having a computer program stored therein, the processor implementing the computer program to:
  • the real vehicle image is transmitted to a preset real vehicle model for processing, and a direction angle corresponding to the real vehicle image is obtained, wherein the real vehicle model is established by deep neural network learning, and is used to represent the correspondence between the real vehicle image and the direction rotation angle. ;
  • the vehicle is controlled to travel in the corresponding lane according to the direction angle.
  • the processor further implements the following steps when executing the computer program:
  • the training data is received, and a real vehicle model is established based on the training data and the neural network model, and the training data includes a real vehicle image and a direction angle.
  • the processor further implements the following steps when executing the computer program:
  • the model training is performed according to the training data after the pre-processing and the neural network model, and the training result is obtained;
  • the processor further implements the following steps when executing the computer program:
  • the direction angle corresponding to the real vehicle image after the preprocessing is calculated, and the training data after the preprocessing is obtained.
  • the processor further implements the following steps when executing the computer program:
  • the network training model is iteratively trained through the training set data, and the training result is obtained.
  • the processor further implements the following steps when executing the computer program:
  • the preliminary model is verified based on the verification set data to obtain a real vehicle model.
  • the processor further implements the following steps when executing the computer program:
  • the direction angle is sent to the steering control system, and the steering angle is used by the steering control system to control the steering of the vehicle to keep the vehicle in the corresponding lane.
  • a computer readable storage medium having stored thereon a computer program that, when executed by a processor, implements the following steps:
  • the real vehicle image is transmitted to a preset real vehicle model for processing, and a direction angle corresponding to the real vehicle image is obtained, wherein the real vehicle model is established by deep neural network learning, and is used to represent the correspondence between the real vehicle image and the direction rotation angle. ;
  • the vehicle is controlled to travel in the corresponding lane according to the direction angle.
  • the computer program is executed by the processor to also implement the following steps:
  • the training data is received, and a real vehicle model is established based on the training data and the neural network model, and the training data includes a real vehicle image and a direction angle.
  • the computer program is executed by the processor to also implement the following steps:
  • the model training is performed according to the training data after the pre-processing and the neural network model, and the training result is obtained;
  • the computer program is executed by the processor to also implement the following steps:
  • the direction angle corresponding to the real vehicle image after the preprocessing is calculated, and the training data after the preprocessing is obtained.
  • the computer program is executed by the processor to also implement the following steps:
  • the network training model is iteratively trained through the training set data, and the training result is obtained.
  • the computer program is executed by the processor to also implement the following steps:
  • the preliminary model is verified based on the verification set data to obtain a real vehicle model.
  • the computer program is executed by the processor to also implement the following steps:
  • the direction angle is sent to the steering control system, and the steering angle is used by the steering control system to control the steering of the vehicle to keep the vehicle in the corresponding lane.
  • the above computer equipment and storage medium collect a large amount of real vehicle data as training data, and perform deep learning through a deep neural network to establish a corresponding real vehicle inference model, and in actual driving, according to the collected real vehicle image, the real vehicle
  • the model is inferred to obtain a corresponding direction angle, thereby controlling the vehicle to remain in the corresponding lane. It can complete the depiction of road information without manual knowledge, and through deep learning, it can also learn the characteristic information that the internal knowledge can not be obtained, which has deep internal understanding of the lane, which can realize the unclear route, the curvature of the curve and the vehicle. Lane keeping in a congested road environment has the advantage of strong recognition ability.
  • Non-volatile memory can include read only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory.
  • Volatile memory can include random access memory (RAM) or external cache memory.
  • RAM is available in a variety of formats, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronization chain.
  • SRAM static RAM
  • DRAM dynamic RAM
  • SDRAM synchronous DRAM
  • DDRSDRAM double data rate SDRAM
  • ESDRAM enhanced SDRAM
  • Synchlink DRAM SLDRAM
  • Memory Bus Radbus
  • RDRAM Direct RAM
  • DRAM Direct Memory Bus Dynamic RAM
  • RDRAM Memory Bus Dynamic RAM

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Evolutionary Computation (AREA)
  • Artificial Intelligence (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Software Systems (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Computing Systems (AREA)
  • Data Mining & Analysis (AREA)
  • Multimedia (AREA)
  • Biomedical Technology (AREA)
  • General Engineering & Computer Science (AREA)
  • Molecular Biology (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Mathematical Physics (AREA)
  • Medical Informatics (AREA)
  • Automation & Control Theory (AREA)
  • Databases & Information Systems (AREA)
  • Transportation (AREA)
  • Mechanical Engineering (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Biodiversity & Conservation Biology (AREA)
  • Evolutionary Biology (AREA)
  • Neurology (AREA)
  • Aviation & Aerospace Engineering (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Remote Sensing (AREA)
  • Image Analysis (AREA)
  • Image Processing (AREA)

Abstract

本申请涉及一种无人驾驶车道保持方法、装置、计算机设备和存储介质，包括步骤：接收数据采集器对车辆进行采集得到的实车图像；将实车图像传输到预设的实车模型进行处理，得到与实车图像相对应的方向转角，其中，实车模型通过深度神经网络学习建立，用于表征实车图像与方向转角的对应关系；根据方向转角控制车辆保持在对应的车道行驶。上述无人驾驶车道保持方法、装置、计算机设备和存储介质，通过采集大量的实车数据作为训练数据，经过深度神经网络进行深度学习建立对应的实车推断模型，不需要人工知识即可完成对道路信息的刻画，并且通过深度学习还能学习到人工知识无法得到的对车道保持有深刻内在理解的特征信息，具有识别能力强的优点。

Description

无人驾驶车道保持方法、装置、计算机设备和存储介质 技术领域
本申请涉及无人驾驶技术领域，特别是涉及一种无人驾驶车道保持方法、装置、计算机设备和存储介质。
背景技术
随着汽车工业的飞速发展和人们生活水平的提高，汽车作为主要的出行工具走进了千家万户。人们在驾驶过程中，容易受到外界因素的影响而无法很好的使汽车保持车道行驶，容易发生交通事故。研究表明，由于车道偏离导致的交通事故占据交通事故的20％。为了避免这类交通事故的发生，无人驾驶技术得到相应的发展。
传统的无人驾驶中车道保持根据人工知识建立车道模型，在实际行驶过程中，通过采集道路图像提取车道标记，之后根据车道模型计算出车道偏移量，利用转角分段PID(Proportion Integral Derivative，比例积分微分控制器)控制器计算修正车道偏离距离所需要的方向盘转角补偿值，进而对车辆车道偏离进行修正。然而，传统的无人驾驶中车道保持方法由于采用人工知识建立对应的车道模型，所以在路线不清晰、弯道曲率较大和车辆拥堵路段的识别能力不足。
发明内容
基于此，有必要针对上述技术问题，提供一种能够提高在路线不清晰、弯道曲率较大和车辆拥堵路段等状况下的识别能力的无人驾驶车道保持方法、装置、计算机设备和存储介质。
一种无人驾驶车道保持方法，所述方法包括步骤：接收数据采集器对车辆进行采集得到的实车图像；将所述实车图像传输到实车模型进行处理，得到与所述实车图像相对应的方向转角，其中，所述实车模型通过深度神经网络学习建立，用于表征实车图像与方向转角的对应关系；根据所述方向转角控制车辆保持在对应的车道行驶。
在一个实施例中，所述接收数据采集器对车辆进行采集得到的实车图像的步骤之前，还包括：基于卷积神经网络建立相应的神经网络模型；接收训练数据，并根据所述训练数据和所述神经网络模型建立实车模型，所述训练数据包 括实车图像和方向转角。
在一个实施例中，所述接收训练数据，并根据所述训练数据和所述神经网络模型建立实车模型的步骤，包括：接收训练数据，对所述训练数据进行预处理；根据所述预处理之后的训练数据和所述神经网络模型进行模型训练，得到训练结果；根据所述训练结果建立实车模型。
在一个实施例中，所述接收训练数据，对所述训练数据进行预处理，包括：接收训练数据，对所述训练数据中的实车图像进行随机平移、旋转、翻转和裁剪，得到预处理之后的实车图像；计算所述预处理之后的实车图像对应的方向转角，得到预处理之后的训练数据。
在一个实施例中，所述训练数据包括训练集数据，所述根据所述预处理之后的训练数据和所述神经网络模型进行模型训练，得到训练结果的步骤，包括：基于Tensorflow建立与预处理之后的训练数据对应的网络训练模型；根据所述训练集数据和所述神经网络模型，经所述训练集数据对所述网络训练模型进行迭代训练，得到训练结果。
在一个实施例中，所述训练数据还包括验证集数据，所述根据所述训练结果建立实车模型的步骤，包括：根据训练结果建立初步模型；根据验证集数据对所述初步模型进行验证，得到实车模型。
在一个实施例中，所述根据所述方向转角控制车辆保持在对应的车道行驶的步骤，包括：将所述方向转角发送至转向控制***，所述方向转角用于所述转向控制***控制车辆转向，使车辆保持在对应的车道行驶。
一种无人驾驶车道保持装置，所述装置包括：实车图像接收模块，用于接收数据采集器对车辆进行采集得到的实车图像；实车推断模块，用于将所述实车图像传输到实车模型进行处理，得到与所述实车图像相对应的方向转角，其中，所述实车模型通过深度神经网络学习建立，用于表征实车图像与方向转角的对应关系；方向转角控制模块，用于根据所述方向转角控制车辆保持在对应的车道行驶。
一种计算机设备，包括存储器和处理器，所述存储器存储有计算机程序，所述处理器执行所述计算机程序时实现上述任一项所述方法的步骤。
一种计算机可读存储介质，其上存储有计算机程序，所述计算机程序被处理器执行时实现上述一项所述的方法的步骤。
上述无人驾驶车道保持方法、装置、计算机设备和存储介质，通过采集大 量的实车数据作为训练数据，经过深度神经网络进行深度学习建立对应的实车推断模型，在实际行驶过程中，能够根据采集的实车图像，经实车推断模型得到对应的方向转角，从而控制车辆保持在对应的车道行驶。不需要人工知识即可完成对道路信息的刻画，并且通过深度学习还能学习到人工知识无法得到的对车道保持有深刻内在理解的特征信息，能够实现在路线不清晰、弯道曲率较大和车辆拥堵路段环境下的车道保持，具有识别能力强的优点。
附图说明
图1为一个实施例中无人驾驶车道保持方法的流程示意图；
图2为一个实施例中方向转角控制示意图；
图3为一个实施例中建立实车模型的流程示意图；
图4为一个实施例中神经网络结构示意图；
图5为另一个实施例中建立实车模型的流程示意图；
图6为一个实施例中训练数据预处理的流程示意图；
图7为一个实施例中模型训练的流程示意图；
图8为一个实施例中模型验证的流程示意图；
图9为一个实施例中网络训练结构示意图；
图10为一个实施例中无人驾驶车道保持装置结构示意图；
图11为一个实施例中建立实车模型的结构示意图；
图12为另一个实施例中建立实车模型结构示意图；
图13为一个实施例中训练数据预处理结构示意图；
图14为一个实施例中模型训练结构示意图；
图15为一个实施例中模型验证结构示意图；
图16为一个实施例中计算机设备的内部结构图。
具体实施方式
为了使本申请的目的、技术方案及优点更加清楚明白，以下结合附图及实施例，对本申请进行进一步详细说明。应当理解，此处描述的具体实施例仅用以解释本申请，并不用于限定本申请。
在一个实施例中，请参阅图1，一种无人驾驶车道保持方法，包括步骤：
S300，接收数据采集器对车辆进行采集得到的实车图像。具体地，根据数 据采集器实时的采集车辆在行驶过程中的实时道路信息，进一步地，数据采集器可以是摄像头，在车辆的行驶过程中，摄像头以一定的频率进行拍照，得到对应的实车图像。
S400，将实车图像传输到预设的实车模型进行处理，得到与实车图像相对应的方向转角。
其中，实车模型通过深度神经网络学习建立，用于表征实车图像与方向转角的对应关系。具体地，数据采集器采集到实车图像之后，传输到预设的实车模型进行实车推断，得到与所采集的实车图像相对应的方向转角。预设的实车模型是指根据采集的实际驾驶过程中的实车图像与方向转角，进行深度神经网络学习所建立的表征实车图像与方向转角关系的模型。
进一步地，在一个实施例中，实车图像为RGB图像，在进行传输时，将RGB(Red Green Blue，RGB色彩模式)实车图像拆分为R、G、B三个通道，在每一通道上都加上相应的报头文和报头尾。在接收到RGB实车图像时，进行校验，当同一幅图像的R、G、B三个通道均被完整接收时，完成校验，否则校验失败。接收校验成功的RGB实车图像进行归一化后，进行实车模型推断，校验失败的实车图像则将丢弃。在对实车图像进行归一化处理时，将实车图像的RGB值从0～255归一化到[-1,1]。将每一帧的RGB实车图像封装为三帧的socket udp帧，以采样频率为30Hz为例，完成一帧RGB实车图像的传送损耗时间小于200us，满足实时性的要求。socket udp是通用的大数据报通讯方式，在c++和python上都有易于获取的接口函数，可避免c++和python混编带来的复杂调试，易于问题查找，进而缩短开发时间。
步骤S500，根据方向转角控制车辆保持在对应的车道行驶。具体地，根据实车模型推断出所采集的实车图像所对应的方向转角之后，控制车辆根据所得到的转角进行转向，保持在合适的车道行驶。
进一步地，请参阅图2，在一个实施例中，步骤S500包括将方向转角发送至转向控制***，方向转角用于转向控制***控制车辆转向，使车辆保持在对应的车道行驶。具体地，经过实车模型推断得到与所采集的实车图像相对应的方向转角之后，将所得到的方向转角发送到数据采集端，然后数据采集端再将方向转角发送到EPS(Electric Power Steering，电子助力转向***)，从而控制车辆进行转向，使车辆保持在对应的车道行驶。进一步地，实车模型将推断出来的对应方向转角发送到数据采集端时，通过socket(套接字，用于两个程序之 间的数据交换)进行传输，数据采集端通过CAN(Controller Area Network，控制器局域网络)总线将方向转角发送到EPS。通过上述方式完成对车辆转向的闭环控制，具有控制平滑、泛化能力高的优点。
在一个实施例中，请参阅图3，步骤S300之前还包括步骤S100和步骤S200。
步骤S100，基于卷积神经网络建立相应的神经网络模型。在进行车道保持的过程中，根据输入的实车图像来得到对应的方向转角，因此，可以把车道保持看作一个图像处理的问题，由于卷积神经网络在图像分类处理方面具有很强的优势，所以采用卷积神经网络作为神经网络的主要成分。具体地，请参阅图4，在一个实施例中，卷积神经网络共分为14层，一个输入层；5个卷积层，卷积层是稀疏连接，实现图像局部特征提取；3个max_pooling(最大池化)池化层，池化层的主要作用是对特征图下采样，整合特征；3个全连接层，其作用是将“分布式特征表示”映射到样本标记空间，可认为是对之前提取的局部特征的加权和；1个spp_net(空间金字塔池化网络)层，在最后一个卷积层与全连接层之间加入一个spp_net层，使模型适应多种尺寸输入；1个n类的softmax(分类概率函数，将输出值映射为分类概率)输出层，输出长度为n的1维向量，该向量值最大的元素的索引号即为根据输入图像预测出来的方向盘转角值；除了softmax层，每层的输出都要经过激活函数，以此增强网络非线性表达。由于样本间有一定的相似性，因此训练过程中每一个全连接层都设置了dropout，抑制过拟合，droptout用于在一次训练过程中，以一定概率使神经元失效。
进一步地，在一个实施例中，步骤S100之前还进行训练数据的采集。
具体地，在人工驾驶过程中，以一定的频率实时地采集车道保持时前方的实车图像以及方向转角，将采集到的实车图像和方向转角保存。进一步地，以30Hz的频率和1280*1080的像素对实车图像进行采集，并且将采集得到的实车图像以视频的格式进行保存，同时记录下捕获视频的时间戳并以txt文档进行记录，以100Hz的频率对方向转角进行采集，并且将采集得到的方向转角和对应的时间戳以二进制bin文件进行保存。可以理解，实车图像和方向转角的采集频率并不仅限于本实施例中所列采样频率，还可以以其它频率进行实车图像和方向转角的采样，只要方向转角的采样频率大于实车图像的采样频率即可，实车图像的采集像素也并不仅限于1280*1080，文档的保存形式也不仅限于本实施例，只要能对训练数据中的信息进行保存即可。
更进一步地，在一个实施例中，训练数据的采集之后还进行训练数据库的 建立。具体地，将采集得到的数据划分为直道、弯道、左偏纠正和右偏纠正四大类，直道主要用于正常行驶，其它三类主要用于车辆偏离车道后纠正。在正常行驶过程中，大多是直道，因此直道数据具有较大比重，为了进行数据进行平衡，以降采样因子γ(大于1)对直道数据降采样，其它数据保持原有采样频率。由于方向转角的采集频率较高，为了使数据集包含更多的原始信息，我们以方向转角的采集时间为基准，在其之前采集且与之时间最近的图像作为当前方向盘转角对应的图像，将实车图像和方向转角进行同步。以1280*1080的像素对实车图像进行采集时，得到的对于车道保持来说视野过于宽广，并且在进行训练时，图片输入尺寸越大，不仅网络参数会增多，对车道保持而言引入的无关因子也会增多，而为了识别出无关因子，则数据量要成倍的增长，因此以H个像素高度，W个像素长度截取车辆前方道路图像H*W(H<1280，W<1080)，具体的大小可以根据实际情况来进行调整。由于HDF5文件在机器学习及控制软件中较易应用，选择HDF5文件来存储视频和方向转角，并且文件内图像的顺序与对应视频内视频帧的顺序相同。通过建立相应的训练数据库，方便在之后的训练过程中提取训练数据。
步骤S200，接收训练数据，并根据训练数据和神经网络模型建立实车模型。具体地，训练数据包括实车图像和方向转角，基于神经网络模型，根据接收的训练数据进行深度学习，建立实车模型。
进一步地，请参阅图5，在一个实施例中，步骤S200包括步骤S210、步骤S220和步骤S230。步骤S210，接收训练数据，对训练数据进行预处理。具体地，将采集得到的训练数据进行预处理，以扩充训练数据的数量和增加样本的多样性。
更进一步地，请参阅图6，在一个实施例中，步骤S210包括步骤S211和步骤S212。
步骤S211，接收训练数据，对训练数据中的实车图像进行随机平移、旋转、翻转和裁剪，得到预处理之后的实车图像。具体地，将采集得到的每一实车图像进行不同程度的随机平移、旋转以及以一定概率的水平翻转后再将H*W图像裁剪至IN_H*IN_W像素，大尺寸图像变换后再裁剪为小图像，主要是防止裁剪后的图像出现小范围的黑框。对H*W图像进行裁剪时，根据H*W的大小，选择适当的像素进行裁剪，裁剪时尽量缩小其它无关信息在实车图像中的占比，保证道路信息在实车图像中的占比。
步骤S212，计算预处理之后的实车图像对应的方向转角，得到预处理之后的训练数据。具体地，通过计算得到在实车图像进行预处理之后，与之对应的方向转角。通过下述变换公式得到：
steer_out＝sym_symbol*(steer_init+pix_shift*α-pix_rotate*β)
其中，α是随机平移像素对应角度的变换系数，β是图像旋转对应方向盘转角的变换系数。steer_out是对应图像变换后输出的角度值。sym_symbol是图像水平对称的标识，为示性函数，当sym_symbol为-1时表示水平对称，当sym_symbol为1表示不进行水平对称，计算式如下：
Figure PCTCN2018111274-appb-000001
f(-T,T)表示在[-T,T]闭区间等概率生成一个随机整数，T为不为零任意整数，下述公式pix_shift和pix_rotate等式与之类似，M和K均表示不为零任意整数。图像经过水平对称带来的益处是可以平衡样本中车辆不居于车道中间时方向转角的***方向随机平移的像素个数，计算方式如下：
pix_shift＝f(-M,M)
负数表示IN_H*IN_W大小的滑动框在H*W的图上左移，反之右移。pix_rotate是H*W的图像做旋转变换的旋转角度，计算式如下：
pix_rotate＝f(-K,K)
根据上述计算公式，可以得到预处理之后的实车图像对应的方向转角，从而得到预处理之后的训练数据。
步骤S220，根据预处理之后的训练数据和神经网络模型进行模型训练，得到训练结果。具体地，基于神经网络模型根据大量预处理之后的到的训练数据进行模型训练，得到对应的训练结果。
进一步地，在一个实施例中，请参阅图7，步骤S220包括步骤S221和步骤S222。步骤S221，基于Tensorflow建立与预处理之后的训练数据对应的网络训练模型。具体地，Tensorflow是一种将复杂的数据结构传输至人工智能神经网中进行分析和处理过程的智能学习***。基于Tensorflow建立与预处理之后的训练数据对应的网络训练模型，方便对预处理之后的训练数据进行后续的迭代训练。
步骤S222，根据训练集数据和神经网络模型，经训练集数据对网络训练模 型进行迭代训练，得到训练结果。具体地，在进行训练连之前，将训练集数据随机打乱，破坏样本之间的相关性，增加训练结果的可靠性。更进一步地，在一个实施例中，由于得到的训练数据的容量较大，将训练数据分批载入，根据用于训练的服务器的配置的不同，每一批载入的训练数据的也不一样，可以根据实际情况来进行选择，并且，为了便于扩展，可以将存储训练数据和迭代训练分置于不同的服务器，两服务器之间通过socket进行数据传输。可以理解，在服务器配置允许的前提下，可以将训练数据一次性载入网络训练模型，同样也可以将训练数据的存储和迭代训练置于相同的服务器进行。
步骤S230，根据训练结果建立实车模型。具体地，基于Tensorflow网络训练模型，根据接收的训练数据进行相应的训练，得到关于实车图像与方向转角对应关系的训练结果并保存，根据对大量训练数据的训练结果，建立对应的实车模型。
进一步地，在一个实施例中，请参阅图8-图9，训练数据还包括验证集数据，步骤S230包括步骤S231和步骤S232。步骤S231，根据训练结果建立初步模型。具体地，根据训练集数据基于Tensorflow网络训练模型和神经网络模型，对Tensorflow网络训练模型进行迭代训练，得到实车图像与方向转角的对应关系，并根据得到的对应关系建立初步模型。进一步地，模型训练采用mini-batch SGD(Stochastic gradient descent，随机梯度下降)作为优化器，初始学习率为δ，学习率以系数θ进行指数衰减，训练次数达到设定值后进行一次学习率衰减，具体地，设定值根据多次训练累积的经验来确定。
步骤S232，根据验证集数据对初步模型进行验证，得到实车模型。具体地，在根据训练集数据进行迭代训练之后，根据训练结果建立关于实车图形与方向转角对应关系的初步模型，然后基于验证集数据对得到的初步模型进行能力评估，并且根据初步模型在验证集上的损失值或准确率的变化趋势决定是否终止训练。进一步地，为了防止意外导致训练程序中断，每对一定量的训练数据进行训练之后就保存一次模型的训练结果。
更进一步地，在一个实施例中，训练数据还包括测试集数据，在根据训练集数据完成初步训练，验证集对初步模型进行验证得到实车模型之后，通过测试集数据对得到的实车模型进行模型预测，衡量所建立的实车模型的性能和分类能力，得到结果并输出。将得到的训练数据划分为训练集数据、验证集数据和测试集数据，有效地防止模型的过拟合，进一步地提高了所建立的实车模型 的可靠性。
上述无人驾驶车道保持方法，通过采集大量的实车数据作为训练数据，经过深度神经网络进行深度学习建立对应的实车推断模型，在实际行驶过程中，能够根据采集的实车图像，经实车推断模型得到对应的方向转角，从而控制车辆保持在对应的车道行驶。不需要人工知识即可完成对道路信息的刻画，并且通过深度学习还能学习到人工知识无法得到的对车道保持有深刻内在理解的特征信息，能够实现在路线不清晰、弯道曲率较大和车辆拥堵路段环境下的车道保持，具有识别能力强的优点。
请参阅图10，一种无人驾驶车道保持装置，包括实车图像接收模块300、实车推断模块400和方向转角控制模块500。
实车图像接收模块300用于接收数据采集器对车辆进行采集得到的实车图像。具体地，根据数据采集器实时的采集车辆在行驶过程中的实时道路信息，进一步地，数据采集器可以是摄像头，在车辆的行驶过程中，摄像头以一定的频率进行拍照，得到对应的实车图像。
实车推断模块400，用于将实车图像传输到预设的实车模型进行处理，得到与实车图像相对应的方向转角。
其中，实车模型通过深度神经网络学习建立，用于表征实车图像与方向转角的对应关系。具体地，数据采集器采集到实车图像之后，传输到预设的实车模型进行实车推断，得到与所采集的实车图像相对应的方向转角。预设的实车模型是指根据采集的实际驾驶过程中的实车图像与方向转角，进行深度神经网络学习所建立的表征实车图像与方向转角关系的模型。
进一步地，在一个实施例中，实车图像为RGB图像，在进行传输时，将RGB实车图像拆分为R、G、B三个通道，在每一通道上都加上相应的报头文和报头尾。在接收到RGB实车图像时，进行校验，当同一幅图像的R、G、B三个通道均被完整接收时，完成校验，否则校验失败。接收校验成功的RGB实车图像进行归一化后，进行实车模型推断，校验失败的实车图像则将丢弃。在对实车图像进行归一化处理时，将实车图像的RGB值从0～255归一化到[-1,1]。将每一帧的RGB实车图像封装为三帧的socket udp帧，以采样频率为30Hz为例，完成一帧RGB实车图像的传送损耗时间小于200us，满足实时性的要求。socket udp是通用的大数据报通讯方式，在c++和python上都有易于获取的接口函数，可避免c++和python混编带来的复杂调试，易于问题查找，进而缩短开发时间。
方向转角控制模块500，用于根据方向转角控制车辆保持在对应的车道行驶。具体地，根据实车模型推断出所采集的实车图像所对应的方向转角之后，控制车辆根据所得到的转角进行转向，保持在合适的车道行驶。
进一步地，请参阅图2，在一个实施例中，方向转角控制模块500，将方向转角发送至转向控制***，方向转角用于转向控制***控制车辆转向，使车辆保持在对应的车道行驶。具体地，经过实车模型推断得到与所采集的实车图像相对应的方向转角之后，将所得到的方向转角发送到数据采集端，然后数据采集端再将方向转角发送到EPS，从而控制车辆进行转向，使车辆保持在对应的车道行驶。进一步地，实车模型将推断出来的对应方向转角发送到数据采集端时，通过socket(套接字，用于两个程序之间的数据交换)进行传输，数据采集端通过CAN总线将方向转角发送到EPS。通过上述方式完成对车辆转向的闭环控制，具有控制平滑、泛化能力高的优点。
在一个实施例中，请参阅图11，无人驾驶车道保持装置还包括神经网络模型建立模块100和实车模型建立模块200。
神经网络模型建立模块100，用于基于卷积神经网络建立相应的神经网络模型。在进行车道保持的过程中，根据输入的实车图像来得到对应的方向转角，因此，可以把车道保持看作一个图像处理的问题，由于卷积神经网络在图像分类处理方面具有很强的优势，所以采用卷积神经网络作为神经网络的主要成分。具体地，请参阅图4，在一个实施例中，卷积神经网络共分为14层，一个输入层；5个卷积层，卷积层是稀疏连接，实现图像局部特征提取；3个max_pooling池化层，池化层的主要作用是对特征图下采样，整合特征；3个全连接层，其作用是将“分布式特征表示”映射到样本标记空间，可认为是对之前提取的局部特征的加权和；1个spp_net层，在最后一个卷积层与全连接层之间加入一个spp_net层，使模型适应多种尺寸输入；1个n类的softmax输出层，输出长度为n的1维向量，该向量值最大的元素的索引号即为根据输入图像预测出来的方向盘转角值；除了softmax层，每层的输出都要经过激活函数，以此增强网络非线性表达。由于样本间有一定的相似性，因此训练过程中每一个全连接层都设置了dropout，抑制过拟合，droptout用于在一次训练过程中，以一定概率使神经元失效。
进一步地，在一个实施例中，神经网络模型建立模块100基于卷积神经网络建立相应的神经网络模型之前还进行训练数据采集。
具体地，在人工驾驶过程中，以一定的频率实时地采集车道保持时前方的实车图像以及方向转角，将采集到的实车图像和方向转角保存。进一步地，以30Hz的频率和1280*1080的像素对实车图像进行采集，并且将采集得到的实车图像以视频的格式进行保存，同时记录下捕获视频的时间戳并以txt文档进行记录，以100Hz的频率对方向转角进行采集，并且将采集得到的方向转角和对应的时间戳以二进制bin文件进行保存。可以理解，实车图像和方向转角的采集频率并不仅限于本实施例中所列采样频率，还可以以其它频率进行实车图像和方向转角的采样，只要方向转角的采样频率大于实车图像的采样频率即可，实车图像的采集像素也并不仅限于1280*1080，文档的保存形式也不仅限于本实施例，只要能对训练数据中的信息进行保存即可。
更进一步地，在一个实施例中，训练数据的采集之后还进行训练数据库的建立。具体地，将采集得到的数据划分为直道、弯道、左偏纠正和右偏纠正四大类，直道主要用于正常行驶，其它三类主要用于车辆偏离车道后纠正。在正常行驶过程中，大多是直道，因此直道数据具有较大比重，为了进行数据进行平衡，以降采样因子γ(大于1)对直道数据降采样，其它数据保持原有采样频率。由于方向转角的采集频率较高，为了使数据集包含更多的原始信息，我们以方向转角的采集时间为基准，在其之前采集且与之时间最近的图像作为当前方向盘转角对应的图像，将实车图像和方向转角进行同步。以1280*1080的像素对实车图像进行采集时，得到的对于车道保持来说视野过于宽广，并且在进行训练时，图片输入尺寸越大，不仅网络参数会增多，对车道保持而言引入的无关因子也会增多，而为了识别出无关因子，则数据量要成倍的增长，因此以H个像素高度，W个像素长度截取车辆前方道路图像H*W(H<1280，W<1080)，具体的大小可以根据实际情况来进行调整。由于HDF5文件在机器学习及控制软件中较易应用，选择HDF5文件来存储视频和方向转角，并且文件内图像的顺序与对应视频内视频帧的顺序相同。通过建立相应的训练数据库，方便在之后的训练过程中提取训练数据。
实车模型建立模块200，用于接收训练数据，并根据训练数据和神经网络模型建立实车模型。具体地，训练数据包括实车图像和方向转角，基于神经网络模型，根据接收的训练数据进行深度学习，建立实车模型。
进一步地，请参阅12，在一个实施例中，实车模型建立模块200包括预处理模组210、训练模组220和模型建立模组230。预处理模组210，用于接收训 练数据，对训练数据进行预处理。具体地，将采集得到的训练数据进行预处理，以扩充训练数据的数量和增加样本的多样性。
更进一步地，请参阅图13，在一个实施例中，预处理模组210包括实车图像处理单元211和方向转角计算单元212。实车图像处理单元211，用于接收训练数据，对训练数据中的实车图像进行随机平移、旋转、翻转和裁剪，得到预处理之后的实车图像。具体地，将采集得到的每一实车图像进行不同程度的随机平移、旋转以及以一定概率的水平翻转后再将H*W图像裁剪至IN_H*IN_W像素，大尺寸图像变换后再裁剪为小图像，主要是防止裁剪后的图像出现小范围的黑框。对H*W图像进行裁剪时，根据H*W的大小，选择适当的像素进行裁剪，裁剪时尽量缩小其它无关信息在实车图像中的占比，保证道路信息在实车图像中的占比。
方向转角计算单元212，用于计算预处理之后的实车图像对应的方向转角，得到预处理之后的训练数据。具体地，通过计算得到在实车图像进行预处理之后，与之对应的方向转角。通过下述变换公式得到：
steer_out＝sym_symbol*(steer_init+pix_shift*α-pix_rotate*β)
其中，α是随机平移像素对应角度的变换系数，β是图像旋转对应方向盘转角的变换系数。steer_out是对应图像变换后输出的角度值。sym_symbol是图像水平对称的标识，为示性函数，当sym_symbol为-1时表示水平对称，当sym_symbol为1表示不进行水平对称，计算式如下：
Figure PCTCN2018111274-appb-000002
f(-T,T)表示在[-T,T]闭区间等概率生成一个随机整数，T为不为零任意整数，下述公式pix_shift和pix_rotate等式与之类似，M和K均表示不为零任意整数。图像经过水平对称带来的益处是可以平衡样本中车辆不居于车道中间时方向转角的***方向随机平移的像素个数，计算方式如下：
pix_shift＝f(-M,M)
负数表示IN_H*IN_W大小的滑动框在H*W的图上左移，反之右移。pix_rotate是H*W的图像做旋转变换的旋转角度，计算式如下：
pix_rotate＝f(-K,K)
steer_out是对应图像变换后输出的角度值。
根据上述计算公式，可以得到预处理之后的实车图像对应的方向转角，从而得到预处理之后的训练数据。
训练模组220，用于根据预处理之后的训练数据和神经网络模型进行模型训练，得到训练结果。具体地，基于神经网络模型根据大量预处理之后的到的训练数据进行模型训练，得到对应的训练结果。
进一步地，在一个实施例中，请参阅图14，训练模组220包括网络训练模型建立单元221和迭代训练单元222。网络训练模型建立单元221，用于基于Tensorflow建立网络训练模型。具体地，Tensorflow是一种将复杂的数据结构传输至人工智能神经网中进行分析和处理过程的智能学习***。基于Tensorflow建立与预处理之后的训练数据对应的网络训练模型，方便对预处理之后的训练数据进行后续的迭代训练。
迭代训练单元222，用于根据训练集数据和神经网络模型，经训练集数据对网络训练模型行迭代训练，得到训练结果。具体地，在进行训练连之前，将训练集数据随机打乱，破坏样本之间的相关性，增加训练结果的可靠性。更进一步地，在一个实施例中，由于得到的训练数据的容量较大，将训练数据分批载入，根据用于训练的服务器的配置的不同，每一批载入的训练数据的也不一样，可以根据实际情况来进行选择，并且，为了便于扩展，可以将存储训练数据和迭代训练分置于不同的服务器，两服务器之间通过socket进行数据传输。可以理解，在服务器配置允许的前提下，可以将训练数据一次性载入网络训练模型，同样也可以将训练数据的存储和迭代训练置于相同的服务器进行。
模型建立模组230，用于根据训练结果建立实车模型。具体地，基于Tensorflow网络训练模型，根据接收的训练数据进行相应的训练，得到关于实车图像与方向转角对应关系的训练结果并保存，根据对大量训练数据的训练结果，建立对应的实车模型。
进一步地，在一个实施例中，请参阅图15，训练数据还包括验证集数据，模型建立模组230包括初步模型建立单元231、初步模型验证单元232。初步模型建立单元231，用于根据训练结果建立初步模型。具体地，根据训练集数据基于Tensorflow网络训练模型和神经网络模型，对Tensorflow网络训练模型进行迭代训练，得到实车图像与方向转角的对应关系，并根据得到的对应关系建立初步模型。进一步地，模型训练采用mini-batch SGD作为优化器，初始学习率为δ，学习率以系数θ进行指数衰减，训练次数达到设定值后进行一次学习率衰 减，具体地，设定值根据多次训练累积的经验来确定。
初步模型验证单元232，用于根据验证集数据对初步模型进行验证，得到实车模型。具体地，在根据训练集数据进行迭代训练之后，根据训练结果建立关于实车图形与方向转角对应关系的初步模型，然后基于验证集数据对得到的初步模型进行能力评估，并且根据初步模型在验证集上的损失值或准确率的变化趋势决定是否终止训练。进一步地，为了防止意外导致训练程序中断，每对一定量的训练数据进行训练之后就保存一次模型的训练结果。
更进一步地，在一个实施例中，训练数据还包括测试集数据，在根据训练集数据完成初步训练，验证集对初步模型进行验证得到实车模型之后，通过测试集数据对得到的实车模型进行模型预测，衡量所建立的实车模型的性能和分类能力，得到结果并输出。将得到的训练数据划分为训练集数据、验证集数据和测试集数据，有效地防止模型的过拟合，进一步地提高了所建立的实车模型的可靠性。
上述无人驾驶车道保持装置，通过采集大量的实车数据作为训练数据，经过深度神经网络进行深度学习建立对应的实车推断模型，在实际行驶过程中，能够根据采集的实车图像，经实车推断模型得到对应的方向转角，从而控制车辆保持在对应的车道行驶。不需要人工知识即可完成对道路信息的刻画，并且通过深度学习还能学习到人工知识无法得到的对车道保持有深刻内在理解的特征信息，能够实现在路线不清晰、弯道曲率较大和车辆拥堵路段环境下的车道保持，具有识别能力强的优点。
上述无人驾驶车道保持装置中的各个模块可全部或部分通过软件、硬件及其组合来实现。上述各模块可以硬件形式内嵌于或独立于计算机设备中的处理器中，也可以以软件形式存储于计算机设备中的存储器中，以便于处理器调用执行以上各个模块对应的操作。
在一个实施例中，提供了一种计算机设备，该计算机设备可以是服务器，其内部结构图可以如图16所示。该计算机设备包括通过***总线连接的处理器、存储器、网络接口和数据库。其中，该计算机设备的处理器用于提供计算和控制能力。该计算机设备的存储器包括非易失性存储介质、内存储器。该非易失性存储介质存储有操作***、计算机程序和数据库。该内存储器为非易失性存储介质中的操作***和计算机程序的运行提供环境。该计算机设备的数据库用于存储无人驾驶车道保持方法中的数据。该计算机设备的网络接口用于与外部 的终端通过网络连接通信。该计算机程序被处理器执行时以实现一种无人驾驶车道保持方法。
本领域技术人员可以理解，图16中示出的结构，仅仅是与本申请方案相关的部分结构的框图，并不构成对本申请方案所应用于其上的计算机设备的限定，具体的计算机设备可以包括比图中所示更多或更少的部件，或者组合某些部件，或者具有不同的部件布置。
在一个实施例中，提供了一种计算机设备，包括存储器和处理器，存储器中存储有计算机程序，该处理器执行计算机程序时实现以下步骤：
接收数据采集器对车辆进行采集得到的实车图像；
将实车图像传输到预设的实车模型进行处理，得到与实车图像相对应的方向转角，其中，实车模型通过深度神经网络学习建立，用于表征实车图像与方向转角的对应关系；
根据方向转角控制车辆保持在对应的车道行驶。
在一个实施例中，处理器执行计算机程序时还实现以下步骤：
基于卷积神经网络建立相应的神经网络模型；
接收训练数据，并根据训练数据和神经网络模型建立实车模型，训练数据包括实车图像和方向转角。
在一个实施例中，处理器执行计算机程序时还实现以下步骤：
接收训练数据，对训练数据进行预处理，得到预处理之后的训练数据；
根据预处理之后的训练数据和神经网络模型进行模型训练，得到训练结果；
根据训练结果建立实车模型。
在一个实施例中，处理器执行计算机程序时还实现以下步骤：
接收训练数据，对训练数据中的实车图像进行随机平移、旋转、翻转和裁剪，得到预处理之后的实车图像；
计算预处理之后的实车图像对应的方向转角，得到预处理之后的训练数据。
在一个实施例中，处理器执行计算机程序时还实现以下步骤：
基于Tensorflow建立与预处理之后的训练数据对应的网络训练模型；
根据训练集数据和神经网络模型，经训练集数据对网络训练模型进行迭代训练，得到训练结果。
在一个实施例中，处理器执行计算机程序时还实现以下步骤：
根据训练结果建立初步模型；
根据验证集数据对初步模型进行验证，得到实车模型。
在一个实施例中，处理器执行计算机程序时还实现以下步骤：
将方向转角发送至转向控制***，方向转角用于转向控制***控制车辆转向，使车辆保持在对应的车道行驶。
在一个实施例中，提供了一种计算机可读存储介质，其上存储有计算机程序，计算机程序被处理器执行时实现以下步骤：
接收数据采集器对车辆进行采集得到的实车图像；
将实车图像传输到预设的实车模型进行处理，得到与实车图像相对应的方向转角，其中，实车模型通过深度神经网络学习建立，用于表征实车图像与方向转角的对应关系；
根据方向转角控制车辆保持在对应的车道行驶。
在一个实施例中，计算机程序被处理器执行时还实现以下步骤：
基于卷积神经网络建立相应的神经网络模型；
接收训练数据，并根据训练数据和神经网络模型建立实车模型，训练数据包括实车图像和方向转角。
在一个实施例中，计算机程序被处理器执行时还实现以下步骤：
接收训练数据，对训练数据进行预处理，得到预处理之后的训练数据；
根据预处理之后的训练数据和神经网络模型进行模型训练，得到训练结果；
根据训练结果建立实车模型。
在一个实施例中，计算机程序被处理器执行时还实现以下步骤：
接收训练数据，对训练数据中的实车图像进行随机平移、旋转、翻转和裁剪，得到预处理之后的实车图像；
计算预处理之后的实车图像对应的方向转角，得到预处理之后的训练数据。
在一个实施例中，计算机程序被处理器执行时还实现以下步骤：
基于Tensorflow建立与预处理之后的训练数据对应的网络训练模型；
根据训练集数据和神经网络模型，经训练集数据对网络训练模型进行迭代训练，得到训练结果。
在一个实施例中，计算机程序被处理器执行时还实现以下步骤：
根据训练结果建立初步模型；
根据验证集数据对初步模型进行验证，得到实车模型。
在一个实施例中，计算机程序被处理器执行时还实现以下步骤：
将方向转角发送至转向控制***，方向转角用于转向控制***控制车辆转向，使车辆保持在对应的车道行驶。
上述计算机设备和存储介质，通过采集大量的实车数据作为训练数据，经过深度神经网络进行深度学习建立对应的实车推断模型，在实际行驶过程中，能够根据采集的实车图像，经实车推断模型得到对应的方向转角，从而控制车辆保持在对应的车道行驶。不需要人工知识即可完成对道路信息的刻画，并且通过深度学习还能学习到人工知识无法得到的对车道保持有深刻内在理解的特征信息，能够实现在路线不清晰、弯道曲率较大和车辆拥堵路段环境下的车道保持，具有识别能力强的优点。
本领域普通技术人员可以理解实现上述实施例方法中的全部或部分流程，是可以通过计算机程序来指令相关的硬件来完成，所述的计算机程序可存储于一非易失性计算机可读取存储介质中，该计算机程序在执行时，可包括如上述各方法的实施例的流程。其中，本申请所提供的各实施例中所使用的对存储器、存储、数据库或其它介质的任何引用，均可包括非易失性和/或易失性存储器。非易失性存储器可包括只读存储器(ROM)、可编程ROM(PROM)、电可编程ROM(EPROM)、电可擦除可编程ROM(EEPROM)或闪存。易失性存储器可包括随机存取存储器(RAM)或者外部高速缓冲存储器。作为说明而非局限，RAM以多种形式可得，诸如静态RAM(SRAM)、动态RAM(DRAM)、同步DRAM(SDRAM)、双数据率SDRAM(DDRSDRAM)、增强型SDRAM(ESDRAM)、同步链路(Synchlink)DRAM(SLDRAM)、存储器总线(Rambus)直接RAM(RDRAM)、直接存储器总线动态RAM(DRDRAM)、以及存储器总线动态RAM(RDRAM)等。
以上实施例的各技术特征可以进行任意的组合，为使描述简洁，未对上述实施例中的各个技术特征所有可能的组合都进行描述，然而，只要这些技术特征的组合不存在矛盾，都应当认为是本说明书记载的范围。
以上所述实施例仅表达了本申请的几种实施方式，其描述较为具体和详细，但并不能因此而理解为对发明专利范围的限制。应当指出的是，对于本领域的普通技术人员来说，在不脱离本申请构思的前提下，还可以做出若干变形和改进，这些都属于本申请的保护范围。因此，本申请专利的保护范围应以所附权利要求为准。

Claims (10)

  1. 一种无人驾驶车道保持方法，其特征在于，所述方法包括步骤：
    接收数据采集器对车辆进行采集得到的实车图像；
    将所述实车图像传输到预设的实车模型进行处理，得到与所述实车图像相对应的方向转角，其中，所述实车模型通过深度神经网络学习建立，用于表征实车图像与方向转角的对应关系；
    根据所述方向转角控制车辆保持在对应的车道行驶。
  2. 根据权利要求1所述的无人驾驶车道保持方法，其特征在于，所述接收数据采集器对车辆进行采集得到的实车图像的步骤之前，还包括：
    基于卷积神经网络建立相应的神经网络模型；
    接收训练数据，并根据所述训练数据和所述神经网络模型建立实车模型，所述训练数据包括实车图像和方向转角。
  3. 根据权利要求2所述的无人驾驶车道保持方法，其特征在于，所述接收训练数据，并根据所述训练数据和所述神经网络模型建立实车模型的步骤，包括：
    接收训练数据，对所述训练数据进行预处理，得到预处理之后的训练数据；
    根据所述预处理之后的训练数据和所述神经网络模型进行模型训练，得到训练结果；
    根据所述训练结果建立实车模型。
  4. 根据权利要求3所述的无人驾驶车道保持方法，其特征在于，所述接收训练数据，对所述训练数据进行预处理，包括：
    接收训练数据，对所述训练数据中的实车图像进行随机平移、旋转、翻转和裁剪，得到预处理之后的实车图像；
    计算所述预处理之后的实车图像对应的方向转角，得到预处理之后的训练数据。
  5. 根据权利要求3所述的无人驾驶车道保持方法，其特征在于，所述训练数据包括训练集数据，所述根据所述预处理之后的训练数据和所述神经网络模型进行模型训练，得到训练结果的步骤，包括：
    基于Tensorflow建立与预处理之后的训练数据对应的网络训练模型；
    根据所述训练集数据和所述神经网络模型，经所述训练集数据对所述网络训练模型进行迭代训练，得到训练结果。
  6. 根据权利要求5所述的无人驾驶车道保持方法，其特征在于，所述训练数据还包括验证集数据，所述根据所述训练结果建立实车模型的步骤，包括：
    根据训练结果建立初步模型；
    根据所述验证集数据对所述初步模型进行验证，得到实车模型。
  7. 根据权利要求1-6任意一项所述的无人驾驶车道保持方法，其特征在于，所述根据所述方向转角控制车辆保持在对应的车道行驶的步骤，包括：
    将所述方向转角发送至转向控制***，所述方向转角用于所述转向控制***控制车辆转向，使车辆保持在对应的车道行驶。
  8. 一种无人驾驶车道保持装置，其特征在于，所述装置包括：
    实车图像接收模块，用于接收数据采集器对车辆进行采集得到的实车图像；
    实车推断模块，用于将所述实车图像传输到实车模型进行处理，得到与所述实车图像相对应的方向转角，其中，所述实车模型通过深度神经网络学习建立，用于表征实车图像与方向转角的对应关系；
    方向转角控制模块，用于根据所述方向转角控制车辆保持在对应的车道行驶。
  9. 一种计算机设备，包括存储器和处理器，所述存储器存储有计算机程序，其特征在于，所述处理器执行所述计算机程序时实现权利要求1至7中任一项所述方法的步骤。
  10. 一种计算机可读存储介质，其上存储有计算机程序，其特征在于，所述计算机程序被处理器执行时实现权利要求1至7中任一项所述的方法的步骤。
PCT/CN2018/111274 2018-03-23 2018-10-22 无人驾驶车道保持方法、装置、计算机设备和存储介质 WO2019179094A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US16/339,008 US11505187B2 (en) 2018-03-23 2018-10-22 Unmanned lane keeping method and device, computer device, and storage medium

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201810247138.XA CN110298219A (zh) 2018-03-23 2018-03-23 无人驾驶车道保持方法、装置、计算机设备和存储介质
CN201810247138.X 2018-03-23

Publications (1)

Publication Number Publication Date
WO2019179094A1 true WO2019179094A1 (zh) 2019-09-26

Family

ID=67986648

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2018/111274 WO2019179094A1 (zh) 2018-03-23 2018-10-22 无人驾驶车道保持方法、装置、计算机设备和存储介质

Country Status (3)

Country Link
US (1) US11505187B2 (zh)
CN (1) CN110298219A (zh)
WO (1) WO2019179094A1 (zh)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112434588A (zh) * 2020-11-18 2021-03-02 青岛慧拓智能机器有限公司 一种端到端驾驶员高速道路换道意图的推理方法
CN112686147A (zh) * 2020-12-30 2021-04-20 北京纵目安驰智能科技有限公司 车辆与车轮从属关系预测方法、***、存储介质及终端

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11615208B2 (en) * 2018-07-06 2023-03-28 Capital One Services, Llc Systems and methods for synthetic data generation
CN111553188B (zh) * 2020-03-24 2023-06-09 深圳市齐远兴电子有限公司 基于深度学习的端到端自动驾驶车辆转向控制***
CN112092825B (zh) * 2020-08-31 2022-04-29 英博超算(南京)科技有限公司 一种基于机器学习的车道保持方法

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106873566A (zh) * 2017-03-14 2017-06-20 东北大学 一种基于深度学习的无人驾驶物流车
CN107169402A (zh) * 2016-03-08 2017-09-15 福特全球技术公司 车辆车道定位
EP3219564A1 (en) * 2016-03-14 2017-09-20 IMRA Europe S.A.S. Driving prediction with a deep neural network
CN107515607A (zh) * 2017-09-05 2017-12-26 百度在线网络技术（北京）有限公司 用于无人车的控制方法和装置

Family Cites Families (16)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8494716B1 (en) * 2012-06-04 2013-07-23 GM Global Technology Operations LLC Lane keeping system using rear camera
US9481287B2 (en) * 2014-01-21 2016-11-01 Harman International Industries, Inc. Roadway projection system
JP6617534B2 (ja) * 2015-11-30 2019-12-11 株式会社デンソー 運転支援装置
US10635117B2 (en) * 2016-10-25 2020-04-28 International Business Machines Corporation Traffic navigation for a lead vehicle and associated following vehicles
WO2018200522A1 (en) * 2017-04-24 2018-11-01 Mobileye Vision Technologies Ltd. Systems and methods for compression of lane data
US11200433B2 (en) * 2017-05-03 2021-12-14 Mobileye Vision Technologies Ltd. Detection and classification systems and methods for autonomous vehicle navigation
CN107226087B (zh) * 2017-05-26 2019-03-26 西安电子科技大学 一种结构化道路自动驾驶运输车及控制方法
EP3436783A4 (en) * 2017-06-22 2019-02-06 Baidu.com Times Technology (Beijing) Co., Ltd. FRAMEWORK FOR THE ASSESSMENT OF PREVIEWED TRAJECTORIES IN TRAFFIC FORECASTING AUTONOMOUS VEHICLES
US20180373980A1 (en) * 2017-06-27 2018-12-27 drive.ai Inc. Method for training and refining an artificial intelligence
US10668925B2 (en) * 2017-09-05 2020-06-02 Baidu Usa Llc Driver intention-based lane assistant system for autonomous driving vehicles
US10611371B2 (en) * 2017-09-14 2020-04-07 Toyota Motor Engineering & Manufacturing North America, Inc. System and method for vehicle lane change prediction using structural recurrent neural networks
EP3495220B1 (en) * 2017-12-11 2024-04-03 Volvo Car Corporation Path prediction for a vehicle
US10737717B2 (en) * 2018-02-14 2020-08-11 GM Global Technology Operations LLC Trajectory tracking for vehicle lateral control using neural network
KR20190083317A (ko) * 2019-06-20 2019-07-11 엘지전자 주식회사 차량의 차선 변경에 관련된 알림을 제공하는 인공 지능 장치 및 그 방법
US11488395B2 (en) * 2019-10-01 2022-11-01 Toyota Research Institute, Inc. Systems and methods for vehicular navigation
US11741723B2 (en) * 2020-02-27 2023-08-29 Honda Motor Co., Ltd. System for performing intersection scenario retrieval and method thereof

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107169402A (zh) * 2016-03-08 2017-09-15 福特全球技术公司 车辆车道定位
EP3219564A1 (en) * 2016-03-14 2017-09-20 IMRA Europe S.A.S. Driving prediction with a deep neural network
CN106873566A (zh) * 2017-03-14 2017-06-20 东北大学 一种基于深度学习的无人驾驶物流车
CN107515607A (zh) * 2017-09-05 2017-12-26 百度在线网络技术（北京）有限公司 用于无人车的控制方法和装置

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
BAI, CHENJIA: "Research on Autonomous Driving Method Based on Computer Vision and Deep Learning", CHINESE MASTER'S THESES FULL-TEXT DATABASE, 15 February 2018 (2018-02-15) *

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112434588A (zh) * 2020-11-18 2021-03-02 青岛慧拓智能机器有限公司 一种端到端驾驶员高速道路换道意图的推理方法
CN112686147A (zh) * 2020-12-30 2021-04-20 北京纵目安驰智能科技有限公司 车辆与车轮从属关系预测方法、***、存储介质及终端
CN112686147B (zh) * 2020-12-30 2024-05-28 北京纵目安驰智能科技有限公司 车辆与车轮从属关系预测方法、***、存储介质及终端

Also Published As

Publication number Publication date
US11505187B2 (en) 2022-11-22
US20210339743A1 (en) 2021-11-04
CN110298219A (zh) 2019-10-01

Similar Documents

Publication Publication Date Title
WO2019179094A1 (zh) 无人驾驶车道保持方法、装置、计算机设备和存储介质
CN109145759B (zh) 车辆属性识别方法、装置、服务器及存储介质
EP3968179A1 (en) Place recognition method and apparatus, model training method and apparatus for place recognition, and electronic device
WO2021217775A1 (zh) 一种基于神经网络模型融合的废钢评级方法及装置
US11790499B2 (en) Certificate image extraction method and terminal device
CN107506711A (zh) 基于卷积神经网络的双目视觉障碍物检测***及方法
WO2022257254A1 (zh) 图像数据处理方法、装置、设备以及介质
CN111696110B (zh) 场景分割方法及***
CN111091023B (zh) 一种车辆检测方法、装置及电子设备
CN109584299B (zh) 一种定位方法、定位装置、终端及存储介质
US11994408B2 (en) Incremental map building using learnable features and descriptors
WO2021184754A1 (zh) 视频对比方法、装置、计算机设备和存储介质
CN113284144B (zh) 一种基于无人机的隧道检测方法及装置
CN111435446A (zh) 一种基于LeNet车牌识别方法及装置
CN112654998B (zh) 一种车道线检测方法和装置
CN116994236A (zh) 一种基于深度神经网络的低质图像车牌检测方法
CN116091823A (zh) 一种基于快速分组残差模块的单特征无锚框目标检测方法
CN114821530A (zh) 基于深度学习的车道线检测方法及***
CN111414997B (zh) 一种基于人工智能的用于战场目标识别的方法
CN112102379A (zh) 一种无人机多光谱影像配准方法
CN116994068A (zh) 一种基于知识蒸馏的目标检测方法及装置
CN114972470B (zh) 基于双目视觉的路面环境获取方法和***
Cheng et al. Edge-assisted lightweight region-of-interest extraction and transmission for vehicle perception
CN114529455A (zh) 一种基于任务解耦的有参图像超分辨率方法和***
CN115063753B (zh) 一种安全带佩戴检测模型训练方法及安全带佩戴检测方法

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 18911302

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 18911302

Country of ref document: EP

Kind code of ref document: A1