WO2020083103A1 - 一种基于深度神经网络图像识别的车辆定位方法 - Google Patents

一种基于深度神经网络图像识别的车辆定位方法 Download PDF

Info

Publication number
WO2020083103A1
WO2020083103A1 PCT/CN2019/111840 CN2019111840W WO2020083103A1 WO 2020083103 A1 WO2020083103 A1 WO 2020083103A1 CN 2019111840 W CN2019111840 W CN 2019111840W WO 2020083103 A1 WO2020083103 A1 WO 2020083103A1
Authority
WO
WIPO (PCT)
Prior art keywords
neural network
deep neural
road sign
vehicle
coordinate system
Prior art date
Application number
PCT/CN2019/111840
Other languages
English (en)
French (fr)
Inventor
冯江华
胡云卿
袁浩
林军
刘悦
游俊
熊群芳
丁驰
岳伟
Original Assignee
中车株洲电力机车研究所有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 中车株洲电力机车研究所有限公司 filed Critical 中车株洲电力机车研究所有限公司
Priority to SG11202103814PA priority Critical patent/SG11202103814PA/en
Publication of WO2020083103A1 publication Critical patent/WO2020083103A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
    • G06V20/58—Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
    • G06V20/582—Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads of traffic signs
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
    • G01C21/00—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
    • G01C21/26—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 specially adapted for navigation in a road network
    • G01C21/28—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 specially adapted for navigation in a road network with correlation of data from several navigational instruments
    • G01C21/30—Map- or contour-matching
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/24—Aligning, centring, orientation detection or correction of the image
    • G06V10/245—Aligning, centring, orientation detection or correction of the image by locating a pattern; Special marks for positioning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V2201/00—Indexing scheme relating to image or video recognition or understanding
    • G06V2201/09—Recognition of logos

Definitions

  • the invention relates to the technical field of image recognition and positioning, in particular to a vehicle positioning method based on deep neural network image recognition, and a training method of the deep neural network.
  • vehicle positioning technology mainly uses GPS technology and high-precision map matching positioning.
  • the GPS technology has the following problems when it is used: when using the ordinary GPS mode for positioning, the positioning error reaches the meter level, which cannot meet the accuracy requirements of the vehicle; if the GPS RTK mode is used, it is necessary to obtain both satellite information and ground reference positioning information It is necessary to install reference positioning communication equipment along the road. The equipment cost and use cost are high. When the vehicle enters the road section with poor satellite signal, such as dense forest or tunnel, the GPS signal is easy to be lost, thereby losing positioning information.
  • Map data needs to be established and stored on the vehicle in advance.
  • point cloud data or image data of the current environment of the vehicle is obtained through an external laser radar or camera device And match with pre-stored map data.
  • the cost of map making and software and hardware cost of matching calculation are relatively high.
  • a low-cost, high-precision vehicle positioning method is needed to provide reliable data support for vehicle positioning, pit route planning, and speed control.
  • the invention provides a vehicle positioning method based on deep neural network image recognition, and a training method of the deep neural network.
  • the invention can increase the training accuracy of the deep neural network by increasing the number of training samples and optimizing the parameters of the neural network, thereby improving the positioning accuracy of the vehicle, and the required equipment cost and use cost are low.
  • the first aspect of the present invention provides a deep neural network training method for road sign recognition, including the following steps:
  • Road sign graphic setting step set a road sign graphic on the road surface of the station inbound direction, and the distance between the marking point of the road sign graphic and the edge of the station inbound direction is L;
  • Steps for setting the camera install the camera on the vehicle, the optical axis of the camera coincides with the longitudinal centerline of the vehicle body, and the distance between the lens center of the camera and the ground is H
  • Training sample production step calculate the position coordinates of the identification point of the road sign graphic in the image coordinate system in each of the image samples, make a label set, and combine each of the image samples with the corresponding labels Set pairing to form training samples;
  • Steps for building a deep neural network on the basis of the target recognition classification deep neural network, the final classification output layer of the network is modified into an output layer composed of 2 nodes to output the position of the marking point of the road marking graphic coordinate;
  • Deep neural network training step input the training samples to the deep neural network for training.
  • the shooting time is selected at noon on a sunny day and at night on a sunny day.
  • the shooting time is selected at noon on rainy days and night on rainy days.
  • the shooting time is selected at noon on a foggy day and at night on a foggy day.
  • the photographing device photographs an image sample of the road marking pattern every 5 ° within an angle range of 5 ° to 180 ° between the optical axis and the road surface.
  • the lens parameters of the shooting device are selected so that when all the road sign graphics appear in the lens screen, the road sign graphics can occupy more than 20% of the area of the lens screen.
  • the photographing device is installed at the front roof position of the vehicle, and points in the direction of the vehicle.
  • the road sign graphic adopts a triangle, a rectangle, an arc, or other geometric element combinations that are easy to recognize.
  • the road sign graphic is a bar code or a two-dimensional code.
  • the identification point of the road identification graphic is its geometric center.
  • the deep neural network adopts ResNet50 network, and replaces the last classification output layer of the network with two fully connected layers with 1024 nodes, and the fully connected layer is connected with an output layer with 2 nodes output.
  • the deep neural network adopts a ResNet50 network, and replaces the final classification output layer of the network with two fully connected layers with 2048 nodes, and the fully connected layer is connected with an output layer with 2 nodes output.
  • the floating point data output by the two nodes belong to the closed interval of [0, 1], and the pixel coordinates can be obtained by multiplying the output floating point data by the corresponding image width and height.
  • a second aspect of the present invention provides a vehicle positioning method using the above deep neural network training method, including the following steps:
  • Road sign pattern recognition step use the deep neural network after training to identify the road sign pattern photographed during the actual pit stop of the vehicle and obtain its sign point P in the image coordinate system Position coordinates (u, v);
  • Vehicle positioning step determine the distance between the shooting device and the edge of the station in the direction of the station according to the distance between the obtained road marking graphic identification point P and the shooting device, and then combine the shooting device on the vehicle To determine the distance between the vehicle and the edge of the station into the station.
  • the marking point P of the road marking graphic is on the optical axis of the lens of the shooting device;
  • the origin of the coordinate system of the shooting device is set at the position of the imaging aperture of the shooting device, and the horizontal distance between the optical center of the lens of the shooting device and the marking point P of the road marking graphic is Z C ;
  • the positive direction of the Z axis of the camera coordinate system is selected as the forward direction of the vehicle, the positive direction of the Y axis of the camera coordinate system is selected as the downward direction of the vehicle, and the positive X axis of the camera coordinate system is positive Select the right direction of the vehicle;
  • the world coordinate system coincides with the camera coordinate system
  • the origin of the image coordinate system is on the Z axis of the camera coordinate system, and the X axis and Y axis of the image coordinate system are parallel to the X axis and Y axis of the camera coordinate system, respectively;
  • the image sample collection process is carried out in multiple periods under different lighting and weather conditions, which reduces the influence of environmental factors on the training results and improves the environmental adaptability of the deep neural network.
  • the above method can provide the distance data between the vehicle and the station, provide data support for vehicle positioning, inbound route planning, and speed control, and has the advantages of simple operation, low cost, and high reliability.
  • FIG. 1 is a flowchart of a deep neural network training method for road sign recognition
  • Figure 2 is a schematic diagram of shooting in the image sample collection step
  • FIG. 3 is a flowchart of a vehicle positioning method based on a deep neural network after training is completed
  • Figure 4 is a side view of the vehicle during the vehicle entering the station
  • 5 is a plan view of the vehicle during the vehicle entering the station
  • Fig. 6 is a schematic diagram of calculating the edge distance between the vehicle and the station in the direction of the station.
  • FIG. 1 is a flowchart of a deep neural network training method for road sign recognition provided by the present invention, including a road sign graphic setting step 101, a camera setting step 102, an image sample acquisition step 103, a training sample production step 104, Deep neural network construction step 105, deep neural network training step 106.
  • Road sign graphic setting step 101 Set a road sign graphic on the road surface of the station inbound direction, and the distance between the marking point of the road sign graphic and the edge of the station inbound direction is L.
  • the road marking graphics may be, but not limited to, triangles, rectangles, arcs, or other easily identifiable combinations of geometric elements, or text graphics, or bar codes or two-dimensional codes incorporating station-related information.
  • the identification point of the road identification graphic may be a geometric center point, vertex or other geometric feature point of the road identification graphic.
  • Shooting device setting step 102 Install a shooting device on the vehicle.
  • the lens of the shooting device points to the direction of the vehicle. It is installed on the roof of the front of the vehicle or other position where the road sign can be photographed.
  • the longitudinal symmetry centerline of the vehicle body coincides, and the distance H between the lens optical center of the shooting device and the ground is recorded.
  • the lens parameters of the shooting device are selected such that, when all the road sign graphics appear in the lens screen, the road sign graphics can occupy more than 20% of the area of the lens screen, and the larger the area, the sign point of the road sign graphics is The more precise the positioning.
  • Image sample collection step 103 Under different lighting or weather conditions, such as sunny noon and sunny night, rainy noon and rainy night, foggy noon and foggy night, the road sign pattern is photographed using the above-mentioned shooting device.
  • the shooting angle is shown in Figure 2, where the letter A represents the road sign graphic, and the angle between the optical axis of the camera and the road surface is changed in the direction of the vehicle's advance and the direction perpendicular to the direction of the vehicle's direction, so that the camera is in its An image sample of a road marking image is taken every 5 ° within an angle of 5 ° to 180 ° between the optical axis and the road surface.
  • Training sample production step 104 Calculate the position coordinates of the road marking graphics in each image sample in the image coordinate system, make a label set, and pair each image sample with the corresponding label set to form a training sample, so that Then input deep neural network for training.
  • Deep neural network construction step 105 Use a target recognition classification deep neural network, but modify the final classification output layer of the network to an output layer composed of two nodes, the values output by these two nodes are the identification points of the road marking graphics. Coordinates in the image frame. More specifically, the ResNet50 network can be used, and the final classification output layer is removed. According to the required recognition effect, two fully connected layers with 1024 nodes or 2048 nodes are used instead. After the fully connected layer, there are 2 connected In the output layer output by the node, the floating point data output by these two nodes belongs to the closed interval of [0, 1], and the pixel coordinates can be obtained by multiplying the output floating point data by the corresponding image width and height.
  • Deep neural network training step 106 The aforementioned training samples are input to the optimized deep neural network for training. After the training is completed, the deep neural network can be used to identify the road sign graphic and obtain the geometric center position coordinates.
  • FIG. 3 is a flowchart of a vehicle positioning method using the above deep neural network training method provided by the present invention, including a road sign pattern recognition step 201, a road sign pattern positioning step 202, and a vehicle positioning step 203.
  • Road sign pattern recognition step 201 Using the deep neural network after the training is completed, the road sign graphics photographed during the actual pit stop of the vehicle are recognized and the position coordinates (u, v) of the sign point P in the image coordinate system are obtained .
  • Road sign graphic positioning step 202 Calculate the coordinates (X w , Y w , Z w ) of the road sign graphic point P in the world coordinate system through the transformation relationship between the image coordinate system and the world coordinate system, thereby obtaining the road sign graphic The distance between the marking point P and the camera.
  • the transformation between the above image coordinate system and the world coordinate system can be described using a small hole imaging model.
  • Z C represents the horizontal distance between the road marking graphic marking point P and the optical center of the camera lens
  • d x , d y , u 0 , v 0 , f are the internal lens parameters related to the camera lens, specific for:
  • d x , d y represent the physical length of the unit pixel in the X and Y directions of the image coordinate system; u 0 , v 0 respectively represent the origin of the image coordinate system and the origin of the camera coordinate system in the X and Y directions Offset; f represents the lens imaging focal length.
  • R represents the rotation relationship between the world coordinate system and the camera coordinate system
  • formula (2) is used to calculate:
  • ⁇ , ⁇ , and ⁇ respectively represent the angles required to rotate around the X axis, Y axis, and Z axis when the world coordinate system and the camera coordinate system coincide.
  • T represents the translation relationship between the world coordinate system and the camera coordinate system
  • formula (3) is used to calculate:
  • t x , t y , and t z represent the translation amounts of the world coordinate system and the camera coordinate system on the X axis, Y axis, and Z axis, respectively.
  • the above parameters d x , d y , u 0 , v 0 , f, ⁇ , ⁇ , ⁇ , t x , t y , t z can be calibrated using but not limited to the conditions described below.
  • the shooting device is installed at the front roof of the vehicle and points in the forward direction.
  • the axis of the lens optical center of the shooting device coincides with the geometric symmetry centerline of the longitudinal axis of the vehicle.
  • the marking point P of the road marking on the road surface in front of the vehicle is on the axis of the lens optical center of the shooting device, and the horizontal distance from the lens optical center of the shooting device is Z C .
  • Set the origin of the camera coordinate system to the location of the imaging aperture of the camera.
  • the world coordinate system coincides with the camera coordinate system, and the forward direction of the vehicle is selected as the positive direction of the Z axis, the downward direction of the vehicle is the positive direction of the Y axis, and the right direction of the vehicle is the positive direction of the X axis.
  • the origin of the image coordinate system is on the Z axis of the camera coordinate system, and the X axis and Y axis of the image coordinate system are parallel to the X axis and Y axis of the camera coordinate system, respectively.
  • Vehicle positioning step 203 As shown in FIG. 6, after obtaining the horizontal distance Z C of the marking point P of the road marking graphic from the optical center of the lens of the shooting device, and then combining the distance L of the marking point P of the road marking graphic from the edge of the station in the direction of stop , You can calculate the horizontal distance L CZ of the optical center of the camera and the edge of the station in the direction of the station:
  • the distance between the vehicle and the edge of the station entering direction is determined, so as to realize vehicle positioning.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Remote Sensing (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Evolutionary Computation (AREA)
  • Computational Linguistics (AREA)
  • Evolutionary Biology (AREA)
  • Molecular Biology (AREA)
  • General Health & Medical Sciences (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Biophysics (AREA)
  • Biomedical Technology (AREA)
  • Health & Medical Sciences (AREA)
  • Computing Systems (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Automation & Control Theory (AREA)
  • Multimedia (AREA)
  • Image Analysis (AREA)
  • Traffic Control Systems (AREA)

Abstract

一种基于深度神经网络图像识别的车辆定位方法，以及该深度神经网络的训练方法。所述训练方法包括道路标识图形设置（101）、拍摄装置设置（102）、图像样本采集（103）、训练样本制作（104）、深度神经网络搭建（105）以及深度神经网络训练（106）。其中图像样本采集过程在不同光照及天气条件下的多个时段内进行，能够提高深度神经网络的环境适应性。此外，在车辆前进方向和与前进方向垂直的方向上每隔一定角度拍摄一张采样图像，获得的训练样本数据量大，能够提高深度神经网络的训练精度，从而提高车辆定位的精度。

Description

一种基于深度神经网络图像识别的车辆定位方法 技术领域
本发明涉及图像识别及定位技术领域，具体为一种基于深度神经网络图像识别的车辆定位方法，以及该深度神经网络的训练方法。
背景技术
目前，公共交通车辆在进入站点前，司机仅靠目视来判断车辆与站点之间的距离，无法实现精确的进站路线规划和速度控制。为了能让车辆在进站前实现精确的进站路线规划和速度控制，需要精确定位车辆与站点之间的距离。目前，车辆定位技术主要采用GPS技术和高精地图匹配定位。
GPS技术在使用时存在以下几个问题：使用普通GPS模式定位时，定位误差达到米级，不能满足车辆靠站精度要求；若使用GPS的RTK模式，需要同时获取卫星信息和地面的参考定位信息，需要在道路沿线布设参考定位通信设备，设备成本以及使用成本较高；当车辆进入卫星信号较差路段例如密林，隧道内时，GPS信号容易丢失，从而失去定位信息。
高精地图匹配定位一般采用点云数据匹配或立体视觉匹配，需要预先建立地图数据并存储于车辆上，车辆运行时，通过外部激光雷达或摄像装置来获取车辆当前环境的点云数据或图像数据，并与预先存储的地图数据进行匹配。该定位方法地图制作成本以及匹配计算的软硬件成本较高。
因此，需要一种低成本、高精度的车辆定位方法，为车辆定位、进站路线规划以及速度控制提供可靠的数据支持。
发明内容
本发明提供了一种基于深度神经网络图像识别的车辆定位方法，以及该深度神经网络的训练方法。本发明通过增大训练样本量、优化神经网络参数，能够提高深度神经网络的训练精度，从而提高车辆的定位精度，且所需的设备成本和使用成本较低。
本发明第一方面提供了一种用于道路标识识别的深度神经网络训练方法，包括以下步骤：
(1)道路标识图形设置步骤：在站点进站方向的路面设置道路标识图形，所述道路标识图形的标识点距离所述站点进站方向边缘的距离为L；
(2)拍摄装置设置步骤：在车辆上安装拍摄装置，所述拍摄装置的光轴线与所述车辆车身的纵向对称中心线重合，所述拍摄装置的镜头光心到地面的距离为H；
(3)图像样本采集步骤：在不同光照或天气条件下，使用所述拍摄装置对所述道路标识图形进行拍摄，分别在所述车辆的前进方向以及与所述车辆前进方向垂直的方向上，改变所述拍摄装置的光轴与路面的夹角角度，使得所述拍摄装置在其光轴与路面夹角呈一定范围内每隔一定角度拍摄一张所述道路标识图形的图像样本；
(4)训练样本制作步骤：计算每张所述图像样本中所述道路标识图形的标识点在图像坐标系中的位置坐标，制作成标签集，并将每张所述图像样本与相应的标签集配对，形成训练样本；
(5)深度神经网络搭建步骤：在目标识别分类深度神经网络的基础上，将所述网络最后的分类输出层修改为2个节点构成的输出层，以输出所述道路标识图形的标识点位置坐标；
(6)深度神经网络训练步骤：将所述训练样本输入到所述深度神经网络进行训练。
优选地，所述图像样本采集步骤中，拍摄时间选择在晴天正午和晴天夜晚。
优选地，所述图像样本采集步骤中，拍摄时间选择在雨天正午和雨天夜晚。
优选地，所述图像样本采集步骤中，拍摄时间选择在雾天正午和雾天夜晚。
优选地，所述图像样本采集步骤中，所述拍摄装置在其光轴与路面夹角呈5°到180°的范围内每隔5°拍摄一张所述道路标识图形的图像样本。
优选地，所述拍摄装置的镜头参数选择为，当所述道路标识图形全部出现在镜头画面中时，所述道路标识图形能够占据所述镜头画面20％以上的面积。
优选地，所述拍摄装置安装于所述车辆前部车顶位置，并指向所述车辆前进方向。
优选地，所述道路标识图形采用三角形，或矩形，或圆弧，或其他易于识别的几何元素组合。
优选地，所述道路标识图形采用条形码或二维码。
优选地，所述道路标识图形的标识点为其几何中心。
优选地，所述深度神经网络采用ResNet50网络，将所述网络最后的分类输出层替换为两层具有1024个节点的全连接层，所述全连接层后连接具有2个节点输出的输出层。
优选地，所述深度神经网络采用ResNet50网络，将所述网络最后的分类输出层替换为两层具有2048个节点的全连接层，所述全连接层后连接具有2个节点输出的输出层。
优选地，所述2个节点输出的浮点数据属于[0，1]的闭区间，将所述输出浮点数据与相应的图像宽度和高度相乘即可获得像素坐标。
本发明第二方面提供了一种使用上述深度神经网络训练方法的车辆定位方法，包括以下步骤：
(1)道路标识图形识别步骤：利用训练完成后的所述深度神经网络，对所述车辆实际进站过程中拍摄到的所述道路标识图形进行识别并获取其标识点P在图像坐标系中的位置坐标(u，v)；
(2)道路标识图形定位步骤：通过图像坐标系与世界坐标系的变换关系，计算出所述道路标识图形标识点P在世界坐标系中的坐标(X w，Y w，Z w)，从而获得所述道路标识图形标识点P与所述拍摄装置的距离；
(3)车辆定位步骤：根据所获得的道路标识图形标识点P与所述拍摄装置的距离，确定所述拍摄装置与站点进站方向边缘的距离，再结合所述拍摄装置在所述车辆上的安装位置，确定所述车辆与站点进站方向边缘的距离。
优选地，所述道路标识图形的标识点P在所述拍摄装置镜头光心轴线上；
所述拍摄装置坐标系原点设定在所述拍摄装置成像小孔位置，所述拍摄装置镜头光心与所述道路标识图形标识点P的水平距离为Z C；
所述拍摄装置坐标系的Z轴正向选取为所述车辆前进方向，所述拍摄装置坐标系的Y轴正向选取为所述车辆向下方向，所述拍摄装置坐标系的X轴正向选取为所述车辆向右方向；
所述世界坐标系与所述拍摄装置坐标系重合；
所述图像坐标系的原点在所述拍摄装置坐标系Z轴上，所述图像坐标系的X轴和Y轴分别与所述拍摄装置坐标系的X轴和Y轴平行；
根据公式：
Figure PCTCN2019111840-appb-000001
得到所述拍摄装置与所述道路标识图形标识点Pi的水平距离Z C；
再根据公式：
L cz＝Z c+L
得到所述拍摄装置与站点进站方向边缘的水平距离L CZ。
本发明的优点在于：
(1)图像样本的采集过程在不同光照及天气条件下的多个时段内进行，减小了环境因素对训练结果的影响，提高了深度神经网络的环境适应性。
(2)在车辆前进方向和与前进方向垂直的方向上每隔一定角度拍摄一张采样图像，获得的训练样本数据量大，提高了深度神经网络的训练精度，从而提高了后续车辆定位的精度。
(3)利用训练好的深度神经网络来识别设置于站点前方的道路标识图形，并通过图像坐标系与世界坐标系的转换，推算出车载拍摄装置的空间位置进而定位车辆位置。上述方法能够提供车辆与站点之间的距离数据，为车辆定位、进站路线规划以及速度控制提供数据支持，具有操作简单、成本较低、可靠性高的优势。
附图说明
本发明的以上内容以及下面的具体实施方式在结合附图阅读时会得到更好的理解。需要说明的是，附图仅作为所请求保护的发明的示例。在附图中，相同的附图标记代表相同或类似的元素。
图1为用于道路标识识别的深度神经网络训练方法的流程图；
图2为图像样本采集步骤中的拍摄示意图；
图3为一种基于训练完成后的深度神经网络的车辆定位方法的流程图；
图4为车辆进站过程中的车辆侧视图；
图5为车辆进站过程中的车辆俯视图；
图6为车辆与站点进站方向边缘距离计算示意图。
具体实施方式
以下结合附图和实施例对本发明作进一步的详细说明。
图1为本发明提供的一种用于道路标识识别的深度神经网络训练方法的流程图，包括道路标识图形设置步骤101、拍摄装置设置步骤102、图像样本采集步骤103、训练样本制作步骤104、深度神经网络搭建步骤105、深度神经网络训练步骤106。
道路标识图形设置步骤101：在站点进站方向的路面设置道路标识图形，该道路标识图形的标识点距离站点进站方向边缘的距离为L。该道路标识图形可采用但不限于三角形、矩形、圆弧，或其他易于识别的几何元素组合，或文字图形，或融入了车站相关信息的条形码或二维码。该道路标识图形的标识点可以是该道路标识图形的几何中心点、顶点或其他几何特征点。
拍摄装置设置步骤102：在车辆上安装拍摄装置，该拍摄装置的镜头指向车辆前进方向，安装在车辆前部车顶位置或其他能够拍摄到道路标识图形的位置，并使拍摄装置的光轴线与车辆车身的纵向对称中心线重合，记录该拍摄装置的镜头光心到地面的距离H。该拍摄装置的镜头参数选择为，当前述道路标识图形全部出现在镜头画面中时，道路标识图形能够占据镜头画面20％以上的面积，占据的面积越大，道路标识图形的标识点在画面中的定位就越精确。
图像样本采集步骤103：在不同光照或天气条件下，例如晴天正午和晴天夜晚，雨天正午和雨天夜晚，雾天正午和雾天夜晚，使用上述拍摄装置对道路标识图形进行拍摄。拍摄角度如图2所示，其中字母A表示道路标识图形，分别在车辆的前进方向以及与车辆前进方向垂直的方向上，改变拍摄装置的光轴与路面的夹角角度，使得拍摄装置在其光轴与路面夹角呈5°到180°的范围内每隔5°拍摄一张道路标识图形的图像样本。
训练样本制作步骤104：计算每张图像样本中道路标识图形的标识点在图像坐标系中的位置坐标，制作成标签集，并将每张图像样本与相应的标签集配对，形成训练样本，以便后续输入深度神经网络进行训练。
深度神经网络搭建步骤105：使用一目标识别分类深度神经网络，但将网络最后的分类输出层修改成两个节点构成的输出层，这两个节点输出的数值即为道路标识图形的标识点在图像画面中的坐标。更具体地，可采用ResNet50网络，并将最后的分类输出层去除，根据所需的识别效果，使用两层具有1024个节点或2048个节点的全连接层代替，全连接层后连接具有2个节点输出的输出层，这两个节点输出的浮点数据属于[0，1]的闭区间，将输出的浮点数据乘以各自对应的图像宽度和高度即可获得像素坐标。
深度神经网络训练步骤106：将前述训练样本输入到优化后的深度神经网络进行训练，训练完成后，即可使用该深度神经网络对道路标识图形进行识别并获取其几何中心位置坐标。
图3为本发明提供的一种使用上述深度神经网络训练方法的车辆定位方法的流程图，包括道路标识图形识别步骤201、道路标识图形定位步骤202、车辆定位步骤203。
道路标识图形识别步骤201：利用训练完成后的深度神经网络，对车辆实际进站过程中拍摄到的道路标识图形进行识别并获取其标识点P在图像坐标系中的位置坐标(u，v)。
道路标识图形定位步骤202：通过图像坐标系与世界坐标系的变换关系，计算出道路标识图形标识点P在世界坐标系中的坐标(X w，Y w，Z w)，从而获得道路标识图形标识点P与拍摄装置的距离。上述图像坐标系与世界坐标系的变换可采用小孔成像模型来描述。世界坐标系中的一点P w，其在世界坐标系中的坐标为(X w，Y w，Z w)，通过镜头成像到二维图像坐标系中的P i点，其坐标为(u，v)，则P w与P i点的坐标可以使用公式(1)进行换算：
Figure PCTCN2019111840-appb-000002
(1)式中，Z C表示道路标识图形标识点P与拍摄装置镜头光心的水平距离；d x，d y，u 0，v 0，f是与拍摄装置镜头有关的镜头内部参数，具体为：
d x，d y分别表示图像坐标系X方向和Y方向单位像素的物理长度；u 0，v 0分别表示图像坐标系中，图像坐标系原点与拍摄装置坐标系原点在X方向和Y方向的偏移；f表示镜头成像焦距。
(1)式中，R表示世界坐标系与拍摄装置坐标系的旋转关系，采用(2)式计算：
Figure PCTCN2019111840-appb-000003
其中α、β、γ分别表示世界坐标系与拍摄装置坐标系重合时，需要围绕X轴、Y轴、Z轴转动的角度。
(1)式中，T表示世界坐标系与拍摄装置坐标系的平移关系，采用(3)式计算：
T＝[t x t y t z] T       (3)
其中t x、t y、t z分别表示世界坐标系与拍摄装置坐标系在X轴、Y轴、Z轴的平移量。
具体实施时，以上参数d x，d y，u 0，v 0，f，α，β，γ，t x，t y，t z可以采用但不限于以下所描述的情况进行标定。
如图4、图5所示，拍摄装置安装于车辆前部车顶位置，并指向前进方向。拍摄装置的镜头光心轴线与车辆纵轴几何对称中心线重合，车辆前方路面的道路标识图形的标识点P在拍摄装置镜头光心轴线上，其与拍摄装置镜头光心的水平距离为Z C。将拍摄装置坐标系原点设定在拍摄装置成像小孔位置。为简化计算，假设世界坐标系与拍摄装置坐标系重合，且选取车辆前进方向为Z轴正向，车辆向下为Y轴正方向，车辆向右为X轴正向。图像坐标系的原点在拍摄装置坐标系的Z轴上，图像坐标系的X轴和Y轴分别与拍摄装置坐标系的X轴和Y轴平行。
根据以上条件，可知P点在世界坐标系中的坐标为(X w，Y w，Z w)，在拍摄装置坐标系中的坐标为(X c，Y c，Z c)，并且X w＝X c＝0，Y w＝H，Z w＝Z c。P点经过拍摄装置小孔成像后，在图像坐标系中的坐标为(u，v)，且u＝0，u 0＝0，v 0＝0。拍摄装置坐标系与世界坐标系的平移参数t x＝t y＝t z＝0。从而式(1)可以化简为：
Figure PCTCN2019111840-appb-000004
即
Figure PCTCN2019111840-appb-000005
车辆定位步骤203：如图6所示，获得道路标识图形的标识点P距离拍摄装置镜头光心的水平距离Z C后，再结合道路标识图形的标识点P距离站点进站方向边缘的距离L，可以计算得到拍摄装置镜头光心与站点进站方向边缘的水平距离L CZ：
L cz＝Z c+L       (6)
再结合该拍摄装置在车辆上的安装位置，确定车辆与站点进站方向边缘的距离，从而实现车辆定位。
这里基于的术语和表述方式只是用于描述，本发明并不应局限于这些术语和表述。使用这些术语和表述并不意味着排除任何示意和描述(或其中部分)的等效特征，应认 识到可能存在的各种修改也应包含在权利要求范围内。其他修改、变化和替换也可能存在。相应的，权利要求应视为覆盖所有这些等效物。
同样，需要指出的是，虽然本发明已参照当前的具体实施例来描述，但是本技术领域中的普通技术人员应当认识到，以上的实施例仅是用来说明本发明，在没有脱离本发明精神的情况下还可做出各种等效的变化或替换，因此，只要在本发明的实质精神范围内对上述实施例的变化、变型都将落在本申请的权利要求书的范围内。

Claims (15)

  1. 一种用于道路标识识别的深度神经网络训练方法，其特征在于，所述方法包括：
    道路标识图形设置步骤：在站点进站方向的路面设置道路标识图形，所述道路标识图形的标识点距离所述站点进站方向边缘的距离为L；
    拍摄装置设置步骤：在车辆上安装拍摄装置，所述拍摄装置的光轴线与所述车辆车身的纵向对称中心线重合，所述拍摄装置的镜头光心到地面的距离为H；
    图像样本采集步骤：在不同光照或天气条件下，使用所述拍摄装置对所述道路标识图形进行拍摄，分别在所述车辆的前进方向以及与所述车辆前进方向垂直的方向上，改变所述拍摄装置的光轴与路面的夹角角度，使得所述拍摄装置在其光轴与路面夹角呈一定范围内每隔一定角度拍摄一张所述道路标识图形的图像样本；
    训练样本制作步骤：计算每张所述图像样本中所述道路标识图形的标识点在图像坐标系中的位置坐标，制作成标签集，并将每张所述图像样本与相应的标签集配对，形成训练样本；
    深度神经网络搭建步骤：在目标识别分类深度神经网络的基础上，将所述网络最后的分类输出层修改为2个节点构成的输出层，以输出所述道路标识图形的标识点位置坐标；
    深度神经网络训练步骤：将所述训练样本输入到所述深度神经网络进行训练。
  2. 根据权利要求1所述的一种用于道路标识识别的深度神经网络训练方法，其特征在于，所述图像样本采集步骤中，拍摄时间选择在晴天正午和晴天夜晚。
  3. 根据权利要求1所述的一种用于道路标识识别的深度神经网络训练方法，其特征在于，所述图像样本采集步骤中，拍摄时间选择在雨天正午和雨天夜晚。
  4. 根据权利要求1所述的一种用于道路标识识别的深度神经网络训练方法，其特征在于，所述图像样本采集步骤中，拍摄时间选择在雾天正午和雾天夜晚。
  5. 根据权利要求1～4所述的一种用于道路标识识别的深度神经网络训练方法，其特征在于，所述图像样本采集步骤中，所述拍摄装置在其光轴与路面夹角呈5°到180°的范围内每隔5°拍摄一张所述道路标识图形的图像样本。
  6. 根据权利要求1所述的一种用于道路标识识别的深度神经网络训练方法，其特征在于，所述拍摄装置的镜头参数选择为，当所述道路标识图形全部出现在镜头画面中时，所述道路标识图形能够占据所述镜头画面20％以上的面积。
  7. 根据权利要求6所述的一种用于道路标识识别的深度神经网络训练方法，其特征在于，所述拍摄装置安装于所述车辆前部车顶位置，并指向所述车辆前进方向。
  8. 根据权利要求1所述的一种用于道路标识识别的深度神经网络训练方法，其特征在于，所述道路标识图形采用三角形，或矩形，或圆弧，或其他易于识别的几何元素组合。
  9. 根据权利要求1所述的一种用于道路标识识别的深度神经网络训练方法，其特征在于，所述道路标识图形采用条形码或二维码。
  10. 根据权利要求1所述的一种用于道路标识识别的深度神经网络训练方法，其特征在于，所述道路标识图形的标识点为其几何中心。
  11. 根据权利要求1所述的一种用于道路标识识别的深度神经网络训练方法，其特征在于，所述深度神经网络采用ResNet50网络，将所述网络最后的分类输出层替换为两层具有1024个节点的全连接层，所述全连接层后连接具有2个节点输出的输出层。
  12. 根据权利要求1所述的一种用于道路标识识别的深度神经网络训练方法，其特征在于，所述深度神经网络采用ResNet50网络，将所述网络最后的分类输出层替换为两层具有2048个节点的全连接层，所述全连接层后连接具有2个节点输出的输出层。
  13. 根据权利要求11～12所述的一种用于道路标识识别的深度神经网络训练方法，其特征在于，所述2个节点输出的浮点数据属于[0，1]的闭区间，将所述输出浮点数据与相应的图像宽度和高度相乘即可获得像素坐标。
  14. 一种使用权利要求1所述的深度神经网络训练方法的车辆定位方法，其特征在于，所述方法包括：
    道路标识图形识别步骤：利用训练完成后的所述深度神经网络，对所述车辆实际进站过程中拍摄到的所述道路标识图形进行识别并获取其标识点P在图像坐标系中的位置坐标(u，v)；
    道路标识图形定位步骤：通过图像坐标系与世界坐标系的变换关系，计算出所述道路标识图形标识点P在世界坐标系中的坐标(X w，Y w，Z w)，从而获得所述道路标识图形标识点P与所述拍摄装置的距离；
    车辆定位步骤：根据所获得的道路标识图形标识点P与所述拍摄装置的距离，确定所述拍摄装置与站点进站方向边缘的距离，再结合所述拍摄装置在所述车辆上的安装位置，确定所述车辆与站点进站方向边缘的距离。
  15. 根据权利要求14所述的一种车辆定位方法，其特征在于，所述道路标识图形的标识点P在所述拍摄装置镜头光心轴线上；
    所述拍摄装置坐标系原点设定在所述拍摄装置成像小孔位置，所述拍摄装置镜头光心与所述道路标识图形标识点P的水平距离为Z C；
    所述拍摄装置坐标系的Z轴正向选取为所述车辆前进方向，所述拍摄装置坐标系的Y轴正向选取为所述车辆向下方向，所述拍摄装置坐标系的X轴正向选取为所述车辆向右方向；
    所述世界坐标系与所述拍摄装置坐标系重合；
    所述图像坐标系的原点在所述拍摄装置坐标系Z轴上，所述图像坐标系的X轴和Y轴分别与所述拍摄装置坐标系的X轴和Y轴平行；
    根据公式：
    Figure PCTCN2019111840-appb-100001
    得到所述拍摄装置与所述道路标识图形标识点P的水平距离Z C；
    再根据公式：
    L cz＝Z c+L
    得到所述拍摄装置与站点进站方向边缘的水平距离L CZ。
PCT/CN2019/111840 2018-10-24 2019-10-18 一种基于深度神经网络图像识别的车辆定位方法 WO2020083103A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
SG11202103814PA SG11202103814PA (en) 2018-10-24 2019-10-18 Vehicle positioning method based on deep neural network image recognition

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201811245274.1 2018-10-24
CN201811245274.1A CN109446973B (zh) 2018-10-24 2018-10-24 一种基于深度神经网络图像识别的车辆定位方法

Publications (1)

Publication Number Publication Date
WO2020083103A1 true WO2020083103A1 (zh) 2020-04-30

Family

ID=65547888

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2019/111840 WO2020083103A1 (zh) 2018-10-24 2019-10-18 一种基于深度神经网络图像识别的车辆定位方法

Country Status (3)

Country Link
CN (1) CN109446973B (zh)
SG (1) SG11202103814PA (zh)
WO (1) WO2020083103A1 (zh)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111914691A (zh) * 2020-07-15 2020-11-10 北京埃福瑞科技有限公司 一种轨道交通车辆定位方法及***
CN113378735A (zh) * 2021-06-18 2021-09-10 北京东土科技股份有限公司 一种道路标识线识别方法、装置、电子设备及存储介质

Families Citing this family (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109446973B (zh) * 2018-10-24 2021-01-22 中车株洲电力机车研究所有限公司 一种基于深度神经网络图像识别的车辆定位方法
CN110726414B (zh) * 2019-10-25 2021-07-27 百度在线网络技术（北京）有限公司 用于输出信息的方法和装置
CN111161227B (zh) * 2019-12-20 2022-09-06 成都数之联科技股份有限公司 一种基于深度神经网络的靶心定位方法及***
CN113496594A (zh) * 2020-04-03 2021-10-12 郑州宇通客车股份有限公司 一种公交车进站控制方法、装置及***
CN112699823A (zh) * 2021-01-05 2021-04-23 浙江得图网络有限公司 一种用于共享电动车的定点还车方法
CN112950922B (zh) * 2021-01-26 2022-06-10 浙江得图网络有限公司 一种共享电动车的定点还车方法
CN117256009A (zh) * 2021-08-19 2023-12-19 浙江吉利控股集团有限公司 基于环境匹配的车辆定位方法、装置、车辆及存储介质

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN202350794U (zh) * 2011-11-29 2012-07-25 高德软件有限公司 一种导航数据采集装置
CN103925927A (zh) * 2014-04-18 2014-07-16 中国科学院软件研究所 一种基于车载视频的交通标识定位方法
CN108009518A (zh) * 2017-12-19 2018-05-08 大连理工大学 一种基于快速二分卷积神经网络的层次化交通标识识别方法
US20180211120A1 (en) * 2017-01-25 2018-07-26 Ford Global Technologies, Llc Training An Automatic Traffic Light Detection Model Using Simulated Images
CN109446973A (zh) * 2018-10-24 2019-03-08 中车株洲电力机车研究所有限公司 一种基于深度神经网络图像识别的车辆定位方法

Family Cites Families (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9940553B2 (en) * 2013-02-22 2018-04-10 Microsoft Technology Licensing, Llc Camera/object pose from predicted coordinates
CN105718860B (zh) * 2016-01-15 2019-09-10 武汉光庭科技有限公司 基于驾驶安全地图及双目交通标志识别的定位方法及***
US9773196B2 (en) * 2016-01-25 2017-09-26 Adobe Systems Incorporated Utilizing deep learning for automatic digital image segmentation and stylization
CN106326858A (zh) * 2016-08-23 2017-01-11 北京航空航天大学 一种基于深度学习的公路交通标志自动识别与管理***
CN106403926B (zh) * 2016-08-30 2020-09-11 上海擎朗智能科技有限公司 一种定位方法和***
CN106845547B (zh) * 2017-01-23 2018-08-14 重庆邮电大学 一种基于摄像头的智能汽车定位与道路标识识别***及方法
CN107563419B (zh) * 2017-08-22 2020-09-04 交控科技股份有限公司 图像匹配和二维码相结合的列车定位方法
CN107703936A (zh) * 2017-09-22 2018-02-16 南京轻力舟智能科技有限公司 基于卷积神经网络的自动导航小车***及小车定位方法

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN202350794U (zh) * 2011-11-29 2012-07-25 高德软件有限公司 一种导航数据采集装置
CN103925927A (zh) * 2014-04-18 2014-07-16 中国科学院软件研究所 一种基于车载视频的交通标识定位方法
US20180211120A1 (en) * 2017-01-25 2018-07-26 Ford Global Technologies, Llc Training An Automatic Traffic Light Detection Model Using Simulated Images
CN108009518A (zh) * 2017-12-19 2018-05-08 大连理工大学 一种基于快速二分卷积神经网络的层次化交通标识识别方法
CN109446973A (zh) * 2018-10-24 2019-03-08 中车株洲电力机车研究所有限公司 一种基于深度神经网络图像识别的车辆定位方法

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111914691A (zh) * 2020-07-15 2020-11-10 北京埃福瑞科技有限公司 一种轨道交通车辆定位方法及***
CN111914691B (zh) * 2020-07-15 2024-03-19 北京埃福瑞科技有限公司 一种轨道交通车辆定位方法及***
CN113378735A (zh) * 2021-06-18 2021-09-10 北京东土科技股份有限公司 一种道路标识线识别方法、装置、电子设备及存储介质
CN113378735B (zh) * 2021-06-18 2023-04-07 北京东土科技股份有限公司 一种道路标识线识别方法、装置、电子设备及存储介质

Also Published As

Publication number Publication date
CN109446973A (zh) 2019-03-08
CN109446973B (zh) 2021-01-22
SG11202103814PA (en) 2021-05-28

Similar Documents

Publication Publication Date Title
WO2020083103A1 (zh) 一种基于深度神经网络图像识别的车辆定位方法
CN109945858B (zh) 用于低速泊车驾驶场景的多传感融合定位方法
CN106651953B (zh) 一种基于交通指示牌的车辆位姿估计方法
CN108802785B (zh) 基于高精度矢量地图和单目视觉传感器的车辆自定位方法
CN106441319B (zh) 一种无人驾驶车辆车道级导航地图的生成***及方法
US10860871B2 (en) Integrated sensor calibration in natural scenes
CN108256413B (zh) 可通行区域检测方法及装置、存储介质、电子设备
US10127461B2 (en) Visual odometry for low illumination conditions using fixed light sources
CN106525057A (zh) 高精度道路地图的生成***
US11625851B2 (en) Geographic object detection apparatus and geographic object detection method
CN109815300B (zh) 一种车辆定位方法
CN105930819A (zh) 基于单目视觉和gps组合导航***的实时城区交通灯识别***
JP2021508815A (ja) 妨害物体の検出に基づいて高精細度マップを補正するためのシステムおよび方法
CN109212545A (zh) 基于主动视觉的多信源目标跟踪测量***及跟踪方法
CN109583409A (zh) 一种面向认知地图的智能车定位方法及***
CN112740225B (zh) 一种路面要素确定方法及装置
WO2022041706A1 (zh) 一种定位方法、定位***和车辆
CN110018503B (zh) 车辆的定位方法及定位***
CN112446915B (zh) 一种基于图像组的建图方法及装置
CN113673386A (zh) 一种交通信号灯在先验地图中的标注方法
CN112444251B (zh) 车辆行车位置确定方法、装置、存储介质及计算机设备
CN110135387B (zh) 一种基于传感器融合的图像快速识别方法
Lin et al. Semi-automatic road tracking by template matching and distance transform
CN115127547B (zh) 一种基于捷联惯导***和图像定位的隧道检测车定位方法
CN116630559A (zh) 一种轻量级道路语义地图的构建方法

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19876299

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 19876299

Country of ref document: EP

Kind code of ref document: A1