TWI764542B - Autonomous intelligent vehicle real-time visual localization and uncertainty estimation system - Google Patents

Autonomous intelligent vehicle real-time visual localization and uncertainty estimation system

Info

Publication number
TWI764542B
TWI764542B TW110103077A TW110103077A TWI764542B TW I764542 B TWI764542 B TW I764542B TW 110103077 A TW110103077 A TW 110103077A TW 110103077 A TW110103077 A TW 110103077A TW I764542 B TWI764542 B TW I764542B
Authority
TW
Taiwan
Prior art keywords
positioning
intelligent vehicle
real
estimation system
time visual
Prior art date
Application number
TW110103077A
Other languages
Chinese (zh)
Other versions
TW202230202A (en
Inventor
李綱
陳俊翰
陳俊儒
Original Assignee
國立臺灣大學
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 國立臺灣大學 filed Critical 國立臺灣大學
Priority to TW110103077A priority Critical patent/TWI764542B/en
Application granted granted Critical
Publication of TWI764542B publication Critical patent/TWI764542B/en
Publication of TW202230202A publication Critical patent/TW202230202A/en

Links

Images

Landscapes

  • Navigation (AREA)
  • Image Analysis (AREA)

Abstract

Disclosed is an autonomous intelligent vehicle real-time visual localization and uncertainty estimation system, comprising: an image inputting device, a feature extracting device, and a localizing and estimating device. The autonomous intelligent vehicle real-time visual localization and uncertainty estimation system can obtain initial position information of an autonomous intelligent vehicle without relying on previous position information of the autonomous intelligent vehicle so as to solve the lost, kidnapped, and initial position problems.

Description

智慧型載具之實時視覺定位與不確定性估測系統Real-time Visual Positioning and Uncertainty Estimation System for Intelligent Vehicles

本發明相關於一種智慧型載具之定位系統，特別是相關於一種智慧型載具之實時視覺定位與不確定性估測系統。The present invention relates to a positioning system for an intelligent vehicle, in particular to a real-time visual positioning and uncertainty estimation system for an intelligent vehicle.

智慧型載具可包括自動駕駛汽車（self-driving cars）及自主移動機器人（autonomous mobile robots；AMR）的二個主要類別，兩者在實現上都需要實時、準確且高強健性（robust）的定位系統。一般而言，自主移動機器人之定位通常依賴二維光學雷達，但是在許多情況下，例如走廊之類的無特徵（或稱特徵稀疏；featureless）環境下，二維光學雷達無法獲得足夠的特徵或地標進行定位。自動駕駛汽車則通常是利用三維光學雷達或全球定位系統（Global Positioning System；GPS）／衛星導航系統（Global Navigation Satellite System；GNSS）來定位，雖然此種基於三維光學雷達的定位算法可提供準確的定位結果，但在實際應用中會遇到初始定位（initial pose）問題、綁架（kidnapped）問題、高計算成本及高成本。Intelligent vehicles can include two main categories of self-driving cars (self-driving cars) and autonomous mobile robots (AMRs), both of which require real-time, accurate, and robust GPS. Generally speaking, the localization of autonomous mobile robots usually relies on 2D lidar, but in many cases, such as corridors and other featureless (or featureless; featureless) environments, 2D lidar cannot obtain enough features or Placemarks for positioning. Autonomous vehicles are usually positioned using 3D LiDAR or Global Positioning System (GPS)/Global Navigation Satellite System (GNSS), although such 3D LiDAR-based positioning algorithms can provide accurate Positioning results, but in practical applications will encounter the initial pose problem, kidnapped problem, high computational cost and high cost.

因此，本發明的目的即在提供一種智慧型載具之實時視覺定位與不確定性估測系統，以在達到低成本、可靠、實時且準確的定位的同時，解決初始定位及綁架等的習知技術之問題。Therefore, the purpose of the present invention is to provide a real-time visual positioning and uncertainty estimation system for an intelligent vehicle, so as to achieve low-cost, reliable, real-time and accurate positioning, while solving the conventional problems of initial positioning and kidnapping. know technical issues.

本發明為解決習知技術之問題所採用之技術手段係提供一種智慧型載具之實時視覺定位與不確定性估測系統，係以不依靠一智慧型載具的一前時刻位置資訊的方式定位得出該智慧型載具的一初始定位位置資訊，該實時視覺定位與不確定性估測系統包含：一圖像輸入裝置，設置於該智慧型載具，且係經配置而取得在預定的一定位環境中的關於該智慧型載具之一當前視覺圖像資訊；一特徵擷取裝置，連接於該圖像輸入裝置，該特徵擷取裝置係經配置而以一MobileNetV2電腦視覺神經網路模型對於該當前視覺圖像資訊進行一圖像特徵擷取運算；以及一定位估測裝置，連接於該特徵擷取裝置，該定位估測裝置係經配置而以包含有全域平均池化層、丟棄層、及全連接層的一深度學習迴歸演算模組對於經過該圖像特徵擷取運算的該當前視覺圖像資訊予以執行一迴歸分析，而運算輸出關於該智慧型載具的一當前定位位置資訊作為該初始定位位置資訊，並且估測輸出關於該當前定位位置資訊的信賴程度的一定位不確定性估測資訊。The technical means adopted by the present invention to solve the problems of the prior art is to provide a real-time visual positioning and uncertainty estimation system for an intelligent vehicle in a manner that does not rely on the previous position information of an intelligent vehicle An initial positioning position information of the intelligent vehicle is obtained by positioning, and the real-time visual positioning and uncertainty estimation system includes: an image input device, which is arranged on the intelligent vehicle and is configured to obtain the predetermined position. a current visual image information about the intelligent vehicle in a positioning environment; a feature extraction device connected to the image input device, the feature extraction device is configured to use a MobileNetV2 computer vision neural network the road model performs an image feature extraction operation on the current visual image information; and a location estimation device connected to the feature extraction device, the location estimation device is configured to include a global average pooling layer , a drop layer, and a deep learning regression algorithm module of the fully connected layer perform a regression analysis on the current visual image information subjected to the image feature extraction operation, and the operation outputs a current information about the intelligent vehicle The positioning position information is used as the initial positioning position information, and a positioning uncertainty estimation information about the reliability of the current positioning position information is estimated and output.

在本發明的一實施例中係提供一種智慧型載具之實時視覺定位與不確定性估測系統，其中該圖像輸入裝置係為一相機。In an embodiment of the present invention, a real-time visual positioning and uncertainty estimation system for an intelligent vehicle is provided, wherein the image input device is a camera.

在本發明的一實施例中係提供一種智慧型載具之實時視覺定位與不確定性估測系統，其中該圖像輸入裝置係為一單眼相機。In an embodiment of the present invention, a real-time visual positioning and uncertainty estimation system for an intelligent vehicle is provided, wherein the image input device is a monocular camera.

在本發明的一實施例中係提供一種智慧型載具之實時視覺定位與不確定性估測系統，其中該定位環境係對於二維光學雷達為一無特徵環境。In an embodiment of the present invention, a real-time visual positioning and uncertainty estimation system for an intelligent vehicle is provided, wherein the positioning environment is a featureless environment for a two-dimensional optical radar.

在本發明的一實施例中係提供一種智慧型載具之實時視覺定位與不確定性估測系統，其中該當前視覺圖像資訊係為單一RGB圖像。In an embodiment of the present invention, a real-time visual positioning and uncertainty estimation system for an intelligent vehicle is provided, wherein the current visual image information is a single RGB image.

在本發明的一實施例中係提供一種智慧型載具之實時視覺定位與不確定性估測系統，其中該智慧型載具係為一自主移動機器人。In an embodiment of the present invention, a real-time visual positioning and uncertainty estimation system for an intelligent vehicle is provided, wherein the intelligent vehicle is an autonomous mobile robot.

經由本發明所採用之技術手段，本發明的智慧型載具之實時視覺定位與不確定性估測系統能夠在達到低成本、可靠、實時且準確的定位的同時，解決初始定位及綁架等問題。Through the technical means adopted in the present invention, the real-time visual positioning and uncertainty estimation system of the intelligent vehicle of the present invention can achieve low-cost, reliable, real-time and accurate positioning, and at the same time solve the problems of initial positioning and kidnapping. .

相較於以二維光學雷達所進行的定位在無特徵環境中無法獲得足夠的特徵或地標，本發明的智慧型載具之實時視覺定位與不確定性估測系統能夠利用圖像輸入裝置獲取多種特徵，例如：告示板、水管、燈，甚至是遠方的消失點（vanishing point），這些特徵是建築物與生俱來的，並且是有用的定位特徵，故能夠使定位性能提高。Compared with the positioning performed by the two-dimensional optical radar, which cannot obtain enough features or landmarks in a featureless environment, the real-time visual positioning and uncertainty estimation system of the intelligent vehicle of the present invention can be obtained by using an image input device. Various features, such as: notice boards, water pipes, lights, and even distant vanishing points, are inherent in buildings and are useful positioning features that can improve positioning performance.

相較於以三維光學雷達所進行的定位會遇到的初始定位問題、綁架問題、高計算成本及高成本，本發明的智慧型載具之實時視覺定位與不確定性估測系統能夠從圖像估測出智慧型載具的當前定位位置資訊及定位不確定性估測資訊，而無須進行額外的特徵工程或圖形最佳化。並且，本發明的智慧型載具之實時視覺定位與不確定性估測系統能夠在預定的定位環境中全局重新定位，即，在不依靠智慧型載具的前時刻位置資訊的前提下實現對於該智慧型載具的初始定位，從而解決迷路、綁架及初始定位問題。Compared with the initial positioning problem, kidnapping problem, high computational cost and high cost encountered in the positioning performed by the three-dimensional optical radar, the real-time visual positioning and uncertainty estimation system of the intelligent vehicle of the present invention can be obtained from the figure. Such as estimating the current positioning position information and positioning uncertainty estimation information of intelligent vehicles without additional feature engineering or graphics optimization. In addition, the real-time visual positioning and uncertainty estimation system of the intelligent vehicle of the present invention can reposition globally in a predetermined positioning environment, that is, without relying on the previous position information of the intelligent vehicle. The initial positioning of the intelligent vehicle, so as to solve the problems of getting lost, kidnapping and initial positioning.

以下根據第1圖至第5圖，而說明本發明的實施方式。該說明並非為限制本發明的實施方式，而為本發明之實施例的一種。Embodiments of the present invention will be described below with reference to FIGS. 1 to 5 . This description is not intended to limit the embodiments of the present invention, but is an example of the present invention.

如第1圖至第5圖所示，依據本發明的一實施例的一智慧型載具之實時視覺定位與不確定性估測系統100包含：一圖像輸入裝置1、一特徵擷取裝置2、及一定位估測裝置3。As shown in FIGS. 1 to 5, a real-time visual positioning and uncertainty estimation system 100 for an intelligent vehicle according to an embodiment of the present invention includes: an image input device 1, and a feature extraction device 2. and a positioning estimation device 3.

具體而言，在本發明中，該智慧型載具之實時視覺定位與不確定性估測系統100係以不依靠一智慧型載具（圖未示）的一前時刻位置資訊的方式定位得出該智慧型載具的一初始定位位置資訊。該智慧型載具在本實施例中為一自主移動機器人，但亦可為一自動駕駛汽車或其它智慧型載具。Specifically, in the present invention, the real-time visual positioning and uncertainty estimation system 100 of the intelligent vehicle is located in a way that does not rely on the previous position information of an intelligent vehicle (not shown). An initial positioning position information of the intelligent vehicle is output. The intelligent vehicle is an autonomous mobile robot in this embodiment, but can also be an autonomous vehicle or other intelligent vehicles.

如第1圖至第3圖所示，在本實施例的該智慧型載具之實時視覺定位與不確定性估測系統100中，該圖像輸入裝置1係設置於該智慧型載具，且係經配置而取得在預定的一定位環境E中的關於該智慧型載具之一當前視覺圖像資訊I0。As shown in FIG. 1 to FIG. 3, in the real-time visual positioning and uncertainty estimation system 100 of the intelligent vehicle of the present embodiment, the image input device 1 is disposed on the intelligent vehicle, and is configured to obtain current visual image information I0 about one of the intelligent vehicles in a predetermined positioning environment E.

具體而言，如第1圖所示，在本實施例中，該圖像輸入裝置1係為一相機，特別是一單眼相機（monocular camera）。該圖像輸入裝置1設置於該智慧型載具，故無論該智慧型載具被初始放置或是被綁架到預定的該定位環境E中的任何位置，該圖像輸入裝置1都能夠取得該智慧型載具在當前所在位置處的視覺圖像，即，該當前視覺圖像資訊I0。較佳地，該當前視覺圖像資訊I0係為單一RGB圖像，即，單張的RGB點陣圖。Specifically, as shown in FIG. 1 , in this embodiment, the image input device 1 is a camera, especially a monocular camera. The image input device 1 is set on the intelligent vehicle, so no matter the intelligent vehicle is initially placed or kidnapped to any position in the predetermined positioning environment E, the image input device 1 can obtain the The visual image of the intelligent vehicle at the current location, that is, the current visual image information I0. Preferably, the current visual image information I0 is a single RGB image, that is, a single RGB bitmap.

如第2圖及第3圖所示，在本實施例中，該定位環境E為一走廊，其對於二維光學雷達為一無特徵環境，即，二維光學雷達無法獲得足夠的特徵或地標以進行定位的環境。對於本發明的該智慧型載具之實時視覺定位與不確定性估測系統100而言，即使在這樣的定位環境中，該圖像輸入裝置1仍然能夠獲得具有各種有用的定位特徵的該當前視覺圖像資訊I0。如第3圖所示，在該定位環境E為一走廊時，定位特徵可以例如是消防栓、監視器、天花板燈、消失線（vanishing line）。As shown in Figures 2 and 3, in this embodiment, the positioning environment E is a corridor, which is a featureless environment for the 2D LiDAR, that is, the 2D LiDAR cannot obtain enough features or landmarks environment for positioning. For the real-time visual positioning and uncertainty estimation system 100 of the intelligent vehicle of the present invention, even in such a positioning environment, the image input device 1 can still obtain the current position with various useful positioning features. Visual image information I0. As shown in FIG. 3 , when the positioning environment E is a corridor, the positioning features may be, for example, fire hydrants, monitors, ceiling lights, and vanishing lines.

如第1圖及第4圖所示，在本實施例的該智慧型載具之實時視覺定位與不確定性估測系統100中，該特徵擷取裝置2係連接於該圖像輸入裝置1，該特徵擷取裝置2係經配置而以一MobileNetV2電腦視覺神經網路模型20對於該當前視覺圖像資訊I0進行一圖像特徵擷取運算。As shown in FIG. 1 and FIG. 4 , in the real-time visual positioning and uncertainty estimation system 100 of the intelligent vehicle of this embodiment, the feature extraction device 2 is connected to the image input device 1 , the feature extraction device 2 is configured to perform an image feature extraction operation on the current visual image information I0 with a MobileNetV2 computer vision neural network model 20 .

具體而言，該MobileNetV2電腦視覺神經網路模型20是由「Google」所推出的第二代行動裝置版電腦視覺神經網路模型，其架構如第4圖及下面的表1所示，其透過深度可分離卷積（depthwise separable convolution）的方式來達到壓縮模型的目的，以減少參數並提升運算速度，除此之外更具備層間的線性轉換方式（linear bottleneck）以及瓶頸（bottleneck）之間的捷徑連接（shortcut connections）的二項特性。另外，第4圖中的「ReLU6」是指限制最大輸出值為「6」的線性整流函數（Rectified Linear Unit；ReLU）。「卷積1×1」及「卷積3×3」則分別代表以1×1卷積核進行卷積運算，以及以3×3卷積核進行卷積運算，其中卷積核（Convolution Kernels）的功用主要是將圖像切割成許多小塊，以擷取出圖像的特徵資訊。關於該MobileNetV2電腦視覺神經網路模型20的細節技術為本發明所屬技術領域中具有通常知識者依據本領域通常知識並配合參照第4圖及表1所能瞭解，故僅簡要說明如上，不再作進一步贅述。Specifically, the MobileNetV2 computer vision neural network model 20 is a second-generation mobile device version computer vision neural network model launched by "Google", and its architecture is shown in Figure 4 and Table 1 below. Depthwise separable convolution is used to achieve the purpose of compressing the model to reduce parameters and improve operation speed. In addition, it also has a linear bottleneck between layers and a bottleneck between the bottlenecks. Binomial properties of shortcut connections. In addition, "ReLU6" in FIG. 4 refers to a linear rectification function (Rectified Linear Unit; ReLU) that restricts the maximum output value to "6". "Convolution 1×1" and "Convolution 3×3" respectively represent convolution operations with 1×1 convolution kernels and convolution operations with 3×3 convolution kernels. ) is mainly to cut the image into many small pieces to extract the characteristic information of the image. The detailed technology of the MobileNetV2 computer vision neural network model 20 can be understood by those with ordinary knowledge in the technical field to which the present invention belongs, based on the ordinary knowledge in the field and with reference to FIG. to elaborate further.

〔表1〕MobileNetV2電腦視覺神經網路模型結構 輸入 層（類型） 膨脹係數 輸出通道數 重覆次數 步長 224 2×3 二維卷積 3×3 - 32 1 2 112 2×32 瓶頸 1 16 1 1 112 2×16 瓶頸 6 24 2 2 56 2×24 瓶頸 6 32 3 2 28 2×32 瓶頸 6 64 4 2 14 2×64 瓶頸 6 96 3 1 14 2×96 瓶頸 6 160 3 2 7 2×160 瓶頸 6 320 1 1 7 2×320 二維卷積 1×1 - 120 1 1 7 2×1280 - - - - - [Table 1] MobileNetV2 computer vision neural network model structure enter layer (type) Coefficient of expansion Number of output channels repetitions step size 224 2 × 3 2D Convolution 3×3 - 32 1 2 112 2 × 32 bottleneck 1 16 1 1 112 2 × 16 bottleneck 6 twenty four 2 2 56 2 × 24 bottleneck 6 32 3 2 28 2 × 32 bottleneck 6 64 4 2 14 2 × 64 bottleneck 6 96 3 1 14 2 × 96 bottleneck 6 160 3 2 7 2 × 160 bottleneck 6 320 1 1 7 2 × 320 2D convolution 1×1 - 120 1 1 7 2 × 1280 - - - - -

如第1圖及第5圖所示，在本實施例的該智慧型載具之實時視覺定位與不確定性估測系統100中，該定位估測裝置3係連接於該特徵擷取裝置2，該定位估測裝置3係經配置而以包含有全域平均池化（global average pooling）層、丟棄（dropout）層、及全連接（fully connected）層的一深度學習迴歸演算模組30對於經過該圖像特徵擷取運算的該當前視覺圖像資訊I0予以執行一迴歸分析，而運算輸出關於該智慧型載具的一當前定位位置資訊I1作為該初始定位位置資訊，並且估測輸出關於該當前定位位置資訊I1的信賴程度的一定位不確定性估測資訊I2。As shown in FIG. 1 and FIG. 5 , in the real-time visual positioning and uncertainty estimation system 100 of the smart vehicle of this embodiment, the positioning estimation device 3 is connected to the feature extraction device 2 , the location estimation device 3 is configured to include a deep learning regression algorithm 30 including a global average pooling layer, a dropout layer, and a fully connected layer. The current visual image information I0 of the image feature extraction operation is subjected to a regression analysis, and the operation outputs a current positioning position information I1 about the intelligent vehicle as the initial positioning position information, and the estimated output is about the A positioning uncertainty estimation information I2 of the trust level of the current positioning position information I1.

具體而言，該定位估測裝置3的示意架構如第5圖所示，模型結構則如下面的表2所示。其中，全域平均池化層用於總和空間資訊並使輸入的空間資訊更為壯健（robust）。丟棄層不僅用於避免過適（overfitting），更用於估測深度學習視覺定位的不確定性（uncertainty），即，輸出該定位不確定性估測資訊I2。全連接層用於學習如何將特徵向量投射至姿態座標。該定位估測裝置3的最終輸出則為所估測出的姿態，即，該當前定位位置資訊I1。關於全域平均池化層、丟棄層、及全連接層的細節技術為本發明所屬技術領域中具有通常知識者依據本領域通常知識所能瞭解，故僅針對彼等在該定位估測裝置3中的作用簡要說明如上，不再作細節內容的贅述。Specifically, the schematic structure of the positioning estimation device 3 is shown in FIG. 5 , and the model structure is shown in Table 2 below. Among them, the global average pooling layer is used to sum up the spatial information and make the input spatial information more robust. The discarding layer is not only used to avoid overfitting, but also used to estimate the uncertainty of deep learning visual positioning, that is, to output the positioning uncertainty estimation information I2. Fully connected layers are used to learn how to project feature vectors to pose coordinates. The final output of the positioning estimation device 3 is the estimated attitude, that is, the current positioning position information I1. The detailed techniques of the global average pooling layer, the discarding layer, and the fully connected layer are known to those with ordinary knowledge in the technical field to which the present invention pertains according to the ordinary knowledge in the field, so they are only described in the positioning estimation device 3 for them. The role of the function is briefly described as above, and the details will not be repeated.

〔表2〕定位估測裝置的模型結構 輸入 層（類型） 輸出 7 2＊1280 全域平均池化 1×1×1280 1×1×1280 丟棄 1×1×1280 1×1×1280 全連接 1280×2048 1×1×2048 1×1×2048 全連接 2048×3 1×1×3 1×1×2048 全連接 2048×4 1×1×4 [Table 2] Model structure of the positioning estimation device enter layer (type) output 7 2 *1280 global average pooling 1×1×1280 1×1×1280 throw away 1×1×1280 1×1×1280 Fully connected 1280×2048 1×1×2048 1×1×2048 Fully connected 2048×3 1×1×3 1×1×2048 Fully connected 2048×4 1×1×4

藉由上述結構，本發明的該智慧型載具之實時視覺定位與不確定性估測系統100能夠在達到低成本、可靠、實時且準確的定位的同時，解決初始定位及綁架等問題。With the above structure, the real-time visual positioning and uncertainty estimation system 100 of the intelligent vehicle of the present invention can achieve low-cost, reliable, real-time and accurate positioning, and at the same time solve the problems of initial positioning and kidnapping.

具體而言，相較於以二維光學雷達所進行的定位在無特徵環境中無法獲得足夠的特徵或地標，本發明的該智慧型載具之實時視覺定位與不確定性估測系統100能夠利用該圖像輸入裝置1獲取多種特徵，例如：告示板、水管、燈，甚至是遠方的消失點（vanishing point），這些特徵是建築物與生俱來的，並且是有用的定位特徵，故能夠使定位性能提高。Specifically, the real-time visual positioning and uncertainty estimation system 100 of the intelligent vehicle of the present invention is capable of obtaining sufficient features or landmarks in a featureless environment compared to the positioning performed by the two-dimensional optical radar. Use the image input device 1 to obtain various features, such as: notice boards, water pipes, lights, and even distant vanishing points. These features are inherent in buildings and are useful positioning features, so The positioning performance can be improved.

再者，相較於以三維光學雷達所進行的定位會遇到的初始定位問題、綁架問題、高計算成本及高成本，本發明的該智慧型載具之實時視覺定位與不確定性估測系統100能夠從圖像（即，該當前視覺圖像資訊I0）估測出該智慧型載具的當前定位位置資訊I1及定位不確定性估測資訊I2，而無須進行額外的特徵工程或圖形最佳化。並且，本發明的該智慧型載具之實時視覺定位與不確定性估測系統100能夠在預定的定位環境E中全局重新定位，即，在不依靠該智慧型載具的前時刻位置資訊的前提下實現對於該智慧型載具的初始定位，從而解決迷路、綁架及初始定位問題。Furthermore, compared with the initial positioning problem, kidnapping problem, high computational cost and high cost encountered by the positioning performed by the three-dimensional optical radar, the real-time visual positioning and uncertainty estimation of the intelligent vehicle of the present invention The system 100 can estimate the current positioning position information I1 and positioning uncertainty estimation information I2 of the intelligent vehicle from the image (ie, the current visual image information I0 ) without additional feature engineering or graphics optimize. Moreover, the real-time visual positioning and uncertainty estimation system 100 of the intelligent vehicle of the present invention can reposition globally in the predetermined positioning environment E, that is, without relying on the previous position information of the intelligent vehicle Under the premise, the initial positioning of the intelligent vehicle can be realized, so as to solve the problems of getting lost, kidnapping and initial positioning.

以上之敘述以及說明僅為本發明之較佳實施例之說明，對於此項技術具有通常知識者當可依據以下所界定申請專利範圍以及上述之說明而作其他之修改，惟此些修改仍應是為本發明之發明精神而在本發明之權利範圍中。The above descriptions and descriptions are only descriptions of preferred embodiments of the present invention. Those with ordinary knowledge in the art can make other modifications according to the scope of the patent application defined below and the above descriptions, but these modifications should still be It is within the scope of the right of the present invention for the inventive spirit of the present invention.

100:智慧型載具之實時視覺定位與不確定性估測系統 1:圖像輸入裝置 2:特徵擷取裝置 20:MobileNetV2電腦視覺神經網路模型 3:定位估測裝置 30:深度學習迴歸演算模組 E:定位環境 I0:當前視覺圖像資訊 I1:當前定位位置資訊 I2:定位不確定性估測資訊100: Real-time Visual Positioning and Uncertainty Estimation System for Intelligent Vehicles 1: Image input device 2: Feature extraction device 20: MobileNetV2 computer vision neural network model 3: Positioning Estimation Device 30: Deep Learning Regression Calculus Module E: Positioning environment I0: Current visual image information I1: Current positioning location information I2: Positioning Uncertainty Estimation Information

［第1圖］為顯示根據本發明的一實施例的智慧型載具之實時視覺定位與不確定性估測系統的示意圖； ［第2圖］為顯示根據本發明的實施例的智慧型載具之實時視覺定位與不確定性估測系統的定位環境的示意圖； ［第3圖］為顯示根據本發明的實施例的智慧型載具之實時視覺定位與不確定性估測系統的圖像輸入裝置所取得的當前視覺圖像資訊的示意圖； ［第4圖］為顯示根據本發明的實施例的智慧型載具之實時視覺定位與不確定性估測系統的特徵擷取裝置的MobileNetV2電腦視覺神經網路模型的架構示意圖； ［第5圖］為顯示根據本發明的實施例的智慧型載具之實時視覺定位與不確定性估測系統的定位估測裝置的架構示意圖。 [FIG. 1] is a schematic diagram showing a real-time visual positioning and uncertainty estimation system of an intelligent vehicle according to an embodiment of the present invention; [Fig. 2] is a schematic diagram showing the positioning environment of the real-time visual positioning and uncertainty estimation system of the intelligent vehicle according to the embodiment of the present invention; [Fig. 3] is a schematic diagram showing the current visual image information obtained by the image input device of the real-time visual positioning and uncertainty estimation system of the intelligent vehicle according to the embodiment of the present invention; [FIG. 4] is a schematic diagram showing the architecture of the MobileNetV2 computer vision neural network model of the feature extraction device of the real-time visual positioning and uncertainty estimation system of the intelligent vehicle according to an embodiment of the present invention; [FIG. 5] is a schematic diagram showing the structure of a positioning estimation device of a real-time visual positioning and uncertainty estimation system for an intelligent vehicle according to an embodiment of the present invention.

100:智慧型載具之實時視覺定位與不確定性估測系統 100: Real-time Visual Positioning and Uncertainty Estimation System for Intelligent Vehicles

1:圖像輸入裝置 1: Image input device

2:特徵擷取裝置 2: Feature extraction device

20:MobileNetV2電腦視覺神經網路模型 20: MobileNetV2 computer vision neural network model

3:定位估測裝置 3: Positioning Estimation Device

30:深度學習迴歸演算模組 30: Deep Learning Regression Calculus Module

I0:當前視覺圖像資訊 I0: Current visual image information

I1:當前定位位置資訊 I1: Current positioning location information

I2:定位不確定性估測資訊 I2: Positioning Uncertainty Estimation Information

Claims (6)

一種智慧型載具之實時視覺定位與不確定性估測系統，係以不依靠一智慧型載具的一前時刻位置資訊的方式定位得出該智慧型載具的一初始定位位置資訊，該實時視覺定位與不確定性估測系統包含： 一圖像輸入裝置，設置於該智慧型載具，且係經配置而取得在預定的一定位環境中的關於該智慧型載具之一當前視覺圖像資訊； 一特徵擷取裝置，連接於該圖像輸入裝置，該特徵擷取裝置係經配置而以一MobileNetV2電腦視覺神經網路模型對於該當前視覺圖像資訊進行一圖像特徵擷取運算；以及 一定位估測裝置，連接於該特徵擷取裝置，該定位估測裝置係經配置而以包含有全域平均池化層、丟棄層、及全連接層的一深度學習迴歸演算模組對於經過該圖像特徵擷取運算的該當前視覺圖像資訊予以執行一迴歸分析，而運算輸出關於該智慧型載具的一當前定位位置資訊作為該初始定位位置資訊，並且估測輸出關於該當前定位位置資訊的信賴程度的一定位不確定性估測資訊。 A real-time visual positioning and uncertainty estimation system for an intelligent vehicle, which locates and obtains an initial positioning position information of the intelligent vehicle in a way that does not rely on the previous position information of the intelligent vehicle. The real-time visual positioning and uncertainty estimation system includes: an image input device disposed on the intelligent vehicle and configured to obtain a current visual image information about the intelligent vehicle in a predetermined positioning environment; a feature extraction device connected to the image input device, the feature extraction device is configured to perform an image feature extraction operation on the current visual image information with a MobileNetV2 computer vision neural network model; and a location estimation device connected to the feature extraction device, the location estimation device is configured to use a deep learning regression algorithm module including a global average pooling layer, a drop layer, and a fully connected layer The current visual image information of the image feature extraction operation is subjected to a regression analysis, and the operation outputs a current positioning position information about the intelligent vehicle as the initial positioning position information, and the estimation output is about the current positioning position A certain degree of uncertainty in the confidence level of the information to estimate the information. 如請求項1所述之智慧型載具之實時視覺定位與不確定性估測系統，其中該圖像輸入裝置係為一相機。The real-time visual positioning and uncertainty estimation system for an intelligent vehicle according to claim 1, wherein the image input device is a camera. 如請求項2所述之智慧型載具之實時視覺定位與不確定性估測系統，其中該圖像輸入裝置係為一單眼相機。The real-time visual positioning and uncertainty estimation system for an intelligent vehicle according to claim 2, wherein the image input device is a monocular camera. 如請求項1所述之智慧型載具之實時視覺定位與不確定性估測系統，其中該定位環境係對於二維光學雷達為一無特徵環境。The real-time visual positioning and uncertainty estimation system for an intelligent vehicle according to claim 1, wherein the positioning environment is a featureless environment for a two-dimensional optical radar. 如請求項1或4所述之智慧型載具之實時視覺定位與不確定性估測系統，其中該當前視覺圖像資訊係為單一RGB圖像。The real-time visual positioning and uncertainty estimation system for an intelligent vehicle according to claim 1 or 4, wherein the current visual image information is a single RGB image. 如請求項1所述之智慧型載具之實時視覺定位與不確定性估測系統，其中該智慧型載具係為一自主移動機器人。The real-time visual positioning and uncertainty estimation system for an intelligent vehicle according to claim 1, wherein the intelligent vehicle is an autonomous mobile robot.
TW110103077A 2021-01-27 2021-01-27 Autonomous intelligent vehicle real-time visual localization and uncertainty estimation system TWI764542B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
TW110103077A TWI764542B (en) 2021-01-27 2021-01-27 Autonomous intelligent vehicle real-time visual localization and uncertainty estimation system

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
TW110103077A TWI764542B (en) 2021-01-27 2021-01-27 Autonomous intelligent vehicle real-time visual localization and uncertainty estimation system

Publications (2)

Publication Number Publication Date
TWI764542B true TWI764542B (en) 2022-05-11
TW202230202A TW202230202A (en) 2022-08-01

Family

ID=82594194

Family Applications (1)

Application Number Title Priority Date Filing Date
TW110103077A TWI764542B (en) 2021-01-27 2021-01-27 Autonomous intelligent vehicle real-time visual localization and uncertainty estimation system

Country Status (1)

Country Link
TW (1) TWI764542B (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
TWI795306B (en) * 2022-06-14 2023-03-01 艾歐圖科技股份有限公司 Localization failure detection system for autonomous mobile robots using deep learning based 6d pose regression

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
TW201245666A (en) * 2011-05-03 2012-11-16 Ind Tech Res Inst Tracking systems and methods
US20190370648A1 (en) * 2018-05-29 2019-12-05 Google Llc Neural architecture search for dense image prediction tasks
CN111401215A (en) * 2020-03-12 2020-07-10 杭州涂鸦信息技术有限公司 Method and system for detecting multi-class targets
CN111627065A (en) * 2020-05-15 2020-09-04 Oppo广东移动通信有限公司 Visual positioning method and device and storage medium

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
TW201245666A (en) * 2011-05-03 2012-11-16 Ind Tech Res Inst Tracking systems and methods
US20190370648A1 (en) * 2018-05-29 2019-12-05 Google Llc Neural architecture search for dense image prediction tasks
CN111401215A (en) * 2020-03-12 2020-07-10 杭州涂鸦信息技术有限公司 Method and system for detecting multi-class targets
CN111627065A (en) * 2020-05-15 2020-09-04 Oppo广东移动通信有限公司 Visual positioning method and device and storage medium

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
TWI795306B (en) * 2022-06-14 2023-03-01 艾歐圖科技股份有限公司 Localization failure detection system for autonomous mobile robots using deep learning based 6d pose regression

Also Published As

Publication number Publication date
TW202230202A (en) 2022-08-01

Similar Documents

Publication Publication Date Title
US10549430B2 (en) Mapping method, localization method, robot system, and robot
US10949712B2 (en) Information processing method and information processing device
US10679072B2 (en) Moving-object position estimating system, information processing apparatus and moving-object position estimating method
CN108168539B (en) Blind person navigation method, device and system based on computer vision
Lin et al. Stereo-based simultaneous localization, mapping and moving object tracking
US11373418B2 (en) Information processing apparatus, information processing method, program, and mobile object
US20180286056A1 (en) Information processing device, information processing method, and program
US11111785B2 (en) Method and device for acquiring three-dimensional coordinates of ore based on mining process
CN111736586B (en) Method for automatically driving vehicle position for path planning and device thereof
JP2020508528A (en) Imaging system
Ataer-Cansizoglu et al. Calibration of non-overlapping cameras using an external SLAM system
US11119498B2 (en) Systems and methods for simulation utilizing a segmentable monolithic mesh
RU2016143549A (en) METHOD FOR LOCALIZING ROBOT IN THE PLANE OF LOCALIZATION
Wang et al. Radarloc: Learning to relocalize in fmcw radar
TWI764542B (en) Autonomous intelligent vehicle real-time visual localization and uncertainty estimation system
US20180239351A1 (en) Autonomous mobile device
Alves et al. Localization and navigation of a mobile robot in an office-like environment
CN113378605B (en) Multi-source information fusion method and device, electronic equipment and storage medium
GB2578721A (en) Method and system for processing image data utilizing deep neural network
CN117152249A (en) Multi-unmanned aerial vehicle collaborative mapping and perception method and system based on semantic consistency
Gao et al. Multi-view sensor fusion by integrating model-based estimation and graph learning for collaborative object localization
US11561553B1 (en) System and method of providing a multi-modal localization for an object
Han et al. Localization of a mobile robot using multiple ceiling lights
Hwang et al. Monocular vision-based global localization using position and orientation of ceiling features
Jose et al. Effective utilization and analysis of ros on embedded platform for implementing autonomous car vision and navigation modules