TWI408342B - Real-time augmented reality device, real-time augmented reality method and computer program product thereof - Google Patents
Real-time augmented reality device, real-time augmented reality method and computer program product thereof Download PDFInfo
- Publication number
- TWI408342B TWI408342B TW99108329A TW99108329A TWI408342B TW I408342 B TWI408342 B TW I408342B TW 99108329 A TW99108329 A TW 99108329A TW 99108329 A TW99108329 A TW 99108329A TW I408342 B TWI408342 B TW I408342B
- Authority
- TW
- Taiwan
- Prior art keywords
- instant
- microprocessor
- augmented reality
- real
- virtual
- Prior art date
Links
Landscapes
- Processing Or Creating Images (AREA)
Abstract
Description
本發明係關於一種即時擴增實境裝置、即時擴增實境方法及其電腦程式產品。具體而言,本發明係關於一種可根據導航資訊、即時實景影像以及即時使用者影像,產生一指引資訊之擴增實境裝置、擴增實境方法及其電腦程式產品。The invention relates to an instant augmented reality device, an instant augmented reality method and a computer program product thereof. Specifically, the present invention relates to an augmented reality device, an augmented reality method, and a computer program product that can generate a guidance information according to navigation information, real-time live image, and instant user image.
近幾年來,隨著定位導航技術快速成熟,各式各樣輔助定位導航技術的顯示器亦大量見於人們日常生活中,例如抬頭顯示器(HUD)、微型顯示眼鏡(HMD)以及一內含LCD/OLED顯示器之玻璃窗板等等,其中抬頭顯示器(HUD)更已被消費市場廣泛接受,下文將說明習知抬頭顯示器(HUD)定位導航系統之運作機制。In recent years, with the rapid maturity of positioning and navigation technology, a variety of displays that assist in positioning and navigation technology are also widely seen in people's daily lives, such as head-up display (HUD), miniature display glasses (HMD), and an embedded LCD/OLED. The glazing of the display and the like, among which the head-up display (HUD) has been widely accepted by the consumer market, the operation mechanism of the conventional head-up display (HUD) positioning navigation system will be explained below.
抬頭顯示器(HUD)係為一種可幫助駕駛員認清所有需要的駕駛資訊,同時保持視線在前方的顯示技術。HUD是利用光學反射的原理,將重要的駕駛相關資訊投射在一片玻璃上,這片玻璃位於駕駛者前端,駕駛者透過HUD往前方看的時候,能夠輕易將外界的景象與HUD顯示的資料融合在一起。其設計用意係使駕駛者不需低頭查看儀表,僅需保持抬頭姿勢即可透過HUD看到顯示資料。The Head-Up Display (HUD) is a display technology that helps the driver recognize all the driving information needed while keeping the line of sight in front. The HUD uses the principle of optical reflection to project important driving related information on a piece of glass. This glass is located at the front of the driver. When the driver looks through the HUD, it can easily display the external scene and the information displayed by the HUD. mix together. The design is intended to allow the driver to view the display through the HUD without having to look down at the meter.
惟習知抬頭顯示器(HUD)定位導航系統,顯示面板必定介於駕駛員的眼睛與後方景物之間,同時顯示面板與駕駛員的眼睛距離,遠小於與後方景物之距離。HUD顯示的資訊與後方景物通常具有一定關連性,因此,若使用固定式HUD,當駕駛員的頭部偏移(例如上半身向左及右方傾斜)時,HUD上顯示的資訊自駕駛員的視角觀看,就會與原本的景物產生偏移,另外,若使用移動式HUD,當駕駛員的視線方向改變時,HUD顯示的資訊同樣也會與原後方景物產生偏移。However, the conventional head-up display (HUD) positioning navigation system, the display panel must be between the driver's eyes and the rear scene, and the distance between the display panel and the driver's eyes is much smaller than the distance from the rear scene. The information displayed by the HUD is usually related to the rear scene. Therefore, if a fixed HUD is used, when the driver's head is offset (for example, the upper body is tilted to the left and right), the information displayed on the HUD is from the driver's When viewed from the perspective, it will be offset from the original scene. In addition, if the mobile HUD is used, when the driver's line of sight direction changes, the information displayed by the HUD will also be offset from the original scene.
综上所述,如何克服HUD顯示的資訊會與後方景物產生偏移之缺點,進而使抬頭顯示器(HUD)定位導航系統具備根據駕駛員視線角度,自動修正顯示偏移的能力,實為該領域之技術者極需解決之課題。In summary, how to overcome the shortcomings of the information displayed by the HUD and the rear scene, so that the head-up display (HUD) positioning navigation system has the ability to automatically correct the display offset according to the driver's line of sight angle, which is actually the field. The problem that the technicians need to solve is extremely urgent.
本發明之一目的在於提供一種即時擴增實境(augmented reality)裝置,此即時擴增實境裝置係可與一導航裝置、一實景影像擷取裝置以及一使用者影像擷取裝置搭配使用,該導航裝置係可根據導航裝置之一目前位置,產生一導航資訊,實景影像擷取裝置用以擷取一包含一實景物件之即時實景影像,使用者影像擷取裝置用以擷取一包含一人臉物件之即時使用者影像。An object of the present invention is to provide an augmented reality device, which can be used in conjunction with a navigation device, a real image capturing device, and a user image capturing device. The navigation device generates a navigation information according to the current position of the navigation device, and the real image capturing device is configured to capture an instant real image including a real object, and the user image capturing device is configured to capture one person. Instant user image of the face object.
本發明之即時擴增實境裝置包含一傳送/接收介面、一儲存器以及一微處理器,傳送/接收介面係與導航裝置、影像擷取裝置以及使用者影像擷取裝置呈電性連接,微處理器係與傳送/接收介面以及儲存器呈電性連接,傳送/接收介面用以接收導航資訊、即時實景影像以及即時使用者影像,儲存器用以儲存實景物件之一實際尺寸以及人臉物件之一預設位置以及一預設視線角度,微處理器用以判斷人臉物件於即時使用者影像中所具有之一虛擬視線角度、判斷人臉物件於即時使用者影像中所具有之一虛擬位置、判斷實景物件於即時實景影像中所具有之一虛擬尺寸以及根據實際尺寸、預設位置、預設視線角度、虛擬視線角度、虛擬位置、虛擬尺寸以及導航資訊,產生一指引資訊。The instant augmented reality device of the present invention comprises a transmitting/receiving interface, a storage and a microprocessor, and the transmitting/receiving interface is electrically connected to the navigation device, the image capturing device and the user image capturing device. The microprocessor is electrically connected to the transmitting/receiving interface and the storage, and the transmitting/receiving interface is for receiving navigation information, real-time live image and instant user image, and the storage is used for storing the actual size of one of the real objects and the face object. a preset position and a preset line of sight angle, the microprocessor is configured to determine a virtual line of sight angle of the face object in the instant user image, and determine that the face object has a virtual position in the instant user image And determining a virtual size of the real object in the real-time live image and generating a guiding information according to the actual size, the preset position, the preset line of sight angle, the virtual line of sight angle, the virtual position, the virtual size, and the navigation information.
本發明之另一目的在於提供一種用於前述即時擴增實境裝置之即時擴增實境方法,該即時擴增實境方法包含下列步驟:(A)令一傳送/接收介面接收導航資訊、即時實景影像以及即時使用者影像;(B)令一微處理器判斷人臉物件於即時使用者影像中所具有之一虛擬視線角度;(C)令該微處理器判斷該人臉物件於該即時使用者影像所具有之一虛擬位置;(D)令該微處理器判斷該實景物件於該即時實景影像中所具有之一虛擬尺寸;以及(E)令該微處理器根據該實際尺寸、該預設位置、該預設視線角度、該虛擬視線角度、該虛擬位置、該虛擬尺寸以及該導航資訊,產生一指引資訊。Another object of the present invention is to provide an instant augmented reality method for the aforementioned instant augmented reality device, the instant augmented reality method comprising the following steps: (A) causing a transmitting/receiving interface to receive navigation information, (B) causing a microprocessor to determine a virtual line of sight angle of the face object in the instant user image; (C) causing the microprocessor to determine the face object in the The virtual user image has a virtual location; (D) causing the microprocessor to determine that the real object has a virtual size in the live real image; and (E) causing the microprocessor to The preset position, the preset line of sight angle, the virtual line of sight angle, the virtual position, the virtual size, and the navigation information generate a guidance information.
本發明之又一目的在於提供一種電腦程式產品,內儲一種用以執行用於前述即時擴增實境裝置之即時擴增實境方法之程式,該程式被載入即時擴增實境裝置後執行:一程式指令A,令一傳送/接收介面接收該導航資訊、該即時實景影像以及該即時使用者影像;一程式指令B,令一微處理器判斷該人臉物件於即時使用者影像中所具有之一虛擬視線角度;一程式指令C,令該微處理器判斷該人臉物件於該即時使用者影像所具有之一虛擬位置;一程式指令D,令該微處理器判斷該實景物件於該即時實景影像中所具有之一虛擬尺寸;以及一程式指令E,令該微處理器根據該實際尺寸、該預設位置、該預設視線角度、該虛擬視線角度、該虛擬位置、該虛擬尺寸以及該導航資訊,產生一指引資訊。It is still another object of the present invention to provide a computer program product storing a program for executing an instant augmented reality method for the aforementioned instant augmented reality device, the program being loaded into an instant augmented reality device Executing: a program command A for a transmitting/receiving interface to receive the navigation information, the live view image and the instant user image; a program command B for causing a microprocessor to determine the face object in the instant user image Having a virtual line of sight angle; a program command C, the microprocessor determines that the face object has a virtual position in the instant user image; a program command D causes the microprocessor to determine the real object Having a virtual size in the live view image; and a program command E for the microprocessor to determine the actual size, the preset position, the preset line of sight angle, the virtual line of sight angle, the virtual position, the The virtual size and the navigation information generate a guide information.
综上所述,本發明之即時擴增實境裝置係可根據包含一人臉物件之即時使用者影像、包含一實景物件之即時實景影像以及導航資訊,產生符合駕駛人視覺辨識習慣之指引資訊,換言之,該指引資訊係應因駕駛者頭部偏移以及視線改變做修正而產生,藉此,習知HUD顯示資訊會與原後方景物產生偏移之缺點得以被有效克服,進而增加抬頭顯示器(HUD)定位導航系統之整體附加價值。In summary, the instant augmented reality device of the present invention can generate guidance information conforming to the driver's visual recognition habit according to an instant user image including a human face object, an instant real-life image including a real scene object, and navigation information. In other words, the guidance information should be generated by the driver's head shift and the change of the line of sight. Therefore, the disadvantage that the conventional HUD display information is offset from the original rear view can be effectively overcome, thereby increasing the head-up display ( HUD) The overall added value of the positioning navigation system.
以下將透過實施例來解釋本發明內容,本發明的實施例並非用以限制本發明須在如實施例所述之任何特定的環境、應用或特殊方式方能實施。因此,以下實施例之描述僅為說明目的,並非本發明之限制。須說明者,以下實施例及圖式中,與本發明非直接相關之元件已省略而未繪示;且圖式中各元件間之尺寸關係僅為求容易瞭解,非用以限制實際比例。The present invention is not limited by the embodiment, and the embodiment of the present invention is not intended to limit the invention to any specific environment, application or special mode as described in the embodiments. Therefore, the description of the following examples is for illustrative purposes only and is not a limitation of the invention. It should be noted that, in the following embodiments and drawings, components that are not directly related to the present invention have been omitted and are not shown; and the dimensional relationships between the components in the drawings are merely for ease of understanding and are not intended to limit the actual ratio.
本發明之第一實施例如第1圖所示,其係為一即時擴增實境導航顯示系統1之示意圖,即時擴增實境導航顯示系統1係包含一即時擴增實境裝置11、一實景影像擷取裝置13、一導航裝置15、一使用者影像擷取裝置17以及一顯示裝置19。於本實施例中,即時擴增實境導航顯示系統1係用於一車輛,而於其它實施例中,即時擴增實境導航顯示系統1亦可視使用者之實際需求,應用於其它駕駛工具,例如:飛機、船以及機車...等等,並不以此限制本發明之應用範圍。以下將說明擴增實境裝置11如何搭配實景影像擷取裝置13、導航裝置15、使用者影像擷取裝置17以及顯示裝置19實現即時擴增實境導航顯示系統1。The first embodiment of the present invention is shown in FIG. 1 , which is a schematic diagram of an instant augmented reality navigation display system 1 . The instant augmented reality navigation display system 1 includes an instant augmented reality device 11 . The live image capturing device 13, a navigation device 15, a user image capturing device 17, and a display device 19. In this embodiment, the instant augmented reality navigation display system 1 is used for one vehicle, and in other embodiments, the instant augmented reality navigation display system 1 can also be applied to other driving tools according to the actual needs of the user. For example, aircraft, boats, and locomotives, etc., do not limit the scope of application of the present invention. Hereinafter, how the augmented reality device 11 is matched with the live image capturing device 13, the navigation device 15, the user image capturing device 17, and the display device 19 realizes the instant augmented reality navigation display system 1.
即時擴增實境導航顯示系統1之導航裝置15係根據其目前位置,產生一導航資訊150,實景影像擷取裝置13用以擷取一包含一物件之即時實景影像130,使用者影像擷取裝置17則用以擷取一包含一人臉物件之即時使用者影像170,而擴增實境裝置11儲存有該實景物件之一實際尺寸1130以及該人臉物件之一預設位置1132以及一預設視線角度1134,且根據實際尺寸1130、預設位置1132、預設視線角度1134、即時實景影像130、即時使用者影像170以及導航資訊150,產生與傳送一指引資訊117至顯示裝置19,俾顯示裝置19可顯示指引資訊117,供車輛駕駛人參考。The navigation device 15 of the real-time augmented reality navigation display system 1 generates a navigation information 150 according to the current position thereof, and the real-time image capturing device 13 captures an instant real-time image 130 containing an object, and the user images are captured. The device 17 is configured to capture an instant user image 170 including a human face object, and the augmented reality device 11 stores an actual size 1130 of the real object and a preset position 1132 of the facial object and a pre- The line of sight angle 1134 is set, and according to the actual size 1130, the preset position 1132, the preset line of sight angle 1134, the real-time live image 130, the instant user image 170, and the navigation information 150, a guide information 117 is generated and transmitted to the display device 19, The display device 19 can display the guide information 117 for reference by the driver of the vehicle.
於本實施例中,導航裝置15係運用GPS技術確定導航裝置15本身或其所安裝處的經緯度、方向、速度、高度等資訊,並利用慣性導航如電子羅盤、加速計、陀螺儀等,輔助計算GPS資訊更新週期之間的資訊,再利用定位資訊與地圖資訊,確認交通工具所處位置,並決定行進路徑,以產生導航資訊150,其係屬一二維資訊,於其它實施例中,導航裝置15可利用其它定位技術產生導航資訊150,並不以此為限。In the present embodiment, the navigation device 15 uses GPS technology to determine the latitude, longitude, direction, speed, height, and the like of the navigation device 15 itself or its installation, and uses inertial navigation such as an electronic compass, an accelerometer, a gyroscope, etc., to assist Calculating the information between the GPS information update periods, and then using the positioning information and the map information to confirm the location of the vehicle and determining the travel path to generate the navigation information 150, which is a two-dimensional information. In other embodiments, The navigation device 15 can generate the navigation information 150 by using other positioning technologies, and is not limited thereto.
再者,實景影像擷取裝置13所擷取之即時實景影像130係可為直接、即時方式輸入,舉例而言,如裝載於車輛前方之攝影機,另外,此即時實景影像亦可以間接、非即時方式輸入,舉例而言,如模擬駕駛艙將可用記錄影像,或是由記錄影像衍生的電腦3D影像作為即時實景影像130。Furthermore, the real-time live image 130 captured by the real-time image capturing device 13 can be input in a direct, instant manner. For example, if the camera is mounted in front of the vehicle, the live-action image can also be indirect or non-instant. Mode input, for example, a simulated cockpit will be available for recording images, or a computer 3D image derived from recorded images as an instant live image 130.
為方便後續說明,於本實施例中,即時擴增實境導航顯示系統1係安裝於一行駛中之車輛,導航裝置15所產生之導航資訊150可視為包含該行駛中之車輛之目前位置,而實景影像擷取裝置13所擷取之即時實景影像130則可視為該行駛中車輛之周圍環境景象,例如道路以及路樹等等,即時實景影像130為車輛駕駛人所視之道路影像,而即時實景影像130所包含之物件係可為駕駛人自車輛前方車窗看到的道路分隔線。以下將說明擴增實境裝置11係如何產生指引資訊117。For convenience of the following description, in the present embodiment, the instant augmented reality navigation display system 1 is installed on a traveling vehicle, and the navigation information 150 generated by the navigation device 15 can be regarded as including the current position of the traveling vehicle. The real-time live image 130 captured by the real-time image capturing device 13 can be regarded as the surrounding environment of the vehicle in motion, such as roads and road trees, etc. The real-time live image 130 is a road image viewed by the driver of the vehicle, and The object contained in the live-action image 130 is a road divider that the driver sees from the front window of the vehicle. How the augmented reality device 11 generates the guidance information 117 will be explained below.
由第1圖可知,即時擴增實境裝置11包含一傳送/接收介面111、一儲存器113、一投影器114以及一微處理器115,傳送/接收介面111係與導航裝置15、實景影像擷取裝置13、使用者影像擷取裝置17以及顯示裝置19呈電性連接,微處理器115係與傳送/接收介面111、投影器114以及儲存器113呈電性連接,儲存器113用以儲存該實景物件(即道路分隔線)之一實際尺寸1130以及該人臉物件之一預設位置1132以及一預設視線角度1134。As can be seen from FIG. 1, the instant augmented reality device 11 includes a transmitting/receiving interface 111, a storage 113, a projector 114, and a microprocessor 115. The transmitting/receiving interface 111 is connected to the navigation device 15 and the live image. The capturing device 13, the user image capturing device 17 and the display device 19 are electrically connected. The microprocessor 115 is electrically connected to the transmitting/receiving interface 111, the projector 114 and the storage 113. The storage device 113 is used for the storage device 113. One of the actual size 1130 of the real object (ie, the road divider) and one of the preset positions 1132 of the face object and a preset line of sight angle 1134 are stored.
於導航裝置15產生導航資訊150、實景影像擷取裝置13擷取出包含該道路分隔線之即時實景影像130以及使用者影像擷取裝置17擷取出包含人臉物件之即時使用者影像170後,傳送/接收介面111係可接收導航資訊150、即時使用者影像170以及即時實景影像130,接下來微處理器115係可根據一物件邊緣辨識法,判斷該道路分隔線於即時實景影像130中所具有之一虛擬尺寸(可為該道路分隔線之虛擬長度以及虛擬寬度),以供後續處理之用,需注意者,本實施例所採用之物件邊緣辨識法係可透過習知技術達成,且於其它實施例中,微處理器115亦可根據其它判斷方式,判斷出該道路分隔線於即時實景影像130中所具有之虛擬長度與 虛擬寬度,並不以此為限。The navigation device 15 generates the navigation information 150, and the real-time image capturing device 13 extracts the real-time live image 130 including the road dividing line, and the user image capturing device 17 extracts the instant user image 170 including the human face object, and transmits the image. The receiving interface 111 can receive the navigation information 150, the instant user image 170, and the instant real-time image 130. Then, the microprocessor 115 can determine that the road dividing line has the real-time live image 130 according to an object edge identification method. One of the virtual dimensions (which may be the virtual length and the virtual width of the road divider) for subsequent processing. It should be noted that the object edge identification method used in this embodiment can be achieved through conventional techniques. In other embodiments, the microprocessor 115 may also determine the virtual length of the road divider in the live view image 130 according to other determination modes. Virtual width, not limited to this.
除了判斷該道路分隔線於即時實景影像130中所具有之虛擬尺寸,微處理器115更用以判斷該人臉物件於即時使用者影像170中所具有之一虛擬視線角度,以及判斷該人臉物件於即時使用者影像170中所具有之一虛擬位置。需注意者,微處理器115判斷虛擬尺寸、虛擬視線角度以及虛擬位置之順序係可視實際應用而改變,並不以此為限。In addition to determining the virtual size of the road divider in the live view image 130, the microprocessor 115 is further configured to determine a virtual line of sight angle of the face object in the instant user image 170, and determine the face. The object has a virtual location in the instant user image 170. It should be noted that the order in which the microprocessor 115 determines the virtual size, the virtual line of sight angle, and the virtual position may vary depending on the actual application, and is not limited thereto.
具體而言,由於當車輛行駛於路面時,駕駛人之雙眼視線以及頭部將會產生平移以及上下游動等動作,為準確產生指引資訊117,微處理器115係可由即時使用者影像170中,判斷出一駕駛人目前之視線角度,即即時使用者影像170所具有之虛擬視線角度,另一方面,當駕駛人之臉部於車內位置產生移動,如將頭向右方車窗觀望以及將頭伸至後方查看,微處理器115係可由即時使用者影像170中,判斷一移動後的臉部位置,即為即時使用者影像170所具有之虛擬位置。Specifically, since the driver's binocular line of sight and the head will produce translational and upstream and downstream movements when the vehicle is traveling on the road surface, the microprocessor 115 can be used by the instant user image 170 in order to accurately generate the guidance information 117. In the middle, the driver's current line of sight angle is determined, that is, the virtual line of sight angle of the instant user image 170, and on the other hand, when the driver's face moves in the interior position, such as turning the head to the right window Looking at the camera and extending the head to the rear, the microprocessor 115 can determine the position of the face after the movement by the instant user image 170, that is, the virtual position of the instant user image 170.
接下來,為得知人臉物件之視線角度變化程度,微處理器115將根據判斷出之虛擬視線角度以及儲存器113所儲存之預設視線角度1134,產生一視線角度差,微處理器115再根據儲存器113所儲存之預設位置1132以及判斷出之虛擬位置,產生一位置差,以得知人臉物件之位置變化程度。Next, in order to know the degree of change of the line of sight angle of the face object, the microprocessor 115 will generate a line of sight angle difference according to the determined virtual line of sight angle and the preset line of sight angle 1134 stored by the memory 113, and the microprocessor 115 According to the preset position 1132 stored in the storage 113 and the determined virtual position, a position difference is generated to know the degree of change of the position of the face object.
微處理器115亦根據該實景物件之實際尺寸1130(包含實景物件之實際長度與實際寬度)以及判斷出之虛擬尺寸(包含虛擬長度以及虛擬寬度),計算出實景影像擷取裝置13之一影像擷取方向與一水平面間之一仰角,再根據實際長度、實際寬度、虛擬長度、虛擬寬度以及導航資訊150,計算出該影像擷取方向與導航裝置15之一行進方向間之一偏角,接下來,微處理器115根據該仰角、該偏角、該視線角度差、該位置差以及導航資訊150,產生一指引資訊117。The microprocessor 115 also calculates an image of the real image capturing device 13 according to the actual size 1130 of the real object (including the actual length and actual width of the real object) and the determined virtual size (including the virtual length and the virtual width). Taking an elevation angle between the direction and a horizontal plane, and calculating an off angle between the image capturing direction and the traveling direction of the navigation device 15 according to the actual length, the actual width, the virtual length, the virtual width, and the navigation information 150, Next, the microprocessor 115 generates a guidance information 117 based on the elevation angle, the declination, the line-of-sight angle difference, the position difference, and the navigation information 150.
詳言之,根據道路法規的規範,道路分隔線之實際長度以及實際寬度係為固定,於判斷出道路分隔線之虛擬長度與虛擬寬度後,微處理器115係利用實際長度與虛擬長度、實際寬度以及虛擬寬度的比例,計算出實景影像擷取裝置13之影像擷取方向與水平面間之仰角,且微處理器115更利用實際長度與虛擬長度、實際寬度以及虛擬寬度的比例以及導航資訊150,計算出實景影像擷取裝置13之影像擷取方向與導航裝置15之行進方向間之偏角,微處理器115根據該仰角、該偏角、該視線角度差、該位置差以及導航資訊150,即可產生考量垂直視角深度的指引資訊117。In detail, according to the regulations of road regulations, the actual length and actual width of the road divider are fixed. After determining the virtual length and virtual width of the road divider, the microprocessor 115 uses the actual length and the virtual length, and the actual The ratio of the width and the virtual width is calculated, and the elevation angle between the image capturing direction and the horizontal plane of the real image capturing device 13 is calculated, and the microprocessor 115 further utilizes the ratio of the actual length to the virtual length, the actual width, and the virtual width, and the navigation information 150. Calculating an off angle between the image capturing direction of the real image capturing device 13 and the traveling direction of the navigation device 15, the microprocessor 115 according to the elevation angle, the off angle, the line of sight angle difference, the position difference, and the navigation information 150 , guidance information 117 that takes into account the depth of the vertical viewing angle can be generated.
更具體而言,請參閱第2圖,其係為一裝載即時擴增實境導航顯示系統1之車輛21行駛於路面的示意圖,路面上有一具有實際長度D與實際寬度W之道路分隔線23,實景影像擷取裝置13係自位置27看出去的影像擷取方向擷取即時實景影像130,即時實景影像130中所包含之道路分隔線將隨著車輛的行進方向以及道路所處之地形或沿伸之方向而改變,簡言之,即是即時實景影像130中之道路分隔線的虛擬長度以及虛擬寬度係隨著車輛的行進方向以及道路所處之地形或沿伸之方向而改變。More specifically, please refer to FIG. 2, which is a schematic diagram of a vehicle 21 carrying an instant augmented reality navigation display system 1 traveling on a road surface having a road separation line 23 having an actual length D and an actual width W. The real-time image capturing device 13 captures the real-time live image 130 from the image capturing direction seen from the position 27, and the road dividing line included in the real-time real-life image 130 will follow the traveling direction of the vehicle and the terrain of the road or In the direction of the extension, in short, the virtual length of the road dividing line in the real-time live image 130 and the virtual width change with the direction of travel of the vehicle and the terrain or direction along which the road is located.
透過微處理器115連續的判斷道路分隔線的虛擬長度以及虛擬寬度,目前影像擷取方向與導航裝置15行進方向之偏角,以及影像擷取方向與水平面間之仰角將可連續即時計算出,以使微處理器115將導航裝置15二維的導航資訊轉換為三維之指引資訊,換言之,微處理器115係將導航裝置所呈現的二維地圖中的距離轉換成三維投射影像中的深度,當指引資訊為一箭頭符號時,由於指引資訊係根據仰角、偏角、該視線角度差、該位置差以及導航資訊150即時產生時,箭頭符號遇到突然之岔路時,仍能正確的落在岔路中央位置,不使其偏移,以指示駕駛者選擇正確的路口轉彎。Through the microprocessor 115, the virtual length and the virtual width of the road dividing line are continuously determined. The angle between the current image capturing direction and the traveling direction of the navigation device 15 and the elevation angle between the image capturing direction and the horizontal plane can be continuously calculated in real time. To enable the microprocessor 115 to convert the two-dimensional navigation information of the navigation device 15 into three-dimensional guidance information, in other words, the microprocessor 115 converts the distance in the two-dimensional map presented by the navigation device into a depth in the three-dimensional projection image. When the guidance information is an arrow symbol, since the guidance information is generated immediately according to the elevation angle, the declination angle, the line-of-sight angle difference, the position difference, and the navigation information 150, the arrow symbol still falls correctly when it encounters a sudden road. The center of the road is not offset, to instruct the driver to choose the right intersection to turn.
需強調的是,微處理器115係根據一領域轉換(Domain Transform)方式,輔以根據仰角、偏角、視線角度差、位置差以及導航資訊150,產生指引資訊117,換言之,係利用仰角、偏角、視線角度差、位置差等數據,可計算出一矩陣,將導航資訊150根據該矩陣做領域轉換,以將用以指引道路方向之箭頭符號,利用該矩陣使其上下部壓縮,成為一考量垂直視角深度後的指引資訊117。另一方面,於本實施例中,顯示裝置19可為一透視式抬頭顯示(Head up Display)裝置,其係可與傳送/接收介面111呈電性連接,於指引資訊117產生後,微處理器115透過傳送/接收介面111,傳送指引資訊117至該透視式抬頭顯示裝置,俾該透視式抬頭顯示裝置可顯示指引資訊117。再者,於本實施例中,顯示裝置19亦可為一投影式抬頭顯示裝置,微處理器115可透過投影器114,投射指引資訊117至投影式抬頭顯示裝置,俾投影式抬頭顯示裝置可顯示指引資訊117。需注意者,當顯示裝置19為透視式抬頭 顯示裝置時,即時擴增實境裝置11之投影器114可省略,以節省即時擴增實境裝置11之硬體成本。It should be emphasized that the microprocessor 115 generates the guidance information 117 according to a domain transform method, supplemented by the elevation angle, the off-angle, the line-of-sight angle difference, the position difference, and the navigation information 150, in other words, using the elevation angle, Deviation, line-of-sight angle difference, position difference and other data, a matrix can be calculated, and the navigation information 150 is converted according to the matrix, so that the arrow symbol for guiding the direction of the road is used, and the matrix is used to compress the upper and lower portions. The guidance information 117 after considering the vertical depth of view. On the other hand, in the embodiment, the display device 19 can be a perspective-up display device, which can be electrically connected to the transmission/reception interface 111. After the guidance information 117 is generated, the micro-processing is performed. The device 115 transmits the guidance information 117 to the perspective head-up display device through the transmission/reception interface 111, and the perspective-type head-up display device can display the guidance information 117. Furthermore, in the embodiment, the display device 19 can also be a projection type head-up display device, and the microprocessor 115 can transmit the guidance information 117 to the projection type head-up display device through the projector 114, and the projection type head-up display device can be Display guide information 117. It should be noted that when the display device 19 is a see-through head When the device is displayed, the projector 114 of the instant augmented reality device 11 can be omitted to save the hardware cost of the instant augmented reality device 11.
微處理器115更可將指引資訊117合成於即時實景影像130上,以產生導航影像119,微處理器115透過可傳送/接收介面111,傳送導航影像119至顯示裝置19,俾顯示裝置19可顯示導航影像119,以供車輛駕駛人參考。The microprocessor 115 can further synthesize the guidance information 117 on the real-time live image 130 to generate the navigation image 119. The microprocessor 115 transmits the navigation image 119 to the display device 19 through the transmit/receive interface 111. The display device 19 can be A navigation image 119 is displayed for reference by the driver of the vehicle.
具體而言,導航影像119係為微處理器115合成即時實景影像130以及指引資訊117所產生,換言之,若指引資訊117係為一箭頭符號,則即時實景影像130將與箭頭符號合成在一起,而駕駛者係所看到之導航影像119係即時實景影像130搭配考量垂直視角深度的指引資訊117所產生,需特別說明者,指引資訊117更可為其它圖形,並不以此限制本發明之範圍。Specifically, the navigation image 119 is generated by the microprocessor 115 synthesizing the real-time live image 130 and the guide information 117. In other words, if the guide information 117 is an arrow symbol, the live-action image 130 will be combined with the arrow symbol. The navigation image 119 that the driver sees is generated by the real-time live image 130 with the guidance information 117 considering the vertical depth of view. The special information is required. The guidance information 117 can be other graphics, and the present invention is not limited thereto. range.
本發明之第二實施例如第2A-2E圖所示,其係為一用於如第一實施例所述之即時擴增實境裝置之即時擴增實境方法之流程圖,即時擴增實境裝置係可與一導航裝置、一實景影像擷取裝置以及一使用者影像裝置搭配使用,導航裝置係可根據其一目前位置,產生一導航資訊,實景影像擷取裝置用以擷取一包含一實景物件之即時實景影像,使用者影像擷取裝置用以擷取一包含一人臉物件之即時使用者影像。A second embodiment of the present invention, as shown in FIG. 2A-2E, is a flow chart of an instant augmented reality method for an instant augmented reality device as described in the first embodiment. The navigation device can be used with a navigation device, a real image capturing device and a user image device. The navigation device can generate a navigation information according to a current location, and the real image capturing device can capture an inclusion An instant real-time image of the real object, the user image capturing device is used to capture an instant user image containing a human face object.
此外,該即時擴增實境裝置包含一傳送/接收介面、一投影器、一儲存器以及一微處理器,該微處理器係與該傳送/接收介面以及該儲存器呈電性連接,該傳送/接收介面係與該導航裝置、該影像擷取裝置以及該使用者影像擷取裝置呈電性連接,該儲存器用以儲存該實景物件之一實際尺寸以及該人臉物件之一預設位置以及一預設視線角度。In addition, the instant augmented reality device includes a transmitting/receiving interface, a projector, a storage, and a microprocessor, and the microprocessor is electrically connected to the transmitting/receiving interface and the storage. The transmitting/receiving interface is electrically connected to the navigation device, the image capturing device, and the user image capturing device, and the storage device is configured to store an actual size of the real object and a preset position of the human face object. And a preset line of sight angle.
第二實施例即時擴增實境方法所採之技術手段實質上與第一實施例即時擴增實境裝置所採知技術手段相同,此項技術領域具有通常知識者將可根據第一實施例所揭示之內容,輕易得知第二實施例即時擴增實境方法係如何實現,以下將只簡述即時擴增實境方法。The technical means adopted by the instant augmented reality method of the second embodiment is substantially the same as the technical means adopted by the instant augmented reality device of the first embodiment, and those skilled in the art will be able to according to the first embodiment. What is disclosed, it is easy to know how the instant augmented reality method of the second embodiment is implemented, and only the instant augmented reality method will be briefly described below.
本實施例之即時擴增實境方法係包含以下步驟,請先參閱第3A圖,執行步驟301,令一傳送/接收介面接收該導航資訊、該即時實景影像以及該即時使用者影像,執行步驟302,令一微處理器判斷人臉物件於即時使用者影像中所具有之一虛擬視線角度,再執行步驟303,令微處理器判斷該人臉物件於該即時使用者影像所具有之一虛擬位置。The instant augmented reality method of the present embodiment includes the following steps. Please refer to FIG. 3A first, and step 301 is executed to enable a transmitting/receiving interface to receive the navigation information, the live view image, and the instant user image, and perform the steps. 302. Let a microprocessor determine that the human face object has a virtual line of sight angle in the instant user image, and then perform step 303 to enable the microprocessor to determine that the human face object has one of the instant user images. position.
執行步驟304,令該微處理器判斷該實景物件於該即時實景影像中所具有之一虛擬尺寸,接下來請參閱第3B圖,執行步驟305,令微處理器根據虛擬視線角度以及預設視線角度,產生一視線角度差,執行步驟306,令微處理器根據預設位置以及虛擬位置,產生一位置差,執行步驟307,令微處理器根據實際尺寸以及虛擬尺寸,計算出實景影像擷取裝置之一影像擷取方向與一水平面間之一仰角,執行步驟308,令微處理器根據實際尺寸、虛擬尺寸以及導航資訊,計算出影像擷取方向與導航裝置之一行進方向之一偏角。Step 304 is executed to enable the microprocessor to determine that the real object has a virtual size in the live view image. Next, refer to FIG. 3B, and step 305 is executed to enable the microprocessor to view the virtual line of sight and the preset line of sight. Angle, generating a line of sight angle difference, step 306 is executed to cause the microprocessor to generate a position difference according to the preset position and the virtual position, and step 307 is executed to enable the microprocessor to calculate the live image image according to the actual size and the virtual size. Step 308 is performed by one of the image capturing directions and a horizontal plane of the device, and the microprocessor is configured to calculate a deviation angle between the image capturing direction and one of the traveling directions of the navigation device according to the actual size, the virtual size, and the navigation information. .
請參閱第3C圖,執行步驟309,令微處理器根據仰角、偏角、視線角度差、位置差以及導航資訊,產生指引資訊,接下來,如該顯示裝置係為一透視式抬頭顯示裝置,則執行步驟310,令微處理器透過傳送/接收介面,傳送指引資訊至透視式抬頭顯示裝置,俾透視式抬頭顯示裝置可顯示指引資訊。Referring to FIG. 3C, step 309 is executed to enable the microprocessor to generate guidance information according to the elevation angle, the off-angle, the line-of-sight angle difference, the position difference, and the navigation information. Next, if the display device is a perspective head-up display device, Then, in step 310, the microprocessor transmits the guidance information to the perspective head-up display device through the transmission/reception interface, and the perspective-type head-up display device can display the guidance information.
如該顯示裝置係為一投影式抬頭顯示裝置,請參閱第3D圖,於步驟309後,執行步驟311,令微處理器更用以透過投影器,投射指引資訊至投影式抬頭顯示裝置,俾投影式抬頭顯示裝置可顯示指引資訊。If the display device is a projection type head-up display device, please refer to FIG. 3D. After step 309, step 311 is executed to enable the microprocessor to use the projector to project guidance information to the projection head-up display device. The projection head-up display device can display guidance information.
此外,請參閱第3E圖,於步驟309後,執行步驟312,令微處理器將指引資訊合成於即時實景影像上,以產生一導航影像,最後,執行步驟313,令微處理器透過傳送/接收介面,傳送導航影像至顯示裝置,俾顯示裝置可顯示導航影像。In addition, referring to FIG. 3E, after step 309, step 312 is executed to enable the microprocessor to synthesize the guidance information on the live real-time image to generate a navigation image. Finally, step 313 is executed to enable the microprocessor to transmit/ The receiving interface transmits a navigation image to the display device, and the display device can display the navigation image.
除了上述步驟,第二實施例亦能執行第一實施例所描述之操作及功能,所屬技術領域具有通常知識者可直接瞭解第二實施例如何基於上述第一實施例以執行此等操作及功能,故不贅述。In addition to the above steps, the second embodiment can also perform the operations and functions described in the first embodiment, and those skilled in the art can directly understand how the second embodiment performs the operations and functions based on the above-described first embodiment. Therefore, I will not repeat them.
此外,第二實施例所描述之即時擴增實境方法可由一電腦程式產品執行,當即時擴增實境裝置經由一電腦載入該電腦程式產品並執行該電腦程式產品所包含之複數個指令後,即可完成第二實施例所述之即時擴增實境方法。前述之電腦程式產品可儲存於電腦可讀取記錄媒體中,例如唯讀記憶體(read only memory;ROM)、快閃記憶體、軟碟、硬碟、光碟、隨身碟、磁帶、可由網路存取之資料庫或熟習此項技藝者所習知且具有相同功能之任何其它儲存媒體中。In addition, the instant augmented reality method described in the second embodiment can be executed by a computer program product, when the instant augmented reality device loads the computer program product via a computer and executes a plurality of instructions included in the computer program product. After that, the instant augmented reality method described in the second embodiment can be completed. The aforementioned computer program product can be stored in a computer readable recording medium, such as read only memory (ROM), flash memory, floppy disk, hard disk, optical disk, flash drive, tape, network available Access to the database or any other storage medium known to those skilled in the art and having the same function.
综上所述,本發明之即時擴增實境裝置係可根據包含一人臉物件之即時使用者影像、包含一實景物件之即時實景影像以及導航資訊,產生符合駕駛人視覺辨識習慣之指引資訊,換言之,該指引資訊係應因駕駛者頭部偏移以及視線改變做修正而產生,藉此,習知HUD顯示資訊會與原後方景物產生偏移之缺點得以被有效克服,進而增加抬頭顯示器(HUD)定位導航系統之整體附加價值。In summary, the instant augmented reality device of the present invention can generate guidance information conforming to the driver's visual recognition habit according to an instant user image including a human face object, an instant real-life image including a real scene object, and navigation information. In other words, the guidance information should be generated by the driver's head shift and the change of the line of sight. Therefore, the disadvantage that the conventional HUD display information is offset from the original rear view can be effectively overcome, thereby increasing the head-up display ( HUD) The overall added value of the positioning navigation system.
上述之實施例僅用來例舉本發明之實施態樣,以及闡釋本發明之技術特徵,並非用來限制本發明之保護範疇。任何熟悉此技術者可輕易完成之改變或均等性之安排均屬於本發明所主張之範圍,本發明之權利保護範圍應以申請專利範圍為準。The embodiments described above are only intended to illustrate the embodiments of the present invention, and to explain the technical features of the present invention, and are not intended to limit the scope of protection of the present invention. Any changes or equivalents that can be easily made by those skilled in the art are within the scope of the invention. The scope of the invention should be determined by the scope of the claims.
1...即時擴增實境導航顯示系統1. . . Instant Augmented Reality Navigation Display System
11...即時擴增實境裝置11. . . Instant augmented reality device
111...傳送/接收介面111. . . Transmit/receive interface
113...儲存器113. . . Storage
1130...實際尺寸1130. . . Actual size
1132...預設位置1132. . . Preset position
1134...預設視線角度1134. . . Default line of sight
115...微處理器115. . . microprocessor
114‧‧‧投影器114‧‧‧Projector
117‧‧‧指引資訊117‧‧‧Guide information
119‧‧‧導航影像119‧‧‧Navigation images
13‧‧‧實景影像擷取裝置13‧‧‧Real image capture device
130‧‧‧即時實景影像130‧‧‧ Real-time live image
15‧‧‧導航裝置15‧‧‧ navigation device
150‧‧‧導航資訊150‧‧‧ Navigation Information
17‧‧‧使用者影像擷取裝置17‧‧‧User image capture device
170‧‧‧即時使用者影像170‧‧‧ Instant user image
19‧‧‧顯示裝置19‧‧‧ display device
21‧‧‧車輛21‧‧‧ Vehicles
27‧‧‧實景影像擷取裝置之位置27‧‧‧Location of the real image capture device
第1圖係為本發明之第一實施例之示意圖;Figure 1 is a schematic view of a first embodiment of the present invention;
第2圖係為裝載第一實施例之即時擴增實境導航顯示系統之車輛行駛於路面之示意圖;以及Figure 2 is a schematic view showing the vehicle on the road surface loaded with the instant augmented reality navigation display system of the first embodiment;
第3A圖至第3E圖係為本發明之第二實施例之流程圖。3A to 3E are flowcharts of a second embodiment of the present invention.
1‧‧‧即時擴增實境導航顯示系統1‧‧‧ Instant Augmented Reality Navigation Display System
11‧‧‧即時擴增實境裝置11‧‧‧ Instant Augmented Reality Device
111‧‧‧傳送/接收介面111‧‧‧Transfer/receive interface
113‧‧‧儲存器113‧‧‧Storage
1130‧‧‧實際尺寸1130‧‧‧ actual size
1132‧‧‧預設位置1132‧‧‧Preset location
1134‧‧‧預設視線角度1134‧‧‧Preset line of sight
115‧‧‧微處理器115‧‧‧Microprocessor
114‧‧‧投影器114‧‧‧Projector
117‧‧‧指引資訊117‧‧‧Guide information
119‧‧‧導航影像119‧‧‧Navigation images
13‧‧‧實景影像擷取裝置13‧‧‧Real image capture device
130‧‧‧即時實景影像130‧‧‧ Real-time live image
15‧‧‧導航裝置15‧‧‧ navigation device
150‧‧‧導航資訊150‧‧‧ Navigation Information
17‧‧‧使用者影像擷取裝置17‧‧‧User image capture device
170‧‧‧即時使用者影像170‧‧‧ Instant user image
19‧‧‧顯示裝置19‧‧‧ display device
Claims (21)
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
TW99108329A TWI408342B (en) | 2010-03-22 | 2010-03-22 | Real-time augmented reality device, real-time augmented reality method and computer program product thereof |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
TW99108329A TWI408342B (en) | 2010-03-22 | 2010-03-22 | Real-time augmented reality device, real-time augmented reality method and computer program product thereof |
Publications (2)
Publication Number | Publication Date |
---|---|
TW201132936A TW201132936A (en) | 2011-10-01 |
TWI408342B true TWI408342B (en) | 2013-09-11 |
Family
ID=46750996
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
TW99108329A TWI408342B (en) | 2010-03-22 | 2010-03-22 | Real-time augmented reality device, real-time augmented reality method and computer program product thereof |
Country Status (1)
Country | Link |
---|---|
TW (1) | TWI408342B (en) |
Cited By (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US10282915B1 (en) | 2017-12-27 | 2019-05-07 | Industrial Technology Research Institute | Superimposition device of virtual guiding indication and reality image and the superimposition method thereof |
TWI798789B (en) * | 2020-08-18 | 2023-04-11 | 日商優橋有限公司 | Navigation device, navigation system, navigation method, and media storaged the navigation program |
Families Citing this family (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
TWI691891B (en) | 2018-09-07 | 2020-04-21 | 財團法人工業技術研究院 | Method and apparatus for displaying information of multiple objects |
Citations (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US7511736B2 (en) * | 2003-03-10 | 2009-03-31 | Charles Benton | Augmented reality navigation system |
TW201011259A (en) * | 2008-09-12 | 2010-03-16 | Wistron Corp | Method capable of generating real-time 3D map images and navigation system thereof |
-
2010
- 2010-03-22 TW TW99108329A patent/TWI408342B/en not_active IP Right Cessation
Patent Citations (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US7511736B2 (en) * | 2003-03-10 | 2009-03-31 | Charles Benton | Augmented reality navigation system |
TW201011259A (en) * | 2008-09-12 | 2010-03-16 | Wistron Corp | Method capable of generating real-time 3D map images and navigation system thereof |
Cited By (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US10282915B1 (en) | 2017-12-27 | 2019-05-07 | Industrial Technology Research Institute | Superimposition device of virtual guiding indication and reality image and the superimposition method thereof |
TWI798789B (en) * | 2020-08-18 | 2023-04-11 | 日商優橋有限公司 | Navigation device, navigation system, navigation method, and media storaged the navigation program |
Also Published As
Publication number | Publication date |
---|---|
TW201132936A (en) | 2011-10-01 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US11935197B2 (en) | Adaptive vehicle augmented reality display using stereographic imagery | |
EP3338136B1 (en) | Augmented reality in vehicle platforms | |
JP6780642B2 (en) | Information processing equipment, information processing methods and programs | |
US9934614B2 (en) | Fixed size augmented reality objects | |
KR102079097B1 (en) | Device and method for implementing augmented reality using transparent display | |
US8503762B2 (en) | Projecting location based elements over a heads up display | |
JP6304628B2 (en) | Display device and display method | |
JP6536856B2 (en) | Vehicle display device | |
TWI408339B (en) | Real-time augmented reality device, real-time augmented reality methode and computer program product thereof | |
US20150161762A1 (en) | Information processing apparatus, information processing method, and program | |
US10532814B2 (en) | Augmented reality travel route planning | |
US9639986B2 (en) | Head mounted display and method of controlling thereof | |
WO2019224922A1 (en) | Head-up display control device, head-up display system, and head-up display control method | |
KR20110114114A (en) | Real 3d navigation implementing method | |
TWI453462B (en) | Telescopic observation for virtual reality system and method thereof using intelligent electronic device | |
CN109990797A (en) | A kind of control method of the augmented reality navigation display for HUD | |
CN102200445B (en) | Real-time augmented reality device and method thereof | |
US11410330B2 (en) | Methods, devices, and systems for determining field of view and producing augmented reality | |
JP6345381B2 (en) | Augmented reality system | |
TWI408342B (en) | Real-time augmented reality device, real-time augmented reality method and computer program product thereof | |
TWI799000B (en) | Method, processing device, and display system for information display | |
TW201919390A (en) | Display system and method thereof | |
WO2022172335A1 (en) | Virtual guide display device, virtual guide display system, and virtual guide display method | |
US20240236257A1 (en) | Method, processing device, and system for information display | |
JP2024040034A (en) | Processor, image processing apparatus, spectacle-type information display device, image processing method, and image processing program |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
MM4A | Annulment or lapse of patent due to non-payment of fees |