TWI844902B - Method, processing device, and display system for information display - Google Patents

Method, processing device, and display system for information display Download PDF

Info

Publication number
TWI844902B
TWI844902B TW111128679A TW111128679A TWI844902B TW I844902 B TWI844902 B TW I844902B TW 111128679 A TW111128679 A TW 111128679A TW 111128679 A TW111128679 A TW 111128679A TW I844902 B TWI844902 B TW I844902B
Authority
TW
Taiwan
Prior art keywords
display
information
user
target
acquisition device
Prior art date
Application number
TW111128679A
Other languages
Chinese (zh)
Other versions
TW202311817A (en
Inventor
羅翌瑋
陳健龍
鄭莛薰
趙玉如
林郁欣
Original Assignee
財團法人工業技術研究院
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 財團法人工業技術研究院 filed Critical 財團法人工業技術研究院
Priority to CN202210997341.5A priority Critical patent/CN115794004A/en
Priority to US17/939,900 priority patent/US20230070721A1/en
Publication of TW202311817A publication Critical patent/TW202311817A/en
Application granted granted Critical
Publication of TWI844902B publication Critical patent/TWI844902B/en

Links

Images

Abstract

A method, a processing device, and a system for information display are proposed, and the system includes a light transmissive display. A first information extraction device extracts spatial position information of a user, and a second information extraction device extracts spatial position information of a target object. The processing device performs the following steps. Display position information of virtual information of the target object on the display is determined according to the spatial position information of the user and the spatial position information of the target object. The display position information includes a first display reference position corresponding to a previous time and a second display position corresponding to a current time. An actual display position of the virtual information on the display corresponding to the current time is determined according to the distance between the first display reference position and the second display reference position. The virtual information is displayed on the display according to the actual display position.

Description

資訊顯示方法及其處理裝置與資訊顯示系統Information display method and processing device thereof and information display system

本發明是有關於一種資訊顯示技術。 The present invention relates to an information display technology.

隨著影像處理技術與空間定位技術的發展，透明顯示器的應用已逐漸受到重視。此類的技術可讓顯示器搭配實體物件，再輔以虛擬相關資訊，並且依照使用者的需求來產生互動式的體驗，可使資訊以更為直觀的方式呈現。 With the development of image processing technology and spatial positioning technology, the application of transparent displays has gradually received attention. This type of technology allows displays to be paired with physical objects, supplemented with virtual related information, and to generate interactive experiences based on user needs, allowing information to be presented in a more intuitive way.

再者，關聯於實體物件的虛擬資訊可顯示於透明顯示器的特定位置上，讓使用者可透過透明顯示器同時觀看到實體物件與疊加於實體物件上或一側的虛擬資訊。然而，當實體物件動態移動時，可能發生虛擬資訊偏離實體物件的情況，進而導致透明顯示器所顯示的虛擬資訊無法即時地跟隨實體物件移動。此外，由於實體物件的移動不一定可被預測或可能是多變的，因此可能造成基於即時物件辨識而顯示的虛擬資訊出現晃動或堆疊的情況，導致 觀看者閱讀虛擬資訊的困難度與不適。 Furthermore, virtual information related to physical objects can be displayed at a specific position of the transparent display, so that users can simultaneously view the physical object and the virtual information superimposed on or on one side of the physical object through the transparent display. However, when the physical object moves dynamically, the virtual information may deviate from the physical object, resulting in the virtual information displayed by the transparent display being unable to follow the movement of the physical object in real time. In addition, since the movement of the physical object may not be predictable or may be variable, the virtual information displayed based on real-time object recognition may shake or overlap, resulting in difficulty and discomfort for the viewer to read the virtual information.

本揭露提供一種資訊顯示方法及其處理裝置與資訊顯示系統。 This disclosure provides an information display method and its processing device and information display system.

在本揭露的一範例實施例中，上述的資訊顯示系統包括可透光的顯示器、第一資訊擷取裝置、第二資訊擷取裝置以及處理裝置，其中處理裝置連接於顯示器、第一資訊擷取裝置以及第二資訊擷取裝置。第一資訊擷取裝置用以擷取使用者的空間位置資訊，第二資訊擷取裝置用以擷取目標物的空間位置資訊。處理裝置經配置以執行下列步驟。根據使用者的空間位置資訊以及目標物的空間位置資訊決定目標物的虛擬資訊於顯示器上的顯示位置資訊。顯示位置資訊包括對應於先前時間的第一顯示參考位置與對應至當前時間的第二顯示參考位置。根據第一顯示參考位置與第二顯示參考位置之間的距離決定虛擬資訊於顯示器上對應於當前時間的實際顯示位置。根據實際顯示位置顯示虛擬資訊於顯示器上。 In an exemplary embodiment of the present disclosure, the above-mentioned information display system includes a light-transmitting display, a first information acquisition device, a second information acquisition device and a processing device, wherein the processing device is connected to the display, the first information acquisition device and the second information acquisition device. The first information acquisition device is used to acquire the spatial position information of the user, and the second information acquisition device is used to acquire the spatial position information of the target object. The processing device is configured to perform the following steps. The display position information of the virtual information of the target object on the display is determined based on the spatial position information of the user and the spatial position information of the target object. The display position information includes a first display reference position corresponding to a previous time and a second display reference position corresponding to a current time. Determine the actual display position of the virtual information on the display corresponding to the current time according to the distance between the first display reference position and the second display reference position. Display the virtual information on the display according to the actual display position.

在本揭露的一範例實施例中，上述的資訊顯示方法適用於具有可透光的顯示器、第一資訊擷取裝置、第二資訊擷取裝置以及處理裝置的資訊顯示系統，並且包括下列步驟。利用第一資訊擷取裝置擷取使用者的空間位置資訊。利用第二資訊擷取裝置擷取目標物的空間位置資訊。根據使用者的空間位置資訊以及目標物的空間位置資訊決定目標物的虛擬資訊於顯示器上的顯示位置資 訊。顯示位置資訊包括對應於先前時間的第一顯示參考位置與對應至當前時間的第二顯示參考位置。根據第一顯示參考位置與第二顯示參考位置之間的距離決定虛擬資訊於顯示器上對應於當前時間的實際顯示位置。根據實際顯示位置顯示虛擬資訊於顯示器上。 In an exemplary embodiment of the present disclosure, the above-mentioned information display method is applicable to an information display system having a light-transmissive display, a first information acquisition device, a second information acquisition device, and a processing device, and includes the following steps. The spatial position information of the user is acquired by the first information acquisition device. The spatial position information of the target object is acquired by the second information acquisition device. The display position information of the virtual information of the target object on the display is determined based on the spatial position information of the user and the spatial position information of the target object. The display position information includes a first display reference position corresponding to a previous time and a second display reference position corresponding to a current time. Determine the actual display position of the virtual information on the display corresponding to the current time according to the distance between the first display reference position and the second display reference position. Display the virtual information on the display according to the actual display position.

在本揭露的一範例實施例中，上述的處理裝置連接於可透光的顯示器、第一資訊擷取裝置以及第二資訊擷取裝置。第一資訊擷取裝置用以擷取使用者的空間位置資訊，第二資訊擷取裝置用以擷取目標物的空間位置資訊。處理裝置包括記憶體與連接記憶體的處理器。記憶體用以儲存資料，而處理器經配置以執行下列步驟。利用第一資訊擷取裝置擷取使用者的空間位置資訊。利用第二資訊擷取裝置擷取目標物的空間位置資訊。根據使用者的空間位置資訊以及目標物的空間位置資訊決定目標物的虛擬資訊於顯示器上的顯示位置資訊。顯示位置資訊包括對應於先前時間的第一顯示參考位置與對應至當前時間的第二顯示參考位置。根據第一顯示參考位置與第二顯示參考位置之間的距離決定虛擬資訊於顯示器上對應於當前時間的實際顯示位置。根據實際顯示位置顯示虛擬資訊於顯示器上。 In an exemplary embodiment of the present disclosure, the above-mentioned processing device is connected to a light-transmissive display, a first information acquisition device, and a second information acquisition device. The first information acquisition device is used to capture the spatial position information of the user, and the second information acquisition device is used to capture the spatial position information of the target object. The processing device includes a memory and a processor connected to the memory. The memory is used to store data, and the processor is configured to perform the following steps. Use the first information acquisition device to capture the spatial position information of the user. Use the second information acquisition device to capture the spatial position information of the target object. Determine the display position information of the virtual information of the target object on the display based on the spatial position information of the user and the spatial position information of the target object. The display position information includes a first display reference position corresponding to the previous time and a second display reference position corresponding to the current time. The actual display position of the virtual information on the display corresponding to the current time is determined according to the distance between the first display reference position and the second display reference position. The virtual information is displayed on the display according to the actual display position.

為讓本揭露能更明顯易懂，下文特舉實施例，並配合所附圖式作詳細說明如下。 In order to make this disclosure more clear and easy to understand, the following is a specific example and a detailed description with the attached drawings.

100:資訊顯示系統 100: Information display system

110:顯示器 110: Display

120:第一資訊擷取裝置 120: First information acquisition device

130:第二資訊擷取裝置 130: Second information acquisition device

140:處理裝置 140: Processing device

141:記憶體 141:Memory

142:處理器 142:Processor

Vf1:虛擬資訊 Vf1: Virtual information

RF1:參考顯示物件框 RF1: Reference display object frame

U1:使用者 U1: User

Obj1:目標物 Obj1: Target object

CX1、CX2:鏡頭光軸 CX1, CX2: lens optical axis

HX1:顯示平面法線 HX1: Display plane normals

IP1:交點位置 IP1: intersection position

IP1_p:第一顯示參考位置 IP1_p: First display reference position

IP1_c:第二顯示參考位置 IP1_c: Second display reference position

BL:左邊界 BL: Left Boundary

UL:上邊界 UL: Upper limit

Q:交點 Q: Intersection point

L1:中線 L1: Center line

Roi1:物件邊界框 Roi1: object bounding box

Roi_p:第一參考顯示物件框 Roi_p: First reference display object frame

Roi_c:第二參考顯示物件框 Roi_c: Second reference display object frame

S210~S250、S241~S243、S402~S422、S702~S718:步驟 S210~S250, S241~S243, S402~S422, S702~S718: Steps

圖1A是根據本揭露一範例實施例所繪示的資訊顯示系統的方塊圖。 FIG. 1A is a block diagram of an information display system according to an exemplary embodiment of the present disclosure.

圖1B是根據本揭露一範例實施例所繪示的資訊顯示系統的示意圖。 FIG. 1B is a schematic diagram of an information display system according to an exemplary embodiment of the present disclosure.

圖2是根據本揭露一範例實施例所繪示的資訊顯示方法的流程圖。 FIG2 is a flow chart of an information display method according to an exemplary embodiment of the present disclosure.

圖3是根據本揭露一範例實施例所繪示的決定對應於當前時間的實際顯示位置的流程圖。 FIG3 is a flow chart of determining the actual display position corresponding to the current time according to an exemplary embodiment of the present disclosure.

圖4是根據本揭露一範例實施例所繪示的資訊顯示方法的流程圖。 FIG4 is a flow chart of an information display method according to an exemplary embodiment of the present disclosure.

圖5A是根據本揭露一範例實施例所繪示的第一影像感測器與第二影像感測器的拍攝方向的示意圖。 FIG5A is a schematic diagram showing the shooting directions of the first image sensor and the second image sensor according to an exemplary embodiment of the present disclosure.

圖5B是根據本揭露一範例實施例所繪示的決定顯示位置資訊的示意圖。 FIG. 5B is a schematic diagram showing the determination of displaying location information according to an exemplary embodiment of the present disclosure.

圖6A是根據本揭露一範例實施例所繪示的不更新實際顯示位置的示意圖。 FIG. 6A is a schematic diagram showing a non-updated actual display position according to an exemplary embodiment of the present disclosure.

圖6B是根據本揭露一範例實施例所繪示的更新實際顯示位置的示意圖。 FIG. 6B is a schematic diagram of updating the actual display position according to an exemplary embodiment of the present disclosure.

圖7是根據本揭露一範例實施例所繪示的資訊顯示方法的流程圖。 FIG7 is a flow chart of an information display method according to an exemplary embodiment of the present disclosure.

圖8A與圖8B是根據本揭露一範例實施例所繪示的決定顯示 位置資訊的示意圖。 FIG8A and FIG8B are schematic diagrams showing the determination of displaying location information according to an exemplary embodiment of the present disclosure.

圖9A是根據本揭露一範例實施例所繪示的不更新實際顯示位置的示意圖。 FIG. 9A is a schematic diagram showing a non-updated actual display position according to an exemplary embodiment of the present disclosure.

圖9B是根據本揭露一範例實施例所繪示的更新實際顯示位置的示意圖。 FIG. 9B is a schematic diagram showing an updated actual display position according to an exemplary embodiment of the present disclosure.

本揭露的部份範例實施例接下來將會配合附圖來詳細描述，以下的描述所引用的元件符號，當不同附圖出現相同的元件符號將視為相同或相似的元件。這些範例實施例只是本揭露的一部份，並未揭示所有本揭露的可實施方式。更確切的說，這些範例實施例僅為本揭露的專利申請範圍中的方法、裝置以及系統的範例。 Some exemplary embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings. When the same element symbols appear in different drawings, they will be regarded as the same or similar elements. These exemplary embodiments are only part of the present disclosure and do not disclose all possible implementations of the present disclosure. More specifically, these exemplary embodiments are only examples of methods, devices and systems within the scope of the patent application of the present disclosure.

圖1A是根據本揭露一範例實施例所繪示的資訊顯示系統的方塊圖。首先圖1A先介紹系統中的各個構件以及配置關係，詳細功能將配合後續範例實施例的流程圖一併揭露。 FIG. 1A is a block diagram of an information display system according to an exemplary embodiment of the present disclosure. First, FIG. 1A introduces the components and configuration relationships in the system, and the detailed functions will be disclosed together with the flow chart of the subsequent exemplary embodiment.

請參照圖1A，本範例實施例中的資訊顯示系統100可包括顯示器110、第一資訊擷取裝置120、第二資訊擷取裝置130以及處理裝置140，其中處理裝置140可以是以無線、有線或電性連接於顯示器110、第一資訊擷取裝置120以及第二資訊擷取裝置130。 Referring to FIG. 1A , the information display system 100 in this exemplary embodiment may include a display 110 , a first information acquisition device 120 , a second information acquisition device 130 , and a processing device 140 , wherein the processing device 140 may be wirelessly, wiredly, or electrically connected to the display 110 , the first information acquisition device 120 , and the second information acquisition device 130 .

顯示器110可用以顯示資訊，其包括例如是液晶顯示器(Liquid crystal display，LCD)、場色序(Field sequential color) 液晶顯示器、發光二極體(Light emitting diode，LED)顯示器、電濕潤顯示器等穿透式可透光顯示器，或者是投影式可透光顯示器。 The display 110 can be used to display information, and includes, for example, a liquid crystal display (LCD), a field sequential color display, a light emitting diode (LED) display, an electro-wetting display, or a transmissive light display, or a projection-type transmissive light display.

第一資訊擷取裝置120用以擷取使用者的空間位置資訊，而第二資訊擷取裝置130用以擷取實體場景中目標物的空間位置資訊。第一資訊擷取裝置120可以例如是至少一個影像感測器或者是至少一個影像感測器搭配至少一個深度感測器來對使用者進行影像辨識定位，其中影像感測器可包括可見光感測器或非可見光感測器如紅外線感測器等。此外，第一資訊擷取裝置120更可以例如是光學***來對使用者進行光學空間定位。只要是可以定位出使用者所在位置資訊的裝置或其組合，皆屬於第一資訊擷取裝置120的範疇。 The first information acquisition device 120 is used to acquire the spatial position information of the user, and the second information acquisition device 130 is used to acquire the spatial position information of the target object in the physical scene. The first information acquisition device 120 may be, for example, at least one image sensor or at least one image sensor with at least one depth sensor to perform image recognition and positioning of the user, wherein the image sensor may include a visible light sensor or a non-visible light sensor such as an infrared sensor. In addition, the first information acquisition device 120 may be, for example, an optical positioner to perform optical spatial positioning of the user. Any device or combination thereof that can locate the user's location information belongs to the scope of the first information acquisition device 120.

第二資訊擷取裝置130則可以例如是至少一個影像感測器或者是至少一個影像感測器搭配至少一個深度感測器來對目標物進行影像辨識定位，其中影像感測器可包括可見光感測器或非可見光感測器如紅外線感測器等。只要是可以定位出目標物所在位置資訊的裝置或其組合，皆屬於第二資訊擷取裝置130的範疇。 The second information acquisition device 130 can be, for example, at least one image sensor or at least one image sensor combined with at least one depth sensor to perform image recognition and positioning of the target, wherein the image sensor may include a visible light sensor or a non-visible light sensor such as an infrared sensor. Any device or combination thereof that can locate the location information of the target falls within the scope of the second information acquisition device 130.

於本揭露實施例中，上述的影像感測器可用以擷取影像並且包括具有透鏡以及感光元件的攝像鏡頭。上述的深度感測器可用以偵測深度資訊，其可以利用主動式深度感測技術以及被動式深度感測技術來實現。主動式深度感測技術可藉由主動發出光源、紅外線、超音波、雷射等作為訊號搭配時差測距技術來計算深 度資訊。被動式深度感測技術可以藉由兩個影像感測器以不同視角擷取其前方的兩張影像，以利用兩張影像的視差來計算深度資訊。 In the disclosed embodiment, the above-mentioned image sensor can be used to capture images and includes a camera lens having a lens and a photosensitive element. The above-mentioned depth sensor can be used to detect depth information, which can be realized by using active depth sensing technology and passive depth sensing technology. Active depth sensing technology can calculate depth information by actively emitting light, infrared, ultrasound, laser, etc. as signals and using time difference ranging technology. Passive depth sensing technology can use two image sensors to capture two images in front of it at different viewing angles to calculate depth information using the parallax of the two images.

處理裝置140用以控制資訊顯示系統100的作動，其可包括記憶體141以及處理器142。記憶體141可以例如是任意型式的固定式或可移動式隨機存取記憶體(random access memory，RAM)、唯讀記憶體(read-only memory，ROM)、快閃記憶體(flash memory)、硬碟或其他類似裝置、積體電路及其組合。處理器142可以例如是中央處理單元(central processing unit，CPU)、應用處理器(application processor，AP)，或是其他可程式化之一般用途或特殊用途的微處理器(microprocessor)、數位訊號處理器(digital signal processor，DSP)、影像訊號處理器(image signal processor，ISP)、圖形處理器(graphics processing unit，GPU)或其他類似裝置、積體電路及其組合。以下範例實施例中將搭配資訊顯示系統100的各元件來說明處理裝置140執行資訊顯示方法流程的細節。 The processing device 140 is used to control the operation of the information display system 100, and may include a memory 141 and a processor 142. The memory 141 may be, for example, any type of fixed or removable random access memory (RAM), read-only memory (ROM), flash memory, hard disk or other similar devices, integrated circuits and combinations thereof. The processor 142 may be, for example, a central processing unit (CPU), an application processor (AP), or other programmable general-purpose or special-purpose microprocessor, a digital signal processor (DSP), an image signal processor (ISP), a graphics processing unit (GPU) or other similar devices, integrated circuits and combinations thereof. The following exemplary embodiments will be used with the components of the information display system 100 to illustrate the details of the processing device 140 executing the information display method flow.

在本範例實施例中，處理裝置140可以是內建於顯示器110或連接顯示器110的計算機裝置。第一資訊擷取裝置120以及第二資訊擷取裝置130可以是分別設置於資訊顯示系統100所屬場域相對於顯示器110的相對兩側等，用以對使用者以及目標物進行定位，並且透過各自的通訊介面以有線或是無線的方式傳輸資訊至處理裝置140。於一些實施例中，第一資訊擷取裝置120與第二資訊擷取裝置130也可各自具有處理器與記憶體，並具有可 根據影像資料進行物件辨識與物件追蹤的計算能力。在另一範例實施例中，資訊顯示系統100可為單一整合系統，其可以實作成頭戴式顯示裝置、智慧型手機、平板電腦等，本揭露不在此設限。以下範例實施例中將搭配資訊顯示系統100的各元件來說明資訊顯示方法的實施細節。 In this exemplary embodiment, the processing device 140 may be a computer device built into the display 110 or connected to the display 110. The first information acquisition device 120 and the second information acquisition device 130 may be respectively disposed on opposite sides of the display 110 in the field of the information display system 100, etc., to locate the user and the target object, and transmit information to the processing device 140 in a wired or wireless manner through their respective communication interfaces. In some embodiments, the first information acquisition device 120 and the second information acquisition device 130 may also each have a processor and a memory, and have computing capabilities for object recognition and object tracking based on image data. In another exemplary embodiment, the information display system 100 may be a single integrated system, which may be implemented as a head-mounted display device, a smart phone, a tablet computer, etc., but the present disclosure is not limited thereto. The following exemplary embodiments will illustrate the implementation details of the information display method in conjunction with the various components of the information display system 100.

圖1B是根據本揭露一範例實施例所繪示的資訊顯示系統的示意圖。請參照圖1B，使用者U1與目標物Obj1分別位於顯示器110的不同側。使用者U1可透過顯示器110觀看疊加有目標物Obj1的虛擬資訊Vf1的實體場景。虛擬資訊Vf1可視為基於目標物Obj1而擴增的擴增實境內容。於一些實施例中，第一資訊擷取裝置120可包括用以拍攝使用者U1的第一影像感測器，且第二資訊擷取裝置130包括用以拍攝目標物Obj1的第二影像感測器。第一資訊擷取裝置120可基於所擷取的使用者影像獲取使用者U1的空間位置資訊，而第二資訊擷取裝置130可基於所擷取的目標物影像獲取目標物Obj1的空間位置資訊。 FIG1B is a schematic diagram of an information display system according to an exemplary embodiment of the present disclosure. Referring to FIG1B , the user U1 and the target object Obj1 are located on different sides of the display 110. The user U1 can view the physical scene superimposed with the virtual information Vf1 of the target object Obj1 through the display 110. The virtual information Vf1 can be regarded as an augmented reality content expanded based on the target object Obj1. In some embodiments, the first information capture device 120 may include a first image sensor for photographing the user U1, and the second information capture device 130 includes a second image sensor for photographing the target object Obj1. The first information acquisition device 120 can obtain the spatial location information of the user U1 based on the captured user image, and the second information acquisition device 130 can obtain the spatial location information of the target object Obj1 based on the captured target object image.

處理裝置140可根據使用者U1的空間位置資訊以及目標物Obj1的空間位置資訊來決定虛擬資訊Vf1於顯示器110上的顯示位置資訊。於圖1B的範例中，上述顯示位置資訊實施為顯示平面上的一個參考顯示物件框RF1。於其他實施例中，顯示位置資訊可實施為顯示器110的顯示平面上的一個點。具體來說，顯示位置資訊可視為使用者觀看目標物Obj1時視線投射於顯示平面上的落點或區域。藉此，處理裝置140可利用顯示位置資訊(例如參 考顯示物件框RF1)為參考基準來顯示虛擬資訊Vf1。更具體而言，基於各式需求或不同應用，處理裝置140可根據顯示位置資訊決定虛擬資訊Vf1的實際顯示位置，以讓使用者U1可透過顯示器110看到位於目標物Obj1附近的虛擬資訊Vf1或看到疊加於目標物Obj1上的虛擬資訊Vf1。 The processing device 140 can determine the display position information of the virtual information Vf1 on the display 110 according to the spatial position information of the user U1 and the spatial position information of the target object Obj1. In the example of FIG. 1B , the display position information is implemented as a reference display object frame RF1 on the display plane. In other embodiments, the display position information can be implemented as a point on the display plane of the display 110. Specifically, the display position information can be regarded as the point or area where the user's line of sight is projected on the display plane when viewing the target object Obj1. In this way, the processing device 140 can use the display position information (e.g., the reference display object frame RF1) as a reference to display the virtual information Vf1. More specifically, based on various needs or different applications, the processing device 140 can determine the actual display position of the virtual information Vf1 according to the display position information, so that the user U1 can see the virtual information Vf1 near the target object Obj1 or the virtual information Vf1 superimposed on the target object Obj1 through the display 110.

值得一提的是，於一些實施例中，使用者U1及/或目標物Obj1都可能會動態移動。舉例而言，目標物Obj1可以是飼養於水族箱裡的魚隻，而虛擬資訊Vf1可以是魚隻的種類名稱或介紹資訊。於是，第一資訊擷取裝置120可對著使用者U1持續進行拍攝而產生包括分別對應至不同時間點的多張使用者影像的視頻序列，第一資訊擷取裝置120可根據包括多張使用者影像的視頻序列來追蹤使用者U1的空間位置資訊。同理，第二資訊擷取裝置130可對著目標物Obj1持續進行拍攝而產生包括分別對應至不同時間點的多張目標物影像的視頻序列，第二資訊擷取裝置130可根據包括多張目標物影像的視頻序列來追蹤目標物Obj1的空間位置資訊。由於處理裝置140可基於目標物Obj1與使用者U1的追蹤結果不斷地更新虛擬資訊Vf1的顯示位置資訊，因此顯示器110所顯示的虛擬資訊Vf1可反應於使用者U1的移動與/或目標物Obj1的移動而對應移動，以達到虛擬資訊Vf1跟隨目標物Obj1的視覺效果。 It is worth mentioning that in some embodiments, the user U1 and/or the target object Obj1 may move dynamically. For example, the target object Obj1 may be a fish kept in an aquarium, and the virtual information Vf1 may be the species name or introduction information of the fish. Therefore, the first information acquisition device 120 may continuously shoot the user U1 to generate a video sequence including multiple user images corresponding to different time points, and the first information acquisition device 120 may track the spatial position information of the user U1 according to the video sequence including the multiple user images. Similarly, the second information acquisition device 130 can continuously shoot the target object Obj1 to generate a video sequence including multiple target object images corresponding to different time points. The second information acquisition device 130 can track the spatial position information of the target object Obj1 according to the video sequence including the multiple target object images. Since the processing device 140 can continuously update the display position information of the virtual information Vf1 based on the tracking results of the target object Obj1 and the user U1, the virtual information Vf1 displayed by the display 110 can respond to the movement of the user U1 and/or the movement of the target object Obj1 and move accordingly, so as to achieve the visual effect that the virtual information Vf1 follows the target object Obj1.

圖2是根據本揭露一範例實施例所繪示的資訊顯示方法的流程圖，請同時參照圖1A、圖1B以及圖2，而圖2的方法流程 可由圖1A與圖1B的資訊顯示系統100來實現。在此，使用者U1可透過資訊顯示系統100的顯示器110來觀看目標物Obj1及其虛擬資訊Vf1。 FIG. 2 is a flow chart of an information display method according to an exemplary embodiment of the present disclosure. Please refer to FIG. 1A, FIG. 1B and FIG. 2 at the same time. The method flow of FIG. 2 can be implemented by the information display system 100 of FIG. 1A and FIG. 1B. Here, the user U1 can view the target object Obj1 and its virtual information Vf1 through the display 110 of the information display system 100.

於步驟S210，資訊顯示系統100中利用第一資訊擷取裝置120擷取使用者U1的空間位置資訊。於步驟S220，資訊顯示系統100中利用第二資訊擷取裝置130擷取目標物Obj1的空間位置資訊。如同前述，第一資訊擷取裝置120以及第二資訊擷取裝置130例如是可針對使用者U1以及目標物Obj1的所在位置進行定位的影像感測器、深度感測器或其組合。 In step S210, the information display system 100 uses the first information acquisition device 120 to acquire the spatial position information of the user U1. In step S220, the information display system 100 uses the second information acquisition device 130 to acquire the spatial position information of the target object Obj1. As mentioned above, the first information acquisition device 120 and the second information acquisition device 130 are, for example, image sensors, depth sensors, or a combination thereof that can locate the positions of the user U1 and the target object Obj1.

於步驟S230，資訊顯示系統100的處理裝置140根據使用者U1的空間位置資訊以及目標物Obj1的空間位置資訊決定目標物Obj1的虛擬資訊Vf1於顯示器110上的顯示位置資訊。顯示位置資訊包括對應於先前時間的第一顯示參考位置與對應至當前時間的第二顯示參考位置。具體而言，處理裝置140可根據使用者U1的空間位置資訊以及目標物Obj1的空間位置資訊持續地計算出分別對應至多個連續時間的多個顯示參考位置。換言之，第一資訊擷取裝置120與第二資訊擷取裝置130可持續更新目標物Obj1與使用者U1的空間位置資訊，而處理裝置140可據以持續更新虛擬資訊Vf1的顯示位置資訊。 In step S230, the processing device 140 of the information display system 100 determines the display position information of the virtual information Vf1 of the target object Obj1 on the display 110 according to the spatial position information of the user U1 and the spatial position information of the target object Obj1. The display position information includes a first display reference position corresponding to the previous time and a second display reference position corresponding to the current time. Specifically, the processing device 140 can continuously calculate multiple display reference positions corresponding to multiple continuous times according to the spatial position information of the user U1 and the spatial position information of the target object Obj1. In other words, the first information acquisition device 120 and the second information acquisition device 130 can continuously update the spatial location information of the target object Obj1 and the user U1, and the processing device 140 can continuously update the display location information of the virtual information Vf1 accordingly.

於步驟S240，資訊顯示系統100的處理裝置140根據第一顯示參考位置與第二顯示參考位置之間的距離決定虛擬資訊Vf1於顯示器110上對應於當前時間的實際顯示位置。接著，於步 驟S250，資訊顯示系統100的處理裝置140根據實際顯示位置顯示虛擬資訊Vf1於顯示器110上。 In step S240, the processing device 140 of the information display system 100 determines the actual display position of the virtual information Vf1 on the display 110 corresponding to the current time according to the distance between the first display reference position and the second display reference position. Then, in step S250, the processing device 140 of the information display system 100 displays the virtual information Vf1 on the display 110 according to the actual display position.

也就是說，於處理裝置140決定虛擬資訊Vf1對應於當前時間的實際顯示位置的過程中，處理裝置140會計算先前時間的第一顯示參考位置與當前時間的第二顯示參考位置之間的距離。接著，處理裝置140再根據上述距離來決定利用當前時間的第二顯示參考位置更新虛擬資訊Vf1於顯示器110上的實際顯示位置，或決定不要更新虛擬資訊Vf1的實際顯示位置。 That is, when the processing device 140 determines the actual display position of the virtual information Vf1 corresponding to the current time, the processing device 140 calculates the distance between the first display reference position at the previous time and the second display reference position at the current time. Then, the processing device 140 determines whether to update the actual display position of the virtual information Vf1 on the display 110 using the second display reference position at the current time according to the above distance, or decides not to update the actual display position of the virtual information Vf1.

圖3是根據本揭露一範例實施例所繪示的決定對應於當前時間的實際顯示位置的流程圖。請參照圖3，圖2的步驟S240可實施為步驟S241~步驟S243。於步驟S241，處理裝置140判斷對應於先前時間的第一顯示參考位置與對應於當前時間的第二顯示參考位置之間的距離是否大於預設臨界值。 FIG3 is a flowchart of determining the actual display position corresponding to the current time according to an exemplary embodiment of the present disclosure. Referring to FIG3, step S240 of FIG2 can be implemented as steps S241 to S243. In step S241, the processing device 140 determines whether the distance between the first display reference position corresponding to the previous time and the second display reference position corresponding to the current time is greater than a preset critical value.

若第一顯示參考位置與第二顯示參考位置之間的距離大於預設臨界值，代表使用者U1所看到的目標物Obj1的位移是明顯的。因此，反應於第一顯示參考位置與第二顯示參考位置之間的距離大於預設臨界值(步驟S241判斷為是)，於步驟S242，處理裝置140根據對應至當前時間的第二顯示參考位置更新虛擬資訊Vf1於顯示器110上對應於當前時間的實際顯示位置。 If the distance between the first display reference position and the second display reference position is greater than the preset critical value, it means that the displacement of the target object Obj1 seen by the user U1 is obvious. Therefore, in response to the distance between the first display reference position and the second display reference position being greater than the preset critical value (step S241 is judged as yes), in step S242, the processing device 140 updates the virtual information Vf1 on the display 110 corresponding to the actual display position of the current time according to the second display reference position corresponding to the current time.

另一方面，若第一顯示參考位置與第二顯示參考位置之間的距離未大於預設臨界值，代表使用者U1所看到的目標物Obj1的位移較小。因此，反應於第一顯示參考位置與第二顯示參考位置 之間的距離未大於預設臨界值(步驟S241判斷為否)，於步驟S243，處理裝置140不更新虛擬資訊Vf1於顯示器110上對應於當前時間的實際顯示位置，亦即對應至當前時間的實際顯示位置會相同於對應至先前時間的實際顯示位置。於一實施例中，虛擬資訊Vf1可根據基於先前時間的第一顯示參考位置所決定的實際顯示位置來進行顯示。也就是說，若第一顯示參考位置與第二顯示參考位置之間的距離未大於預設臨界值，虛擬資訊Vf1於顯示器110上的顯示位置可以不更動。藉此，虛擬資訊Vf1的實際顯示位置不會因為目標物Obj1輕微的晃動或移動而變化，大幅提昇使用者U1觀看虛擬資訊Vf1的舒適度。意即，虛擬資訊Vf1可平穩地顯示於顯示器110上，而減少晃動或殘影堆疊的情況發生。 On the other hand, if the distance between the first display reference position and the second display reference position is not greater than the preset critical value, it means that the displacement of the target object Obj1 seen by the user U1 is smaller. Therefore, in response to the fact that the distance between the first display reference position and the second display reference position is not greater than the preset critical value (step S241 is judged as no), in step S243, the processing device 140 does not update the actual display position of the virtual information Vf1 on the display 110 corresponding to the current time, that is, the actual display position corresponding to the current time will be the same as the actual display position corresponding to the previous time. In one embodiment, the virtual information Vf1 can be displayed according to the actual display position determined based on the first display reference position at the previous time. That is, if the distance between the first display reference position and the second display reference position is not greater than the preset critical value, the display position of the virtual information Vf1 on the display 110 may not be changed. In this way, the actual display position of the virtual information Vf1 will not change due to a slight shake or movement of the target object Obj1, which greatly improves the comfort of the user U1 in viewing the virtual information Vf1. In other words, the virtual information Vf1 can be displayed smoothly on the display 110, and the occurrence of shaking or afterimage stacking is reduced.

為了方便明瞭，以下將搭配顯示系統100列舉實施例以說明本揭露計算顯示位置資訊的實施方式。 For the sake of convenience and clarity, the following will be combined with a display system 100 to illustrate the implementation method of calculating and displaying location information disclosed in this disclosure.

圖4是根據本揭露一範例實施例所繪示的資訊顯示方法的流程圖。請參照圖4，於步驟S402，第一資訊擷取裝置120擷取使用者影像，其中使用者個數可為一個以上。於步驟S404，第一資訊擷取裝置120對多張使用者影像進行人臉偵測而獲取各使用者影像中的人臉位置。第一資訊擷取裝置120可透過各種人臉偵測技術與人臉特徵偵測技術從各張使用者影像中偵測出人臉區塊以及人臉區塊中的臉部特徵。人臉偵測技術例如是利用像素值分析的傳統人臉偵測演算法或應用機器學習的深度學習演算法等等，本揭露對此不限制。上述臉部特徵例如是雙眼。於一些實施例 中，第一資訊擷取裝置120可將人臉區塊的中心點作為人臉位置或將雙眼中心點作為人臉位置，但本揭露對此不限制。 FIG4 is a flow chart of an information display method according to an exemplary embodiment of the present disclosure. Referring to FIG4, in step S402, the first information capture device 120 captures user images, wherein the number of users may be more than one. In step S404, the first information capture device 120 performs face detection on multiple user images to obtain the face position in each user image. The first information capture device 120 can detect face blocks and facial features in face blocks from each user image through various face detection techniques and facial feature detection techniques. Face detection techniques include, for example, traditional face detection algorithms that utilize pixel value analysis or deep learning algorithms that apply machine learning, etc., and the present disclosure is not limited thereto. The facial features are, for example, eyes. In some embodiments, the first information acquisition device 120 may use the center point of the face block as the face position or the center point of the eyes as the face position, but the present disclosure is not limited to this.

於步驟S406，第一資訊擷取裝置120根據人臉位置進行座標轉換而獲取使用者U1的空間位置資訊。具體來說，基於使用者影像獲取的人臉位置是屬於二維的畫素座標。第一資訊擷取裝置120可根據使用者U1的深度資訊與使用者影像中的人臉位置進行座標轉換，以產生使用者U1的空間位置資訊。於圖4的實施例中，使用者U1的空間位置資訊可包括相機座標系下的使用者相機座標。 In step S406, the first information acquisition device 120 performs coordinate conversion based on the face position to obtain the spatial position information of the user U1. Specifically, the face position obtained based on the user image is a two-dimensional pixel coordinate. The first information acquisition device 120 can perform coordinate conversion based on the depth information of the user U1 and the face position in the user image to generate the spatial position information of the user U1. In the embodiment of FIG. 4, the spatial position information of the user U1 may include the user camera coordinates under the camera coordinate system.

接著，於步驟S408，處理裝置140根據第一資訊擷取裝置120的第一影像感測器的拍攝方向將使用者相機座標轉換為相對於顯示器110的使用者位置座標。於一實施例中，第一影像感測器的拍攝方向可包括垂直方向上的拍攝俯仰角。舉例而言，圖5A是根據本揭露一範例實施例所繪示的第一影像感測器與第二影像感測器的拍攝方向的示意圖。請參照圖5A，用以拍攝使用者U1的第一影像感測器具有拍攝俯仰角θ1。拍攝俯仰角θ1為第一影像感測器之鏡頭光軸CX1與顯示器110之顯示平面法線HX1之間的夾角。於一實施例中，假設使用者相機座標為(cx1,cy1,cz1)，則處理裝置140可根據下列公式(1)將使用者相機座標(cx1,cy1,cz1)轉換為相對於顯示器110的使用者位置座標(tx1,ty1,tz1)。 Then, in step S408, the processing device 140 converts the user camera coordinates into user position coordinates relative to the display 110 according to the shooting direction of the first image sensor of the first information capture device 120. In one embodiment, the shooting direction of the first image sensor may include a shooting pitch angle in the vertical direction. For example, Figure 5A is a schematic diagram of the shooting directions of the first image sensor and the second image sensor according to an exemplary embodiment of the present disclosure. Referring to Figure 5A, the first image sensor used to shoot the user U1 has a shooting pitch angle θ1. The shooting pitch angle θ1 is the angle between the lens optical axis CX1 of the first image sensor and the display plane normal HX1 of the display 110. In one embodiment, assuming that the user camera coordinates are (cx 1 , cy 1 , cz 1 ), the processing device 140 may convert the user camera coordinates (cx 1 , cy 1 , cz 1 ) into the user position coordinates (t x1 , ty1 , t z1 ) relative to the display 110 according to the following formula (1).

Figure 111128679-A0305-02-0015-1
Figure 111128679-A0305-02-0015-1

於一實施例中，相對於顯示器110的使用者位置座標(tx1,ty1,tz1)所屬的參考座標系的三座標軸分別為顯示器110的顯示X軸、顯示Y軸以及顯示平面法線。拍攝俯仰角θ1為顯示器110的顯示平面法線與第一影像感測器之鏡頭光軸CX1之間的夾角。 In one embodiment, the three coordinate axes of the reference coordinate system to which the user position coordinates ( tx1 , ty1 , tz1 ) relative to the display 110 belong are the display X axis, the display Y axis, and the display plane normal of the display 110. The shooting pitch angle θ1 is the angle between the display plane normal of the display 110 and the lens optical axis CX1 of the first image sensor.

另一方面，於步驟S410，第二資訊擷取裝置130擷取目標物影像，其中目標物個數可為一個以上。於步驟S412，第二資訊擷取裝置130對多張目標物影像進行物件偵測而獲取目標物於各目標物影像中的物件邊界框。第二資訊擷取裝置130可利用深度學習演算法中的卷積神經網路(Convolution Neural Network，CNN)模型來進行物件偵測，上述卷積神經網路模型例如是用以進行物件偵測的R-CNN、Fast R-CNN、Faster R-CNN、YOLO或SSD等等，但本揭露對此不限制。第二資訊擷取裝置130所使用的物件偵測模型可輸出目標物Obj1的物件邊界框(Bonding box)的位置與尺寸以及目標物Obj1的分類結果。於一實施例中，處理裝置140可根據目標物Obj1的分類結果而從資料庫挑選對應的虛擬資訊Vf1進行顯示。 On the other hand, in step S410, the second information acquisition device 130 captures target images, wherein the number of targets may be more than one. In step S412, the second information acquisition device 130 performs object detection on multiple target images and obtains an object bounding box of the target in each target image. The second information acquisition device 130 may use a convolution neural network (CNN) model in a deep learning algorithm to perform object detection. The convolution neural network model is, for example, R-CNN, Fast R-CNN, Faster R-CNN, YOLO or SSD, etc., which are used for object detection, but the present disclosure is not limited thereto. The object detection model used by the second information acquisition device 130 can output the position and size of the object bounding box of the target object Obj1 and the classification result of the target object Obj1. In one embodiment, the processing device 140 can select the corresponding virtual information Vf1 from the database for display according to the classification result of the target object Obj1.

於步驟S414，第二資訊擷取裝置130根據物件邊界框進行座標轉換而獲取目標物Obj1的空間位置資訊。具體來說，基於目標物影像獲取的物件邊界框的位置屬於二維的畫素座標，上述物件邊界框的基準點可以是物件邊界框的頂點位置、中心點位置或框邊界，本揭露對此不限制。第二資訊擷取裝置130可根據目標物Obj1的深度資訊與目標物影像中的物件邊界框進行座標轉換， 以產生目標物Obj1的空間位置資訊。於圖4所示的實施例中，目標物Obj1的空間位置資訊包括相機座標系下的目標物相機座標。 In step S414, the second information acquisition device 130 performs coordinate conversion based on the object bounding box to obtain the spatial position information of the target object Obj1. Specifically, the position of the object bounding box obtained based on the target image belongs to the two-dimensional pixel coordinates. The reference point of the object bounding box can be the vertex position, center point position or frame boundary of the object bounding box, and the present disclosure is not limited to this. The second information acquisition device 130 can perform coordinate conversion based on the depth information of the target object Obj1 and the object bounding box in the target image to generate the spatial position information of the target object Obj1. In the embodiment shown in FIG4, the spatial position information of the target object Obj1 includes the target camera coordinates under the camera coordinate system.

接著，於步驟S416，處理裝置140根據第二影像感測器的拍攝方向將目標物相機座標轉換為相對於顯示器110的目標物位置座標。於一實施例中，第二影像感測器的拍攝方向可包括垂直方向上的拍攝俯仰角。舉例而言，請再參照圖5A，用以拍攝目標物Obj1的第二影像感測器具有拍攝俯仰角θ2。拍攝俯仰角θ2為第二影像感測器之鏡頭光軸CX2與顯示平面法線HX1之間的夾角。假設目標物相機座標為(cx2,cy2,cz2)，則處理裝置140可根據下列公式(2)將目標物相機座標(cx2,cy2,cz2)轉換為相對於顯示器110的目標物位置座標(tx2,ty2,tz2)。 Next, in step S416, the processing device 140 converts the target camera coordinates into target position coordinates relative to the display 110 according to the shooting direction of the second image sensor. In one embodiment, the shooting direction of the second image sensor may include a shooting pitch angle in the vertical direction. For example, please refer to FIG. 5A again, the second image sensor used to shoot the target Obj1 has a shooting pitch angle θ2. The shooting pitch angle θ2 is the angle between the lens optical axis CX2 of the second image sensor and the display plane normal HX1. Assuming that the target camera coordinates are (cx 2 , cy 2 , cz 2 ), the processing device 140 may convert the target camera coordinates (cx 2 , cy 2 , cz 2 ) into target position coordinates (t x2 , ty2 , t z2 ) relative to the display 110 according to the following formula (2).

Figure 111128679-A0305-02-0017-2
Figure 111128679-A0305-02-0017-2

於一實施例中，相對於顯示器110的目標物位置座標(tx2,ty2,tz2)所屬的參考座標系相同於使用者位置座標(tx1,ty1,tz1)所屬的參考座標系。拍攝俯仰角θ2為顯示器110的顯示平面法線與第二影像感測器之鏡頭光軸CX2之間的夾角。 In one embodiment, the reference coordinate system to which the target position coordinates ( tx2 , ty2 , tz2 ) relative to the display 110 belong is the same as the reference coordinate system to which the user position coordinates ( tx1 , ty1 , tz1 ) belong. The shooting pitch angle θ2 is the angle between the display plane normal of the display 110 and the lens optical axis CX2 of the second image sensor.

於一些實施例中，第一影像感測器與第二影像感測器的拍攝方向可透過設置於第一影像感測器與第二影像感測器上的慣性感測器而獲取。以圖5A的範例來說，慣性感測器的感測值可用以計算出圖5A中的拍攝俯仰角θ1、θ2。舉例而言，上述慣性感測器例如是加速度感測器等等。 In some embodiments, the shooting directions of the first image sensor and the second image sensor can be obtained through inertial sensors disposed on the first image sensor and the second image sensor. Taking the example of FIG. 5A as an example, the sensing value of the inertial sensor can be used to calculate the shooting pitch angles θ1 and θ2 in FIG. 5A. For example, the inertial sensor is an acceleration sensor, etc.

於步驟S418，處理裝置140根據目標物位置座標與使用者位置座標之間的連線決定虛擬資訊Vf1於顯示器110上的顯示位置資訊。圖5B是根據本揭露一範例實施例所繪示的決定顯示位置資訊的示意圖。請參照圖5B，顯示位置資訊可包括目標物位置座標(tx2,ty2,tz2)與使用者位置座標(tx1,ty1,tz1)之間的連線相交於顯示器110的顯示平面的交點位置IP1。根據公式(1)與公式(2)所計算的使用者位置座標(tx1,ty1,tz1)與目標物位置座標(tx2,ty2,tz2)，位於顯示平面(即Z=0)上的交點位置IP1的空間座標(X,Y,Z)可根據下列公式(3)而計算出來。 In step S418, the processing device 140 determines the display position information of the virtual information Vf1 on the display 110 according to the connection line between the target position coordinates and the user position coordinates. FIG5B is a schematic diagram of determining the display position information according to an exemplary embodiment of the present disclosure. Referring to FIG5B, the display position information may include the intersection position IP1 where the connection line between the target position coordinates ( tx2 , ty2 , tz2 ) and the user position coordinates ( tx1 , ty1 , tz1 ) intersects on the display plane of the display 110. Based on the user position coordinates ( tx1 , ty1 , tz1 ) and the target position coordinates ( tx2 , ty2 , tz2 ) calculated by formula (1) and formula (2), the spatial coordinates (X, Y, Z) of the intersection position IP1 on the display plane (i.e., Z=0) can be calculated according to the following formula (3).

Figure 111128679-A0305-02-0018-3
Figure 111128679-A0305-02-0018-3

之後，處理裝置140可根據顯示器110的尺寸與解析度獲取顯示位置資訊，而顯示位置資訊為顯示器110的畫素座標。更詳細來說，處理裝置140可根據顯示器110的尺寸與解析度將交點位置IP1的空間座標轉換為顯示器110上的畫素座標，以獲取虛擬資訊Vf1於顯示器110上的顯示位置資訊，亦即由畫素座標來表示的交點位置IP1。 Afterwards, the processing device 140 can obtain display position information according to the size and resolution of the display 110, and the display position information is the pixel coordinates of the display 110. More specifically, the processing device 140 can convert the spatial coordinates of the intersection position IP1 into pixel coordinates on the display 110 according to the size and resolution of the display 110 to obtain the display position information of the virtual information Vf1 on the display 110, that is, the intersection position IP1 represented by pixel coordinates.

如同前述，顯示位置資訊包括對應至先前時間的第一顯示參考位置與對應至當前時間的第二顯示參考位置。於步驟S420，處理裝置140根據第一顯示參考位置與第二顯示參考位置之間的距離決定虛擬資訊Vf1於顯示器110上對應於當前時間的實際顯 示位置。於步驟S422，處理裝置140根據實際顯示位置顯示虛擬資訊Vf1於顯示器110上。 As mentioned above, the display position information includes a first display reference position corresponding to the previous time and a second display reference position corresponding to the current time. In step S420, the processing device 140 determines the actual display position of the virtual information Vf1 on the display 110 corresponding to the current time according to the distance between the first display reference position and the second display reference position. In step S422, the processing device 140 displays the virtual information Vf1 on the display 110 according to the actual display position.

詳細而言，圖6A是根據本揭露一範例實施例所繪示的不更新實際顯示位置的示意圖。圖6B是根據本揭露一範例實施例所繪示的更新實際顯示位置的示意圖。請參照圖6A與6B，於本實施範例中，顯示位置資訊包括對應至先前時間的第一顯示參考位置IP1_p與對應至當前時間的第二顯示參考位置IP1_c。第一顯示參考位置IP1_p與第二顯示參考位置IP1_c可以是根據圖4實施例之計算方式而決定的交點位置。更具體而言，基於對應至先前時間的先前使用者影像與先前目標物影像，處理裝置140可根據兩空間座標的連線而獲取顯示平面上的第一顯示參考位置IP1_p。接著，舉例而言，處理裝置140可將對應至先前時間的實際顯示位置設置為第一顯示參考位置IP1_p上方M個像素的位置，M值可以依據實際需求調整，例如參考使用者的視野範圍，使實際顯示位置落入使用者視野範圍內。之後，基於對應至當前時間的當前使用者影像與當前目標物影像，處理裝置140可根據兩空間座標的連線而獲取顯示平面上的第二顯示參考位置IP1_c。處理裝置140判斷第一顯示參考位置IP1_p與第二顯示參考位置IP1_c之間的距離△d1是否大於預設臨界值。 In detail, FIG. 6A is a schematic diagram of not updating the actual display position according to an exemplary embodiment of the present disclosure. FIG. 6B is a schematic diagram of updating the actual display position according to an exemplary embodiment of the present disclosure. Please refer to FIG. 6A and 6B. In this exemplary embodiment, the display position information includes a first display reference position IP1_p corresponding to a previous time and a second display reference position IP1_c corresponding to a current time. The first display reference position IP1_p and the second display reference position IP1_c can be intersection positions determined according to the calculation method of the embodiment of FIG. 4. More specifically, based on the previous user image and the previous target image corresponding to the previous time, the processing device 140 can obtain the first display reference position IP1_p on the display plane according to the connection line of the two spatial coordinates. Then, for example, the processing device 140 may set the actual display position corresponding to the previous time to the position M pixels above the first display reference position IP1_p. The M value may be adjusted according to actual needs, such as referring to the user's field of view so that the actual display position falls within the user's field of view. Afterwards, based on the current user image and the current target image corresponding to the current time, the processing device 140 may obtain the second display reference position IP1_c on the display plane according to the connection line of the two spatial coordinates. The processing device 140 determines whether the distance △d1 between the first display reference position IP1_p and the second display reference position IP1_c is greater than a preset critical value.

若距離△d1未大於預設臨界值，請參照圖6A，處理裝置140可不更新虛擬資訊Vf1於顯示器110上對應於當前時間的實際顯示位置，致使虛擬資訊Vf1將維持於顯示於第一顯示參考位 置IP1_p正上方M個像素的位置。相反地，若距離△d1大於預設臨界值，請參照圖6B，處理裝置140根據對應至當前時間的第二顯示參考位置IP1_c更新虛擬資訊Vf1於顯示器110上對應於當前時間的實際顯示位置，像是處理裝置140可將對應至當前時間的實際顯示位置設置為第二顯示參考位置IP1_c上方M個像素的位置。 If the distance △d1 is not greater than the preset critical value, please refer to FIG. 6A , the processing device 140 may not update the actual display position of the virtual information Vf1 on the display 110 corresponding to the current time, so that the virtual information Vf1 will remain displayed at a position M pixels above the first display reference position IP1_p. On the contrary, if the distance △d1 is greater than the preset critical value, please refer to FIG. 6B , the processing device 140 updates the actual display position of the virtual information Vf1 on the display 110 corresponding to the current time according to the second display reference position IP1_c corresponding to the current time, such as the processing device 140 may set the actual display position corresponding to the current time to a position M pixels above the second display reference position IP1_c.

圖7是根據本揭露一範例實施例所繪示的資訊顯示方法的流程圖。請參照圖7，於步驟S702，第一資訊擷取裝置120擷取使用者影像，其中使用者個數可為一個以上。於步驟S704，第一資訊擷取裝置120對多張使用者影像進行人臉偵測而獲取各使用者影像中的人臉位置。步驟S702與步驟S704的細節可參照圖4中步驟S402與步驟S404的類似相關說明，於此不再贅述。 FIG. 7 is a flow chart of an information display method according to an exemplary embodiment of the present disclosure. Referring to FIG. 7 , in step S702, the first information acquisition device 120 captures user images, wherein the number of users may be more than one. In step S704, the first information acquisition device 120 performs face detection on multiple user images to obtain the face position in each user image. The details of step S702 and step S704 can refer to the similar related descriptions of step S402 and step S404 in FIG. 4 , and will not be repeated here.

於步驟S706，第一資訊擷取裝置120根據人臉位置進行座標轉換而獲取使用者U1的空間位置資訊。於此，第一資訊擷取裝置120可將人臉位置轉換為對應至預設影像解析度的畫素座標。於圖7的實施例中，使用者U1的空間位置資訊可包括使用者U1相對於第一影像感測器的水平偏移角以及垂直偏移角。詳細而言，根據人臉位置的二維畫素座標以及第一影像感測器的水平視角(Field of View，FOV)，第一資訊擷取裝置120可計算出使用者U1相對於鏡頭光軸的水平偏移角。根據人臉位置的二維畫素座標以及第一影像感測器的垂直FOV，第一資訊擷取裝置120可計算出使用者U1相對於鏡頭光軸的垂直偏移角。 In step S706, the first information capture device 120 performs coordinate conversion based on the face position to obtain the spatial position information of the user U1. Here, the first information capture device 120 can convert the face position into pixel coordinates corresponding to the preset image resolution. In the embodiment of Figure 7, the spatial position information of the user U1 may include the horizontal offset angle and the vertical offset angle of the user U1 relative to the first image sensor. In detail, based on the two-dimensional pixel coordinates of the face position and the horizontal field of view (FOV) of the first image sensor, the first information capture device 120 can calculate the horizontal offset angle of the user U1 relative to the lens optical axis. Based on the two-dimensional pixel coordinates of the face position and the vertical FOV of the first image sensor, the first information acquisition device 120 can calculate the vertical offset angle of the user U1 relative to the lens optical axis.

於步驟S708，第二資訊擷取裝置130擷取目標物影像。於步驟S710，第二資訊擷取裝置130對多張目標物影像進行物件偵測而獲取目標物於各目標物影像中的物件邊界框。步驟S708與步驟S710的細節可參照圖4中步驟S410與步驟S412的類似相關說明，於此不再贅述。 In step S708, the second information acquisition device 130 captures the target image. In step S710, the second information acquisition device 130 performs object detection on multiple target images and obtains the object boundary frame of the target in each target image. The details of step S708 and step S710 can refer to the similar related descriptions of step S410 and step S412 in Figure 4, which will not be repeated here.

於步驟S712，第二資訊擷取裝置130根據物件邊界框進行座標轉換而獲取目標物Obj1的空間位置資訊。於此，第二資訊擷取裝置130可將物件邊界框轉換為對應至預設影像解析度的畫素座標。於圖7的實施例中，目標物Obj1的空間位置資訊可包括目標物Obj1相對於第一影像感測器的水平偏移角以及垂直偏移角。詳細而言，根據目標物影像中物件邊界框之垂直框邊界的畫素座標以及第一影像感測器的水平FOV，第一資訊擷取裝置120可計算出目標物Obj1相對於鏡頭光軸的水平偏移角，亦即物件邊界框之左右框邊界各自相對於鏡頭光軸的水平偏移角。根據目標物影像中物件邊界框之水平框邊界的畫素座標以及第一影像感測器的垂直FOV，第一資訊擷取裝置120可計算出目標物Obj1相對於鏡頭光軸的垂直偏移角，亦即物件邊界框之上下框邊界各自相對於鏡頭光軸的垂直偏移角。 In step S712, the second information acquisition device 130 performs coordinate conversion according to the object bounding box to obtain the spatial position information of the target object Obj1. Here, the second information acquisition device 130 can convert the object bounding box into pixel coordinates corresponding to the preset image resolution. In the embodiment of FIG. 7, the spatial position information of the target object Obj1 may include the horizontal offset angle and the vertical offset angle of the target object Obj1 relative to the first image sensor. Specifically, according to the pixel coordinates of the vertical frame border of the object bounding box in the target image and the horizontal FOV of the first image sensor, the first information acquisition device 120 can calculate the horizontal offset angle of the target object Obj1 relative to the lens optical axis, that is, the horizontal offset angles of the left and right frame borders of the object bounding box relative to the lens optical axis. According to the pixel coordinates of the horizontal frame border of the object bounding box in the target image and the vertical FOV of the first image sensor, the first information acquisition device 120 can calculate the vertical offset angle of the target object Obj1 relative to the lens optical axis, that is, the vertical offset angles of the upper and lower frame borders of the object bounding box relative to the lens optical axis.

於步驟S714，處理裝置140根據目標物Obj1相對於第二影像感測器的水平偏移角以及垂直偏移角、使用者U1相對於第一影像感測器的水平偏移角以及垂直偏移角、第一影像感測器的拍攝方向以及第二影像感測器的拍攝方向，決定虛擬資訊Vf1於 顯示器110上的顯示位置資訊。 In step S714, the processing device 140 determines the display position information of the virtual information Vf1 on the display 110 according to the horizontal offset angle and vertical offset angle of the target Obj1 relative to the second image sensor, the horizontal offset angle and vertical offset angle of the user U1 relative to the first image sensor, the shooting direction of the first image sensor, and the shooting direction of the second image sensor.

詳細而言，圖8A與圖8B是根據本揭露一範例實施例所繪示的決定顯示位置資訊的示意圖。於一實施例中，第一影像感測器、第二影像感測器與顯示器110之間的相對位置關係可先定義。舉例而言，若以顯示器110上緣的中心點為原點且顯示平面為XY平面，則第一影像感測器之鏡頭的位置以及第二影像感測器之鏡頭的位置可根據實際裝設位置而定義出來。 In detail, FIG. 8A and FIG. 8B are schematic diagrams of determining display position information according to an exemplary embodiment of the present disclosure. In one embodiment, the relative position relationship between the first image sensor, the second image sensor and the display 110 can be defined first. For example, if the center point of the upper edge of the display 110 is taken as the origin and the display plane is the XY plane, the position of the lens of the first image sensor and the position of the lens of the second image sensor can be defined according to the actual installation position.

請先參照圖8A，以下將先以目標物Obj1之物件邊界框Roi1的左邊界BL為範例進行說明。第一資訊擷取裝置120可將使用者U1相對於第一影像感測器的水平偏移角θ3以及使用者U1距離第一影像感測器的深度資訊d1提供給處理裝置140。第二資訊擷取裝置130可將目標物Obj1相對於第二影像感測器的水平偏移角θ4以及目標物Obj1距離第二影像感測器的深度資訊d2提供給處理裝置140。 Please refer to FIG. 8A . The following will first take the left boundary BL of the object bounding box Roi1 of the target object Obj1 as an example for explanation. The first information acquisition device 120 can provide the horizontal offset angle θ3 of the user U1 relative to the first image sensor and the depth information d1 of the user U1 from the first image sensor to the processing device 140. The second information acquisition device 130 can provide the horizontal offset angle θ4 of the target object Obj1 relative to the second image sensor and the depth information d2 of the target object Obj1 from the second image sensor to the processing device 140.

於是，處理裝置140可獲取角度θ5(θ5=180-θ3-θ4)，並基於餘弦定理而根據深度資訊d1與深度資訊d2計算出使用者U1與左邊界BL之間的距離d3。接著，處理裝置140可基於正弦定理而依據角度θ5、深度資訊d1、深度資訊d2與距離d3計算出角度θ6，並根據角度θ6與水平偏移角θ3計算出角度θ7。於是，處理裝置140可計算使用者U1與顯示器110之中線L1之間的橫向偏移距離為d1*sinθ3，以及計算左邊界BL與顯示器110之中線L1之間的橫向偏移距離為d2*sinθ4。此外，處理裝置140還可計 算出顯示平面上左邊界BL的交點Q的橫向偏移距離Qx1等於d1*sinθ3+d1*cosθ3*tanθ7。同理，物件邊界框Roi1右邊界的交點Q的橫向偏移距離亦可以類似的方式計算出。 Therefore, the processing device 140 can obtain the angle θ5 (θ5=180-θ3-θ4), and calculate the distance d3 between the user U1 and the left boundary BL according to the depth information d1 and the depth information d2 based on the cosine theorem. Then, the processing device 140 can calculate the angle θ6 according to the angle θ5, the depth information d1, the depth information d2 and the distance d3 based on the sine theorem, and calculate the angle θ7 according to the angle θ6 and the horizontal offset angle θ3. Therefore, the processing device 140 can calculate the horizontal offset distance between the user U1 and the center line L1 of the display 110 as d1*sinθ3, and calculate the horizontal offset distance between the left boundary BL and the center line L1 of the display 110 as d2*sinθ4. In addition, the processing device 140 can also calculate that the horizontal offset distance Qx1 of the intersection point Q of the left boundary BL on the display plane is equal to d1*sinθ3+d1*cosθ3*tanθ7. Similarly, the horizontal offset distance of the intersection point Q of the right boundary of the object bounding box Roi1 can also be calculated in a similar manner.

接著，請先參照圖8B，以下將先以目標物Obj1之物件邊界框Roi1的上邊界UL為範例進行說明。第一資訊擷取裝置120可將第一影像感測器的拍攝俯仰角θ1與使用者U1相對於第一影像感測器的垂直偏移角θ9提供給處理裝置140。第二資訊擷取裝置130可將第二影像感測器的拍攝俯仰角θ2與目標物Obj1相對於第二影像感測器的垂直偏移角θ8提供給處理裝置140。 Next, please refer to FIG. 8B . The following will first use the upper boundary UL of the object bounding box Roi1 of the target Obj1 as an example for explanation. The first information acquisition device 120 can provide the shooting pitch angle θ1 of the first image sensor and the vertical offset angle θ9 of the user U1 relative to the first image sensor to the processing device 140. The second information acquisition device 130 can provide the shooting pitch angle θ2 of the second image sensor and the vertical offset angle θ8 of the target Obj1 relative to the second image sensor to the processing device 140.

於是，依據深度資訊d2、拍攝俯仰角θ2與垂直偏移角θ8，處理裝置140可計算上邊界UL與顯示器110的上緣之間的距離為d2sin(θ2+θ8)。依據深度資訊d1、拍攝俯仰角θ1與垂直偏移角θ9，處理裝置140可計算使用者U1與顯示器110的上緣之間的距離為d1sin(θ1+θ9)。於一實施例中，處理裝置140還可依據兩影像感測器之鏡頭與顯示器110的上緣之間的距離來進一步校正計算前述計算結果，而獲取使用者U1與目標物Obj與顯示器110的上緣之間的實際距離。之後，處理裝置140可基於下列公式(4)計算出顯示平面上上邊界UL的交點Q的直向偏移距離Eu1。 Therefore, based on the depth information d2, the shooting pitch angle θ2 and the vertical offset angle θ8, the processing device 140 can calculate the distance between the upper boundary UL and the upper edge of the display 110 as d2sin(θ2+θ8). Based on the depth information d1, the shooting pitch angle θ1 and the vertical offset angle θ9, the processing device 140 can calculate the distance between the user U1 and the upper edge of the display 110 as d1sin(θ1+θ9). In one embodiment, the processing device 140 can further calibrate the above calculation results based on the distance between the lenses of the two image sensors and the upper edge of the display 110, and obtain the actual distance between the user U1 and the target object Obj and the upper edge of the display 110. Afterwards, the processing device 140 can calculate the vertical offset distance Eu1 of the intersection point Q of the upper boundary UL on the display plane based on the following formula (4).

Figure 111128679-A0305-02-0023-4
Figure 111128679-A0305-02-0023-4

其中，使用者U1與顯示器110的上緣之間的實際距離A1可表示為d1×sin(θ1+θ9)-△dcl1，△dcl1為第一影像感測器之鏡頭與顯示器110的上緣之間的距離。目標物Obj與顯示器110的上緣之間 的實際距離A2可表示為d2×sin(θ2+θ8)-△dcl2，△dcl2為第二影像感測器之鏡頭與顯示器110的上緣之間的距離。同理，物件邊界框Roi1下邊界的交點的直向偏移距離亦可以類似的方式計算出。 The actual distance A1 between the user U1 and the upper edge of the display 110 can be expressed as d 1×sin( θ 1+ θ 9)-△ dcl 1, △ dcl 1 is the distance between the lens of the first image sensor and the upper edge of the display 110. The actual distance A2 between the target Obj and the upper edge of the display 110 can be expressed as d 2×sin( θ 2+ θ 8)-△ dcl 2, △ dcl 2 is the distance between the lens of the second image sensor and the upper edge of the display 110. Similarly, the vertical offset distance of the intersection of the lower boundary of the object bounding box Roi1 can also be calculated in a similar manner.

基於上述說明，處理裝置140可獲取顯示平面上對應於物件邊界框Roi1的參考顯示物件框。詳細而言，基於圖8A所示之計算方法，處理裝置140可獲取顯示平面上參考顯示物件框的左右邊界分別與顯示器110之中線L1之間的橫向偏移距離。基於圖8B所示之計算方法，處理裝置140可獲取顯示平面上參考顯示物件框的上下邊界分別與顯示器110上緣之間的直向偏移距離。換言之，顯示平面上對應於物件邊界框Roi1的參考顯示物件框可由平行於顯示X軸的兩直線方程式與平行於顯示Y軸的兩直線方程式來表示。其中，參考顯示物件框的四個框頂點可由這些直線方程式的交點獲取。 Based on the above description, the processing device 140 can obtain the reference display object frame corresponding to the object bounding box Roi1 on the display plane. In detail, based on the calculation method shown in FIG8A, the processing device 140 can obtain the horizontal offset distance between the left and right boundaries of the reference display object frame on the display plane and the center line L1 of the display 110. Based on the calculation method shown in FIG8B, the processing device 140 can obtain the vertical offset distance between the upper and lower boundaries of the reference display object frame on the display plane and the upper edge of the display 110. In other words, the reference display object frame corresponding to the object bounding box Roi1 on the display plane can be represented by two straight line equations parallel to the display X-axis and two straight line equations parallel to the display Y-axis. Among them, the four vertices of the reference display object frame can be obtained from the intersection points of these straight line equations.

之後，處理裝置140可根據顯示器110的尺寸與解析度獲取顯示位置資訊，而顯示位置資訊可為顯示器110的畫素座標。更詳細來說，處理裝置140可根據顯示器110的尺寸與解析度將參考顯示物件框轉換為顯示器110上的畫素座標，以獲取虛擬資訊Vf1於顯示器110上的顯示位置資訊。亦即，由畫素座標系來表示顯示平面上的參考顯示物件框。 Afterwards, the processing device 140 may obtain display position information according to the size and resolution of the display 110, and the display position information may be the pixel coordinates of the display 110. More specifically, the processing device 140 may convert the reference display object frame into the pixel coordinates on the display 110 according to the size and resolution of the display 110 to obtain the display position information of the virtual information Vf1 on the display 110. That is, the reference display object frame on the display plane is represented by the pixel coordinate system.

回到圖7，於步驟S716，處理裝置140比較對應於先前時間的第一參考顯示物件框以及對應於當前時間的第二參考顯示物件框之間的距離與預設臨界值，以決定虛擬資訊Vf1於顯示器 110上對應於當前時間的實際顯示位置。於步驟S718，處理裝置140根據實際顯示位置顯示虛擬資訊Vf1於顯示器110上。 Returning to FIG. 7 , in step S716, the processing device 140 compares the distance between the first reference display object frame corresponding to the previous time and the second reference display object frame corresponding to the current time with a preset threshold value to determine the actual display position of the virtual information Vf1 on the display 110 corresponding to the current time. In step S718, the processing device 140 displays the virtual information Vf1 on the display 110 according to the actual display position.

詳細而言，圖9A是根據本揭露一範例實施例所繪示的不更新實際顯示位置的示意圖。圖9B是根據本揭露一範例實施例所繪示的更新實際顯示位置的示意圖。請參照圖9A與9B，於本實施範例中，顯示位置資訊包括於對應於目標物影像中物件邊界框的參考顯示物件框。參考顯示物件框可包括對應於先前時間的第一參考顯示物件框Roi_p以及對應於當前時間的第二參考顯示物件框Roi_c。第一顯示參考位置包括第一參考顯示物件框Roi_p，而第二顯示參考位置包括第二參考顯示物件框Roi_c。 In detail, FIG. 9A is a schematic diagram of not updating the actual display position according to an exemplary embodiment of the present disclosure. FIG. 9B is a schematic diagram of updating the actual display position according to an exemplary embodiment of the present disclosure. Referring to FIG. 9A and FIG. 9B, in this exemplary embodiment, the display position information includes a reference display object frame corresponding to the object boundary frame in the target image. The reference display object frame may include a first reference display object frame Roi_p corresponding to a previous time and a second reference display object frame Roi_c corresponding to a current time. The first display reference position includes the first reference display object frame Roi_p, and the second display reference position includes the second reference display object frame Roi_c.

對應於先前時間的第一參考顯示物件框Roi_p以及對應於當前時間的第二參考顯示物件框Roi_c可以是根據圖7、圖8A與圖8B所示實施例之計算方式而決定。更具體而言，基於對應至先前時間的先前使用者影像與先前目標物影像，處理裝置140可獲取顯示平面上的第一參考顯示物件框Roi_p。接著，舉例而言，假設第一參考顯示物件框Roi_p的左上頂點的畫素座標為(x1,y1)，處理裝置140可將對應至先前時間的實際顯示位置設置為畫素座標為(x1+αw1,y1+βh1)。其中，w1為第一參考顯示物件框Roi_p的寬度，h1為第一參考顯示物件框Roi_p的高度。α與β可視實際而設置，本揭露對此不限制。 The first reference display object frame Roi_p corresponding to the previous time and the second reference display object frame Roi_c corresponding to the current time can be determined according to the calculation method of the embodiments shown in Figures 7, 8A and 8B. More specifically, based on the previous user image and the previous target image corresponding to the previous time, the processing device 140 can obtain the first reference display object frame Roi_p on the display plane. Then, for example, assuming that the pixel coordinates of the upper left vertex of the first reference display object frame Roi_p are (x1, y1), the processing device 140 can set the actual display position corresponding to the previous time to the pixel coordinates (x1+αw1, y1+βh1). Wherein, w1 is the width of the first reference display object frame Roi_p, and h1 is the height of the first reference display object frame Roi_p. α and β can be set according to actual conditions, and this disclosure does not limit this.

之後，基於對應至當前時間的當前使用者影像與當前目標物影像，處理裝置140可獲取顯示平面上的第二參考顯示物件 框Roi_c。處理裝置140比較對應於先前時間的第一參考顯示物件框Roi_p以及對應於當前時間的第二參考顯示物件框Roi_c之間的距離△d2與預設臨界值。 Afterwards, based on the current user image and the current target image corresponding to the current time, the processing device 140 can obtain the second reference display object frame Roi_c on the display plane. The processing device 140 compares the distance △d2 between the first reference display object frame Roi_p corresponding to the previous time and the second reference display object frame Roi_c corresponding to the current time with the preset critical value.

處理裝置140計算第一參考顯示物件框Roi_p的第一參考點與第二參考顯示物件框Roi_c的第二參考點之間的距離。第一參考點與第二參考點可例如是框中心點或框頂點。於圖9A與9B的範例中，第一參考點與第二參考點分別為第一參考顯示物件框Roi_p的左上頂點與第二參考顯示物件框Roi_c的左上頂點。 The processing device 140 calculates the distance between the first reference point of the first reference display object frame Roi_p and the second reference point of the second reference display object frame Roi_c. The first reference point and the second reference point may be, for example, the center point or the vertex of the frame. In the examples of FIGS. 9A and 9B , the first reference point and the second reference point are the upper left vertex of the first reference display object frame Roi_p and the upper left vertex of the second reference display object frame Roi_c, respectively.

於一實施例中，此預設臨界值可以是根據第二參考顯示物件框Roi_c的尺寸而決定。舉例而言，預設臨界值可等於min(γ*w2,γ*h2)。其中，w2為第二參考顯示物件框Roi_c的寬度，h2為第二參考顯示物件框Roi_c的高度。γ可為介於0到1之間的數值，其可視實際而設置，本揭露對此不限制。min(˙)代表取最小值的函數。由此可見，預設臨界值可反應於參考物件框的尺寸而彈性設置。預設臨界值與參考物件框的尺寸可成正比關係。 In one embodiment, the preset critical value may be determined based on the size of the second reference display object frame Roi_c. For example, the preset critical value may be equal to min(γ*w2,γ*h2). Wherein, w2 is the width of the second reference display object frame Roi_c, and h2 is the height of the second reference display object frame Roi_c. γ may be a value between 0 and 1, which may be set according to actual conditions, and the present disclosure is not limited thereto. min(˙) represents a function that takes the minimum value. It can be seen that the preset critical value may be flexibly set in response to the size of the reference object frame. The preset critical value may be proportional to the size of the reference object frame.

若距離△d2未大於預設臨界值，請參照圖9A，處理裝置140不更新虛擬資訊Vf1於顯示器110上對應於當前時間的實際顯示位置，虛擬資訊Vf1可維持於顯示於第一參考顯示物件框Roi_p的上方。相反地，若距離△d2大於預設臨界值，請參照圖9B，處理裝置140根據對應至當前時間的第二參考顯示物件框Roi_c更新虛擬資訊Vf1於顯示器110上對應於當前時間的實際顯示位置。舉例而言，假設第二參考顯示物件框Roi_c的左上頂點 的畫素座標為(x2,y2)，處理裝置140可將對應至當前時間的實際顯示位置設置為畫素座標為(x2+αw2,y2+βh2)。其中，w2為第二參考顯示物件框Roi_c的寬度，h2為第二參考顯示物件框Roi_c的高度。α與β可視實際而設置，本揭露對此不限制。 If the distance △d2 is not greater than the preset threshold value, please refer to FIG9A , the processing device 140 does not update the actual display position of the virtual information Vf1 on the display 110 corresponding to the current time, and the virtual information Vf1 can be maintained to be displayed above the first reference display object frame Roi_p. On the contrary, if the distance △d2 is greater than the preset threshold value, please refer to FIG9B , the processing device 140 updates the actual display position of the virtual information Vf1 on the display 110 corresponding to the current time according to the second reference display object frame Roi_c corresponding to the current time. For example, assuming that the pixel coordinates of the upper left vertex of the second reference display object frame Roi_c are (x2, y2), the processing device 140 can set the actual display position corresponding to the current time to the pixel coordinates (x2+αw2, y2+βh2). Among them, w2 is the width of the second reference display object frame Roi_c, and h2 is the height of the second reference display object frame Roi_c. α and β can be set according to actual conditions, and this disclosure is not limited to this.

於一實施例中，反應於顯示位置資訊位於顯示器110的顯示邊緣區域，處理裝置140往遠離顯示器110之邊緣的方向調整顯示位置資訊。詳細而言，在處理裝置140獲取某一時間點的顯示位置資訊之後，若顯示位置資訊位於顯示器110的顯示邊緣區域，代表虛擬資訊可能會顯示於顯示器110的顯示邊緣區域或甚至是顯示不完全，導致使用者U1觀看上的困難度。於是，處理裝置140將進一步調整顯示位置資訊，例如處理裝置140可往遠離顯示器110之邊緣的方向調整顯示位置資訊。 In one embodiment, in response to the display position information being located in the display edge area of the display 110, the processing device 140 adjusts the display position information in a direction away from the edge of the display 110. In detail, after the processing device 140 obtains the display position information at a certain time point, if the display position information is located in the display edge area of the display 110, it means that the virtual information may be displayed in the display edge area of the display 110 or even incompletely displayed, resulting in difficulty for the user U1 to view. Therefore, the processing device 140 will further adjust the display position information, for example, the processing device 140 may adjust the display position information in a direction away from the edge of the display 110.

舉例而言，當基於圖5B的計算方式而計算出來的交點位置位於顯示器110的顯示邊緣區域時，處理裝置140可將交點位置往遠離顯示器110之邊緣的方向調整，再根據調整後的交點位置決定實際顯示位置。或者，當基於圖8A與8B的計算方式而計算出來的參考顯示物件框位於顯示器110的顯示邊緣區域時，處理裝置140可將參考顯示物件框往遠離顯示器110之邊緣的方向調整，再根據調整後的參考顯示物件框決定實際顯示位置。 For example, when the intersection position calculated based on the calculation method of FIG. 5B is located in the display edge area of the display 110, the processing device 140 can adjust the intersection position away from the edge of the display 110, and then determine the actual display position according to the adjusted intersection position. Alternatively, when the reference display object frame calculated based on the calculation method of FIGS. 8A and 8B is located in the display edge area of the display 110, the processing device 140 can adjust the reference display object frame away from the edge of the display 110, and then determine the actual display position according to the adjusted reference display object frame.

於一實施例中，處理裝置140可根據下列公式(5)來調整顯示位置資訊於垂直軸向上的座標分量。 In one embodiment, the processing device 140 may adjust the coordinate component of the display position information in the vertical axis according to the following formula (5).

Y=P y -F＊0.1＊(P y -c) 公式(5) Y = P y - F ＊0.1＊( P y - c ) Formula (5)

其中，P y 為顯示位置資訊的原始Y座標、F為調整權重、c為常數。F例如是介於1至100之間的數值，其可視實際而設置，本揭露對此不限制。常數c可根據原始Y座標所在顯示區域而決定。舉例而言，假設顯示位置資訊的原始Y座標位於顯示範圍的中心區域，則常數c可設置為等於原始Y座標，即不調整原始Y座標。假設顯示位置資訊的原始Y座標位於顯示範圍的邊緣區域，則常數c可設置為不等於原始Y座標，即往遠離顯示器110之邊緣的方向調整原始Y座標。同理，顯示位置資訊於水平軸向上的座標分量亦可以類似的方式進行調整。 Wherein, Py is the original Y coordinate of the display position information, F is the adjustment weight, and c is a constant. F is, for example, a value between 1 and 100, which can be set according to actual conditions, and the present disclosure is not limited thereto. The constant c can be determined according to the display area where the original Y coordinate is located. For example, assuming that the original Y coordinate of the display position information is located in the central area of the display range, the constant c can be set to be equal to the original Y coordinate, that is, the original Y coordinate is not adjusted. Assuming that the original Y coordinate of the display position information is located in the edge area of the display range, the constant c can be set to be unequal to the original Y coordinate, that is, the original Y coordinate is adjusted in the direction away from the edge of the display 110. Similarly, the coordinate component of the display position information in the horizontal axis can also be adjusted in a similar manner.

於一些實施例中，處理裝置140還可根據使用者的姿態資訊來決定虛擬資訊於顯示器110上的顯示位置資訊。第一資訊擷取裝置120可用以擷取使用者的姿態資訊。具體而言，第一資訊擷取裝置120可透過各類人體姿態辨識技術來辨識使用者的四肢、軀幹及/或頭部所呈現的姿態。像是，第一資訊擷取裝置120可根據影像資料辨識人體骨架或人體特徵點等等，從而識別出使用者的姿態。於一些實施例中，第一資訊擷取裝置120可對使用者影像進行人臉偵測而獲取各使用者影像中的人臉位置與人臉特徵點。本發明對於人臉特徵點的數量並不加以限制，其例如可包括嘴角特徵點、眉心特徵點或眼角特徵點等等。於一些實施例中，第一資訊擷取裝置120或處理裝置140可根據多幀使用者影像中的人臉特徵點來辨識出使用者的臉部中心位移量與頭部轉動角度。使用者的姿態資訊包括臉部中心位移量與頭部轉動角度。 In some embodiments, the processing device 140 may also determine the display position information of the virtual information on the display 110 according to the posture information of the user. The first information acquisition device 120 may be used to acquire the posture information of the user. Specifically, the first information acquisition device 120 may recognize the posture presented by the user's limbs, trunk and/or head through various human posture recognition technologies. For example, the first information acquisition device 120 may recognize the human skeleton or human body feature points according to the image data, thereby identifying the user's posture. In some embodiments, the first information acquisition device 120 may perform face detection on the user image to obtain the face position and face feature points in each user image. The present invention does not limit the number of facial feature points, which may include, for example, mouth corner feature points, eyebrow center feature points, or eye corner feature points. In some embodiments, the first information acquisition device 120 or the processing device 140 may identify the user's facial center displacement and head rotation angle based on the facial feature points in multiple frames of user images. The user's posture information includes the facial center displacement and the head rotation angle.

於一些實施例中，處理裝置140可根據使用者的姿態資訊來判斷使用者的頭部移動或轉動。反應於根據使用者的姿態資訊判定使用者的頭部移動，處理裝置140可根據使用者的空間位置資訊以及目標物的空間位置資訊決定目標物的虛擬資訊於顯示器110上的顯示位置資訊。亦即，當處理裝置140判定使用者的頭部移動時，處理裝置140可根據如圖2、圖4或圖7所示的流程來決定虛擬資訊的實際顯示位置。另一方面，反應於根據使用者的姿態資訊判定使用者的頭部轉動，處理裝置140可根據使用者的空間位置資訊與姿態資訊以及目標物的空間位置資訊決定目標物的虛擬資訊於顯示器110上的顯示位置資訊。當處理裝置140判定使用者的頭部轉動時，處理裝置140可根據使用者的頭部轉動角度來更新目標物的虛擬資訊於顯示器110上的顯示位置資訊。舉例而言，當使用者的頭部並沒有移動時，處理裝置140可利用使用者的頭部轉動角度來更新先前時間點所決定的顯示位置資訊。 In some embodiments, the processing device 140 may determine the user's head movement or rotation according to the user's posture information. In response to determining the user's head movement according to the user's posture information, the processing device 140 may determine the display position information of the virtual information of the target object on the display 110 according to the user's spatial position information and the spatial position information of the target object. That is, when the processing device 140 determines the user's head movement, the processing device 140 may determine the actual display position of the virtual information according to the process shown in FIG. 2 , FIG. 4 , or FIG. 7 . On the other hand, in response to determining the user's head rotation based on the user's posture information, the processing device 140 may determine the display position information of the virtual information of the target on the display 110 based on the user's spatial position information and posture information and the spatial position information of the target. When the processing device 140 determines that the user's head rotates, the processing device 140 may update the display position information of the virtual information of the target on the display 110 based on the user's head rotation angle. For example, when the user's head does not move, the processing device 140 may use the user's head rotation angle to update the display position information determined at a previous time point.

於一些實施例中，處理裝置140可根據姿態資訊中的臉部中心位移量來估計出一頭部行為指標。上述頭部行為指標可為臉部中心位移量除以頭部半徑基準值。舉例而言，頭部半徑基準值可為80mm。透過比對頭部行為指標以及頭部轉動角度的變化量，處理裝置140可判斷使用者的頭部移動或轉動，以反應於使用者的頭部出現移動或轉動來決定虛擬資訊於顯示器110上的顯示位置資訊。 In some embodiments, the processing device 140 can estimate a head behavior indicator based on the facial center displacement in the posture information. The head behavior indicator can be the facial center displacement divided by the head radius reference value. For example, the head radius reference value can be 80mm. By comparing the head behavior indicator and the change in the head rotation angle, the processing device 140 can determine the user's head movement or rotation, and determine the display position information of the virtual information on the display 110 in response to the movement or rotation of the user's head.

於一些實施例中，若頭部行為指標小於等於頭部轉動角度的變化量，處理裝置140判定使用者的頭部移動，亦即使用者頭部自第一空間位置移動至第二空間位置。於是，可執行如圖2、圖4或圖7所示的流程來決定虛擬資訊的實際顯示位置。若頭部行為指標大於頭部轉動角度的變化量，處理裝置140判定使用者的頭部轉動，亦即使用者頭部轉向發生改變。於是，處理裝置140可根據使用者的空間位置資訊與頭部轉動角度以及目標物的空間位置資訊決定目標物的虛擬資訊於顯示器110上的顯示位置資訊。像是，處理裝置140可利用使用者的頭部轉動角度來更新先前時間點所決定的顯示位置資訊，並據以決定虛擬資訊的實際顯示位置。或者，處理裝置140可利用使用者的頭部轉動角度與使用者的空間位置資訊來識別另一目標物，以根據另一目標物的空間位置資訊以及使用者的空間位置資訊決定另一目標物的另一虛擬資訊於顯示器110上的顯示位置資訊。 In some embodiments, if the head behavior indicator is less than or equal to the change in the head rotation angle, the processing device 140 determines that the user's head moves, that is, the user's head moves from the first spatial position to the second spatial position. Therefore, the process shown in Figure 2, Figure 4 or Figure 7 can be executed to determine the actual display position of the virtual information. If the head behavior indicator is greater than the change in the head rotation angle, the processing device 140 determines that the user's head rotates, that is, the user's head direction changes. Therefore, the processing device 140 can determine the display position information of the virtual information of the target object on the display 110 based on the user's spatial position information and the head rotation angle and the spatial position information of the target object. For example, the processing device 140 can use the user's head rotation angle to update the display position information determined at a previous time point, and determine the actual display position of the virtual information accordingly. Alternatively, the processing device 140 can use the user's head rotation angle and the user's spatial position information to identify another target object, and determine the display position information of another virtual information of the other target object on the display 110 according to the spatial position information of the other target object and the spatial position information of the user.

本揭露的範例實施例所提出的資訊顯示方法及其處理裝置與資訊顯示系統，可即時地依據使用者位置與目標物位置，計算使視線投射於顯示平面上的顯示位置資訊，並據以產生虛擬資訊的實際顯示位置。藉此，即便目標物的移動多變或不可預測，根據實際顯示位置所顯示的虛擬資訊可即時地跟隨目標物的移動。此外，本揭露的範例實施例可依據先前時間的第一顯示參考位置與當前時間的第二顯示參考位置之間的距離決定是否更新虛擬資訊於顯示器上對應於當前時間的實際顯示位置。藉此，基於 即時物件辨識而顯示的虛擬資訊可穩定且順暢地進行顯示，大幅提升使用者的觀看體驗。 The information display method and processing device and information display system proposed in the exemplary embodiment of the present disclosure can calculate the display position information that projects the line of sight on the display plane in real time according to the user position and the target object position, and generate the actual display position of the virtual information accordingly. In this way, even if the movement of the target object is changeable or unpredictable, the virtual information displayed according to the actual display position can follow the movement of the target object in real time. In addition, the exemplary embodiment of the present disclosure can determine whether to update the virtual information on the display corresponding to the actual display position at the current time according to the distance between the first display reference position at the previous time and the second display reference position at the current time. In this way, virtual information displayed based on real-time object recognition can be displayed stably and smoothly, greatly improving the user's viewing experience.

雖然本揭露已以範例實施例揭露如上，然其並非用以限定本揭露，任何所屬技術領域中具有通常知識者，在不脫離本揭露的精神和範圍內，當可作些許的更動與潤飾，故本揭露的保護範圍當視後附的申請專利範圍及其均等範圍所界定者為準。 Although the present disclosure has been disclosed as above by exemplary embodiments, it is not intended to limit the present disclosure. Any person with ordinary knowledge in the relevant technical field may make some changes and modifications without departing from the spirit and scope of the present disclosure. Therefore, the protection scope of the present disclosure shall be defined by the scope of the attached patent application and its equivalent scope.

S210~S250:步驟 S210~S250: Steps

Claims (20)

一種資訊顯示系統，包括：可透光的顯示器；第一資訊擷取裝置，用以擷取一使用者的空間位置資訊；第二資訊擷取裝置，用以擷取一目標物的空間位置資訊；處理裝置，連接於所述顯示器、所述第一資訊擷取裝置以及所述第二資訊擷取裝置，經配置以：根據所述使用者的空間位置資訊以及所述目標物的空間位置資訊決定所述目標物的一虛擬資訊於所述顯示器上的顯示位置資訊，其中所述顯示位置資訊包括對應於先前時間的第一顯示參考位置與對應至當前時間的第二顯示參考位置；根據所述第一顯示參考位置與所述第二顯示參考位置之間的距離決定是否更新所述虛擬資訊於所述顯示器上對應於所述當前時間的實際顯示位置；以及根據所述實際顯示位置顯示所述虛擬資訊於所述顯示器上。 An information display system includes: a light-transmissive display; a first information acquisition device for acquiring spatial position information of a user; a second information acquisition device for acquiring spatial position information of a target object; a processing device connected to the display, the first information acquisition device and the second information acquisition device, and configured to: determine virtual information of the target object according to the spatial position information of the user and the spatial position information of the target object Display position information on the display, wherein the display position information includes a first display reference position corresponding to a previous time and a second display reference position corresponding to a current time; determining whether to update the virtual information on the display to an actual display position corresponding to the current time according to the distance between the first display reference position and the second display reference position; and displaying the virtual information on the display according to the actual display position. 如請求項1所述的資訊顯示系統，其中所述處理裝置經配置以：反應於所述第一顯示參考位置與所述第二顯示參考位置之間的距離大於一預設臨界值，根據對應至所述當前時間的所述第二顯示參考位置更新所述虛擬資訊於所述顯示器上對應於所述當前時間的所述實際顯示位置；以及反應於所述第一顯示參考位置與所述第二顯示參考位置之間 的距離未大於一預設臨界值，不更新所述虛擬資訊於所述顯示器上對應於所述當前時間的所述實際顯示位置。 An information display system as described in claim 1, wherein the processing device is configured to: in response to the distance between the first display reference position and the second display reference position being greater than a preset threshold value, update the virtual information at the actual display position on the display corresponding to the current time according to the second display reference position corresponding to the current time; and in response to the distance between the first display reference position and the second display reference position being not greater than a preset threshold value, not update the virtual information at the actual display position on the display corresponding to the current time. 如請求項1所述的資訊顯示系統，其中所述第一資訊擷取裝置用以擷取該使用者的姿態資訊，而所述處理裝置更經配置以：根據所述使用者的姿態資訊判定所述使用者的頭部移動或轉動；以及反應於根據所述使用者的姿態資訊判定所述使用者的頭部移動，根據所述使用者的空間位置資訊以及所述目標物的空間位置資訊決定所述目標物的所述虛擬資訊於所述顯示器上的所述顯示位置資訊，或反應於根據所述使用者的姿態資訊判定所述使用者的頭部轉動，根據所述使用者的空間位置資訊與姿態資訊以及所述目標物的空間位置資訊決定所述目標物的所述虛擬資訊於所述顯示器上的所述顯示位置資訊。 The information display system as described in claim 1, wherein the first information acquisition device is used to acquire the posture information of the user, and the processing device is further configured to: determine the movement or rotation of the user's head according to the posture information of the user; and in response to determining the movement of the user's head according to the posture information of the user, determine the display position information of the virtual information of the target on the display according to the spatial position information of the user and the spatial position information of the target, or in response to determining the rotation of the user's head according to the posture information of the user, determine the display position information of the virtual information of the target on the display according to the spatial position information and posture information of the user and the spatial position information of the target. 如請求項1所述的資訊顯示系統，其中所述第一資訊擷取裝置包括用以拍攝所述使用者的第一影像感測器，且所述第二資訊擷取裝置包括用以拍攝所述目標物的第二影像感測器。 An information display system as described in claim 1, wherein the first information acquisition device includes a first image sensor for photographing the user, and the second information acquisition device includes a second image sensor for photographing the target object. 如請求項4所述的資訊顯示系統，其中所述第二資訊擷取裝置對多張目標物影像進行物件偵測而獲取所述目標物於各所述目標物影像中的物件邊界框，且所述第二資訊擷取裝置根據所述物件邊界框進行座標轉換而獲取所述目標物的空間位置資訊， 其中所述第一資訊擷取裝置對多張使用者影像進行人臉偵測而獲取各所述使用者影像中的人臉位置，且所述第一資訊擷取裝置根據所述人臉位置進行座標轉換而獲取所述使用者的空間位置資訊。 An information display system as described in claim 4, wherein the second information acquisition device performs object detection on multiple target images to obtain an object bounding box of the target in each of the target images, and the second information acquisition device performs coordinate conversion based on the object bounding box to obtain spatial position information of the target, wherein the first information acquisition device performs face detection on multiple user images to obtain a face position in each of the user images, and the first information acquisition device performs coordinate conversion based on the face position to obtain spatial position information of the user. 如請求項4所述的資訊顯示系統，其中所述目標物的空間位置資訊包括相機座標系下的目標物相機座標，所述使用者的空間位置資訊包括相機座標系下的使用者相機座標，其中所述處理裝置根據所述第一影像感測器的拍攝方向將所述使用者相機座標轉換為相對於所述顯示器的使用者位置座標，並根據所述第二影像感測器的拍攝方向將所述目標物相機座標轉換為相對於所述顯示器的目標物位置座標，其中所述處理裝置根據所述目標物位置座標與所述使用者位置座標之間的連線決定所述虛擬資訊於所述顯示器上的所述顯示位置資訊。 An information display system as described in claim 4, wherein the spatial position information of the target object includes the target camera coordinates in a camera coordinate system, and the spatial position information of the user includes the user camera coordinates in a camera coordinate system, wherein the processing device converts the user camera coordinates into user position coordinates relative to the display according to the shooting direction of the first image sensor, and converts the target camera coordinates into target position coordinates relative to the display according to the shooting direction of the second image sensor, wherein the processing device determines the display position information of the virtual information on the display according to the connection between the target position coordinates and the user position coordinates. 如請求項6所述的資訊顯示系統，其中所述顯示位置資訊包括所述連線相交於所述顯示器的顯示平面的一交點位置。 An information display system as described in claim 6, wherein the display position information includes an intersection position where the connecting line intersects the display plane of the display. 如請求項4所述的資訊顯示系統，其中所述目標物的空間位置資訊包括所述目標物相對於所述第二影像感測器的水平偏移角以及垂直偏移角，所述使用者的空間位置資訊包括所述使用者相對於所述第一影像感測器的水平偏移角以及垂直偏移角，其中所述處理裝置根據所述目標物相對於所述第二影像感測器的水平偏移角以及垂直偏移角、所述使用者相對於所述第一影 像感測器的水平偏移角以及垂直偏移角、所述第一影像感測器的拍攝方向以及所述第二影像感測器的拍攝方向，決定所述虛擬資訊於所述顯示器上的所述顯示位置資訊。 An information display system as described in claim 4, wherein the spatial position information of the target includes the horizontal offset angle and the vertical offset angle of the target relative to the second image sensor, and the spatial position information of the user includes the horizontal offset angle and the vertical offset angle of the user relative to the first image sensor, wherein the processing device determines the display position information of the virtual information on the display according to the horizontal offset angle and the vertical offset angle of the target relative to the second image sensor, the horizontal offset angle and the vertical offset angle of the user relative to the first image sensor, the shooting direction of the first image sensor, and the shooting direction of the second image sensor. 如請求項8所述的資訊顯示系統，其中所述第二資訊擷取裝置對多張目標物影像進行物件偵測而獲取所述目標物於各所述目標物影像中的一物件邊界框，所述顯示位置資訊包括於所述顯示器的顯示平面上對應於各所述物件邊界框的參考顯示物件框，其中所述參考顯示物件框包括對應於所述先前時間的第一參考顯示物件框以及對應於所述當前時間的第二參考顯示物件框，所述第一顯示參考位置包括所述第一參考顯示物件框，而所述第二顯示參考位置包括所述第二參考顯示物件框，其中所述處理裝置比較對應於所述先前時間的所述第一參考顯示物件框以及對應於所述當前時間的所述第二參考顯示物件框之間的距離與一預設臨界值，以決定所述虛擬資訊於所述顯示器上對應於所述當前時間的所述實際顯示位置。 An information display system as described in claim 8, wherein the second information acquisition device performs object detection on a plurality of target images to obtain an object bounding box of the target in each of the target images, and the display position information includes a reference display object frame corresponding to each of the object bounding boxes on a display plane of the display, wherein the reference display object frame includes a first reference display object frame corresponding to the previous time and a second reference display object frame corresponding to the current time. Display object frame, the first display reference position includes the first reference display object frame, and the second display reference position includes the second reference display object frame, wherein the processing device compares the distance between the first reference display object frame corresponding to the previous time and the second reference display object frame corresponding to the current time with a preset threshold value to determine the actual display position of the virtual information on the display corresponding to the current time. 如請求項9所述的資訊顯示系統，其中所述預設臨界值是根據所述第二參考顯示物件框的尺寸而決定。 An information display system as described in claim 9, wherein the preset threshold value is determined based on the size of the second reference display object frame. 如請求項9所述的資訊顯示系統，其中所述處理裝置計算所述第一參考顯示物件框的第一參考點與所述第二參考顯示物件框的第二參考點之間的所述距離，所述第一參考點與所述第二參考點為框中心點或框頂點。 An information display system as described in claim 9, wherein the processing device calculates the distance between the first reference point of the first reference display object frame and the second reference point of the second reference display object frame, and the first reference point and the second reference point are frame center points or frame vertices. 如請求項1所述的資訊顯示系統，其中反應於所述顯示位置資訊位於所述顯示器的顯示邊緣區域，所述處理裝置往遠離所述顯示器之邊緣的方向調整所述顯示位置資訊。 An information display system as described in claim 1, wherein in response to the display position information being located in a display edge region of the display, the processing device adjusts the display position information in a direction away from the edge of the display. 如請求項1所述的資訊顯示系統，其中所述實際顯示位置與所述顯示位置資訊為所述顯示器的畫素座標，所述處理裝置根據所述顯示器的尺寸與解析度獲取所述顯示位置資訊。 An information display system as described in claim 1, wherein the actual display position and the display position information are pixel coordinates of the display, and the processing device obtains the display position information according to the size and resolution of the display. 一種資訊顯示方法，適用於具有可透光的顯示器、第一資訊擷取裝置、第二資訊擷取裝置以及處理裝置的資訊顯示系統，而所述方法包括：利用所述第一資訊擷取裝置擷取一使用者的空間位置資訊；利用所述第二資訊擷取裝置擷取一目標物的空間位置資訊；根據所述使用者的空間位置資訊以及所述目標物的空間位置資訊決定所述目標物的一虛擬資訊於所述顯示器上的顯示位置資訊，其中所述顯示位置資訊包括對應於先前時間的第一顯示參考位置與對應至當前時間的第二顯示參考位置；根據所述第一顯示參考位置與所述第二顯示參考位置之間的距離決定是否更新所述虛擬資訊於所述顯示器上對應於所述當前時間的實際顯示位置；以及根據所述實際顯示位置顯示所述虛擬資訊於所述顯示器上。 An information display method is applicable to an information display system having a light-transmitting display, a first information acquisition device, a second information acquisition device, and a processing device, and the method comprises: using the first information acquisition device to acquire spatial position information of a user; using the second information acquisition device to acquire spatial position information of a target object; determining virtual information of the target object according to the spatial position information of the user and the spatial position information of the target object; Display position information on the display, wherein the display position information includes a first display reference position corresponding to a previous time and a second display reference position corresponding to a current time; determining whether to update the virtual information on the display to an actual display position corresponding to the current time according to the distance between the first display reference position and the second display reference position; and displaying the virtual information on the display according to the actual display position. 如請求項14所述的資訊顯示方法，其中根據所述第一顯示參考位置與所述第二顯示參考位置之間的所述距離決定 是否更新所述虛擬資訊在所述當前時間於所述顯示器上的所述實際顯示位置的步驟包括：反應於所述第一顯示參考位置與所述第二顯示參考位置之間的距離大於一預設臨界值，根據對應至所述當前時間的所述第二顯示參考位置決定所述虛擬資訊於所述顯示器上對應於所述當前時間的所述實際顯示位置；以及反應於所述第一顯示參考位置與所述第二顯示參考位置之間的距離未大於所述預設臨界值，根據對應於所述先前時間的所述第一顯示參考位置決定所述虛擬資訊於所述顯示器上對應於所述當前時間的所述實際顯示位置。 The information display method of claim 14, wherein the step of determining whether to update the actual display position of the virtual information on the display at the current time based on the distance between the first display reference position and the second display reference position comprises: in response to the distance between the first display reference position and the second display reference position being greater than a preset threshold value, based on the actual display position corresponding to the current time The second display reference position determines the actual display position of the virtual information on the display corresponding to the current time; and in response to the distance between the first display reference position and the second display reference position being not greater than the preset threshold value, the actual display position of the virtual information on the display corresponding to the current time is determined according to the first display reference position corresponding to the previous time. 如請求項14所述的資訊顯示方法，其中所述方法更包括：利用所述第一資訊擷取裝置擷取一使用者的姿態資訊，其中根據所述使用者的空間位置資訊以及所述目標物的空間位置資訊決定所述目標物的所述虛擬資訊於所述顯示器上的所述顯示位置資訊的步驟包括：根據所述使用者的姿態資訊判定所述使用者的頭部移動或轉動；以及反應於根據所述使用者的姿態資訊判定所述使用者的頭部移動，根據所述使用者的空間位置資訊以及所述目標物的空間位置資訊決定所述目標物的所述虛擬資訊於所述顯示器上的所述顯示位置資訊，或反應於根據所述使用者的姿態資訊判定所述使用者 的頭部轉動，根據所述使用者的空間位置資訊與姿態資訊以及所述目標物的空間位置資訊決定所述目標物的所述虛擬資訊於所述顯示器上的所述顯示位置資訊。 The information display method as described in claim 14, wherein the method further comprises: utilizing the first information acquisition device to acquire posture information of a user, wherein the step of determining the display position information of the virtual information of the target object on the display according to the spatial position information of the user and the spatial position information of the target object comprises: determining the movement or rotation of the user's head according to the posture information of the user; and responding to the position information determined according to the posture information of the user. In response to the movement of the user's head, the display position information of the virtual information of the target on the display is determined according to the spatial position information of the user and the spatial position information of the target, or in response to the rotation of the user's head determined according to the user's posture information, the display position information of the virtual information of the target on the display is determined according to the spatial position information and posture information of the user and the spatial position information of the target. 如請求項14所述的資訊顯示方法，其中所述第一資訊擷取裝置包括用以拍攝所述使用者的第一影像感測器，且所述第二資訊擷取裝置包括用以拍攝所述目標物的第二影像感測器，其中利用所述第二資訊擷取裝置擷取所述目標物的空間位置資訊的步驟包括：利用所述第二資訊擷取裝置對多張目標物影像進行物件偵測而獲取所述目標物於各所述目標物影像中的物件邊界框，並根據所述物件邊界框進行座標轉換而獲取所述目標物的空間位置資訊，其中利用所述第一資訊擷取裝置擷取所述使用者的空間位置資訊的步驟包括：利用所述第一資訊擷取裝置對多張使用者影像進行人臉偵測而獲取各所述使用者影像中的人臉位置，並根據所述人臉位置進行座標轉換而獲取所述使用者的空間位置資訊。 The information display method as described in claim 14, wherein the first information acquisition device includes a first image sensor for photographing the user, and the second information acquisition device includes a second image sensor for photographing the target object, wherein the step of using the second information acquisition device to acquire the spatial position information of the target object includes: using the second information acquisition device to perform object detection on a plurality of target object images to acquire the target object at each of the target objects; The object bounding box in the object image is obtained, and the spatial position information of the target object is obtained by performing coordinate conversion according to the object bounding box, wherein the step of using the first information acquisition device to acquire the spatial position information of the user includes: using the first information acquisition device to perform face detection on multiple user images to acquire the face position in each of the user images, and performing coordinate conversion according to the face position to acquire the spatial position information of the user. 如請求項17所述的資訊顯示方法，其中所述第二資訊擷取裝置對多張目標物影像進行物件偵測而獲取各所述目標物於所述目標物影像中的一物件邊界框，所述顯示位置資訊包括於所述顯示器的顯示平面上對應於各所述物件邊界框的參考顯示物件框， 其中所述參考顯示物件框包括對應於所述先前時間的第一參考顯示物件框以及對應於所述當前時間的第二參考顯示物件框，所述第一顯示參考位置包括所述第一參考顯示物件框，而所述第二顯示參考位置包括所述第二參考顯示物件框，其中根據所述第一顯示參考位置與所述第二顯示參考位置之間的所述距離決定所述虛擬資訊在所述當前時間於所述顯示器上的所述實際顯示位置的步驟包括：比較對應於所述先前時間的所述第一參考顯示物件框以及對應於所述當前時間的所述第二參考顯示物件框之間的距離與一預設臨界值，以決定所述虛擬資訊於所述顯示器上對應於所述當前時間的實際顯示位置。 The information display method as described in claim 17, wherein the second information acquisition device performs object detection on multiple target images to obtain an object boundary frame of each target in the target image, and the display position information includes a reference display object frame corresponding to each object boundary frame on the display plane of the display, wherein the reference display object frame includes a first reference display object frame corresponding to the previous time and a second reference display object frame corresponding to the current time, and the first display reference position includes the first reference display object frame , and the second display reference position includes the second reference display object frame, wherein the step of determining the actual display position of the virtual information on the display at the current time according to the distance between the first display reference position and the second display reference position includes: comparing the distance between the first reference display object frame corresponding to the previous time and the second reference display object frame corresponding to the current time with a preset threshold value to determine the actual display position of the virtual information on the display corresponding to the current time. 如請求項14所述的資訊顯示方法，其中反應於所述顯示位置資訊位於所述顯示器的顯示邊緣區域，所述處理裝置往遠離所述顯示器之邊緣的方向調整所述顯示位置資訊。 In the information display method as described in claim 14, in response to the display position information being located in the display edge area of the display, the processing device adjusts the display position information in a direction away from the edge of the display. 一種處理裝置，連接於可透光的顯示器、第一資訊擷取裝置以及第二資訊擷取裝置，其中所述第一資訊擷取裝置用以擷取一使用者的空間位置資訊，所述第二資訊擷取裝置用以擷取一目標物的空間位置資訊，該處理裝置包括：記憶體，用以儲存資料；以及處理器，連接所述記憶體並經配置：自所述第一資訊擷取裝置取得所述使用者的空間位置資訊；自所述第二資訊擷取裝置取得所述目標物的空間位置資訊； 根據所述使用者的空間位置資訊以及所述目標物的空間位置資訊決定所述目標物的一虛擬資訊於所述顯示器上的顯示位置資訊，其中所述顯示位置資訊包括對應於先前時間的第一顯示參考位置與對應至當前時間的第二顯示參考位置；根據所述第一顯示參考位置與所述第二顯示參考位置之間的距離決定是否更新所述虛擬資訊於所述顯示器上對應於所述當前時間的實際顯示位置；以及根據所述實際顯示位置顯示所述虛擬資訊於所述顯示器上。 A processing device is connected to a light-transmissive display, a first information acquisition device and a second information acquisition device, wherein the first information acquisition device is used to acquire spatial position information of a user, and the second information acquisition device is used to acquire spatial position information of a target object. The processing device comprises: a memory for storing data; and a processor connected to the memory and configured to: acquire the spatial position information of the user from the first information acquisition device; acquire the spatial position information of the target object from the second information acquisition device; According to the user Determine the display position information of the virtual information of the target on the display according to the spatial position information of the target and the spatial position information of the target, wherein the display position information includes a first display reference position corresponding to the previous time and a second display reference position corresponding to the current time; determine whether to update the virtual information on the display to the actual display position corresponding to the current time according to the distance between the first display reference position and the second display reference position; and display the virtual information on the display according to the actual display position.
TW111128679A 2021-09-09 2022-07-29 Method, processing device, and display system for information display TWI844902B (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
CN202210997341.5A CN115794004A (en) 2021-09-09 2022-08-19 Information display method, processing device thereof and information display system
US17/939,900 US20230070721A1 (en) 2021-09-09 2022-09-07 Method, processing device, and display system for information display

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US202163242044P 2021-09-09 2021-09-09
US63/242,044 2021-09-09

Publications (2)

Publication Number Publication Date
TW202311817A TW202311817A (en) 2023-03-16
TWI844902B true TWI844902B (en) 2024-06-11

Family

ID=

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20200333940A1 (en) 2017-02-24 2020-10-22 Vrad Inc. Virtual reality-based radiology practice apparatus and method

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20200333940A1 (en) 2017-02-24 2020-10-22 Vrad Inc. Virtual reality-based radiology practice apparatus and method

Similar Documents

Publication Publication Date Title
TWI722280B (en) Controller tracking for multiple degrees of freedom
US9779512B2 (en) Automatic generation of virtual materials from real-world materials
US11436742B2 (en) Systems and methods for reducing a search area for identifying correspondences between images
TW202009786A (en) Electronic apparatus operated by head movement and operation method thereof
US10782780B2 (en) Remote perception of depth and shape of objects and surfaces
US11568555B2 (en) Dense depth computations aided by sparse feature matching
US11451760B2 (en) Systems and methods for correcting rolling shutter artifacts
US12020448B2 (en) Systems and methods for updating continuous image alignment of separate cameras
JP2021060627A (en) Information processing apparatus, information processing method, and program
US10992928B1 (en) Calibration system for concurrent calibration of device sensors
US11430086B2 (en) Upsampling low temporal resolution depth maps
US11450014B2 (en) Systems and methods for continuous image alignment of separate cameras
WO2019156990A1 (en) Remote perception of depth and shape of objects and surfaces
CN117751393A (en) Method and system for detecting obstacle elements using vision assistance device
TWI844902B (en) Method, processing device, and display system for information display
US11516452B2 (en) Systems and methods for temporal corrections for parallax reprojection
US20230070721A1 (en) Method, processing device, and display system for information display
TW202311817A (en) Method, processing device, and display system for information display
TWI779332B (en) Augmented reality system and display method for anchor virtual object thereof
JP6468078B2 (en) Gaze calibration program, gaze calibration apparatus, and gaze calibration method
TWI823740B (en) Active interactive navigation system and active interactive navigation method
CN113703161A (en) Augmented reality system and anchoring display method thereof