TW202311908A - Edge side rendering operation method and system for real-time mr interactive application - Google Patents
Edge side rendering operation method and system for real-time mr interactive application Download PDFInfo
- Publication number
- TW202311908A TW202311908A TW110134251A TW110134251A TW202311908A TW 202311908 A TW202311908 A TW 202311908A TW 110134251 A TW110134251 A TW 110134251A TW 110134251 A TW110134251 A TW 110134251A TW 202311908 A TW202311908 A TW 202311908A
- Authority
- TW
- Taiwan
- Prior art keywords
- mixed reality
- management server
- central management
- mobile edge
- edge computing
- Prior art date
Links
Images
Landscapes
- Magnetic Resonance Imaging Apparatus (AREA)
- Apparatus For Radiation Diagnosis (AREA)
- Processing Or Creating Images (AREA)
Abstract
Description
本案屬於混合實境的領域,尤指一種應用於混合實境系統之邊緣渲染運算方法及混合實境系統。This case belongs to the field of mixed reality, especially an edge rendering calculation method applied to a mixed reality system and a mixed reality system.
隨著顯示器、陀螺儀、空間感應器等關鍵零組件的科技先進發展、頭戴顯示器的製造成本降低與微型輕量化,使得包含虛擬實境(Virtual Reality, VR)及擴增實境(Augmented Reality, AR)之混合實境(Mixed Reality, MR)(亦可視為延展實境,即Extended Reality, XR)開始普及,提供使用者完全不同於一般平面顯示器的視覺感官享受。而如何改善混合實境裝置在雲端互動體驗上所遇到的延遲問題,並在畫面品質與傳輸時間當中取得平衡是目前研發的重點。With the advanced technology development of key components such as displays, gyroscopes, and spatial sensors, and the reduction in manufacturing costs and miniaturization of head-mounted displays, virtual reality (VR) and augmented reality (Augmented Reality) , AR) Mixed Reality (Mixed Reality, MR) (also known as Extended Reality, namely Extended Reality, XR) has begun to popularize, providing users with a visual sensory enjoyment that is completely different from ordinary flat-panel displays. How to improve the delay problem encountered by the mixed reality device in the cloud interactive experience, and to achieve a balance between image quality and transmission time is the focus of current research and development.
近年來,由於雲端運算與網路之需求大幅增長,於是第五代行動通訊技術5G也被提出。第五代行動通訊技術5G需要具備低延遲、高負載量與大量裝置連結的特性,而為了實現上述低延遲的特性,一種新的網路架構稱為行動邊緣運算(Mobile Edge Computing,MEC)於是誕生。行動邊緣運算是一種分散式運算的架構,將數據資料的運算,由資料中心移往行動邊緣運算裝置來處理,即行動邊緣運算的技術將原本完全由資料中心處理的大型服務加以分解,切割成更小與更容易管理的部份,以分散到行動邊緣運算裝置去處理。相較於雲端系統,行動邊緣運算裝置更接近於終端裝置,因此可以加快資料的處理與傳送速度,且可以減少延遲。In recent years, due to the sharp increase in demand for cloud computing and networks, the fifth generation of mobile communication technology 5G has also been proposed. The fifth generation of mobile communication technology 5G needs to have the characteristics of low latency, high load and a large number of device connections. In order to achieve the above low latency characteristics, a new network architecture is called mobile edge computing (Mobile Edge Computing, MEC) born. Mobile edge computing is a distributed computing architecture, which moves the computing of data from the data center to the mobile edge computing device for processing, that is, the technology of mobile edge computing decomposes the large-scale services that were originally completely processed by the data center, and cuts them into Smaller and more manageable parts are distributed to mobile edge computing devices for processing. Compared with cloud systems, mobile edge computing devices are closer to terminal devices, so data processing and transmission speeds can be accelerated, and delays can be reduced.
然而在現有的混合實境平台或裝置中,目前都還是只專注於提供傳統遊戲的雲端服務,並未採用行動邊緣運算的技術,使得混合實境裝置並無法有效地發揮第五代行動通訊技術(5G)所帶來的優勢,進而在延遲減少的情況下達成即時互動、串流渲染結果。However, in the existing mixed reality platforms or devices, they still only focus on providing cloud services for traditional games, and do not use mobile edge computing technology, which makes mixed reality devices unable to effectively utilize the fifth generation mobile communication technology (5G) to achieve real-time interaction and streaming rendering results with reduced latency.
本案為一種應用於混合實境系統之邊緣渲染運算方法及其混合實境系統,藉行動邊緣運算裝置的設置與使用,使得混合實境裝置可有效地發揮第五代行動通訊技術所帶來的優勢,進而在延遲減少的情況下達成即時互動、串流渲染結果。This case is an edge rendering calculation method applied to a mixed reality system and its mixed reality system. By setting and using a mobile edge computing device, the mixed reality device can effectively play the role brought by the fifth generation mobile communication technology. Advantages, and then achieve real-time interactive, streaming rendering results with reduced latency.
為達上述目的,本案之一實施例為一種邊緣渲染運算方法,應用於混合實境系統,混合實境系統包含至少一混合實境裝置、中央管理伺服器及至少一行動邊緣運算裝置,邊緣渲染運算方法包含:執行運作程序,運作程序包含: 啟動至少一混合實境裝置,並執行混合實境裝置之用戶應用程式,用戶應用程式透過應用程式介面建立混合實境裝置與中央管理伺服器之間的連線;中央管理伺服器發送可提供的混合實境應用程式之一清單至混合實境裝置,且由混合實境裝置顯示清單;由混合實境裝置從清單中選擇其中之一混合實境應用程式,並透過應用程序介面將所選擇的混合實境應用程式所對應的識別碼傳送至中央管理伺服器;中央管理伺服器在接獲識別碼後輸出啟動訊號至與混合實境裝置最相近之行動邊緣運算裝置,作為選定行動邊緣運算裝置,其中啟動訊號包含混合實境裝置所選擇的混合實境應用程式的資訊;選定行動邊緣運算裝置在接獲啟動訊號後輸出回傳訊號至中央管理伺服器,並執行用戶連線程序等待與混合實境裝置連線;中央管理伺服器於接獲回傳訊號後將選定行動邊緣運算裝置的網際協定位址傳送至混合實境裝置,使混合實境裝置與選定行動邊緣運算裝置嘗試建立連線;選定行動邊緣運算裝置判斷是否與混合實境裝置連線成功。 To achieve the above purpose, one embodiment of this case is an edge rendering calculation method, which is applied to a mixed reality system. The mixed reality system includes at least one mixed reality device, a central management server, and at least one mobile edge computing device. The edge rendering The calculation method includes: executing the operation program, and the operation program includes: Start at least one mixed reality device and execute the user application program of the mixed reality device. The user application program establishes a connection between the mixed reality device and the central management server through the API; the central management server sends the available A list of mixed reality applications is sent to the mixed reality device, and the list is displayed by the mixed reality device; one of the mixed reality applications is selected from the list by the mixed reality device, and the selected one is sent through the API The identification code corresponding to the mixed reality application is sent to the central management server; after receiving the identification code, the central management server outputs an activation signal to the mobile edge computing device closest to the mixed reality device as the selected mobile edge computing device , wherein the activation signal includes the information of the mixed reality application program selected by the mixed reality device; the selected mobile edge computing device outputs a return signal to the central management server after receiving the activation signal, and executes the user connection procedure waiting and mixing Reality device connection: After receiving the return signal, the central management server sends the IP address of the selected mobile edge computing device to the mixed reality device, so that the mixed reality device and the selected mobile edge computing device try to establish a connection ; Select the mobile edge computing device to determine whether the connection with the mixed reality device is successful.
為達上述目的,本案之另一實施例為混合實境系統,包含:混合實境裝置;中央管理伺服器;以及至少一行動邊緣運算裝置;其中混合實境系統用以執行前述之邊緣渲染運算方法。To achieve the above purpose, another embodiment of the present case is a mixed reality system, comprising: a mixed reality device; a central management server; and at least one mobile edge computing device; wherein the mixed reality system is used to perform the aforementioned edge rendering calculation method.
體現本案特徵與優點的一些典型實施例將在後段的說明中詳細敘述。應理解的是本案能夠在不同的態樣上具有各種的變化,其皆不脫離本案的範圍,且其中的說明及圖式在本質上系當作說明之用,而非用於限制本案。Some typical embodiments embodying the features and advantages of the present application will be described in detail in the description in the following paragraphs. It should be understood that this case can have various changes in different aspects without departing from the scope of this case, and the descriptions and drawings therein are used as illustrations in nature rather than limiting this case.
請參閱第1圖,其係為本案較佳實施例之混合實境系統的系統架構圖。本實施例之混合實境系統1包含至少一混合實境裝置2、中央管理伺服器3及至少一行動邊緣運算裝置4,其中混合實境裝置2、中央管理伺服器3及行動邊緣運算裝置4三者間可為但不限於以第五代行動通訊技術5G進行連線通訊。此外,混合實境裝置2與行動邊緣運算裝置4的個數更可分別為複數個,其中複數個混合實境裝置2的所在位置可不同,而複數個行動邊緣運算裝置4的所在位置亦可不同,且任一行動邊緣運算裝置4相較於其它行動邊緣運算裝置4會與複數個混合實境裝置2的其中之一混合實境裝置2最相鄰。Please refer to Figure 1, which is a system architecture diagram of the mixed reality system of the preferred embodiment of this case. The
中央管理伺服器3用以管理所有行動邊緣運算裝置4的資源,且可處理混合實境裝置2與行動邊緣運算裝置4之間的連線。於一些實施例中,中央管理伺服器3具有Portal入口服務的功能,以提供授權驗證與應用程式操作服務等。此外,中央管理伺服器3更具有混合實境應用程式的資料,並可提供關於可執行的混合實境應用程式的清單。The
混合實境裝置2可視為擴增實境裝置的一種,且混合實境裝置2可為但不限於頭戴式顯示器或眼鏡等,其中,混合實境裝置2可對所在環境進行感測,以提供即時資訊,即時資訊包含即時影音資訊、即時聲音資訊及/或即時加速度資訊等。The mixed
於一些實施例中,每一混合實境裝置2具有用戶應用程式,當混合實境裝置2執行用戶應用程式時,用戶應用程式便透過混合實境裝置2之應用程式介面(Application Programming Interface, API)讓混合實境裝置2與中央管理伺服器3之間建立連線,藉此混合實境裝置2便可透過中央管理伺服器3再與行動邊緣運算裝置4進行連線。In some embodiments, each
行動邊緣運算裝置4可與中央管理伺服器3保持連線,且將混合實境裝置2所感測到的即時資訊進行運算,藉此混合實境裝置2可利用行動邊緣運算裝置4的運算結果而顯示出渲染了虛擬影音的擴增實境影音。此外,於本實施例中,混合實境裝置2所感測到的即時資訊將提供給與自身距離最相近之行動邊緣運算裝置4來進行運算。行動邊緣運算裝置4更可儲存至少一混合實境應用程式。The mobile
以下將以第2圖所示之邊緣渲染運算方法來更進一步說明混合實境裝置2、中央管理伺服器3及行動邊緣運算裝置4三者間的作動。請參閱第2A圖、第2B圖,並配合第1圖,其中第2A圖及第2B圖為應用於第1圖所示之混合實境系統之本案較佳實施例之邊緣渲染運算方法執行運作程序時的步驟流程圖。本案之邊緣渲染運算方法包含執行運作程序,運作程序的步驟包含如下。The following will use the edge rendering computing method shown in FIG. 2 to further illustrate the actions among the
步驟S1,啟動混合實境裝置2,並執行混合實境裝置2的用戶應用程式,用戶應用程式透過混合實境裝置2的應用程式介面建立混合實境裝置2與中央管理伺服器3之間的連線。Step S1, start the
步驟S2,中央管理伺服器3發送可提供的混合實境應用程式之一清單至混合實境裝置2,且由混合實境裝置顯示清單。In step S2, the
步驟S3,使用者操作混合實境裝置2,以由混合實境裝置2從清單中選擇其中之一混合實境應用程式,並透過應用程序介面將所選擇的混合實境應用程式所對應的識別碼傳送至中央管理伺服器3。Step S3, the user operates the
步驟S4,中央管理伺服器3在接獲識別碼後輸出啟動訊號至與混合實境裝置2最相近之行動邊緣運算裝置4,以作為選定行動邊緣運算裝置,其中啟動訊號包含混合實境裝置2所選擇的混合實境應用程式的資訊。Step S4, after receiving the identification code, the
步驟S5,選定行動邊緣運算裝置在接獲啟動訊號後輸出回傳訊號至中央管理伺服器3,且選定行動邊緣運算裝置更執行用戶連線程序等待與混合實境裝置2連線。In step S5, the selected mobile edge computing device outputs a return signal to the
步驟S6,中央管理伺服器3於接獲回傳訊號後將選定行動邊緣運算裝置的網際協定位址傳送至混合實境裝置2,使混合實境裝置2與選定行動邊緣運算裝置嘗試建立連線。Step S6, the
步驟S7,選定行動邊緣運算裝置判斷是否與混合實境裝置2連線成功。In step S7, the selected mobile edge computing device determines whether the connection with the
由上可知,本案之應用於混合實境系統之邊緣渲染運算方法及其混合實境系統由於使用了行動邊緣運算裝置,使得混合實境裝置所感測的即時資訊可由行動邊緣運算裝置來進行運算,而非透過資料中心或是其它運算裝置來運算,因此可以加快資料的處理與傳送速度,並減少混合實境裝置的顯示延遲,以提升使用者的體驗效果。It can be seen from the above that the edge rendering calculation method applied to the mixed reality system and the mixed reality system of this case use the mobile edge computing device, so that the real-time information sensed by the mixed reality device can be calculated by the mobile edge computing device. Instead of computing through a data center or other computing devices, it can speed up the processing and transmission of data, and reduce the display delay of the mixed reality device, so as to improve the user experience.
於一些實施例中,運作程序更包含步驟S8及步驟S9。步驟S8,當混合實境裝置2判斷出與選定行動邊緣運算裝置連線成功時,選定行動邊緣運算裝置依據啟動訊號執行混合實境裝置2所選擇的混合實境應用程式,並將混合實境裝置2所感測到的即時資訊進行運算,使執行中的混合實境應用程式利用運算結果產生暫時影音,而暫時影音透過編碼及解碼後形成擴增實境影音,並由混合實境裝置2顯示擴增實境影音。In some embodiments, the operation procedure further includes step S8 and step S9. Step S8, when the
步驟S9,在選定行動邊緣運算裝置執行混合實境裝置2所選擇的混合實境應用程式後,選定行動邊緣運算裝置傳送更新資訊至中央管理伺服器3進行更新,其中更新資訊包含與選定行動邊緣運算裝置之資訊及混合實境裝置2之資訊。Step S9, after the selected mobile edge computing device executes the mixed reality application program selected by the
於一些實施例中,運作程序更包含步驟S9,當混合實境裝置判斷出與選定行動邊緣運算裝置連線失敗時,選定行動邊緣運算裝置輸出連線失敗訊號至中央管理伺服器3,使中央管理伺服器3通知混合實境裝置2啟動中止。In some embodiments, the operation procedure further includes step S9, when the mixed reality device determines that the connection with the selected mobile edge computing device fails, the selected mobile edge computing device outputs a connection failure signal to the
請參閱第3圖,並配合第1圖,其中第3圖為第1圖所示之混合實境裝置及行動邊緣運算裝置的細部結構示意圖。混合實境裝置2包含感測模組20、渲染模組21、輸入控制模組22、影音解碼模組23、第一通訊模組24、解壓縮模組25及顯示模組26。第一通訊模組24用以利用可為但不限於第五代行動通訊技術5G進行資料上傳與資料下載。感測模組20用以感測混合實境裝置2的所在環境,以提供即時資訊,且該即時資訊可藉由第一通訊模組24傳送至行動邊緣運算裝置4,其中感測模組20包含相機模組200、聲音模組201以及加速度計202,故感測模組20提供之即時資訊中可藉由相機模組200的感測而包含即時影像資訊,並藉由聲音模組201的感測而包含即時聲音資訊,以及藉由加速度計202的感測而包含即時加速度資訊。影音解碼模組23用以對所接收到的影音進行解碼,並提供給渲染模組21。渲染模組21對影音解碼模組23所提供的解碼影音進行渲染,以形成擴增實境影音。顯示模組26用以顯示由渲染模組21所形成之擴增實境影音。輸入控制模組22用以依據使用者的輸入操作產生第一控制訊號,第一控制訊號可利用第一通訊模組24傳送至行動邊緣運算裝置4,其中輸入控制模組22更包含藍芽裝置220,藍芽裝置220用以與使用者所操作的輸入裝置(未圖示)進行藍芽通訊,使輸入裝置依據使用者的輸入操作所產生的訊號及資料等可經由藍芽裝置220傳送至輸入控制模組22,進而讓輸入控制模組22產生對應的第一控制訊號。解壓縮模組25用以將通訊模組24所下載的已壓縮影音資料(例如從行動邊緣運算裝置4所下載的已壓縮且編碼的暫時影音)進行解壓縮,並傳送給影音解碼模組23進行解碼。Please refer to Figure 3, and cooperate with Figure 1, wherein Figure 3 is a schematic diagram of the detailed structure of the mixed reality device and the mobile edge computing device shown in Figure 1. The
行動邊緣運算裝置4包含第二通訊模組40、運算模組41、控制模擬模組42、壓縮模組43、應用程式模組44及影音編碼模組45。第二通訊模組40用以利用可為但不限於第五代行動通訊技術5G進行資料上傳與資料下載,其中第二通訊模組40可與第一通訊模組24相通訊。運算模組41可藉由第二通訊模組40接收混合實境裝置2之感測模組20所提供之即時資訊,並將即時資訊進行運算。控制模擬模組42可藉由第二通訊模組40接收混合實境裝置2之輸入控制模組22所提供之第一控制訊號,並依據預設的訊號格式而將第一控制訊號進行轉換,以產生為統一格式的第二控制訊號,且於一些實施例中,控制模擬模組42包含控制訊號接收裝置420及訊號映射裝置421,控制訊號接收裝置420接收混合實境裝置2之輸入控制模組22所提供之第一控制訊號,訊號映射裝置421則預設訊號格式,並依據預設的訊號格式將第一控制訊號進行轉換,以產生為統一格式的第二控制訊號。應用程式模組44用以儲存至少一混合實境應用程式,且可依據控制模擬模組42產生之第二控制訊號而執行所儲存之混合實境應用程式,並使執行的混合實境應用程式利用運算模組41的運算結果產生暫時影音,且於一些實施例中應用程式模組44包含應用程式庫440及運行庫441,其中應用程式庫440用以儲存至少一混合實境應用程式,運行庫441則用以驅使應用程式庫440所儲存的任一混合實境應用程式執行,以利用運算模組41的運算結果產生暫時影音。影音編碼模組45用以將應用程式模組44產生的暫時影音進行編碼。壓縮模組43用以將影音編碼模組45所傳來的已編碼的暫時影音進行壓縮,而已壓縮且編碼的暫時影音可再透過第二通訊模組40傳送至混合實境裝置2。The mobile
請參閱第4圖,並配合第1圖,其中第4圖為應用於第1圖所示之混合實境系統之本案較佳實施例之邊緣渲染運算方法執行串流分享程序時的步驟流程圖。於一些實施例,本案之混合實境系統1更包含串流伺服器5,串流伺服器5可與中央管理伺服器3保持連線,且用以利用串流技術進行影音傳輸,且可儲存串流格式之影音檔案。此外,每一混合實境裝置2更具有分享應用程式,當混合實境裝置2執行分享應用程式時,分享應用程式便透過應用程式介面通知中央管理伺服器3對應的混合實境裝置2即將進行畫面分享。又行動邊緣運算裝置4更可進行影音串流傳輸運作,例如將混合實境應用程式利用運算結果產生的暫時影音以串流方式傳輸至串流伺服器5。Please refer to Figure 4 and cooperate with Figure 1, wherein Figure 4 is a flow chart of the steps of the edge rendering calculation method of the preferred embodiment of this case applied to the mixed reality system shown in Figure 1 when executing the stream sharing program . In some embodiments, the
更甚者,本案之邊緣渲染運算方法更包含執行串流分享程序,串流分享程序的步驟包含如下。What's more, the edge rendering calculation method in this case further includes executing a stream sharing program, and the steps of the stream sharing program include the following.
步驟S10,執行混合實境裝置2之分享應用程式,分享應用程式透過應用程式介面通知中央管理伺服器2混合實境裝置2即將分享畫面。Step S10, execute the sharing application program of the
步驟S11,中央管理伺服器3通知選定行動邊緣運算裝置開始進行串流傳輸運作。In step S11, the
步驟S12,選定行動邊緣運算裝置將暫時影音以串流方式傳輸至串流伺服器5。In step S12, the selected mobile edge computing device transmits the temporary audio and video to the streaming server 5 in a streaming manner.
步驟S13,串流伺服器5於接收到串流格式的暫時影音後,回報中央管理伺服器3。In step S13, the streaming server 5 reports to the
步驟S14,中央管理伺服器3回報混合實境裝置2目前正在進行畫面分享。In step S14, the
請參閱第5圖,並配合第1圖,其中第5圖為應用於第1圖所示之混合實境系統之本案較佳實施例之邊緣渲染運算方法執行觀看程序時的步驟流程圖。於一些實例中,混合實境系統1更包含至少一觀看裝置6,觀看裝置6可為具顯示功能之裝置,例如電視、電腦或手機等,且觀看裝置6可提供用來觀賞串流影音之串流頻道,此外,觀看裝置6更可與中央管理伺服器3相通訊,且可與串流伺服器5建立連線。Please refer to FIG. 5, and cooperate with FIG. 1, wherein FIG. 5 is a flow chart of the steps of the edge rendering calculation method of the preferred embodiment of this case applied to the mixed reality system shown in FIG. 1 when executing the viewing program. In some examples, the
另外,本案之邊緣渲染運算方法更包含執行觀看程序,觀看程序的步驟包含如下。In addition, the edge rendering calculation method in this case further includes executing a viewing program, and the steps of the viewing program include the following.
步驟S20,從觀看裝置6選擇串流頻道,且觀看裝置6將被選擇的串流頻道的資訊通知中央管理伺服器3。In step S20, a streaming channel is selected from the
步驟S21,中央管理伺服器3傳送串流伺服器5的資訊給觀看裝置6。Step S21 , the
步驟S22,觀看裝置6依據中央管理伺服器3所傳送的串流伺服器5的資訊與串流伺服器5建立連線。Step S22 , the
步驟S23,於觀看裝置6與串流伺服器5建立連線完成後,觀看裝置6下載串流伺服器5所提供之串流格式的暫時影音,使觀看裝置6的串流頻道依據串流格式的暫時影音顯示對應的影音畫面。Step S23, after the connection between the
請參閱第6圖,並配合第1圖,其中第6圖為應用於第1圖所示之混合實境系統之本案較佳實施例之邊緣渲染運算方法執行中繼服務程序時的步驟流程圖。於一些實例中,混合實境系統1更包含至少一終端裝置7及至少一中繼伺服器8。每一終端裝置7及每一中繼伺服器8可分別與對應之至少一混合實境裝置2相鄰設,其中終端裝置7可為但不限於包含環境感測器或相機模組等,當終端裝置7為環境感測器時,可感測對應之混合實境裝置2之周遭的環境參數,例如溫度、位移及/或速度等,並對應輸出感測資訊,當終端裝置7為相機模組時,則可感測對應之混合實境裝置2周遭的環境影像,並對應輸出感測資訊。中繼伺服器8可與中央管理伺服器3及對應的終端裝置7相通訊,且可接收並儲存終端裝置7所輸出的感測資訊。Please refer to Figure 6, and cooperate with Figure 1, wherein Figure 6 is a flow chart of the steps of the edge rendering calculation method of the preferred embodiment of this case applied to the mixed reality system shown in Figure 1 when executing the relay service program . In some examples, the
另外,本案之邊緣渲染運算方法更包含中繼服務程序,中繼服務程序的步驟包含如下。In addition, the edge rendering computing method in this case further includes a relay service program, and the steps of the relay service program are as follows.
步驟S30,由與混合實境裝置2相近的至少一終端裝置7對混合實境裝置2周遭的環境進行感測,並輸出感測資訊給與混合實境裝置2相近之中繼伺服器8。Step S30 , at least one terminal device 7 close to the
步驟S31,接收到感測資訊之中繼伺服器8向中央管理伺服器3進行註冊。In step S31 , the
步驟S32,中央管理伺服器3更新已進行註冊之中繼伺服器8的清單。In step S32, the
步驟S33,中央管理伺服器3發送已註冊之中繼伺服器8的連線資訊給選定行動邊緣運算裝置。In step S33, the
步驟S34,選定行動邊緣運算裝置與已註冊之中繼伺服器8建立連線,並從建立連線之中繼伺服器8下載感測資訊。Step S34 , the selected mobile edge computing device establishes a connection with the registered
步驟S35,選定行動邊緣運算裝置將感測資訊配合混合實境裝置2所感測到的即時資訊進行運算,使執行的混合實境應用程式利用運算結果調整擴增實境影音。Step S35 , the selected mobile edge computing device calculates the sensing information with the real-time information sensed by the
綜上所述,本案提供一種應用於混合實境系統之邊緣渲染運算方法及其混合實境系統,邊緣渲染運算方法及其混合實境系統由於使用了行動邊緣運算裝置,使得混合實境裝置所感測的即時資訊可由行動邊緣運算裝置來進行運算,而非透過資料中心或是其它運算裝置來運算,因此可以加快資料的處理與傳送速度,並減少混合實境裝置的顯示延遲,以提升使用者的體驗效果。In summary, this case provides an edge rendering computing method applied to a mixed reality system and its mixed reality system. The edge rendering computing method and its mixed reality system use a mobile edge computing device to make the mixed reality device feel The real-time information measured can be calculated by the mobile edge computing device instead of the data center or other computing devices, so the processing and transmission speed of data can be accelerated, and the display delay of mixed reality devices can be reduced to improve user experience. experience effect.
1:混合實境系統 2:混合實境裝置 3:中央管理伺服器 4:行動邊緣運算裝置 20:感測模組 21:渲染模組 22:輸入控制模組 23:影音解碼模組 24:第一通訊模組 25:解壓縮模組 26:顯示模組 200:相機模組 201:聲音模組 202:加速度計 220:藍芽裝置 40:第二通訊模組 41:運算模組 42:控制模擬模組 43:壓縮模組 44:應用程式模組 45:影音編碼模組 420:控制訊號接收裝置 421:訊號映射裝置 440:應用程式庫 441:運行庫 5:串流伺服器 6:觀看裝置 7:終端裝置 8:中繼伺服器 S1~S9:邊緣渲染運算方法執行運作程序時的步驟 S10~S14:邊緣渲染運算方法執行串流分享程序時的步驟 S20~S23:邊緣渲染運算方法執行觀看程序時的步驟 S30~S35:邊緣渲染運算方法執行中繼服務程序時的步驟 1: Mixed reality system 2: Mixed reality device 3: Central management server 4: Mobile edge computing device 20: Sensing module 21: Rendering module 22: Input control module 23: Audio and video decoding module 24: The first communication module 25: Decompression module 26: Display module 200: Camera module 201: Sound Module 202: accelerometer 220:Bluetooth device 40: Second communication module 41: Operation module 42:Control simulation module 43: Compression module 44: Application Module 45: Video and audio coding module 420: Control signal receiving device 421: Signal mapping device 440: Application Library 441: runtime library 5: Streaming server 6: Watching device 7: Terminal device 8:Relay server S1~S9: Steps when the edge rendering operation method executes the operation program S10~S14: Steps when the edge rendering operation method executes the streaming sharing program S20~S23: Steps when the edge rendering operation method executes the viewing program S30~S35: Steps when the edge rendering operation method executes the relay service program
第1圖為本案較佳實施例之混合實境系統的系統架構圖; 第2A圖及第2B圖為應用於第1圖所示之混合實境系統之本案較佳實施例之邊緣渲染運算方法執行運作程序時的步驟流程圖; 第3圖為第1圖所示之混合實境裝置及行動邊緣運算裝置的細部結構示意圖; 第4圖為應用於第1圖所示之混合實境系統之本案較佳實施例之邊緣渲染運算方法執行串流分享程序時的步驟流程圖; 第5圖為應用於第1圖所示之混合實境系統之本案較佳實施例之邊緣渲染運算方法執行觀看程序時的步驟流程圖; 第6圖為應用於第1圖所示之混合實境系統之本案較佳實施例之邊緣渲染運算方法執行中繼服務程序時的步驟流程圖。 Figure 1 is a system architecture diagram of the mixed reality system of the preferred embodiment of this case; Fig. 2A and Fig. 2B are the flow charts of the steps when the edge rendering calculation method of the preferred embodiment of the present case applied to the mixed reality system shown in Fig. 1 executes the operation procedure; Figure 3 is a schematic diagram of the detailed structure of the mixed reality device and the mobile edge computing device shown in Figure 1; Fig. 4 is a flow chart of steps when the edge rendering calculation method of the preferred embodiment of the present case applied to the mixed reality system shown in Fig. 1 executes the stream sharing program; Fig. 5 is a flow chart of steps when the edge rendering calculation method of the preferred embodiment of the present case applied to the mixed reality system shown in Fig. 1 executes the viewing program; FIG. 6 is a flow chart of steps when the edge rendering calculation method of the preferred embodiment of the present application applied to the mixed reality system shown in FIG. 1 executes the relay service program.
S1~S9:邊緣渲染運算方法執行運作程序時的步驟 S1~S9: Steps when the edge rendering operation method executes the operation program
Claims (10)
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
TW110134251A TWI823146B (en) | 2021-09-14 | 2021-09-14 | Edge side rendering operation method and system for real-time mr interactive application |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
TW110134251A TWI823146B (en) | 2021-09-14 | 2021-09-14 | Edge side rendering operation method and system for real-time mr interactive application |
Publications (2)
Publication Number | Publication Date |
---|---|
TW202311908A true TW202311908A (en) | 2023-03-16 |
TWI823146B TWI823146B (en) | 2023-11-21 |
Family
ID=86690551
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
TW110134251A TWI823146B (en) | 2021-09-14 | 2021-09-14 | Edge side rendering operation method and system for real-time mr interactive application |
Country Status (1)
Country | Link |
---|---|
TW (1) | TWI823146B (en) |
Family Cites Families (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US11140541B2 (en) * | 2017-06-05 | 2021-10-05 | Sony Group Corporation | Method for management of movable edge computing servers |
EP3750335A4 (en) * | 2018-06-01 | 2021-08-11 | Samsung Electronics Co., Ltd. | Systems and methods for better resource utilization in 5g networks using enablement layer |
EP3731495B1 (en) * | 2018-07-13 | 2022-08-31 | Samsung Electronics Co., Ltd. | Method and electronic device for edge computing service |
TWI693845B (en) * | 2018-12-19 | 2020-05-11 | 未來市股份有限公司 | Dispatching method and edge computing system |
AU2020228333B2 (en) * | 2019-02-25 | 2023-05-25 | Niantic, Inc. | Augmented reality mobile edge computing |
CN113347275A (en) * | 2021-07-06 | 2021-09-03 | 北京云端智度科技有限公司 | Edge node scheduling method and system based on geographic coordinates of user terminal |
-
2021
- 2021-09-14 TW TW110134251A patent/TWI823146B/en active
Also Published As
Publication number | Publication date |
---|---|
TWI823146B (en) | 2023-11-21 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US9800939B2 (en) | Virtual desktop services with available applications customized according to user type | |
US20180219929A1 (en) | Method and system for distributed processing, rendering, and displaying of content | |
AU2010260683B2 (en) | Apparatus and method for transmitting and receiving a user interface in a communication system | |
CN113209632B (en) | Cloud game processing method, device, equipment and storage medium | |
JP2017510862A (en) | Drawing system, control method, and storage medium | |
JP2010524056A (en) | Remote control device and control method thereof | |
WO2010008230A2 (en) | Apparatus and method for providing user interface service in a multimedia system | |
WO2019127369A1 (en) | Live broadcast sharing method, and related device and system | |
US20140243083A1 (en) | Apparatus and method of providing cloud service using game platform based on streaming | |
US11272261B2 (en) | Cloud platform capable of providing real-time streaming services for heterogeneous applications including AR, VR, XR, and MR irrespective of specifications of hardware of user | |
KR20150096440A (en) | Distributed cross-platform user interface and application projection | |
KR101942269B1 (en) | Apparatus and method for playing back and seeking media in web browser | |
CN111298427A (en) | Method for reducing picture jitter in virtual reality cloud game system | |
CN111478930A (en) | STB cloud method, system, thin STB, virtual STB, platform and storage medium | |
CN112354176A (en) | Cloud game implementation method, cloud game implementation device, storage medium and electronic equipment | |
CN114546308A (en) | Application interface screen projection method, device, equipment and storage medium | |
CN111935510B (en) | Double-browser application loading method and display equipment | |
WO2024027611A1 (en) | Video live streaming method and apparatus, electronic device and storage medium | |
TWI823146B (en) | Edge side rendering operation method and system for real-time mr interactive application | |
US10514959B2 (en) | Distributed virtual local operating system stored in cloud and its method | |
CN103618968B (en) | A kind of network TV play method and system towards under cloud environment | |
KR20120117383A (en) | Media control device application executing method and system in media displaying device using presentation virtualization | |
US12022160B2 (en) | Live streaming sharing method, and related device and system | |
CN115801878A (en) | Cloud application picture transmission method, equipment and storage medium | |
de Godoy et al. | Multimedia Presentation integrating media with virtual 3D realistic environment produced in Real Time with High Performance Processing |