TWI825838B - Method and computer system for deciding quality of shared screen for remote play - Google Patents
Method and computer system for deciding quality of shared screen for remote play Download PDFInfo
- Publication number
- TWI825838B TWI825838B TW111125857A TW111125857A TWI825838B TW I825838 B TWI825838 B TW I825838B TW 111125857 A TW111125857 A TW 111125857A TW 111125857 A TW111125857 A TW 111125857A TW I825838 B TWI825838 B TW I825838B
- Authority
- TW
- Taiwan
- Prior art keywords
- image quality
- training
- decoding device
- electronic devices
- delay
- Prior art date
Links
- 238000000034 method Methods 0.000 title claims abstract description 43
- 238000012549 training Methods 0.000 claims description 90
- 238000012545 processing Methods 0.000 claims description 42
- 238000013210 evaluation model Methods 0.000 claims description 20
- 230000009467 reduction Effects 0.000 claims description 14
- 230000004044 response Effects 0.000 claims description 14
- 230000003111 delayed effect Effects 0.000 claims description 5
- 230000007423 decrease Effects 0.000 claims 1
- 230000008569 process Effects 0.000 description 6
- 230000001934 delay Effects 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 4
- 230000006870 function Effects 0.000 description 4
- 238000013528 artificial neural network Methods 0.000 description 2
- 229920001621 AMOLED Polymers 0.000 description 1
- 230000009471 action Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 230000001186 cumulative effect Effects 0.000 description 1
- 230000007123 defense Effects 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 230000009191 jumping Effects 0.000 description 1
- 238000013178 mathematical model Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
Images
Landscapes
- Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
- Information Transfer Between Computers (AREA)
Abstract
Description
本發明涉及遠端遊玩(remote play)領域，特別涉及決定遠端遊玩的分享畫面之畫質的技術。The present invention relates to the field of remote play, and in particular to a technology for determining the image quality of a shared screen for remote play.
遠端遊玩是可以讓玩家在有網路連線的任何地方，使用客戶端裝置(特別是行動裝置)控制遠端的遊戲主機(game console)，盡情玩遊戲的一種技術。這樣的技術，可以讓玩家不受地理位置限制，而享受到遊戲主機所提供高運算效能。Remote play is a technology that allows players to use a client device (especially a mobile device) to control a remote game console and play games anywhere there is an Internet connection. This kind of technology allows players to enjoy the high computing performance provided by game consoles without being restricted by geographical location.
在多人對戰的情境中，遊戲主機會將遊戲畫面同步分享給多個客戶端裝置。客戶端裝置在收到遊戲主機所提供的分享畫面後，需對其進行解碼，以顯示該分享畫面。因此，在分享畫面之期間，遊戲運行的流暢度不僅僅取決於遊戲主機的運算效能，而也會受到客戶端裝置的解碼裝置(例如CPU或GPU)的運算能力影響。只要任何一方玩家的客戶端裝置的解碼運算能力低落，便很有可能使遊戲發生卡頓、延遲的現象，甚至使得遊戲難以繼續進行下去。In a multiplayer battle scenario, the game console will simultaneously share the game screen with multiple client devices. After receiving the sharing screen provided by the game console, the client device needs to decode it to display the sharing screen. Therefore, during screen sharing, the smoothness of game operation not only depends on the computing performance of the game console, but is also affected by the computing power of the decoding device (such as CPU or GPU) of the client device. As long as the decoding computing power of any player's client device is low, the game is likely to experience freezes and delays, and may even make it difficult to continue the game.
除了解碼裝置的運算能力外，分享畫面的畫質，如解析度或幀率(frame rate)，也是決定解碼速度之關鍵因素。越高的畫質，會需要越高的解碼運算能力及/或越長的解碼運算時間。因此，對於以上所述遊戲卡頓延遲的問題，調降分享畫面的畫質可以作為一種變通辦法。然而不可否認的是，畫質的調降會衝擊玩家的遊戲體驗。過低的畫質甚至可能使玩家產生不滿情緒，憤而中途離開遊戲。In addition to the computing power of the decoding device, the quality of the shared image, such as resolution or frame rate, is also a key factor in determining the decoding speed. The higher the image quality, the higher the decoding computing power and/or the longer the decoding computing time will be. Therefore, for the above-mentioned problem of game lagging and delay, lowering the image quality of the shared screen can be used as a workaround. However, it is undeniable that the reduction in image quality will impact the player's gaming experience. Too low image quality may even make players dissatisfied and leave the game midway.
目前習知的技術，是以預設的數值，或者遊戲玩家指定的數值，作為分享畫面之畫質。然而，這樣的畫質不一定是能使得遊戲不至於卡頓延遲的分享畫面之最佳畫質。The currently known technology uses a preset value or a value specified by the game player as the image quality of the shared screen. However, this image quality is not necessarily the best image quality for sharing the screen without lagging in the game.
有鑑於此，需要一種方法及電腦系統，用以在多人對戰的遠端遊玩情境中，決定出能使得遊戲不至於卡頓延遲的最佳畫質。In view of this, a method and a computer system are needed to determine the best image quality that can prevent the game from lagging in a remote gaming situation of multi-player battles.
本揭露提供一種決定遠端遊玩的分享畫面之畫質的方法，由電腦系統的處理裝置所執行，包含：於應用程式執行之期間，響應於接收到分享畫面之請求，從每一電子裝置取得解碼裝置資訊；基於既定畫質參數及每一電子裝置的解碼裝置資訊，使用延遲評估模型計算每一電子裝置對應的延遲指數；若所有電子裝置對應的延遲指數皆不大於閾值，則將既定畫質參數作為分享畫面之畫質；以及若任一電子裝置對應的延遲指數大於閾值，則根據每一電子裝置對應的延遲指數計算對應的合適畫質參數，及選取電子裝置的合適畫質參數中的最低者作為分享畫面之畫質。The present disclosure provides a method for determining the image quality of a shared screen for remote play, which is executed by a processing device of a computer system, including: during the execution of an application, in response to receiving a request to share a screen, obtaining from each electronic device Decoding device information; based on the established image quality parameters and the decoding device information of each electronic device, use the delay evaluation model to calculate the delay index corresponding to each electronic device; if the delay index corresponding to all electronic devices is not greater than the threshold, the predetermined image will be The quality parameter is used as the image quality of the shared screen; and if the delay index corresponding to any electronic device is greater than the threshold, the corresponding appropriate image quality parameter is calculated based on the delay index corresponding to each electronic device, and the appropriate image quality parameter of the electronic device is selected. The lowest one is used as the image quality of the shared screen.
在某些實施例中，上述方法更包含使用訓練資料集訓練延遲評估模型。訓練資料集中的每一筆訓練資料包含相互對應的訓練用解碼裝置資訊、訓練用畫質參數及訓練用延遲指數。收集訓練資料集包含：啟動應用程式；根據當下畫質參數將應用程式的畫面分享給多個訓練用電子裝置；響應於偵測到延遲，從每一訓練用電子裝置取得幀處理進度；從幀處理進度最慢的訓練用電子裝置，取得訓練用解碼裝置資訊；將當下畫質參數作為訓練用畫質參數，及記錄下訓練用解碼裝置資訊與訓練用畫質參數的對應關係；調降當下畫質參數，及累加調降次數；響應於應用程式關閉，根據所累計的調降次數推算訓練用解碼裝置資訊與訓練用畫質參數所對應的訓練用延遲指數。In some embodiments, the above method further includes training the delay evaluation model using the training data set. Each piece of training data in the training data set includes corresponding training decoding device information, training image quality parameters and training delay index. Collecting the training data set includes: starting the application; sharing the application screen to multiple training electronic devices according to the current image quality parameters; in response to detecting the delay, obtaining the frame processing progress from each training electronic device; The training electronic device with the slowest processing progress obtains the training decoding device information; uses the current image quality parameters as the training image quality parameters, and records the correspondence between the training decoding device information and the training image quality parameters; lowers the current Image quality parameters, and the accumulated number of downgrades; in response to the application being closed, the training delay index corresponding to the training decoding device information and the training image quality parameters is calculated based on the accumulated number of downgrades.
在某些實施例中，既定畫質參數包含既定解析度及既定幀率，合適畫質參數包含合適解析度及合適幀率，當下畫質參數包含當下解析度及當下幀率。In some embodiments, the predetermined image quality parameters include a predetermined resolution and a predetermined frame rate, the appropriate image quality parameters include an appropriate resolution and an appropriate frame rate, and the current image quality parameters include the current resolution and the current frame rate.
在某些實施例中，調降當下畫質參數包含：若當下所累計的調降次數為奇數，則調降當下幀率；若當下所累計的調降次數為偶數，則調降當下解析度。In some embodiments, reducing the current image quality parameters includes: if the current accumulated number of reductions is an odd number, then reducing the current frame rate; if the current accumulated number of reductions is an even number, then reducing the current resolution .
在某些實施例中，上述方法更包含：於提供分享畫面之期間，響應於偵測到延遲，從每一電子裝置取得幀處理進度；從幀處理進度最慢的電子裝置，取得解碼裝置資訊；從訓練資料集中，找出符合該解碼裝置資訊及分享畫面之畫質的訓練資料；調升該筆訓練資料中的訓練用延遲指數；以及使用更新後的訓練資料集再次訓練延遲評估模型。In some embodiments, the above method further includes: during providing the shared screen, in response to detecting a delay, obtaining frame processing progress from each electronic device; obtaining decoding device information from the electronic device with the slowest frame processing progress ; Find training data from the training data set that matches the decoding device information and the image quality of the shared image; increase the training delay index in the training data; and use the updated training data set to train the delay evaluation model again.
在某些實施例中，解碼裝置資訊包含解碼裝置型號及解碼應用程式介面(API)。In some embodiments, the decoding device information includes a decoding device model and a decoding application programming interface (API).
在某些實施例中，上述方法更包含：對於合適畫質參數最低的電子裝置，根據該電子裝置的解碼裝置資訊，判斷該電子裝置的解碼裝置是否為圖形處理器(GPU)；若電子裝置的解碼裝置為GPU，通知電子裝置對解碼裝置進行升級；以及若電子裝置的解碼裝置不是GPU，通知電子裝置改用GPU。In some embodiments, the above method further includes: for the electronic device with the lowest suitable image quality parameter, determining whether the decoding device of the electronic device is a graphics processing unit (GPU) according to the decoding device information of the electronic device; if the electronic device If the decoding device is a GPU, the electronic device is notified to upgrade the decoding device; and if the decoding device of the electronic device is not a GPU, the electronic device is notified to use the GPU instead.
本揭露提供一種決定遠端遊玩的分享畫面之畫質的電腦系統，包含儲存裝置及處理裝置。儲存裝置儲存訓練延遲評估模型所需的訓練資料集。處理裝置可執行上述方法的各種實施例。The present disclosure provides a computer system that determines the image quality of a shared screen for remote gaming, including a storage device and a processing device. The storage device stores training data sets required for training the latency evaluation model. The processing device may perform various embodiments of the methods described above.
本揭露所提供的方法及電腦系統，引導玩家以最小的畫值調降幅度為代價，即可確保遊戲對戰過程中的流暢度。藉此，玩家得以在遊戲不卡頓延遲之前提下，享受到最高的畫質與最佳的遊戲體驗。The method and computer system provided in this disclosure can guide players to ensure smoothness in the game battle process at the minimum cost of reducing the picture value. With this, players can enjoy the highest image quality and best gaming experience without any lag in the game.
以下敘述列舉本發明的多種實施例，但並非意圖限制本發明內容。實際的發明範圍，是由申請專利範圍所界定。The following description enumerates various embodiments of the present invention, but is not intended to limit the content of the present invention. The actual scope of the invention is defined by the scope of the patent application.
在以下所列舉的各實施例中，將以相同的標號代表相同或相似的元件或件。In each of the embodiments listed below, the same or similar elements or parts will be represented by the same reference numerals.
在本說明書中以及申請專利範圍中的序號，例如「第一」、「第二」等等，僅係為了方便說明，彼此之間並沒有順序上的先後關係。The serial numbers in this specification and the scope of the patent application, such as "first", "second", etc., are only for convenience of explanation, and there is no sequential relationship between them.
第1圖繪示一種典型的遠端遊玩之多人對戰的情境100。在多人對戰的情境100中，多個客戶端裝置102-105可經由網路(例如局部區域網路、廣域網路、虛擬私人網路、網際網路、無線網路及其任何組合)連接至遊戲主機101。遊戲主機101用以執行遊戲的主程式，以及將遊戲畫面同步分享給客戶端裝置102-105。客戶端裝置102-105用以顯示分享畫面給使用者(即遊戲玩家)，以及接收使用者所下達的命令，像是控制遊戲中角色進行某特定動作(例如攻擊、防禦或跳躍)的命令。應注意的是，不同的遊戲可能會有不同數量的對戰玩家人數。因此，雖然第1圖示出四個客戶端裝置，本揭露並不限制遊戲主機所連接客戶端裝置之數量。Figure 1 illustrates a typical
根據本發明之實施例，遊戲主機101可以是任何一種具有處理裝置以執行指令，能夠透過網路(例如局部區域網路、廣域網路、虛擬私人網路、網際網路、無線網路及其任何組合)進行通訊，且通常設置在固定地點的電腦系統，像是個人電腦(如桌上型電腦與筆記型電腦)、伺服器電腦，及/或各種態樣的遊戲機台。上述處理裝置，可以是任何一種用於執行指令的裝置，例如中央處理器(CPU)、微處理器(microprocessor)、控制器、微控制器(microcontroller)、狀態機(state machine)或其任意組合。遊戲主機101可更包含一或多個儲存裝置，例如光儲存元件以及固態儲存裝置如隨機存取記憶體(RAM)、唯讀記憶體(ROM)、可拆卸媒體裝置、記憶卡以及快閃記憶卡、磁碟驅動器等，以儲存遊戲主程式、參數，及遊戲過程中任何需要被儲存的資料。According to embodiments of the present invention, the
根據本發明之實施例，客戶端裝置102-105可以是任何一種具有顯示裝置(如LCD顯示器、LED顯示器、OLED顯示器、AMOLED顯示器或電漿顯示器)以顯示游戲畫面，具有輸入裝置(如鍵盤、滑鼠、搖桿、遊戲手把或觸控式螢幕)以接收玩家下達的指令，並且能夠透過網路(例如局部區域網路、廣域網路、虛擬私人網路、網際網路、無線網路及其任何組合)進行通訊的電子裝置，像是智慧型手機、智慧型電視機、輕薄筆電(Ultrabook)或平板電腦(Tablet)。客戶端裝置102-105更包含解碼裝置，用以將遊戲主機101所提供的分享畫面進行解碼。解碼裝置可例如是設計以執行通用運算的CPU，或者是特別設計以執行圖形運算的圖形處理器(graphic processing unit；GPU)。以處理分享畫面而言，GPU的表現通常會比傳統CPU更為理想。According to embodiments of the present invention, the client devices 102-105 may be any device having a display device (such as an LCD display, an LED display, an OLED display, an AMOLED display or a plasma display) to display the game screen, and an input device (such as a keyboard, mouse, joystick, game controller or touch screen) to receive commands from the player, and can communicate through the network (such as local area network, wide area network, virtual private network, Internet, wireless network and Any combination thereof) electronic devices for communication, such as smartphones, smart TVs, ultrabooks or tablets. The client devices 102-105 further include a decoding device for decoding the shared image provided by the
在多人對戰的情境100中，若客戶端裝置102-105任一者的解碼運算能力低落，該客戶端裝置可能無法及時呈現分享畫面給使用者，因而也無法及時接收到使用者下達的命令。由於遊戲畫面的某些幀(frame)會需要等待使用者下達的命令才能進入下一個幀，此時卡頓延遲的狀況便可能會發生。調降分享畫面的畫質，如解析度或幀率，雖可以改善遊戲進行的流暢度，但也可能衝擊玩家的遊戲體驗。本揭露所提供決定分享畫面之畫質的方法，便是意圖在遊戲流暢度與畫質之間取得最佳平衡點。In the
第2圖是根據本發明之實施例的一種決定分享畫面之畫質的方法200之流程圖。方法200是由遊戲主機(例如第1圖中的遊戲主機101)之處理裝置，在有使用者(即玩家)透過客戶端裝置(例如第1圖中的客戶端裝置102-105任一者)啟動應用程式(即各式各樣的遊戲)並邀請夥伴加入時所執行。如第2圖所示，方法200包含操作201-206。Figure 2 is a flow chart of a
於操作201，響應於接收到分享畫面之請求(例如有玩家邀請到其他玩家，並且請求遊戲主機提供分享畫面給所有玩家)，從多個客戶端裝置的每一者取得解碼裝置資訊。然後，進行操作202。In
在某些實施例中，上述解碼裝置資訊可包含解碼裝置型號及解碼裝置應用程式介面(application programming interface；API)的任一者，或是兩者之組合。In some embodiments, the above decoding device information may include any one of a decoding device model and a decoding device application programming interface (API), or a combination of the two.
於操作202，基於既定畫質參數及每一客戶端裝置的解碼裝置資訊，使用延遲評估模型計算每一客戶端裝置對應的延遲指數。然後，進行操作203。In
在某些實施例中，畫質參數可包含幀率及解析度的任一者，或是兩者之組合。上述既定畫質參數可以是系統預設值，或者是玩家基於主觀判斷所自訂的數值，本揭露並不對此限定。In some embodiments, the image quality parameter may include either frame rate or resolution, or a combination of the two. The above-mentioned predetermined image quality parameters may be system default values, or values customized by players based on subjective judgment. This disclosure is not limited to this.
上述延遲指數，代表遊戲主機預期需要將分享畫面之畫質調降多少單位，才能使該客戶端裝置能及時處理分享畫面，而避免延遲發生。本揭露不限定每單位的具體量值。舉例來說，延遲指數「1」可代表分享畫面須被調降1 fps(frame per second)、3 fps、5 fps，或10 fps的幀率，才能避免延遲。The above-mentioned latency index represents how many units the game console is expected to need to reduce the image quality of the shared screen so that the client device can process the shared screen in time to avoid delays. This disclosure does not limit specific amounts per unit. For example, a latency index of "1" may mean that the shared screen must be reduced to a frame rate of 1 fps (frame per second), 3 fps, 5 fps, or 10 fps to avoid latency.
在某些實施例中，上述延遲評估模型可以是基於神經網路(neural networks)的數學模型。該模型以記載有解碼裝置資訊、畫質參數及延遲指數三者之間對應關係的多筆資料，作為訓練資料集。可使用損失函數(loss function)定義模型基於解碼裝置資訊與畫質參數所推估出的延遲指數，與訓練資料中解碼裝置資訊與畫質參數實際對應的延遲指數之間的差異。於訓練過程中，可使用優化器(optimizer)遞迴地調整模型之參數(例如神經網路之輸出層中的權重分布)，以使損失值被最小化，藉此優化延遲評估模型。在進一步的實施例中，優化器可透過對損失函數作偏微分(partial derivative)計算以取得梯度(gradient)，再根據梯度調整模型之參數，以降低損失值。透過反覆地結果反饋與更新參數的訓練過程，逐步降低損失值，直到損失值收斂至最小值。此時，延遲評估模型便能較精準地從解碼裝置資訊與既定畫質參數，推估出其對應的延遲指數。至於上述訓練資料集的收集，將在之後的說明書內容搭配圖式第3圖加以詳述。In some embodiments, the above delay evaluation model may be a mathematical model based on neural networks. The model uses multiple pieces of data recording the correspondence between decoding device information, image quality parameters and delay index as a training data set. A loss function can be used to define the difference between the delay index estimated by the model based on the decoding device information and image quality parameters, and the delay index actually corresponding to the decoding device information and image quality parameters in the training data. During the training process, an optimizer can be used to recursively adjust the parameters of the model (such as the weight distribution in the output layer of the neural network) so that the loss value is minimized, thereby optimizing the delayed evaluation model. In a further embodiment, the optimizer can obtain the gradient by performing partial derivative calculation on the loss function, and then adjust the parameters of the model according to the gradient to reduce the loss value. Through the training process of repeated result feedback and parameter update, the loss value is gradually reduced until the loss value converges to the minimum value. At this time, the delay evaluation model can more accurately estimate the corresponding delay index from the decoding device information and the established image quality parameters. As for the collection of the above training data set, it will be described in detail in the following instructions and Figure 3 of the diagram.
在某些實施例中，可以對延遲評估模型輸出的結果取概數(例如四捨五入或無條件進位)，作為延遲指數。In some embodiments, the result output by the delay evaluation model may be approximated (eg, rounded or carried unconditionally) as a delay index.
回到第2圖，於操作203，將每一客戶端裝置對應的延遲指數與一閾值做比較。若所有客戶端裝置對應的延遲指數皆不大於閾值，則進行操作204。若有任一客戶端裝置對應的延遲指數大於閾值，則進行操作205。本揭露並不限定上述閾值的大小。Returning to Figure 2, in
於操作204，將既定畫質參數作為分享畫面之畫質。於是，遊戲主機便可根據決定出的畫質提供分享畫面。In
於操作205，根據每一電子裝置對應的延遲指數計算對應的合適畫質參數。然後，進行操作206。In
如先前所述，上述延遲指數代表遊戲主機預期需要將分享畫面之畫質調降多少單位，才能使該客戶端裝置能及時處理分享畫面，而避免延遲發生。至於合適畫質參數，則代表預期足以避免延遲而不必再調降的畫質大小。因此，合適畫質參數等於既定畫質參數減去單位畫質乘以延遲指數的結果。舉例來說，若既定畫質參數為120 fps，單位畫質為5 fps，延遲指數為4，則合適畫質參數為120-5*4=100 fps。As mentioned previously, the above latency index represents how many units the game host is expected to need to reduce the image quality of the shared screen so that the client device can process the shared screen in time to avoid delays. As for the appropriate image quality parameter, it represents the image quality size that is expected to be enough to avoid delays without having to lower it. Therefore, the appropriate image quality parameter is equal to the result of the given image quality parameter minus the unit image quality multiplied by the delay index. For example, if the given image quality parameter is 120 fps, the unit image quality is 5 fps, and the delay index is 4, then the appropriate image quality parameter is 120-5*4=100 fps.
於操作206，選取所有客戶端裝置的合適畫質參數中的最低者，作為分享畫面之畫質。舉例來說，若四個客戶端裝置(如第1圖所示)所對應的合適畫質參數分別是120 fps、115 fps、110 fps、100 fps的幀率，則取100 fps的幀率作為分享畫面之畫質。於是，遊戲主機便可根據決定出的畫質提供分享畫面。在替代的實施例中，可先詢問(例如在進入遊戲前的使用者介面上彈出對話框)使用者是否願意接受方法200所決定出的分享畫面之畫質。若使用者接受該畫質，再實際將該畫質套用到分享畫面。In
在某些實施例中，對於合適畫質參數最低的客戶端裝置，遊戲主機可根據該客戶端裝置的解碼裝置資訊，判斷該客戶端裝置的解碼裝置是否為圖形處理器(GPU)。若該客戶端裝置的解碼裝置為GPU，則通知該客戶端裝置的使用者對解碼裝置進行升級。若該客戶端裝置的解碼裝置不是GPU，則通知該客戶端裝置的使用者改用GPU，像是啟用客戶端裝置的GPU，或者改用帶有GPU的其他電子裝置。In some embodiments, for the client device with the lowest suitable image quality parameters, the game console can determine whether the decoding device of the client device is a graphics processor (GPU) based on the decoding device information of the client device. If the decoding device of the client device is a GPU, the user of the client device is notified to upgrade the decoding device. If the decoding device of the client device is not a GPU, the user of the client device is notified to switch to the GPU, such as enabling the GPU of the client device, or switching to another electronic device with a GPU.
第3圖是根據本發明之實施例收集延遲評估模型之訓練資料集的方法300之流程圖。方法300是由遊戲主機(例如第1圖中的遊戲主機101)之處理裝置，在使用方法200之前所執行。所收集到的訓練資料集，可儲存在遊戲主機的儲存裝置中。如第3圖所示，方法300包含操作301-309。Figure 3 is a flow chart of a
於操作301，啟動應用程式。然後，進行操作302。In
於操作302，根據當下的畫質參數，將應用程式的畫面分享給多個客戶端裝置。然後，進行操作303。In
於操作303，偵測是否發生延遲的狀況。若發生延遲，則進行操作304。若沒有發生延遲，則進行操作308。In
於操作304，從每一客戶端裝置取得幀處理進度，即該客戶端裝置目前處理到第幾個幀。然後，進行操作305。In
於操作305，從幀處理進度最慢的客戶端裝置，取得解碼裝置資訊。舉例來說，若第1圖中的客戶端裝置102-105目前分別是處理到第50、第45、第40、第30個幀，則從處理到第30個幀的客戶端裝置105取得解碼裝置資訊。然後，進行操作306。In
於操作306，記錄下該解碼裝置資訊與當下畫質參數的對應關係。承上述範例，也就是記錄下客戶端裝置105的解碼裝置資訊與當下畫質參數的對應關係。然後，進行操作307。In
於操作307，將當下畫質參數調降1單位，並累加調降次數。然後，回到操作302-303，以將調降後的畫質參數套用至分享畫面，並繼續偵測畫質調降後是否仍再發生延遲的狀況。如果再次發生延遲，操作304-307會再被進行一次，然後再回到操作302-303。換句話說，分享畫面的畫質會再被調降1單位，然後繼續偵測是否仍再發生延遲的狀況，直到應用程式結束前都不再發生延遲為止。In
於操作308，判斷應用程式是否結束。若應用程式結束，則進行操作309。若應用程式尚未結束，則回到操作302。In
於操作309，根據所累計的調降次數推算每一組解碼裝置資訊與當下畫質參數所對應的延遲指數。舉例來說，假設最初的當下畫質參數被設為120 fps的幀率、每次調降5 fps、所累計的調降次數為4，則代表在應用程式結束前，使遊戲得以不再延遲的畫質參數為100 fps。於操作302-307的迴圈被執行4次的過程中，第一次記錄下解碼裝置資訊(以下以”AAA”表示)對應的當下畫質參數為120 fps，距離最後的100 fps還需要被調降4次，因此得到第一筆訓練資料為(“AAA”, 120, 4);第二次記錄下解碼裝置資訊(以下以”BBB”表示)對應的當下畫質參數為115 fps，距離最後的100 fps還需要被調降3次，因此得到第二筆訓練資料為(“BBB”, 115, 3)；第三次記錄下解碼裝置資訊(假設與第一次相同，即”AAA”)對應的當下畫質參數為110 fps，距離最後的100 fps還需要被調降2次，因此得到第三筆訓練資料為(“AAA”, 110, 2)；第四次記錄下解碼裝置資訊(假設與第一次相同，即”AAA”)對應的當下畫質參數為105 fps，距離最後的100 fps還需要被調降1次，因此得到第四筆訓練資料為(“AAA”, 105, 1)。如此一來，可收集到(“AAA”, 120, 4)、(“BBB”, 115, 3)、(“AAA”, 110, 2)及(“AAA”, 105, 1)四筆訓練資料。In
如先前所述，畫質參數可包含幀率及解析度之組合。其中，調降幀率相較於調降解析度而言，對於遊戲體驗的衝擊較小。因此在某些實施例中，會優先考慮調降幀率，其次調降解析度。As mentioned previously, image quality parameters may include a combination of frame rate and resolution. Among them, lowering the frame rate has less impact on the gaming experience than lowering the resolution. Therefore, in some embodiments, the frame rate will be lowered first, and the resolution will be lowered second.
第4圖繪示第3圖中的操作307的一實施例中更詳細的操作步驟之流程圖。如第4圖所示，操作307可再更細分為操作401-404。FIG. 4 illustrates a flowchart of more detailed operational steps in one embodiment of
於操作401，判斷當下累計的調降次數是否為奇數。若調降次數為奇數，則進行操作402。若調降次數為偶數，則進行操作403。In
於操作402，調降當下幀率。然後，進行操作404。In
於操作403，調降當下解析度。然後，進行操作404。In
於操作404，累加調降次數。然後，回到第3圖中的操作302。In
第5圖是根據本發明之實施例對已上線使用中的延遲評估模型進行優化的方法500之流程圖。方法500是由遊戲主機(例如第1圖中的遊戲主機101)之處理裝置所執行。如第5圖所示，方法500包含操作501-505。Figure 5 is a flow chart of a
於操作501，響應於偵測到延遲，從每一客戶端裝置取得幀處理進度。然後，進行操作502。In
於操作502，從幀處理進度最慢的客戶端裝置，取得解碼裝置資訊。然後，進行操作503。In
於操作503，從訓練資料集中，找出符合解碼裝置資訊及分享畫面之畫質的訓練資料。舉例來說，假設幀處理進度最慢的客戶端裝置的解碼裝置資訊為”AAA”，此時分享畫面之畫質為100 fps的幀率，則找出訓練資料集中解碼裝置資訊為”AAA”且畫質參數為100 fps的訓練資料。然後，進行操作504。In
於操作504，調升該筆訓練資料中的延遲指數，本揭露並不限制調升延遲指數的量值。然後，進行操作505。In
於操作505，使用更新後的訓練資料集再次訓練延遲評估模型。In
上述方法可被實現於使用電腦可執行指令。舉例來說，這些指令可包含可以致使通用型電腦、特殊用途電腦或特殊用途處理裝置執行特定功能或一功能的指令和資料。部分所使用的電腦資源可透過網路被存取。舉例來說，電腦可執行性指令可為二元或中間格式指令，像是合語言、韌體或原始碼。The above methods can be implemented using computer-executable instructions. For example, these instructions may include instructions and data that cause a general-purpose computer, a special-purpose computer, or a special-purpose processing device to perform a specific function or functions. Some of the computer resources used may be accessed via the Internet. For example, computer-executable instructions may be binary or intermediate format instructions, such as syntax, firmware, or source code.
本揭露所提供的方法及電腦系統，引導玩家以最小的畫值調降幅度為代價，即可確保遊戲對戰過程中的流暢度。藉此，玩家得以在遊戲不卡頓延遲之前提下，享受到最高的畫質與最佳的遊戲體驗。The method and computer system provided in this disclosure can guide players to ensure smoothness in the game battle process at the minimum cost of reducing the picture value. With this, players can enjoy the highest image quality and best gaming experience without any lag in the game.
以上段落採用多種態樣作敘述。顯然地，本文之教示可以多種方式實現，而在範例中所揭露之任何特定架構或功能僅是一種代表性的情況。根據本文之教示，任何熟知此技藝之人士應理解，可獨立實作本文所揭露之各個態樣，或者合併實作兩種以上之態樣。The above paragraphs are narrated in various ways. Obviously, the teachings of this article can be implemented in a variety of ways, and any specific architecture or functionality disclosed in the examples is only a representative situation. Based on the teachings of this article, anyone familiar with this art should understand that each aspect disclosed in this article can be implemented independently, or two or more aspects can be combined and implemented.
雖然本揭露已以實施例揭露如上，然其並非用以限定本揭露，任何熟習此技藝者，在不脫離本揭露之精神和範圍內，當可作些許之更動與潤飾，因此發明之保護範圍當視後附之申請專利範圍所界定者為準。Although the disclosure has been disclosed above through embodiments, they are not intended to limit the disclosure. Anyone skilled in the art may make slight changes and modifications without departing from the spirit and scope of the disclosure. Therefore, the scope of protection of the invention is The scope of the patent application shall be determined by the attached patent application.
100:多人對戰的情境 101:遊戲主機 102-105:客戶端裝置 200:方法 201-206:操作 300:方法 301-309:操作 401-404:操作 500:方法 501-505:操作 100: Multiplayer battle situation 101:Game console 102-105: Client device 200:Method 201-206: Operation 300:Method 301-309: Operation 401-404: Operation 500:Method 501-505: Operation
本揭露將可從以下示範的實施例之敘述搭配附帶的圖式更佳地理解。此外，應理解的是，在本揭露之流程圖中，各方塊的執行順序可被改變，且/或某些方塊可被改變、刪減或合併。 第1圖繪示一種典型的遠端遊玩之多人對戰的情境。 第2圖是根據本發明之實施例的一種決定分享畫面之畫質的方法之流程圖。 第3圖是根據本發明之實施例收集延遲評估模型之訓練資料集的方法之流程圖。 第4圖繪示調降畫質參數及累加調降次數之更詳細的操作步驟之流程圖。 第5圖是根據本發明之實施例對已上線使用中的延遲評估模型進行優化的方法之流程圖。 The present disclosure will be better understood from the description of the following exemplary embodiments together with the accompanying drawings. In addition, it should be understood that in the flowcharts of the present disclosure, the order of execution of various blocks may be changed, and/or certain blocks may be changed, deleted, or combined. Figure 1 illustrates a typical multiplayer battle situation in remote gaming. Figure 2 is a flowchart of a method for determining the image quality of a shared screen according to an embodiment of the present invention. Figure 3 is a flow chart of a method of collecting a training data set of a delay evaluation model according to an embodiment of the present invention. Figure 4 shows a flowchart of more detailed operation steps for downgrading image quality parameters and accumulating the number of downgrades. Figure 5 is a flow chart of a method for optimizing a delay evaluation model that is already in use according to an embodiment of the present invention.
200:方法 200:Method
201-206:操作 201-206: Operation
Claims (14)
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| TW111125857A TWI825838B (en) | 2022-07-11 | 2022-07-11 | Method and computer system for deciding quality of shared screen for remote play |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| TW111125857A TWI825838B (en) | 2022-07-11 | 2022-07-11 | Method and computer system for deciding quality of shared screen for remote play |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| TWI825838B true TWI825838B (en) | 2023-12-11 |
| TW202402368A TW202402368A (en) | 2024-01-16 |
Family
ID=90053104
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| TW111125857A TWI825838B (en) | 2022-07-11 | 2022-07-11 | Method and computer system for deciding quality of shared screen for remote play |
Country Status (1)
| Country | Link |
|---|---|
| TW (1) | TWI825838B (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| TW201117868A (en) * | 2009-03-23 | 2011-06-01 | Onlive Inc | System and method for accelerated machine switching |
| TW201322743A (en) * | 2011-11-18 | 2013-06-01 | Onlive Inc | Graphical user interface, system and method for controlling a video stream |
| US20140309033A1 (en) * | 2002-12-10 | 2014-10-16 | Stephen G. Perlman | Video Compression System and Method for Reducing the Effects of Packet Loss Over A Communication Channel |
| US20190321725A1 (en) * | 2018-04-02 | 2019-10-24 | Google Llc | Resolution-Based Scaling of Real-time Interactive Graphics |
| CN110812835A (en) * | 2019-11-06 | 2020-02-21 | 腾讯科技（深圳）有限公司 | Cloud game detection method and device, storage medium and electronic device |
| US20220092737A1 (en) * | 2019-09-09 | 2022-03-24 | Nvidia Corporation | Video upsampling using one or more neural networks |
-
2022
- 2022-07-11 TW TW111125857A patent/TWI825838B/en active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20140309033A1 (en) * | 2002-12-10 | 2014-10-16 | Stephen G. Perlman | Video Compression System and Method for Reducing the Effects of Packet Loss Over A Communication Channel |
| TW201117868A (en) * | 2009-03-23 | 2011-06-01 | Onlive Inc | System and method for accelerated machine switching |
| TW201322743A (en) * | 2011-11-18 | 2013-06-01 | Onlive Inc | Graphical user interface, system and method for controlling a video stream |
| US20190321725A1 (en) * | 2018-04-02 | 2019-10-24 | Google Llc | Resolution-Based Scaling of Real-time Interactive Graphics |
| US20220092737A1 (en) * | 2019-09-09 | 2022-03-24 | Nvidia Corporation | Video upsampling using one or more neural networks |
| CN110812835A (en) * | 2019-11-06 | 2020-02-21 | 腾讯科技（深圳）有限公司 | Cloud game detection method and device, storage medium and electronic device |
Also Published As
| Publication number | Publication date |
|---|---|
| TW202402368A (en) | 2024-01-16 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11351469B2 (en) | Method for creating a mini-game | |
| US11229839B2 (en) | Save game load time reduction for cloud gaming | |
| KR102492908B1 (en) | Method and system for providing shortcuts for fast loading when moving between scenes in virtual reality | |
| US9623327B2 (en) | Determining triggers for cloud-based emulated games | |
| JP5969093B2 (en) | Storage medium, system and method for dynamic derivation of matchmaking populations | |
| US9925468B2 (en) | Suspending state of cloud-based legacy applications | |
| US20150258437A1 (en) | Cache-influenced video games | |
| US11064049B2 (en) | Predictive cloud-based presimulation | |
| WO2014194695A1 (en) | Method and server for pvp team matching in computer games | |
| US9658776B2 (en) | Compression of state information for data transfer over cloud-based networks | |
| US20130132509A1 (en) | System And Method For Optimizing Transfers Of Downloadable Content | |
| US20080211822A1 (en) | Method and System For Loading of Image Resource | |
| CN109120987A (en) | A kind of video recording method, device, terminal and computer readable storage medium | |
| US11497993B2 (en) | Method and apparatus for providing game strategy guide | |
| JP7313467B2 (en) | Server load prediction and advanced performance measurement | |
| JP2022502165A (en) | Introducing high input latency in multiplayer programs | |
| TWI825838B (en) | Method and computer system for deciding quality of shared screen for remote play | |
| CN116963809A (en) | In-game dynamic camera angle adjustment | |
| KR20210106152A (en) | Game providing device providing return support system and method of operating the same | |
| US9149725B2 (en) | Method and server for PVP team matching in computer games | |
| KR20120105839A (en) | Game service system and game execution method thereof | |
| JP2020108760A5 (en) | ||
| KR102399661B1 (en) | Apparatus and method for remote connection | |
| KR20180131254A (en) | Apparatus and method for providing game | |
| US11504618B2 (en) | Server-based video help in a video game |