CN117041668A - Method and device for optimizing rendering performance of terminal - Google Patents
Method and device for optimizing rendering performance of terminal Download PDFInfo
- Publication number
- CN117041668A CN117041668A CN202311287713.6A CN202311287713A CN117041668A CN 117041668 A CN117041668 A CN 117041668A CN 202311287713 A CN202311287713 A CN 202311287713A CN 117041668 A CN117041668 A CN 117041668A
- Authority
- CN
- China
- Prior art keywords
- sdk
- decoder
- app
- frame
- terminal
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000009877 rendering Methods 0.000 title claims abstract description 91
- 238000000034 method Methods 0.000 title claims abstract description 43
- 238000004590 computer program Methods 0.000 claims description 4
- 230000015654 memory Effects 0.000 description 6
- 238000010586 diagram Methods 0.000 description 5
- 238000005516 engineering process Methods 0.000 description 5
- 238000004891 communication Methods 0.000 description 3
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 230000006870 function Effects 0.000 description 2
- 230000006378 damage Effects 0.000 description 1
- 230000001066 destructive effect Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
- H04N21/44012—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving rendering scenes according to scene graphs, e.g. MPEG-4 scene graphs
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/443—OS processes, e.g. booting an STB, implementing a Java virtual machine in an STB or power management in an STB
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Software Systems (AREA)
- Information Transfer Between Computers (AREA)
Abstract
The application provides a method and a device for optimizing rendering performance of a terminal, electronic equipment and a storage medium, wherein the method comprises the following steps: after the SDK receives the first frame cloud application picture sent by the cloud server, decoding the first frame cloud application picture through a first decoder, and sending the decoded first frame cloud application picture to a surface texture for off-screen rendering; the SDK sends a first notification message to the APP; the SDK sends an I frame request to the cloud server and receives the I frame sent by the cloud server; the SDK unbinds the first Surface and the EGL, and releases the first decoder, the Surface texture and the EGL; the SDK configures the second decoder by using the first Surface, and starts the second decoder, wherein data decoded by the second decoder directly enter the first Surface for rendering, and through the scheme, the rendering delay of the terminal to the cloud application picture can be reduced.
Description
Technical Field
The present application relates to the field of computer technologies, and in particular, to a method and apparatus for optimizing rendering performance of a terminal, an electronic device, and a storage medium.
Background
Cloud application is an online application technology based on cloud computing technology. Under a cloud application scene, an application program runs in a cloud server, and the cloud server transmits a cloud application picture obtained by rendering to a terminal in a video stream mode through a network so as to be displayed by the terminal.
In the related art, when a user starts a cloud application through an application hall (i.e., APP), a period from when the user clicks a cloud application icon to when the cloud application starts to load a cloud application until a first frame of cloud application picture arrives is a waiting window period, an advertisement or a propaganda video can be played at a terminal in the waiting window period, after the first frame of the cloud application is decoded, an APP (application) creates a Surface object and sends the Surface object to an SDK (software development kit), and the SDK uses the Surface to display the decoded cloud application picture. The starting mode is also called as a half-screen loading mode, in which the decoded data output by the decoder firstly enter Surface texture for processing, then enter Surface view/TextureView for rendering after being processed by a graphics library EGL, and the whole decoding rendering process has more processing flow, so that rendering delay is increased and the efficiency is lower.
Disclosure of Invention
In summary, the embodiment of the application provides a method and a device for optimizing rendering performance of a terminal, electronic equipment and a storage medium, which can reduce rendering delay of the terminal on cloud application pictures.
In a first aspect, an embodiment of the present application provides a method for optimizing rendering performance of a terminal, which is applied to the terminal, and includes:
after the SDK receives the first frame cloud application picture sent by the cloud server, decoding the first frame cloud application picture through a first decoder, and sending the decoded first frame cloud application picture to a surface texture for off-screen rendering, wherein the SDK is integrated in an APP installed on the terminal;
the SDK sends a first notification message to the APP so that the APP hides the preloading interface based on the first notification message and displays a first Surface, wherein the first Surface is created by the APP;
the SDK sends an I frame request to the cloud server and receives the I frame sent by the cloud server;
the SDK unbinds the first Surface and the EGL, and releases the first decoder, the Surface texture and the EGL;
the SDK configures the second decoder by using the first Surface, and starts the second decoder, wherein the data decoded by the second decoder directly enter the first Surface for rendering.
In a second aspect, an embodiment of the present application provides a method for optimizing rendering performance of a terminal, which is applied to the terminal, and includes:
after the SDK receives the first frame cloud application picture sent by the cloud server, decoding the first frame cloud application picture through a first decoder, and sending the decoded first frame cloud application picture to a surface texture for off-screen rendering, wherein the SDK is integrated in an APP installed on the terminal;
the SDK sends a first notification message to the APP so that the APP can hide the preloading interface and the second Surface based on the first notification message and display the first Surface, wherein the first Surface and the second Surface are created by the APP;
the SDK sends an I frame request to the cloud server and receives the I frame sent by the cloud server;
the SDK configures a second decoder by using a second Surface, and starts the second decoder, wherein data decoded by the second decoder directly enter the second Surface for rendering;
the SDK sends a second notification message to the APP so that the APP can hide the first Surface based on the second notification message and display the second Surface;
the SDK unbound the first Surface from the EGL and releases the first decoder, the Surface texture, and the EGL.
In a third aspect, an embodiment of the present application further provides an apparatus for optimizing rendering performance of a terminal, where the apparatus is applied to the terminal, including:
the Software Development Kit (SDK), wherein the SDK is integrated in an APP installed on a terminal, and the SDK specifically executes the following steps:
after the SDK receives the first frame cloud application picture sent by the cloud server, decoding the first frame cloud application picture through a first decoder, and sending the decoded first frame cloud application picture to a surface texture for off-screen rendering;
the SDK sends a first notification message to the APP so that the APP hides the preloading interface based on the first notification message and displays a first Surface, wherein the first Surface is created by the APP;
the SDK sends an I frame request to the cloud server and receives the I frame sent by the cloud server;
the SDK unbinds the first Surface and the EGL, and releases the first decoder, the Surface texture and the EGL;
the SDK configures the second decoder by using the first Surface, and starts the second decoder, wherein the data decoded by the second decoder directly enter the first Surface for rendering.
In a fourth aspect, an embodiment of the present application further provides an apparatus for optimizing rendering performance of a terminal, where the apparatus is applied to the terminal, including:
the Software Development Kit (SDK), wherein the SDK is integrated in an APP installed on a terminal, and the SDK specifically executes the following steps:
after the SDK receives the first frame cloud application picture sent by the cloud server, decoding the first frame cloud application picture through a first decoder, and sending the decoded first frame cloud application picture to a surface texture for off-screen rendering;
the SDK sends a first notification message to the APP so that the APP can hide the preloading interface and the second Surface based on the first notification message and display the first Surface, wherein the first Surface and the second Surface are created by the APP;
the SDK sends an I frame request to the cloud server and receives the I frame sent by the cloud server;
the SDK configures a second decoder by using a second Surface, and starts the second decoder, wherein data decoded by the second decoder directly enter the second Surface for rendering;
the SDK sends a second notification message to the APP so that the APP can hide the first Surface based on the second notification message and display the second Surface;
the SDK unbound the first Surface from the EGL and releases the first decoder, the Surface texture, and the EGL.
In a fifth aspect, embodiments of the present application also provide a computer readable storage medium having stored thereon a computer program which, when executed by a processor, performs the steps of the method of optimizing rendering performance of a terminal according to the first or second aspect.
In a sixth aspect, an embodiment of the present application further provides an electronic device, including: a processor, a storage medium and a bus, the storage medium storing machine-readable instructions executable by the processor, the processor and the storage medium communicating over the bus when the electronic device is running, the processor executing the machine-readable instructions to perform the steps of the method of optimizing rendering performance of a terminal as described in the first or second aspect.
In summary, after decoding of the first frame cloud application picture is completed, by binding Surface to a newly started decoder, data decoded by the decoder can be directly input into the Surface for rendering, compared with decoded data output by the decoder, the decoded data firstly enter Surface texture for processing, then pass through EGL, and finally enter a scheme under a half-screen loading mode of Surface rendering, so that a processing flow of the decoded data is reduced, and the rendering delay of the terminal to the cloud application picture can be reduced.
Drawings
Fig. 1 is a flow chart of a method for optimizing rendering performance of a terminal according to an embodiment of the present application;
fig. 2 is a flowchart of another method for optimizing rendering performance of a terminal according to an embodiment of the present application;
fig. 3 is a schematic structural diagram of an apparatus for optimizing rendering performance of a terminal according to an embodiment of the present application;
fig. 4 is a schematic structural diagram of another apparatus for optimizing rendering performance of a terminal according to an embodiment of the present application;
fig. 5 is a schematic structural diagram of an electronic device according to an embodiment of the present application.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the embodiments of the present application more apparent, the technical solutions of the embodiments of the present application will be clearly and completely described with reference to the accompanying drawings in the embodiments of the present application, and it should be understood that the drawings in the present application are for the purpose of illustration and description only and are not intended to limit the scope of the present application. In addition, it should be understood that the schematic drawings are not drawn to scale. A flowchart, as used in this disclosure, illustrates operations implemented according to some embodiments of the present application. It should be understood that the operations of the flow diagrams may be implemented out of order and that steps without logical context may be performed in reverse order or concurrently. Moreover, one or more other operations may be added to or removed from the flow diagrams by those skilled in the art under the direction of the present disclosure.
In addition, the described embodiments are only some, but not all, embodiments of the application. The components of the embodiments of the present application generally described and illustrated in the figures herein may be arranged and designed in a wide variety of different configurations. Thus, the following detailed description of the embodiments of the application, as presented in the figures, is not intended to limit the scope of the application, as claimed, but is merely representative of selected embodiments of the application. All other embodiments, which can be made by a person skilled in the art without making any inventive effort, are intended to be within the scope of the present application.
It should be noted that the term "comprising" will be used in embodiments of the application to indicate the presence of the features stated hereafter, but not to exclude the addition of other features.
In the related technology of the half screen loading mode, after the cloud application first frame is decoded by a decoder, the SDK informs the APP of creating a Surface and hiding a pre-loading interface. After the APP creates a Surface, the Surface is sent to the SDK. Then, the data decoded by the decoder needs to enter Surface again for rendering through SurfaceTexture, EGL. When the EGL receives the first frame cloud application picture or the EGL sends the first frame cloud application picture to the Surface or after adding the first frame cloud application picture on the Surface, the SDK informs the APP of displaying the Surface. The first frame of the cloud application is decoded by a decoder until the SDK binds Surface to EGL, and the data decoded by the decoder directly enter Surface texture for off-screen rendering, so that a black screen appears; and after the Surface is bound to the EGL, the data decoded by the decoder can enter the Surface to be rendered through SurfaceTexture, EGL in sequence, so that the rendering time is long and the efficiency is low.
In view of this problem, the applicant considers that the data decoded by the decoder can be directly input to the Surface display, but because the configuration method needs to be executed before the decoder is started, and the configuration method needs to input the parameter Surface, the first frame cloud application picture is not decoded yet, the APP is not informed of creating the Surface, the Surface is not created yet, and the configuration method is not executed, so that the decoding is not possible.
In order to solve the above problems, referring to fig. 1, an embodiment of the present application provides a method for optimizing rendering performance of a terminal, which is applied to a terminal, and includes:
s10, after the SDK receives a first frame cloud application picture sent by a cloud server, decoding the first frame cloud application picture through a first decoder, and sending the decoded first frame cloud application picture to a surface texture for off-screen rendering, wherein the SDK is integrated in an APP installed on a terminal;
when the user starts the cloud application through the APP of the terminal, the pre-loading interface and the cloud application are started simultaneously, and at the moment, the pre-loading interface (displaying a loading progress picture, advertisement or propaganda video) is displayed for the user. After the SDK of the terminal receives the first frame cloud application picture sent by the cloud server, a first decoder is started, the first frame cloud application picture is sent to the first decoder (such as an audio and video coding and decoding tool MediaCodec) to be decoded, and after decoding is finished, the decoded first frame cloud application picture can be sent to the surface texture to be rendered off-screen. Before the first decoder is started, the SDK creates a surface texture, the surface texture is transferred to a configuration method, the first decoder is configured by executing the configuration method, and after the configuration is finished, the first decoder is started. After the SDK receives the first frame cloud application picture, decoding is carried out through a first decoder, and the decoded video frame is sent to the SurfaceTexture for off-screen rendering. It should be noted that, when the cloud application is started, the APP may create the first Surface. After the APP creates the first Surface, the first Surface can be sent to the SDK, so that the first Surface is bound to an EGL environment of the SDK, and data decoded by the first decoder sequentially enter the first Surface for rendering through SurfaceTexture, EGL, and therefore the problem of black screen in the related technology of a half-screen loading mode can be avoided.
S11, the SDK sends a first notification message to the APP so that the APP can hide the preloading interface based on the first notification message and display a first Surface, wherein the first Surface is created by the APP;
after decoding the first frame cloud application picture, the SDK sends a first notification message to the APP, after the APP receives the first notification message, the preloaded interface is hidden, the first Surface is displayed, and then the data added to the first Surface by the EGL is rendered.
S12, the SDK sends an I frame request to the cloud server and receives the I frame sent by the cloud server;
it should be noted that, after the first frame cloud application picture decodes, the SDK may request to send an I frame (I frame is also called intra picture or key frame), which is a full frame compressed encoded frame, and when decoding, the cloud server may reconstruct a complete image only with the data of the I frame, and after receiving the I frame request, the cloud server may send an I frame to the SDK. It is understood that the I-frame sent by the cloud server may not be sent based on the I-frame request, i.e., the I-frame received by the SDK may also be sent to the SDK before the cloud server receives the I-frame request.
S13, the SDK unbinds the first Surface and the EGL, and releases the first decoder, the Surface texture and the EGL;
after the SDK receives the I frame, the SDK unbound the first Surface from the EGL and releases the first decoder, the Surface texture, and the EGL.
S14, configuring a second decoder by the SDK through the first Surface, and starting the second decoder, wherein data decoded by the second decoder directly enter the first Surface for rendering.
After releasing the first decoder, the Surface texture and the EGL, the SDK may configure the second decoder with the first Surface, and start the second decoder, so that the data decoded by the second decoder directly enters the first Surface for rendering.
According to the method for optimizing the rendering performance of the terminal, after the decoding of the first frame cloud application picture is completed, the Surface is bound to the newly started decoder, so that the data decoded by the decoder can be directly input into the Surface for rendering, compared with the decoded data output by the decoder, the decoded data firstly enter the Surface text for processing, then pass through the EGL, finally enter the scheme of the Surface rendering in the half-screen loading mode, the processing flow of the decoded data is reduced, and the rendering delay of the terminal to the cloud application picture can be reduced.
Although the above embodiment of the present application solves the problem that the Surface is not created when the decoder executes the configuration method, the process of directly inputting the output of the second decoder to the first Surface rendering after receiving the I frame may have a 1 second stuck phenomenon due to the destructive reconstruction of the decoder, and the scheme shown in fig. 2 is proposed to solve the problem.
Referring to fig. 2, an embodiment of the present application provides a method for optimizing rendering performance of a terminal, which is applied to the terminal, and includes:
s20, after the SDK receives the first frame cloud application picture sent by the cloud server, decoding the first frame cloud application picture through a first decoder, and sending the decoded first frame cloud application picture to a surface texture for off-screen rendering, wherein the SDK is integrated in an APP installed on the terminal;
it should be noted that, when the cloud application is started, the APP may create a first Surface and a second Surface. After the APP creates the first Surface, the first Surface and the second Surface can be sent to the SDK, so that the first Surface is bound to an EGL environment of the SDK, and data decoded by the first decoder enter the first Surface to be rendered through SurfaceTexture, EGL.
S21, the SDK sends a first notification message to the APP so that the APP can hide the preloading interface and the second Surface based on the first notification message and display the first Surface, wherein the first Surface and the second Surface are created by the APP;
after decoding the first frame cloud application picture, the SDK sends a first notification message to the APP, after the APP receives the first notification message, the preloaded interface and the second Surface are hidden, the first Surface is displayed, and then the EGL added data on the first Surface is rendered.
S22, the SDK sends an I frame request to the cloud server and receives the I frame sent by the cloud server;
it should be noted that, after decoding the first frame cloud application picture, the SDK may request to send an I frame to the cloud server, and receive the I frame sent by the cloud server.
S23, configuring a second decoder by the SDK through a second Surface, and starting the second decoder, wherein data decoded by the second decoder directly enter the second Surface for rendering;
it should be noted that, after the SDK receives the I frame, the second decoder is configured by using the second Surface, and the second decoder is started, so that the data output by the second decoder directly enters the second Surface to render.
S24, the SDK sends a second notification message to the APP, so that the APP hides the first Surface based on the second notification message and displays the second Surface;
after the second decoder decodes the first frame of video data, the SDK may send a second notification message to the APP. After the APP receives the second notification message, the APP may conceal the first Surface based on the second notification message and display the second Surface.
S25, the SDK unbinds the first Surface and the EGL, and releases the first decoder, the Surface texture and the EGL.
After the second decoder decodes the first frame of video data, the SDK may unbind the first Surface from the EGL and release the first decoder, the Surface texture, and the EGL.
In this embodiment, each decoder binds one Surface, and since the destruction of the decoder is not involved before the second decoder decodes, the 1 second stuck phenomenon in the process of directly inputting the output of the second decoder to the second Surface rendering after the I frame is received can be avoided.
According to the method for optimizing the rendering performance of the terminal, after the decoding of the first frame cloud application picture is completed, the Surface is bound to the newly started decoder, so that the data decoded by the decoder can be directly input into the Surface for rendering, compared with the decoded data output by the decoder, the decoded data firstly enter the Surface text for processing, then pass through the EGL, finally enter the scheme of the Surface rendering in the half-screen loading mode, the processing flow of the decoded data is reduced, and the rendering delay of the terminal to the cloud application picture can be reduced.
Referring to fig. 3, an apparatus for optimizing rendering performance of a terminal according to an embodiment of the present application is applied to a terminal, and includes:
the software development kit SDK30, wherein the SDK30 is integrated in an APP installed on a terminal, and the SDK30 specifically performs the following steps:
after the SDK30 receives the first frame cloud application picture sent by the cloud server, decoding the first frame cloud application picture through a first decoder, and sending the decoded first frame cloud application picture to a surface texture for off-screen rendering;
the SDK30 sends a first notification message to the APP, so that the APP hides the preloading interface based on the first notification message and displays a first Surface, wherein the first Surface is created by the APP;
the SDK30 sends an I frame request to the cloud server and receives the I frame sent by the cloud server;
the SDK30 unbinds the first Surface from the EGL and releases the first decoder, the Surface texture, and the EGL;
the SDK30 configures the second decoder by using the first Surface, and starts the second decoder, wherein the data decoded by the second decoder directly enters the first Surface for rendering.
According to the device for optimizing the rendering performance of the terminal, after the decoding of the first frame cloud application picture is completed, the Surface is bound to the newly started decoder, so that the data decoded by the decoder can be directly input into the Surface for rendering, compared with the decoded data output by the decoder, the decoded data firstly enter the Surface text for processing, then pass through the EGL, finally enter the scheme of the Surface rendering in the half-screen loading mode, the processing flow of the decoded data is reduced, and the rendering delay of the terminal to the cloud application picture can be reduced.
Based on the foregoing apparatus embodiments, the SDK may also be configured to receive a first Surface sent by the APP.
Referring to fig. 4, an apparatus for optimizing rendering performance of a terminal according to an embodiment of the present application is applied to a terminal, and includes:
the software development kit SDK40, wherein the SDK40 is integrated in an APP installed on a terminal, and the SDK40 specifically performs the following steps:
after the SDK40 receives the first frame cloud application picture sent by the cloud server, decoding the first frame cloud application picture through a first decoder, and sending the decoded first frame cloud application picture to the surface texture for off-screen rendering;
the SDK40 sends a first notification message to the APP so that the APP hides the preloaded interface and the second Surface based on the first notification message and displays the first Surface, wherein the first Surface and the second Surface are created by the APP;
the SDK40 sends an I frame request to the cloud server and receives the I frame sent by the cloud server;
the SDK40 configures a second decoder by using a second Surface, and starts the second decoder, wherein data decoded by the second decoder directly enter the second Surface for rendering;
the SDK40 sends a second notification message to the APP, so that the APP hides the first Surface based on the second notification message and displays the second Surface;
the SDK40 unbound the first Surface from the EGL and releases the first decoder, surface texture, and EGL.
According to the device for optimizing the rendering performance of the terminal, after the decoding of the first frame cloud application picture is completed, the Surface is bound to the newly started decoder, so that the data decoded by the decoder can be directly input into the Surface for rendering, compared with the decoded data output by the decoder, the decoded data firstly enter the Surface text for processing, then pass through the EGL, finally enter the scheme of the Surface rendering in the half-screen loading mode, the processing flow of the decoded data is reduced, and the rendering delay of the terminal to the cloud application picture can be reduced.
Based on the foregoing apparatus embodiments, the SDK may also be configured to receive a first Surface and a second Surface sent by the APP.
The implementation process of the device for optimizing the rendering performance of the terminal provided by the embodiment of the application is consistent with the method for optimizing the rendering performance of the terminal provided by the embodiment of the application, and the effect achieved by the device is the same as the method for optimizing the rendering performance of the terminal provided by the embodiment of the application, and is not repeated here.
As shown in fig. 5, an electronic device provided in an embodiment of the present application includes: a processor 50, a memory 51 and a bus 52, said memory 51 storing machine readable instructions executable by said processor 50, said processor 50 and said memory 51 communicating via the bus 52 when the electronic device is running, said processor 50 executing said machine readable instructions to perform the steps of the method of optimizing rendering performance of a terminal as described above.
Specifically, the above-mentioned memory 51 and the processor 50 can be general-purpose memories and processors, and are not particularly limited herein, and the above-mentioned method of optimizing rendering performance of the terminal can be performed when the processor 50 runs a computer program stored in the memory 51.
Corresponding to the above method for optimizing rendering performance of a terminal, the embodiment of the application further provides a computer readable storage medium, on which a computer program is stored, which when being executed by a processor, performs the steps of the above method for optimizing rendering performance of a terminal.
It will be clear to those skilled in the art that, for convenience and brevity of description, specific working procedures of the above-described system and apparatus may refer to corresponding procedures in the method embodiments, and are not repeated in the present disclosure. In the several embodiments provided by the present application, it should be understood that the disclosed systems, devices, and methods may be implemented in other manners. The above-described apparatus embodiments are merely illustrative, and the division of the modules is merely a logical function division, and there may be additional divisions when actually implemented, and for example, multiple modules or components may be combined or integrated into another system, or some features may be omitted or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be through some communication interface, indirect coupling or communication connection of devices or modules, electrical, mechanical, or other form.
The modules described as separate components may or may not be physically separate, and components shown as modules may or may not be physical units, may be located in one place, or may be distributed over multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
In addition, each functional unit in the embodiments of the present application may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit.
The functions, if implemented in the form of software functional units and sold or used as a stand-alone product, may be stored in a non-volatile computer readable storage medium executable by a processor. Based on this understanding, the technical solution of the present application may be embodied essentially or in a part contributing to the prior art or in a part of the technical solution, in the form of a software product stored in a storage medium, comprising several instructions for causing a computer device (which may be a personal computer, a server, a network device, etc.) to perform all or part of the steps of the method according to the embodiments of the present application. And the aforementioned storage medium includes: a usb disk, a removable hard disk, a ROM, a RAM, a magnetic disk, or an optical disk, etc.
The foregoing is merely illustrative of the present application, and the present application is not limited thereto, and any person skilled in the art will readily appreciate variations or alternatives within the scope of the present application. Therefore, the protection scope of the application is subject to the protection scope of the claims.
Claims (10)
1. A method for optimizing rendering performance of a terminal, applied to the terminal, comprising:
after receiving a first frame cloud application picture sent by a cloud server, a Software Development Kit (SDK) decodes the first frame cloud application picture through a first decoder, and sends the decoded first frame cloud application picture to a surface texture object SurfaceTexture for off-screen rendering, wherein the SDK is integrated in an Application (APP) installed on a terminal;
the SDK sends a first notification message to the APP so that the APP hides the preloading interface based on the first notification message and displays a first Surface object, wherein the first Surface object is recorded as a first Surface, and the first Surface is created by the APP;
the SDK sends an I frame request to the cloud server and receives the I frame sent by the cloud server;
the SDK unbinds the first Surface and the graphic library EGL, and releases a first decoder, surface texture and EGL;
the SDK configures the second decoder by using the first Surface, and starts the second decoder, wherein the data decoded by the second decoder directly enter the first Surface for rendering.
2. The method as recited in claim 1, further comprising:
the SDK receives a first Surface sent by the APP.
3. A method for optimizing rendering performance of a terminal, applied to the terminal, comprising:
after receiving a first frame cloud application picture sent by a cloud server, a Software Development Kit (SDK) decodes the first frame cloud application picture through a first decoder, and sends the decoded first frame cloud application picture to a surface texture object SurfaceTexture for off-screen rendering, wherein the SDK is integrated in an Application (APP) installed on a terminal;
the SDK sends a first notification message to the APP so that the APP can hide the preloaded interface and the second Surface object based on the first notification message and display the first Surface object, wherein the first Surface object is marked as a first Surface, the second Surface object is marked as a second Surface, and the first Surface and the second Surface are created by the APP;
the SDK sends an I frame request to the cloud server and receives the I frame sent by the cloud server;
the SDK configures a second decoder by using a second Surface, and starts the second decoder, wherein data decoded by the second decoder directly enter the second Surface for rendering;
the SDK sends a second notification message to the APP so that the APP can hide the first Surface based on the second notification message and display the second Surface;
the SDK unbound the first Surface from the graphics library EGL and releases the first decoder, surface texture, and EGL.
4. A method as recited in claim 3, further comprising:
the SDK receives a first Surface and a second Surface sent by the APP.
5. An apparatus for optimizing rendering performance of a terminal, applied to the terminal, is characterized by comprising:
the Software Development Kit (SDK), wherein the SDK is integrated in an Application (APP) installed on a terminal, and the SDK specifically executes the following steps:
after the SDK receives the first frame cloud application picture sent by the cloud server, decoding the first frame cloud application picture through a first decoder, and sending the decoded first frame cloud application picture to a surface texture object surface texture for off-screen rendering;
the SDK sends a first notification message to the APP so that the APP hides the preloading interface based on the first notification message and displays a first Surface object, wherein the first Surface object is recorded as a first Surface, and the first Surface is created by the APP;
the SDK sends an I frame request to the cloud server and receives the I frame sent by the cloud server;
the SDK unbinds the first Surface and the graphic library EGL, and releases a first decoder, surface texture and EGL;
the SDK configures the second decoder by using the first Surface, and starts the second decoder, wherein the data decoded by the second decoder directly enter the first Surface for rendering.
6. An apparatus as recited in claim 5, wherein the SDK is further configured to receive a first Surface sent by the APP.
7. An apparatus for optimizing rendering performance of a terminal, applied to the terminal, is characterized by comprising:
the Software Development Kit (SDK), wherein the SDK is integrated in an Application (APP) installed on a terminal, and the SDK specifically executes the following steps:
after the SDK receives the first frame cloud application picture sent by the cloud server, decoding the first frame cloud application picture through a first decoder, and sending the decoded first frame cloud application picture to a surface texture object surface texture for off-screen rendering;
the SDK sends a first notification message to the APP so that the APP can hide the preloaded interface and the second Surface object based on the first notification message and display the first Surface object, wherein the first Surface object is marked as a first Surface, the second Surface object is marked as a second Surface, and the first Surface and the second Surface are created by the APP;
the SDK sends an I frame request to the cloud server and receives the I frame sent by the cloud server;
the SDK configures a second decoder by using a second Surface, and starts the second decoder, wherein data decoded by the second decoder directly enter the second Surface for rendering;
the SDK sends a second notification message to the APP so that the APP can hide the first Surface based on the second notification message and display the second Surface;
the SDK unbound the first Surface from the graphics library EGL and releases the first decoder, surface texture, and EGL.
8. An apparatus as recited in claim 7, wherein the SDK is further configured to receive a first Surface and a second Surface transmitted by the APP.
9. A computer-readable storage medium, characterized in that it has stored thereon a computer program which, when executed by a processor, performs the steps of the method of optimizing rendering performance of a terminal according to any of claims 1 to 4.
10. An electronic device, comprising: a processor, a storage medium and a bus, the storage medium storing machine-readable instructions executable by the processor, the processor and the storage medium communicating over the bus when the electronic device is running, the processor executing the machine-readable instructions to perform the steps of the method of optimizing rendering performance of a terminal as claimed in any one of claims 1 to 4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311287713.6A CN117041668B (en) | 2023-10-08 | 2023-10-08 | Method and device for optimizing rendering performance of terminal |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311287713.6A CN117041668B (en) | 2023-10-08 | 2023-10-08 | Method and device for optimizing rendering performance of terminal |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN117041668A true CN117041668A (en) | 2023-11-10 |
| CN117041668B CN117041668B (en) | 2023-12-08 |
Family
ID=88630297
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311287713.6A Active CN117041668B (en) | 2023-10-08 | 2023-10-08 | Method and device for optimizing rendering performance of terminal |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117041668B (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109922360A (en) * | 2019-03-07 | 2019-06-21 | 腾讯科技（深圳）有限公司 | Method for processing video frequency, device and storage medium |
| CN110147512A (en) * | 2019-05-16 | 2019-08-20 | 腾讯科技（深圳）有限公司 | Player preloading, operation method, device, equipment and medium |
| CN110401866A (en) * | 2018-04-25 | 2019-11-01 | 广州虎牙信息科技有限公司 | Display methods, device, terminal and the storage medium of live video |
| CN110620954A (en) * | 2018-06-20 | 2019-12-27 | 北京优酷科技有限公司 | Video processing method and device for hard solution |
| CN116672704A (en) * | 2022-02-28 | 2023-09-01 | 荣耀终端有限公司 | Image processing method, electronic equipment and storage medium |
-
2023
- 2023-10-08 CN CN202311287713.6A patent/CN117041668B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110401866A (en) * | 2018-04-25 | 2019-11-01 | 广州虎牙信息科技有限公司 | Display methods, device, terminal and the storage medium of live video |
| CN110620954A (en) * | 2018-06-20 | 2019-12-27 | 北京优酷科技有限公司 | Video processing method and device for hard solution |
| CN109922360A (en) * | 2019-03-07 | 2019-06-21 | 腾讯科技（深圳）有限公司 | Method for processing video frequency, device and storage medium |
| CN110147512A (en) * | 2019-05-16 | 2019-08-20 | 腾讯科技（深圳）有限公司 | Player preloading, operation method, device, equipment and medium |
| CN116672704A (en) * | 2022-02-28 | 2023-09-01 | 荣耀终端有限公司 | Image processing method, electronic equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN117041668B (en) | 2023-12-08 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| WO2021008373A1 (en) | Display method and apparatus, electronic device, and computer-readable medium | |
| CN109600666B (en) | Video playing method, device, medium and electronic equipment in game scene | |
| CN110166810B (en) | Video rendering engine switching method, device and equipment and readable storage medium | |
| CN109327727B (en) | Live stream processing method in WebRTC and stream pushing client | |
| CN109922360B (en) | Video processing method, device and storage medium | |
| CN107995482B (en) | Video file processing method and device | |
| CN105426191A (en) | User interface display processing method and device | |
| CN112843676B (en) | Data processing method, device, terminal, server and storage medium | |
| CN110620954B (en) | Video processing method, device and storage medium for hard solution | |
| CN112218148A (en) | Screen recording method and device, computer equipment and computer readable storage medium | |
| CN108733356B (en) | Texture sharing method, computer equipment and storage medium | |
| CN112929740A (en) | Method, device, storage medium and equipment for rendering video stream | |
| CN113473226B (en) | Method and device for improving video rendering efficiency, computer equipment and storage medium | |
| CN117041668B (en) | Method and device for optimizing rendering performance of terminal | |
| CN111355978A (en) | Video file processing method and device, mobile terminal and storage medium | |
| CN111355960B (en) | Method and device for synthesizing video file, mobile terminal and storage medium | |
| CN112565869A (en) | Window fusion method, device and equipment for video redirection | |
| CN113676748A (en) | Cloud game screen capture interaction method and system | |
| CN109302617B (en) | Multi-element-designated video microphone connecting method, device, equipment and storage medium | |
| CN107707930B (en) | Video processing method, device and system | |
| CN113411661A (en) | Method, apparatus, device, storage medium and program product for recording information | |
| CN111866508A (en) | Video processing method, device, medium and electronic equipment | |
| JP7278850B2 (en) | System and method for overlaying multi-source media in video random access memory | |
| CN112911329B (en) | Window live broadcast method, device, electronic equipment and computer readable storage medium | |
| CN116886974B (en) | Method and device for optimizing decoding rendering performance of terminal |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20240130 Address after: 230031 Room 672, 6/F, Building A3A4, Zhong'an Chuanggu Science Park, No. 900, Wangjiang West Road, High-tech Zone, Hefei, Anhui Patentee after: Anhui Haima Cloud Technology Co.,Ltd. Country or region after: China Address before: 301700 room 2d25, Building 29, No.89 Heyuan Road, Jingjin science and Technology Valley Industrial Park, Wuqing District, Tianjin Patentee before: HAIMAYUN (TIANJIN) INFORMATION TECHNOLOGY CO.,LTD. Country or region before: China |
|
| TR01 | Transfer of patent right |