CN112653787A - Display method, display device, storage medium and electronic equipment - Google Patents
Display method, display device, storage medium and electronic equipment Download PDFInfo
- Publication number
- CN112653787A CN112653787A CN202011488205.0A CN202011488205A CN112653787A CN 112653787 A CN112653787 A CN 112653787A CN 202011488205 A CN202011488205 A CN 202011488205A CN 112653787 A CN112653787 A CN 112653787A
- Authority
- CN
- China
- Prior art keywords
- cloud
- equipment
- user
- target image
- display
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 60
- 238000003860 storage Methods 0.000 title claims abstract description 19
- 230000015572 biosynthetic process Effects 0.000 claims abstract description 45
- 238000003786 synthesis reaction Methods 0.000 claims abstract description 45
- 238000012545 processing Methods 0.000 claims abstract description 33
- 230000002194 synthesizing effect Effects 0.000 claims abstract description 17
- 239000000203 mixture Substances 0.000 claims description 22
- 238000009877 rendering Methods 0.000 claims description 19
- 238000004891 communication Methods 0.000 claims description 15
- 238000004590 computer program Methods 0.000 claims description 7
- 230000004044 response Effects 0.000 claims description 4
- 238000004140 cleaning Methods 0.000 claims description 3
- 230000008569 process Effects 0.000 abstract description 17
- 230000003993 interaction Effects 0.000 abstract description 15
- 230000001360 synchronised effect Effects 0.000 abstract description 5
- 230000000875 corresponding effect Effects 0.000 description 25
- 238000010586 diagram Methods 0.000 description 14
- 230000006870 function Effects 0.000 description 12
- 230000005540 biological transmission Effects 0.000 description 5
- 230000000694 effects Effects 0.000 description 4
- 238000012795 verification Methods 0.000 description 4
- DWDGSKGGUZPXMQ-UHFFFAOYSA-N OPPO Chemical compound OPPO DWDGSKGGUZPXMQ-UHFFFAOYSA-N 0.000 description 3
- 238000007726 management method Methods 0.000 description 3
- 239000002131 composite material Substances 0.000 description 2
- 238000013500 data storage Methods 0.000 description 2
- 239000000835 fiber Substances 0.000 description 2
- 238000010606 normalization Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 230000006835 compression Effects 0.000 description 1
- 238000007906 compression Methods 0.000 description 1
- 238000012790 confirmation Methods 0.000 description 1
- 239000000470 constituent Substances 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000002452 interceptive effect Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000004091 panning Methods 0.000 description 1
- 238000003825 pressing Methods 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/08—Network architectures or network communication protocols for network security for authentication of entities
- H04L63/083—Network architectures or network communication protocols for network security for authentication of entities using passwords
Landscapes
- Engineering & Computer Science (AREA)
- Computer Hardware Design (AREA)
- Computer Security & Cryptography (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Information Transfer Between Computers (AREA)
Abstract
The application discloses a display method, a display device, a storage medium and an electronic device. Receiving a connection request sent by user end equipment, wherein the connection request carries display parameter information of the user end equipment; generating a synthesis strategy based on the display parameter information; responding to the connection request, and establishing connection with the user terminal equipment; under the condition of successful connection with the user terminal equipment, receiving a target event sent by the user terminal equipment, and generating a target image based on the target event and a synthesis strategy; and sending the target image to the user terminal equipment so as to enable the user terminal equipment to display the target image. The cloud device can generate a synthesis strategy according to the display parameter information of the user terminal device, send the synthesized target image to the user terminal device and directly display the target image by the user terminal device, so that the user terminal device is prevented from processing and synthesizing the target image again, time consumption and delay of an interaction process are optimized, and more synchronous interaction experience is brought to a user.
Description
Technical Field
The application relates to the technical field of cloud interaction, in particular to a display method, a display device, a storage medium and electronic equipment.
Background
The cloud mobile phone can be understood as a mobile phone which applies a cloud computing technology to network terminal services and realizes the cloud services through a server, and the cloud mobile phone can interact with a user terminal mobile phone to realize functions of data transmission, storage, backup and the like.
When the cloud mobile phone interacts with the user terminal mobile phone, the cloud mobile phone intercepts the screen image, performs drawing, rendering and synthesizing processing operations on the image, and then sends the image to the user terminal mobile phone. However, the user-side mobile phone still needs to process the image to display the image on the screen after receiving the image, and this method easily causes display delay between the cloud mobile phone and the user-side mobile phone, thereby affecting the user's interactive experience.
Disclosure of Invention
The embodiment of the application provides a display method and device, a storage medium and electronic equipment, and when interaction between cloud equipment and user equipment is established, the user equipment can directly display an image sent by the cloud equipment, so that display delay between the user equipment and the cloud equipment is avoided, the effect of display normalization is achieved, and the interaction experience of a user is improved.
In a first aspect, an embodiment of the present application provides a display method, which is applied to a cloud device, and includes:
receiving a connection request sent by user equipment, wherein the connection request carries display parameter information of the user equipment;
generating a synthesis strategy based on the display parameter information;
responding to the connection request, and establishing connection with the user terminal equipment;
under the condition of successful connection with the user terminal equipment, receiving a target event sent by the user terminal equipment, and generating a target image based on the target event and a synthesis strategy;
and sending the target image to the user terminal equipment so as to enable the user terminal equipment to display the target image.
In the embodiment of the application, the cloud device can generate the synthesis strategy according to the display parameter information of the user end device, and under the condition that the connection with the user end device is successful, the synthesized target image can be sent to the user end device and directly displayed by the user end device, so that the user end device is prevented from processing and synthesizing the target image again, the time consumption and delay of the display interaction process are optimized, and more synchronous interaction experience is brought to a user.
In a second aspect, an embodiment of the present application further provides a display method, where the method is applied to a user end device, and includes:
sending a connection request to the cloud end equipment, wherein the connection request carries display parameter information of the user end equipment; the display parameter information is used for enabling the cloud equipment to generate a synthesis strategy;
under the condition of successful connection with the cloud equipment, sending a target event to the cloud equipment so that the cloud equipment generates a target image based on the target event and a synthesis strategy;
and receiving a target image sent by the cloud equipment and displaying the target image.
In the embodiment of the application, the client device can receive and directly display the target image sent by the cloud device under the condition that the connection with the cloud device is successful, so that the time consumption and delay of the display interaction process are optimized, and more synchronous interaction experience is brought to a user.
In a third aspect, an embodiment of the present application provides a display device, where the display device is applied to a cloud device, and the display device includes:
the first receiving module is used for receiving a connection request sent by user end equipment, wherein the connection request carries display parameter information of the user end equipment;
the synthesis module is used for generating a synthesis strategy based on the display parameter information;
the first processing module is used for responding to the connection request and establishing connection with the user side equipment;
the second processing module is used for receiving a target event sent by the user end equipment under the condition of successful connection with the user end equipment and generating a target image based on the target event and a synthesis strategy;
the first sending module is used for sending the target image to the user terminal equipment so as to enable the user terminal equipment to display the target image.
In a fourth aspect, an embodiment of the present application further provides a display device, where the display device is applied to a user end device, and the display device includes:
the second sending module is used for sending a connection request to the cloud end equipment, wherein the connection request carries the display parameter information of the user end equipment; the display parameter information is used for enabling the cloud equipment to generate a synthesis strategy;
the third sending module is used for sending the target event to the cloud equipment under the condition that the connection with the cloud equipment is successful, so that the cloud equipment generates a target image based on the target event and the synthesis strategy;
and the second receiving module is used for receiving the target image sent by the cloud equipment and displaying the target image.
In a fifth aspect, the present application provides a computer-readable storage medium, where a computer program is stored, where the computer program includes program instructions, and when the program instructions are executed by a processor, the display method provided in any one of the first aspect or the second aspect of the present application may be implemented.
In a sixth aspect, an embodiment of the present application provides an electronic device, including a processor, a memory, and a communication interface; the processor is connected with the memory and the communication interface; a memory for storing executable program code; the processor runs a program corresponding to the executable program code by reading the executable program code stored in the memory, so as to execute the display method provided by any one of the first aspect, the first implementation manner of the first aspect, or any one of the second aspect, or the second implementation manner of the second aspect of the embodiments of the present application.
In a seventh aspect, an embodiment of the present application provides a computer program product, which, when run on a display apparatus, causes the display apparatus to execute the display method provided in the first aspect or any implementation manner of the second aspect of the present application embodiment.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings without creative efforts.
Fig. 1 is a schematic diagram of a display system according to an embodiment of the present disclosure;
fig. 2 is a schematic flowchart of a display method according to an embodiment of the present disclosure;
fig. 3 is a schematic diagram illustrating a display login interface according to an embodiment of the present application;
FIG. 4 is a schematic diagram of another display login interface provided in an embodiment of the present application;
fig. 5 is a schematic flowchart of another display method provided in the embodiment of the present application;
fig. 6 is a schematic structural diagram of a display device according to an embodiment of the present disclosure;
fig. 7 is a schematic structural diagram of another display device provided in the embodiment of the present application;
fig. 8 is a schematic structural diagram of an electronic device according to an embodiment of the present disclosure;
fig. 9 is a schematic structural diagram of another electronic device according to an embodiment of the present application.
Detailed Description
The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application.
The terms "first," "second," "third," and the like in the description and claims of this application and in the above-described drawings are used for distinguishing between different objects and not for describing a particular order. Furthermore, the terms "include" and "have," as well as any variations thereof, are intended to cover non-exclusive inclusions. For example, a process, method, system, article, or apparatus that comprises a list of steps or elements is not limited to only those steps or elements listed, but may alternatively include other steps or elements not listed, or inherent to such process, method, article, or apparatus.
Referring to fig. 1, fig. 1 is a schematic diagram illustrating a display system architecture according to an embodiment of the present disclosure.
As shown in fig. 1, the display system may include a client device and a server 20.
The client devices may be one or more client devices, wherein the plurality of client devices may be client device 10a, client device 10b. The user end equipment can be provided with a setting interface for logging in a user account, a user can log in the account by inputting the account and a corresponding password, and the specific user account can be not limited to an OPPO account, a Huawei account or an apple account and the like. The user account can be registered and logged in according to the mobile phone number of the user end device currently used by the user or a mailbox input by the user, and possibly, when the user registers and logs in through the mobile phone number of the user end device, the user end device can obtain the verification code and input the verification code to the corresponding position to log in the user account. Possibly, when the user logs in through the input mailbox, the user mailbox and the corresponding determined login password can be input in the user terminal equipment to log in the user account. The user side equipment is provided with an application program used for establishing connection with the cloud side equipment, and a user can input an account number and a password of the user side equipment login in the application program to establish connection with the cloud side equipment after purchasing the corresponding cloud side equipment according to the account number of the user side equipment login, so that the display synchronization of the cloud side equipment and the user side equipment can be realized. Specifically, the user end device may establish a data connection with the server 20 logged in with the cloud end device corresponding to the user account of the user end device, for example, the user end device sends instructions such as a backup file, a management file, or an operation file to the server 20 through a network.
The client device in the embodiment of the present application may be a mobile phone, a tablet Computer, a desktop Computer, a laptop Computer, a notebook Computer, an Ultra-mobile Personal Computer (UMPC), a handheld Computer, a netbook, a Personal Digital Assistant (PDA), or the like.
The server 20 may store cloud devices corresponding to user accounts logged in by the user end devices, and may store a plurality of cloud devices corresponding to user accounts logged in by the plurality of user end devices. Specifically, when the cloud device of the server 20 is successfully connected to the user account of the corresponding user end device, a data connection may be established with the user account of the user end device, for example, a command such as a backup file, a management file, or an operation file sent by the user end device through a network may be received. The cloud device may not be limited to a cloud cell phone.
The server 20 may be a server capable of providing multiple services, and may implement data interaction between the client device and the cloud device. For example, a request for establishing a connection with the cloud end device sent by the user end device is received through the network, or under the condition that the connection between the cloud end device and a user account of the corresponding user end device is successful, a command such as a backup file, a management file or an operation file sent by the user end device is received through the network.
The server 20 may be, but is not limited to, a hardware server, a virtual server, a cloud server, and the like.
It should be noted that the performance of the server 20 provided in the embodiment of the present application may be better than that of the client device 10, so that the user can complete instructions that are difficult or impossible to execute by the client device on the cloud device.
The network may be a medium providing a communication link between the user end device and the server 20, or may be the internet including a network device and a transmission medium, which is not limited thereto. The transmission medium may be a wired link (such as, but not limited to, coaxial cable, fiber optic cable, and Digital Subscriber Line (DSL), etc.) or a wireless link (such as, but not limited to, wireless fidelity (WIFI), bluetooth, and mobile device network, etc.).
It is understood that the number of client devices and servers 20 in the display system shown in fig. 1 is by way of example only, and that the display system may include any number of client devices and servers 20 in a particular implementation. The embodiment of the present application is not limited to this, and for example, but not limited to, the server 20 may be a server cluster formed by a plurality of servers.
Referring to fig. 2, fig. 2 is a schematic flow chart illustrating a display method according to an embodiment of the present application, and details in the embodiment of the present application are described by taking the above-mentioned client device and the cloud device as examples.
As shown in fig. 2, the display mode includes:
Specifically, the user may log in an existing user account on the user end device, or register the mobile phone number of the current user end device as the user account, so as to complete user account login on the current user end device. For example, refer to a schematic diagram of a display login interface provided in the embodiment of the present application illustrated in fig. 3. As shown in fig. 3, taking a user end device as an OPPO mobile phone as an example, a user selects and enters an account login interface in the setup of the OPPO mobile phone, and a control prompting the user to select a login method and an account login interface corresponding to the login method are displayed on the account login interface. Possibly, when the user selects the short message verification code to log in, the registered mobile phone account and the verification code received by the corresponding mobile phone account can be input below to log in, if the user does not have an account, the registered account can be selected by the user, and the mobile phone number of the current user end device is preferably used as the account to be registered. Possibly, when the user selects the password to log in, the registered mobile phone account or mailbox and the corresponding password can be input below to log in, and if the user forgets the password, the user can select to forget the password control and reset the password of the current account.
Further, after the user logs in the account at the user end device, the user may send a connection request to the cloud end device, for example, the user may download an application program for connecting the cloud end device at the user end device, enter the application program to input a user account and a corresponding account password, send the user account to the cloud end device, and match the user account by the cloud end device. Possibly, the cloud device cannot match the user account, the user account is represented and the cloud device is not purchased, and prompt information can be sent to the user end device by the cloud device. Specifically, referring to fig. 4, a schematic diagram of another display login interface provided in the embodiment of the present application is shown, as shown in fig. 4, a pop-up is displayed on an application interface of a client device, "does not detect a current account number, ask for a purchase? The user can select the confirmation control to jump to the purchase interface and purchase the cloud device corresponding to the current account. After the user purchases successfully, the user can return to the login interface of the application program to log in the account of the user again, so that connection with the cloud device is established.
Possibly, when the cloud device is successfully matched with the user account, the user account is represented to buy the cloud device, and prompt information for representing successful connection can be sent to the user side device to prompt that the current cloud device is available to the user. After receiving prompt information used for representing successful connection and sent by the cloud equipment, the user side equipment can automatically switch to a display mode matched with the cloud equipment, so that the display normalization effect of the user side equipment and the cloud equipment is conveniently realized. The client device may pre-store a processing policy for switching to matching with the cloud device, where the processing policy may include a display mode for performing drawing, rendering, and synthesizing operations on video frames sent by the cloud device and for displaying a synthesized image, for example, when the client device receives a segment of video frames sent by the cloud device, the client device performs drawing, rendering, and synthesizing operations on each received video frame and displays each synthesized video frame to a user through the display mode. In this embodiment of the application, the processing policy matched with the cloud device may not be limited to be stored in the user end device, but may also be stored in the server matched with the user end device, and the user end device may send a processing policy invoking instruction to the server matched with the user end device after being successfully connected with the cloud device.
It should be noted that, when the user end device provided in the embodiment of the present application logs in the user account on the application program used for connecting with the cloud end device, the user end device may not be limited to password login, and may also log in through a short message authentication code or other manners.
The connection request sent by the user end device carries display parameter information of the user end device, the cloud end device analyzes the connection request sent by the user end device to obtain the connection request corresponding to the user account and the display parameter information of the user end device corresponding to the login user account, and the display parameter information can include screen resolution, screen refresh rate and the like of the user end device.
Specifically, the cloud device may generate a composition policy for processing the display image according to the display parameter information after acquiring the display parameter information of the user end device, where the composition policy may include drawing, rendering, and composition operations on the image and a display mode for displaying the composition image. For example, the cloud device may perform drawing and rendering operations on an image according to the screen resolution in the display parameter information of the user end device, and determine a display mode for displaying a composite image according to the screen refresh rate in the display parameter information, and a specific implementation manner of the display mode may be consistent with a manner of generating a composite policy by the existing cloud device, which is not described herein again. According to the embodiment of the application, the synthesis strategy generated by the cloud end equipment through the display parameter information of the user end equipment is matched with the processing strategy stored by the user end equipment when the user end equipment is successfully connected with the cloud end equipment, and the image can be directly displayed by the user end equipment after the synthesized image is sent to the user end equipment by the cloud end equipment, so that repeated image processing operation of the user end equipment is avoided, and the synchronous display effect of the cloud end equipment and the user end equipment is further improved.
It should be noted that, the conventional cloud device outputs a video frame in a manner of capturing a screen image through a cloud (which needs to be subjected to surface warping synthesis), and in the embodiment of the present application, in order to better reduce the time for synthesizing an image, the cloud device may obtain video frame data in a frame buffer readable state through epoll in a screenshot thread, and when new video frame data is detected, copy the new video frame data to a preset video frame pile, and push the preset video frame pile into a video frame queue of a compression thread. The code pressing thread encodes the video frame data through the encoder to generate video data to be transmitted, sends the video data to be transmitted to the client device, and returns the preset video frame piles to the video frame queue, so that double-thread data transmission can be realized, and the data transmission efficiency is improved.
Specifically, after receiving a connection request sent by the user end device, the cloud end device searches and matches a user account in the connection request. The user accounts logged in by the user end devices correspond to the cloud end devices, and after the connection request sent by the user end devices is received and analyzed, the analyzed user accounts can be matched with the user accounts corresponding to the cloud end devices. If the user account number which is consistent with the user account numbers in the plurality of cloud end devices is matched, the fact that the currently received user account number of the user end device purchases the cloud end device is represented, and the user end device can be successfully connected with the user end device. If the user account number which is consistent with the user account numbers in the plurality of cloud end devices cannot be matched, the user account number which represents the currently received user end device does not buy the cloud end device, and purchase request information can be sent to the user end device.
And step 204, the user side equipment sends the target event to the cloud side equipment.
Specifically, under the condition that the client device and the cloud device are successfully connected, the user can execute the target event on the client device, and the client device sends the target event to the cloud device. For example, a user clicks a target application program in a display mode in which the client device is matched with the cloud device, the client device receives a target application program opening instruction of the user and sends the instruction to the cloud device, and the cloud device opens the target application program and synchronously displays an execution interface of the target application program to a display interface of the client device. The application program in the display mode in which the client device and the cloud device are matched may be a default application program of the system (for example, but not limited to, a business hall, an application mall, and the like), and may also include a system-bound application program (for example, but not limited to, a microblog, a browser, and the like).
The target application mentioned in the embodiment of the present application may be an application that is mainstream in the application market, such as, but not limited to, royal glory, love art video, panning, and the like.
Specifically, after receiving a target event of a user, the cloud device may open a corresponding application program according to the target event and generate a target image according to a synthesis strategy. For example, taking the target event as the royal player glory of the user, the user finds the application program glory of the royal and clicks the application program in the display mode that the user end device is matched with the cloud end device, and the user end device receives the target instruction of the user clicking the application program glory of the royal and sends the target instruction to the cloud end device. It should be noted that the user needs to determine whether the target application exists before the client device clicks on the target application. And after receiving a target instruction for clicking the Royal application program, the cloud equipment executes a corresponding action, namely starting the Royal application program. The cloud device performs screenshot on each frame of display content to obtain an initial image in the process of starting a royal glory application program, and performs drawing, rendering and synthesizing operations on the initial image in sequence to generate a target image which can be displayed on the user side device.
It should be noted that the target event executed by the user on the client device may not be limited to opening the target application program, but may also be other execution instructions. For example, after a user opens a royal glory application program, a selection mode and related game operations need to be performed, the user end device receives a selection instruction or a game operation instruction of the user and then sends the selection instruction or the game operation instruction to the cloud end device, the cloud end device generates a display interface corresponding to the user selection instruction or the game operation instruction, the cloud end device performs screenshot according to the display interface of the selection instruction or the game operation instruction, and the cloud end device performs drawing, rendering and synthesizing operations on each frame of image in sequence to generate a target image which can be displayed on the user end device.
And step 206, the cloud device sends the target image to the user terminal device.
Specifically, after the cloud device generates the target image, the cloud device may send the target image to the user end device. Taking a target event as an example of playing a section of video, the cloud device plays the video after receiving a playing instruction corresponding to the playing video, captures each video frame of the video, performs drawing, rendering and synthesizing operations to generate a new video frame, and sends each new video frame to the user end device to be synchronously displayed.
Specifically, after receiving the target image sent by the cloud device, the user end device analyzes the target image and directly displays the target image to the user, so that the user end device does not need to perform drawing, rendering and synthesizing operations on the analyzed image again, time consumption and delay of an interaction process are optimized, and more synchronous interaction experience is brought to the user.
As an embodiment provided by the present application, after the cloud device and the client device are successfully connected, the composition policy generated according to the display parameter information sent by the client device may be stored in the cloud device, and a corresponding relationship is generated with the user account, so that when the cloud device is connected with the client device next time, the composition policy corresponding to the display parameter information is automatically invoked by obtaining the display parameter information sent by the client device, and the connection time between the cloud device and the client device may be optimized.
Referring to fig. 5, fig. 5 is a schematic flow chart illustrating another display method according to an embodiment of the present disclosure.
As shown in fig. 5, the display method includes:
Specifically, step 501 is identical to step 201, and is not described herein again.
Specifically, step 502 is identical to step 202, and is not described herein again.
Specifically, step 503 is identical to step 203, and is not described herein again.
Specifically, step 504 is identical to step 204, and is not described here again.
And 505, the cloud device generates a target image based on the target event and the synthesis strategy.
Specifically, step 505 is identical to step 205, and is not described herein again.
Specifically, step 506 is identical to step 206, and is not described herein again.
Specifically, step 507 is the same as step 207, and is not described herein again.
Specifically, the client device may receive an exit instruction sent by the user to exit the processing policy and the display mode currently matched with the cloud device. Specifically, the exit instruction issued by the user may be a gesture instruction of the user, for example, the user slides a hand from one side of the screen to the other side of the screen facing the screen of the user end device, and after receiving the sliding instruction of the user on the screen, the user end device may automatically pop up a prompt dialog box for prompting the user whether to exit the current mode. Specifically, the exit instruction sent by the user may also be a control instruction acted on the user end device by the user, for example, the user may click a return button disposed on a screen of the user end device or press a power button disposed on a side of the user end device, and the same user end device may automatically pop up a prompt dialog box for prompting the user whether to exit the current mode after receiving the instruction that the user clicks the return button or presses the power button. The exit instruction sent by the user in the embodiment of the present application may be pre-stored in a processing policy matched between the user end device and the cloud end device, and is not limited to the type of the exit instruction sent by the user.
In step 509, the client device sends an exit request to the cloud device.
Specifically, after receiving an exit instruction sent by the user, the user end device may further send an exit request to the cloud end device to prompt the cloud end device to exit the user account of the current user end device.
In step 510, the cloud device responds to the quit request and disconnects from the user end device.
Specifically, after receiving the quit request, the cloud device may automatically disconnect from the user end device, and wait to receive a connection request sent again by the user end device or receive connection requests sent by other user end devices.
Specifically, after the cloud device is disconnected from the user end device, the cloud device can log out the account information of the user end device and clear the corresponding generated synthesis strategy, so that the storage space of the cloud device can be reduced, and under the condition that different display parameter information is carried in a connection request sent by the user end device to the cloud device, the cloud device can conveniently regenerate the synthesis strategy based on the different display parameter information, so that the time consumption and delay of the display interaction process of the cloud device and the user end device are optimized.
As another embodiment provided by the present application, under the condition that the performance parameters of the client device are good, the composition strategy generated by the cloud device based on the display parameter information sent by the client device may not include the image composition operation, for example, the cloud device currently plays a section of video, intercepts each frame of video and performs the drawing and rendering operations on the video, and sends each frame of video after rendering to the client device, and the processing strategy matched with the client device analyzes each frame of video and performs the composition operation on the video, and then displays the video to the user, so that the performance of the client device can be more fully displayed, and meanwhile, the cloud device and the client device can still achieve the effect of optimizing the time consumption and delay of the display interaction process when displaying the interaction.
Referring to fig. 6, fig. 6 is a schematic structural diagram of a display device according to an embodiment of the present disclosure. As shown in fig. 6, the display apparatus 600 is applied to a cloud device, and at least includes a first receiving module 601, a synthesizing module 602, a first processing module 603, a second processing module 604, and a first sending module 605. Wherein:
a first receiving module 601, configured to receive a connection request sent by a user end device, where the connection request carries display parameter information of the user end device;
a composition module 602, configured to generate a composition policy based on the display parameter information;
a first processing module 603, configured to respond to the connection request and establish a connection with the ue;
a second processing module 604, configured to receive a target event sent by the user end device and generate a target image based on the target event and a composition policy under the condition that the connection with the user end device is successful;
the first sending module 605 is configured to send the target image to the user end device, so that the user end device displays the target image.
In some possible embodiments, the second processing module 604 includes:
the first processing unit is used for receiving a target event sent by the user end equipment under the condition of successful connection with the user end equipment and determining an initial image based on the target event;
and the second processing unit is used for sequentially carrying out drawing, rendering and synthesizing operations on the initial image based on the synthesizing strategy to generate a target image.
In some possible embodiments, the display parameter information includes a screen resolution and a screen refresh rate.
In some possible embodiments, the apparatus further comprises:
a third receiving module, configured to receive a quit request sent by the user end device after the first sending module 605 sends the target image to the user end device;
and the third processing module is used for responding to the quit request and disconnecting from the user side equipment.
In some possible embodiments, the apparatus further comprises:
and the fourth processing module is used for logging off the account information of the user terminal equipment and cleaning the synthesis strategy after the third processing module responds to the log-off request and is disconnected with the user terminal equipment.
In some possible embodiments, the apparatus further comprises:
a fourth sending module, configured to send, after the composition module 602 generates the composition policy based on the display parameter information, a prompt message to the user end device, where the prompt message is used to indicate that the user end device is successfully connected.
In some possible embodiments, the apparatus further comprises:
and the fifth processing module is used for storing the synthesis strategy so as to call the synthesis strategy when the cloud equipment receives the display parameter information again.
Referring to fig. 7, fig. 7 is a schematic structural diagram of another display device according to an embodiment of the present disclosure. As shown in fig. 7, the display apparatus 700 is applied to a user end device, and at least includes a second sending module 701, a third sending module 702, and a second receiving module 703.
Wherein:
a second sending module 701, configured to send a connection request to the cloud device, where the connection request carries display parameter information of the user end device; the display parameter information is used for enabling the cloud equipment to generate a synthesis strategy;
a third sending module 702, configured to send a target event to the cloud device when the connection with the cloud device is successful, so that the cloud device generates a target image based on the target event and a synthesis policy;
the second receiving module 703 is configured to receive the target image sent by the cloud device and display the target image.
In some possible embodiments, the target image is determined by the cloud device based on the target event, and the initial image is sequentially subjected to drawing, rendering and composition operations based on the composition policy to generate the target image.
In some possible embodiments, the display parameter information includes a screen resolution and a screen refresh rate.
In some possible embodiments, the apparatus further comprises:
the fourth receiving module is configured to receive an exit instruction after the second receiving module 703 sends the target event to the cloud device;
and the sixth processing module is used for responding to the exit instruction and exiting the display mode matched with the cloud equipment.
In some possible embodiments, the exit instruction comprises a gesture instruction of the user or a control instruction of the user acting on the client device.
In some possible embodiments, the sixth processing module is further configured to send, in response to the exit instruction, an exit request to the cloud device to disconnect from the cloud device.
In some possible embodiments, the apparatus further comprises:
the seventh processing module is configured to receive, after the second sending module 701 sends the connection request to the cloud device, before the third sending module sends the target event to the cloud device, prompt information sent by the cloud device under the condition that the connection with the cloud device is successful, and switch the display mode of the user side device to a display mode matched with the cloud device based on the prompt information; the prompt information is used for representing the successful connection of the user terminal equipment.
Referring to fig. 8, fig. 8 is a schematic structural diagram of an electronic device according to an embodiment of the present disclosure, which is applicable to a cloud device. As shown in fig. 8, the electronic device 800 may include: at least one processor 801, at least one network interface 804, a user interface 803, a memory 805, and at least one communication bus 802.
The communication bus 802 may be used to implement the connection communication of the above components.
The user interface 803 may include keys, and the optional user interface may include a standard wired interface or a wireless interface.
The network interface 804 may optionally include a bluetooth module, an NFC module, a Wi-Fi module, or the like.
The memory 805 may include a RAM and a ROM. Optionally, the memory 805 includes non-transitory computer readable media. The memory 805 may be used to store instructions, programs, code sets, or instruction sets. The memory 805 may include a stored program area and a stored data area, wherein the stored program area may store instructions for implementing an operating system, instructions for at least one function (such as a touch function, a sound playing function, an image playing function, etc.), instructions for implementing the various method embodiments described above, and the like; the storage data area may store data and the like referred to in the above respective method embodiments. The memory 805 may optionally be at least one memory device located remotely from the processor 801 as previously described. As shown in fig. 8, the memory 805, which is a kind of computer storage medium, may include therein an operating system, a network communication module, a user interface module, and a display application program.
In particular, the processor 801 may be configured to invoke a display application stored in the memory 805 and specifically perform the following operations:
receiving a connection request sent by user equipment, wherein the connection request carries display parameter information of the user equipment;
generating a synthesis strategy based on the display parameter information;
responding to the connection request, and establishing connection with the user terminal equipment;
under the condition of successful connection with the user terminal equipment, receiving a target event sent by the user terminal equipment, and generating a target image based on the target event and a synthesis strategy;
and sending the target image to the user terminal equipment so as to enable the user terminal equipment to display the target image.
In some possible embodiments, the processor 801 generates the target image based on the target event and the composition policy, and specifically performs:
determining an initial image based on the target event;
and sequentially carrying out drawing, rendering and synthesizing operations on the initial image based on the synthesizing strategy to generate a target image.
In some possible embodiments, the display parameter information includes a screen resolution and a screen refresh rate.
In some possible embodiments, after sending the target image to the client device, the processor 801 further performs:
receiving an exit request sent by user side equipment;
and responding to the exit request, and disconnecting from the customer premise equipment.
In some possible embodiments, in response to the logout request, after disconnecting from the customer premises device, the processor 801 further performs:
and logging out the account information of the user terminal equipment, and cleaning the synthesis strategy.
In some possible embodiments, after generating the composition policy based on the display parameter information, the processor 801 further performs:
and sending prompt information to the user side equipment, wherein the prompt information is used for representing the successful connection of the user side equipment.
In some possible embodiments, the processor 801 further performs:
and saving the synthesis strategy so that the synthesis strategy is called when the cloud equipment receives the display parameter information again.
Referring to fig. 9, fig. 9 is a schematic structural diagram of another electronic device provided in the embodiment of the present application, which can be applied to a client-side electronic device. As shown in fig. 9, the electronic device 900 may include: at least one processor 901, at least one network interface 904, a user interface 903, memory 905, a display 906, and at least one communication bus 902.
The communication bus 902 can be used for realizing the connection communication of the above components.
The user interface 903 may include keys, and the selectable user interfaces may include standard wired interfaces and wireless interfaces.
The network interface 904 may optionally include a bluetooth module, an NFC module, a Wi-Fi module, or the like.
The memory 905 may include a RAM or a ROM. Optionally, the memory 905 includes a non-transitory computer readable medium. The memory 905 may be used to store instructions, programs, code, sets of codes, or sets of instructions. The memory 905 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as a touch function, a sound playing function, an image playing function, etc.), instructions for implementing the above-described method embodiments, and the like; the storage data area may store data and the like referred to in the above respective method embodiments. The memory 905 may optionally be at least one memory device located remotely from the processor 901. As shown in fig. 9, the memory 905, which is a kind of computer storage medium, may include therein an operating system, a network communication module, a user interface module, and a display application program.
Specifically, the processor 901 may be configured to call a display application program stored in the memory 905, and specifically perform the following operations:
sending a connection request to the cloud end equipment, wherein the connection request carries display parameter information of the user end equipment; the display parameter information is used for enabling the cloud equipment to generate a synthesis strategy;
under the condition of successful connection with the cloud equipment, sending a target event to the cloud equipment so that the cloud equipment generates a target image based on the target event and a synthesis strategy;
and receiving a target image sent by the cloud equipment and displaying the target image.
In some possible embodiments, the target image is determined by the cloud device based on the target event, and the initial image is sequentially subjected to drawing, rendering and composition operations based on the composition policy to generate the target image.
In some possible embodiments, the display parameter information includes a screen resolution and a screen refresh rate.
In some possible embodiments, after receiving the target image sent by the cloud device and displaying the target image, the processor 901 further performs:
receiving an exit instruction;
and responding to the exit instruction, and exiting the display mode matched with the cloud equipment.
In some possible embodiments, the exit instruction comprises a gesture instruction of the user or a control instruction of the user acting on the client device.
In some possible embodiments, in response to the exit instruction, processor 901 further performs:
and sending an exit request to the cloud equipment so as to disconnect the cloud equipment.
In some possible embodiments, after sending the connection request to the cloud device and before sending the target event to the cloud device, the processor 901 further performs:
under the condition of successful connection with the cloud equipment, receiving prompt information sent by the cloud equipment, and switching the display mode of the user side equipment into a display mode matched with the cloud equipment on the basis of the prompt information; the prompt information is used for representing the successful connection of the user terminal equipment.
Embodiments of the present application also provide a computer-readable storage medium, which stores instructions that, when executed on a computer or a processor, cause the computer or the processor to perform one or more of the steps in the embodiments shown in fig. 2 or fig. 5. The respective constituent modules of the above-described display device may be stored in the computer-readable storage medium if they are implemented in the form of software functional units and sold or used as independent products.
In the above embodiments, the implementation may be wholly or partially realized by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When loaded and executed on a computer, cause the processes or functions described in accordance with the embodiments of the application to occur, in whole or in part. The computer may be a general purpose computer, a special purpose computer, a network of computers, or other programmable device. The computer instructions may be stored in or transmitted over a computer-readable storage medium. The computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center by wire (e.g., coaxial cable, fiber optic, Digital Subscriber Line (DSL)), or wirelessly (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device, such as a server, a data center, etc., that incorporates one or more of the available media. The usable medium may be a magnetic medium (e.g., floppy Disk, hard Disk, magnetic tape), an optical medium (e.g., Digital Versatile Disk (DVD)), or a semiconductor medium (e.g., Solid State Disk (SSD)), among others.
It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by a computer program, which can be stored in a computer-readable storage medium, and can include the processes of the embodiments of the methods described above when the program is executed. And the aforementioned storage medium includes: various media that can store program codes, such as ROM, RAM, magnetic or optical disks. The technical features in the present examples and embodiments may be arbitrarily combined without conflict.
The above-described embodiments are merely preferred embodiments of the present application, and are not intended to limit the scope of the present application, and various modifications and improvements made to the technical solutions of the present application by those skilled in the art without departing from the design spirit of the present application should fall within the protection scope defined by the claims of the present application.
Claims (18)
1. A display method is applied to a cloud device, and is characterized by comprising the following steps:
receiving a connection request sent by user end equipment, wherein the connection request carries display parameter information of the user end equipment;
generating a composition strategy based on the display parameter information;
responding to the connection request, and establishing connection with the customer premise equipment;
under the condition that the connection with the user end equipment is successful, receiving a target event sent by the user end equipment, and generating a target image based on the target event and the synthesis strategy;
and sending the target image to the user end equipment so that the user end equipment displays the target image.
2. The method of claim 1, wherein generating a target image based on the target event and the compositing strategy comprises:
determining an initial image based on the target event;
and sequentially carrying out drawing, rendering and synthesizing operations on the initial image based on the synthesizing strategy to generate a target image.
3. The method of claim 1, wherein the display parameter information comprises a screen resolution and a screen refresh rate.
4. The method according to claim 1, further comprising, after the sending the target image to the client device:
receiving an exit request sent by the user end equipment;
and responding to the exit request, and disconnecting from the customer premise equipment.
5. The method according to claim 4, wherein after disconnecting from the client device in response to the logout request, further comprising:
and logging out the account information of the user terminal equipment, and cleaning the synthesis strategy.
6. The method of claim 1, wherein after generating the compositing policy based on the display parameter information, further comprising:
and sending prompt information to the customer premise equipment, wherein the prompt information is used for representing the successful connection of the customer premise equipment.
7. The method of claim 1, further comprising:
and saving the synthesis strategy so as to call the synthesis strategy when the cloud equipment receives the display parameter information again.
8. A display method is applied to customer premise equipment and is characterized by comprising the following steps:
sending a connection request to a cloud device, wherein the connection request carries display parameter information of the user end device; the display parameter information is used for enabling the cloud end equipment to generate a synthesis strategy;
under the condition that the connection with the cloud equipment is successful, sending a target event to the cloud equipment so that the cloud equipment generates a target image based on the target event and the synthesis strategy;
and receiving the target image sent by the cloud equipment and displaying the target image.
9. The method of claim 8, wherein the target image is generated by the cloud device by determining an initial image based on the target event and performing drawing, rendering and compositing operations on the initial image in sequence based on the compositing policy.
10. The method of claim 8, wherein the display parameter information comprises a screen resolution and a screen refresh rate.
11. The method of claim 8, wherein after receiving the target image sent by the cloud device and displaying the target image, further comprising:
receiving an exit instruction;
and responding to the exit instruction, and exiting the display mode matched with the cloud end equipment.
12. The method of claim 11, wherein the exit instruction comprises a gesture instruction of a user or a control instruction of the user acting on the client device.
13. The method of claim 11, wherein after said responding to said exit instruction, further comprising:
and sending an exit request to the cloud equipment so as to be disconnected with the cloud equipment.
14. The method of claim 11, wherein after sending the connection request to the cloud device and before sending the target event to the cloud device, further comprising:
under the condition that the connection with the cloud equipment is successful, receiving prompt information sent by the cloud equipment, and switching the display mode of the user side equipment to the display mode matched with the cloud equipment on the basis of the prompt information; and the prompt information is used for representing the successful connection of the user terminal equipment.
15. The utility model provides a display device, the device is applied to high in the clouds equipment, its characterized in that includes:
the first receiving module is used for receiving a connection request sent by user end equipment, wherein the connection request carries display parameter information of the user end equipment;
a synthesis module for generating a synthesis strategy based on the display parameter information;
the first processing module is used for responding to the connection request and establishing connection with the customer premise equipment;
the second processing module is used for receiving a target event sent by the user end equipment under the condition that the connection with the user end equipment is successful, and generating a target image based on the target event and the synthesis strategy;
the first sending module is used for sending the target image to the user end equipment so as to enable the user end equipment to display the target image.
16. A display device, which is applied to a user end device, is characterized by comprising:
the second sending module is used for sending a connection request to the cloud end equipment, wherein the connection request carries the display parameter information of the user end equipment; the display parameter information is used for enabling the cloud end equipment to generate a synthesis strategy;
the third sending module is used for sending a target event to the cloud equipment under the condition that the connection with the cloud equipment is successful, so that the cloud equipment generates a target image based on the target event and the synthesis strategy;
and the second receiving module is used for receiving the target image sent by the cloud equipment and displaying the target image.
17. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the method according to any one of claims 1-14.
18. An electronic device comprising a processor, a memory, and a communication interface;
the processor is connected with the memory and the communication interface;
the memory for storing executable program code;
the processor runs a program corresponding to the executable program code by reading the executable program code stored in the memory for performing the method of any one of claims 1-14.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011488205.0A CN112653787B (en) | 2020-12-16 | 2020-12-16 | Display method, display device, storage medium and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011488205.0A CN112653787B (en) | 2020-12-16 | 2020-12-16 | Display method, display device, storage medium and electronic equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112653787A true CN112653787A (en) | 2021-04-13 |
| CN112653787B CN112653787B (en) | 2023-05-12 |
Family
ID=75354572
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011488205.0A Active CN112653787B (en) | 2020-12-16 | 2020-12-16 | Display method, display device, storage medium and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112653787B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113452944A (en) * | 2021-08-31 | 2021-09-28 | 江苏北弓智能科技有限公司 | Picture display method of cloud mobile phone |
| CN113691442A (en) * | 2021-08-16 | 2021-11-23 | 北京百度网讯科技有限公司 | Friend recommendation method, device, equipment, storage medium and program product |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20130329124A1 (en) * | 2012-06-08 | 2013-12-12 | Canon Kabushiki Kaisha | Image processing apparatus and image processing method |
| US20160286172A1 (en) * | 2015-03-25 | 2016-09-29 | Vivotek Inc. | Network communication system and method thereof |
| CN107547924A (en) * | 2017-09-12 | 2018-01-05 | 深圳市澳柯瑞科技有限公司 | Management system, management method and the display terminal of display terminal |
| CN108171783A (en) * | 2018-03-20 | 2018-06-15 | 联想(北京)有限公司 | Image rendering method, system and electronic equipment |
| CN110730364A (en) * | 2019-10-14 | 2020-01-24 | 广州微算互联信息技术有限公司 | Cloud mobile phone data transmission method and device and storage medium |
| CN110807111A (en) * | 2019-09-23 | 2020-02-18 | 北京铂石空间科技有限公司 | Three-dimensional graph processing method and device, storage medium and electronic equipment |
| CN110913202A (en) * | 2019-11-26 | 2020-03-24 | 深圳英伦科技股份有限公司 | Three-dimensional display cloud rendering method and system |
| US20200357095A1 (en) * | 2018-01-30 | 2020-11-12 | Shenzhen Skyworth-Rgb Electronic Co., Ltd. | Image processing method, device and system, and computer readable storage medium |
-
2020
- 2020-12-16 CN CN202011488205.0A patent/CN112653787B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20130329124A1 (en) * | 2012-06-08 | 2013-12-12 | Canon Kabushiki Kaisha | Image processing apparatus and image processing method |
| US20160286172A1 (en) * | 2015-03-25 | 2016-09-29 | Vivotek Inc. | Network communication system and method thereof |
| CN107547924A (en) * | 2017-09-12 | 2018-01-05 | 深圳市澳柯瑞科技有限公司 | Management system, management method and the display terminal of display terminal |
| US20200357095A1 (en) * | 2018-01-30 | 2020-11-12 | Shenzhen Skyworth-Rgb Electronic Co., Ltd. | Image processing method, device and system, and computer readable storage medium |
| CN108171783A (en) * | 2018-03-20 | 2018-06-15 | 联想(北京)有限公司 | Image rendering method, system and electronic equipment |
| CN110807111A (en) * | 2019-09-23 | 2020-02-18 | 北京铂石空间科技有限公司 | Three-dimensional graph processing method and device, storage medium and electronic equipment |
| CN110730364A (en) * | 2019-10-14 | 2020-01-24 | 广州微算互联信息技术有限公司 | Cloud mobile phone data transmission method and device and storage medium |
| CN110913202A (en) * | 2019-11-26 | 2020-03-24 | 深圳英伦科技股份有限公司 | Three-dimensional display cloud rendering method and system |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113691442A (en) * | 2021-08-16 | 2021-11-23 | 北京百度网讯科技有限公司 | Friend recommendation method, device, equipment, storage medium and program product |
| CN113691442B (en) * | 2021-08-16 | 2023-02-28 | 北京百度网讯科技有限公司 | Friend recommendation method, device, equipment, storage medium and program product |
| CN113452944A (en) * | 2021-08-31 | 2021-09-28 | 江苏北弓智能科技有限公司 | Picture display method of cloud mobile phone |
| CN113452944B (en) * | 2021-08-31 | 2021-11-02 | 江苏北弓智能科技有限公司 | Picture display method of cloud mobile phone |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112653787B (en) | 2023-05-12 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| JP7263442B2 (en) | System and method for real-time remote control of mobile applications | |
| US11683279B2 (en) | System and method of using conversational agent to collect information and trigger actions | |
| US20210194994A1 (en) | Publication of Applications Using Server-Side Virtual Screen Change Capture | |
| EP1691550A2 (en) | Interactive video | |
| CN109347722B (en) | Interaction system, method, client and background server | |
| CN102780653B (en) | Quick method, client and the system communicated in instant messaging | |
| US20150081764A1 (en) | Remote Virtualization of Mobile Apps | |
| US20150082239A1 (en) | Remote Virtualization of Mobile Apps with Transformed Ad Target Preview | |
| JP6990772B2 (en) | Information push method, storage medium, terminal equipment and server | |
| CN110694267A (en) | Cloud game implementation method and device | |
| CN104461474A (en) | Mobile terminal and screen-shooting method and device therefor | |
| WO2008008806A2 (en) | One-click universal screen sharing | |
| CN111901695B (en) | Video content interception method, device and equipment and computer storage medium | |
| WO2017121180A1 (en) | Multi-screen interaction method, device, server and non-transitory machine-readable storage medium | |
| WO2020143159A1 (en) | User interface processing method and device | |
| WO2018184443A1 (en) | Terminal screen display processing method and apparatus and storage medium | |
| CN112653787B (en) | Display method, display device, storage medium and electronic equipment | |
| CN108694009B (en) | Terminal control method and device | |
| WO2013013554A1 (en) | Program callback method and device | |
| CN110865870B (en) | Application calling method and device based on hook technology | |
| CN111790148A (en) | Information interaction method and device in game scene and computer readable medium | |
| US9537935B2 (en) | Consumer PC in the cloud | |
| CN111773730A (en) | Data processing method and device | |
| CN108574878B (en) | Data interaction method and device | |
| CN112274910A (en) | Virtual key configuration method, virtual key method and related device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |